{ "id": 193424, "title": "3rd place 0.156 public lb, 0.148 private lb", "url": "/competitions/rsna-str-pulmonary-embolism-detection/discussion/193424", "author_name": "DatNT", "post_date": "2020-10-27T02:51:24.433000", "votes": 33, "comment_count": 20, "views": 0, "content": "

Edit: Github added.

\n

First of all, congrats to all the winners.

\n

Our solution is quite straight forward. We mostly use the same tricks from last year's RSNA-IHD with some updated modules for performance improvements, label consistency, and exam-level prediction.

\n

The solution can be divided into 3 main parts:

\n

Part one is single-image training. We used CNN to predict pe_present_on_image probability for each slice and modify the CNN model by adding a FC layer with 7 units before the final binary prediction layer. This layer act as a embeddings-generator layer and the final binary will just be a linear function of these 7 inputs. We chose 7 because they actually are fine-grained labels of pe_present_on_image which are combinations of slice-level labels and exam-level labels (pe_present_on_image and rv_lv_ratio_gte_1 or pe_present_on_image and central pe for example). We also tried to grain the label finer but no luck.

\n

Part two is sequential slice-level models. From each slice in part one, we get a 7-d embeddings vector, concatenating 31 consecutive slice's embeddings vector and we have a 31-by-7 image as embeddings feature map of the center slice (in other words, for each slice, we also look at 15 slices before it and 15 slices after it). For edge cases, padding is used. Part two model is a module combining a simple shallow CNN with no pooling and a sequential model with two bi-directional LSTM layers. We get the output of this model as the final prediction of each slice, and with reversing augmentation and two models (CNN and LSTM), we have 4 outputs. Concatenate them all and we have a final 32-d embeddings vector for each slice.

\n

Part three is the exam-level CNN models. We just stacking the 32-d embeddings vector of all the slice in an exam and chose 1024-by-32 as the common image size. For exams with less than 1024 slices, we zero padding and for exams with more than 1024 slices, we truncate it. This model will predict 9 exam-level labels.

\n

My training source code:
\nhttps://github.com/moewiee/RSNA2020-Team-VinBDI-MedicalImaging

\n

You guys can also take a look at my inference kernel:
\nhttps://www.kaggle.com/moewie94/rsna-2020-inference

", "messages": [ { "id": 1061458, "postDate": "2020-10-27T02:51:24.433Z", "content": "

Edit: Github added.

\n

First of all, congrats to all the winners.

\n

Our solution is quite straight forward. We mostly use the same tricks from last year's RSNA-IHD with some updated modules for performance improvements, label consistency, and exam-level prediction.

\n

The solution can be divided into 3 main parts:

\n

Part one is single-image training. We used CNN to predict pe_present_on_image probability for each slice and modify the CNN model by adding a FC layer with 7 units before the final binary prediction layer. This layer act as a embeddings-generator layer and the final binary will just be a linear function of these 7 inputs. We chose 7 because they actually are fine-grained labels of pe_present_on_image which are combinations of slice-level labels and exam-level labels (pe_present_on_image and rv_lv_ratio_gte_1 or pe_present_on_image and central pe for example). We also tried to grain the label finer but no luck.

\n

Part two is sequential slice-level models. From each slice in part one, we get a 7-d embeddings vector, concatenating 31 consecutive slice's embeddings vector and we have a 31-by-7 image as embeddings feature map of the center slice (in other words, for each slice, we also look at 15 slices before it and 15 slices after it). For edge cases, padding is used. Part two model is a module combining a simple shallow CNN with no pooling and a sequential model with two bi-directional LSTM layers. We get the output of this model as the final prediction of each slice, and with reversing augmentation and two models (CNN and LSTM), we have 4 outputs. Concatenate them all and we have a final 32-d embeddings vector for each slice.

\n

Part three is the exam-level CNN models. We just stacking the 32-d embeddings vector of all the slice in an exam and chose 1024-by-32 as the common image size. For exams with less than 1024 slices, we zero padding and for exams with more than 1024 slices, we truncate it. This model will predict 9 exam-level labels.

\n

My training source code:
\nhttps://github.com/moewiee/RSNA2020-Team-VinBDI-MedicalImaging

\n

You guys can also take a look at my inference kernel:
\nhttps://www.kaggle.com/moewie94/rsna-2020-inference

", "rawMarkdown": "Edit: Github added.\n\nFirst of all, congrats to all the winners.\n\nOur solution is quite straight forward. We mostly use the same tricks from last year's RSNA-IHD with some updated modules for performance improvements, label consistency, and exam-level prediction.\n\nThe solution can be divided into 3 main parts:\n\nPart one is single-image training. We used CNN to predict `pe_present_on_image` probability for each slice and modify the CNN model by adding a FC layer with 7 units before the final binary prediction layer. This layer act as a embeddings-generator layer and the final binary will just be a linear function of these 7 inputs. We chose 7 because they actually are fine-grained labels of `pe_present_on_image` which are combinations of slice-level labels and exam-level labels (`pe_present_on_image` and `rv_lv_ratio_gte_1` or `pe_present_on_image` and `central pe` for example). We also tried to grain the label finer but no luck.\n\nPart two is sequential slice-level models. From each slice in part one, we get a 7-d embeddings vector, concatenating 31 consecutive slice's embeddings vector and we have a 31-by-7 image as embeddings feature map of the center slice (in other words, for each slice, we also look at 15 slices before it and 15 slices after it). For edge cases, padding is used. Part two model is a module combining a simple shallow CNN with no pooling and a sequential model with two bi-directional LSTM layers. We get the output of this model as the final prediction of each slice, and with reversing augmentation and two models (CNN and LSTM), we have 4 outputs. Concatenate them all and we have a final 32-d embeddings vector for each slice.\n\nPart three is the exam-level CNN models. We just stacking the 32-d embeddings vector of all the slice in an exam and chose 1024-by-32 as the common image size. For exams with less than 1024 slices, we zero padding and for exams with more than 1024 slices, we truncate it. This model will predict 9 exam-level labels.\n\nMy training source code:\nhttps://github.com/moewiee/RSNA2020-Team-VinBDI-MedicalImaging\n\nYou guys can also take a look at my inference kernel:\nhttps://www.kaggle.com/moewie94/rsna-2020-inference", "votes": 33 }, { "id": 1061463, "postDate": "2020-10-27T02:59:08.290Z", "content": "

@moewie94 wow, congratulations for being master😁

", "rawMarkdown": "@moewie94 wow, congratulations for being master😁", "votes": 4, "replies": [ { "id": 1062604, "postDate": "2020-10-28T02:49:41.577Z", "content": "

thank you @garybios :) failed bengali, didn't have a chance to team up on herbarium but I still hope we can collab in the future :)

", "rawMarkdown": "thank you @garybios :) failed bengali, didn't have a chance to team up on herbarium but I still hope we can collab in the future :)", "votes": 2 } ] }, { "id": 1064484, "postDate": "2020-10-30T07:30:19.297Z", "content": "

\"\"
\nCongrats, a gaggle for you!

", "rawMarkdown": "![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5168115%2F877e47111764ec535f1b872144061fe7%2Fsuper_reo.png?generation=1604043007238481&alt=media) \nCongrats, a gaggle for you!", "votes": 1 }, { "id": 1062459, "postDate": "2020-10-27T21:10:18.817Z", "content": "

This is a very interesting approach, well done on implementing it. 💪💪

", "rawMarkdown": "This is a very interesting approach, well done on implementing it. 💪💪", "votes": 1, "replies": [ { "id": 1062597, "postDate": "2020-10-28T02:45:31.590Z", "content": "

Thanks, while solving this problem i actually split screen and coding on 1 side, browsing your post and github from last year on the other side to learn :)

", "rawMarkdown": "Thanks, while solving this problem i actually split screen and coding on 1 side, browsing your post and github from last year on the other side to learn :)", "votes": 1 } ] }, { "id": 1061638, "postDate": "2020-10-27T06:48:03.447Z", "content": "

Congratulations, good thinking of reversing augmentation!

", "rawMarkdown": "Congratulations, good thinking of reversing augmentation!", "votes": 1, "replies": [ { "id": 1062600, "postDate": "2020-10-28T02:47:27.947Z", "content": "

actually sometime while scrolling through CTs, i'm quite annoyed by the reversing order thing so make it an augmentation =)

", "rawMarkdown": "actually sometime while scrolling through CTs, i'm quite annoyed by the reversing order thing so make it an augmentation =)", "votes": 1 } ] }, { "id": 1061486, "postDate": "2020-10-27T03:14:52.813Z", "content": "

congrats!!!

", "rawMarkdown": "congrats!!!", "votes": 1, "replies": [ { "id": 1062601, "postDate": "2020-10-28T02:48:05.473Z", "content": "

thank you @underwearfitting :)

", "rawMarkdown": "thank you @underwearfitting :)" } ] }, { "id": 1061459, "postDate": "2020-10-27T02:52:53.980Z", "content": "

Congrats on results and becoming master @moewie94 bro Dat and Dung :D

", "rawMarkdown": "Congrats on results and becoming master @moewie94 bro Dat and Dung :D", "votes": 2, "replies": [ { "id": 1062606, "postDate": "2020-10-28T02:50:47.490Z", "content": "

co dinh apply dot 2 chuong trinh 200AI ben anh khong ? :D

", "rawMarkdown": "co dinh apply dot 2 chuong trinh 200AI ben anh khong ? :D", "votes": 1 }, { "id": 1062612, "postDate": "2020-10-28T03:11:07.570Z", "content": "

em chuan bi phong van cai engineering residency ben vinai a oi :3, BTW len top3 r kia ;)

", "rawMarkdown": "em chuan bi phong van cai engineering residency ben vinai a oi :3, BTW len top3 r kia ;)", "votes": -1 }, { "id": 1064374, "postDate": "2020-10-30T03:51:05.620Z", "content": "

downvoted =)

", "rawMarkdown": "downvoted =)" }, { "id": 1064443, "postDate": "2020-10-30T06:00:22.987Z", "content": "

so sad :((

", "rawMarkdown": "so sad :((" } ] }, { "id": 1062476, "postDate": "2020-10-27T21:43:59.913Z", "content": "

Congrats. Great solution! Very elegant

\n

I love how you used 3 stages and made 7D embeddings. What CNN did you use in stage 1? Did you use sample weights or weighted loss for stage 1 CNN and/or stage 2?

", "rawMarkdown": "Congrats. Great solution! Very elegant\n\nI love how you used 3 stages and made 7D embeddings. What CNN did you use in stage 1? Did you use sample weights or weighted loss for stage 1 CNN and/or stage 2?", "replies": [ { "id": 1062595, "postDate": "2020-10-28T02:43:46.833Z", "content": "

for stage 1 I use pretrained EfficientNets. Should have tried resnet based architecture to enrich my ensemble :( and no, I dont use any weighting for stage 1/2, just simple binary ce

", "rawMarkdown": "for stage 1 I use pretrained EfficientNets. Should have tried resnet based architecture to enrich my ensemble :( and no, I dont use any weighting for stage 1/2, just simple binary ce", "votes": 1 } ] }, { "id": 1061840, "postDate": "2020-10-27T11:18:27.330Z", "content": "

Congratulations on becoming a master.

", "rawMarkdown": "Congratulations on becoming a master.", "replies": [ { "id": 1062599, "postDate": "2020-10-28T02:46:08.870Z", "content": "

thank you @imoore :)

", "rawMarkdown": "thank you @imoore :)" }, { "id": 1062714, "postDate": "2020-10-28T06:02:35.437Z", "content": "

Well, actually concrats on becoming the third :)

", "rawMarkdown": "Well, actually concrats on becoming the third :)" } ] }, { "id": 1063063, "postDate": "2020-10-28T13:24:32.733Z", "content": "

Thanks for sharing bro!!!

", "rawMarkdown": "Thanks for sharing bro!!!" } ], "comments": [ { "id": 1061463, "author_name": "Gary", "author_url": "", "post_date": "2020-10-27T02:59:08.290000", "content": "

@moewie94 wow, congratulations for being master😁

", "votes": 4, "replies": [ { "id": 1062604, "author_name": "DatNT", "author_url": "", "post_date": "2020-10-28T02:49:41.577000", "content": "

thank you @garybios :) failed bengali, didn't have a chance to team up on herbarium but I still hope we can collab in the future :)

", "votes": 2, "replies": [] } ] }, { "id": 1064484, "author_name": "sin", "author_url": "", "post_date": "2020-10-30T07:30:19.297000", "content": "

\"\"
\nCongrats, a gaggle for you!

", "votes": 1, "replies": [] }, { "id": 1062459, "author_name": "Darragh", "author_url": "", "post_date": "2020-10-27T21:10:18.817000", "content": "

This is a very interesting approach, well done on implementing it. 💪💪

", "votes": 1, "replies": [ { "id": 1062597, "author_name": "DatNT", "author_url": "", "post_date": "2020-10-28T02:45:31.590000", "content": "

Thanks, while solving this problem i actually split screen and coding on 1 side, browsing your post and github from last year on the other side to learn :)

", "votes": 1, "replies": [] } ] }, { "id": 1061638, "author_name": "Kerem Turgutlu", "author_url": "", "post_date": "2020-10-27T06:48:03.447000", "content": "

Congratulations, good thinking of reversing augmentation!

", "votes": 1, "replies": [ { "id": 1062600, "author_name": "DatNT", "author_url": "", "post_date": "2020-10-28T02:47:27.947000", "content": "

actually sometime while scrolling through CTs, i'm quite annoyed by the reversing order thing so make it an augmentation =)

", "votes": 1, "replies": [] } ] }, { "id": 1061486, "author_name": "sin", "author_url": "", "post_date": "2020-10-27T03:14:52.813000", "content": "

congrats!!!

", "votes": 1, "replies": [ { "id": 1062601, "author_name": "DatNT", "author_url": "", "post_date": "2020-10-28T02:48:05.473000", "content": "

thank you @underwearfitting :)

", "votes": 0, "replies": [] } ] }, { "id": 1061459, "author_name": "KhanhVD", "author_url": "", "post_date": "2020-10-27T02:52:53.980000", "content": "

Congrats on results and becoming master @moewie94 bro Dat and Dung :D

", "votes": 2, "replies": [ { "id": 1062606, "author_name": "DatNT", "author_url": "", "post_date": "2020-10-28T02:50:47.490000", "content": "

co dinh apply dot 2 chuong trinh 200AI ben anh khong ? :D

", "votes": 1, "replies": [] }, { "id": 1062612, "author_name": "KhanhVD", "author_url": "", "post_date": "2020-10-28T03:11:07.570000", "content": "

em chuan bi phong van cai engineering residency ben vinai a oi :3, BTW len top3 r kia ;)

", "votes": -1, "replies": [] }, { "id": 1064374, "author_name": "DatNT", "author_url": "", "post_date": "2020-10-30T03:51:05.620000", "content": "

downvoted =)

", "votes": 0, "replies": [] }, { "id": 1064443, "author_name": "KhanhVD", "author_url": "", "post_date": "2020-10-30T06:00:22.987000", "content": "

so sad :((

", "votes": 0, "replies": [] } ] }, { "id": 1062476, "author_name": "Chris Deotte", "author_url": "", "post_date": "2020-10-27T21:43:59.913000", "content": "

Congrats. Great solution! Very elegant

\n

I love how you used 3 stages and made 7D embeddings. What CNN did you use in stage 1? Did you use sample weights or weighted loss for stage 1 CNN and/or stage 2?

", "votes": 0, "replies": [ { "id": 1062595, "author_name": "DatNT", "author_url": "", "post_date": "2020-10-28T02:43:46.833000", "content": "

for stage 1 I use pretrained EfficientNets. Should have tried resnet based architecture to enrich my ensemble :( and no, I dont use any weighting for stage 1/2, just simple binary ce

", "votes": 1, "replies": [] } ] }, { "id": 1061840, "author_name": "Eisa", "author_url": "", "post_date": "2020-10-27T11:18:27.330000", "content": "

Congratulations on becoming a master.

", "votes": 0, "replies": [ { "id": 1062599, "author_name": "DatNT", "author_url": "", "post_date": "2020-10-28T02:46:08.870000", "content": "

thank you @imoore :)

", "votes": 0, "replies": [] }, { "id": 1062714, "author_name": "Eisa", "author_url": "", "post_date": "2020-10-28T06:02:35.437000", "content": "

Well, actually concrats on becoming the third :)

", "votes": 0, "replies": [] } ] }, { "id": 1063063, "author_name": "Md. Abdullah Al Mamun", "author_url": "", "post_date": "2020-10-28T13:24:32.733000", "content": "

Thanks for sharing bro!!!

", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1061458": "Edit: Github added.\n\nFirst of all, congrats to all the winners.\n\nOur solution is quite straight forward. We mostly use the same tricks from last year's RSNA-IHD with some updated modules for performance improvements, label consistency, and exam-level prediction.\n\nThe solution can be divided into 3 main parts:\n\nPart one is single-image training. We used CNN to predict `pe_present_on_image` probability for each slice and modify the CNN model by adding a FC layer with 7 units before the final binary prediction layer. This layer act as a embeddings-generator layer and the final binary will just be a linear function of these 7 inputs. We chose 7 because they actually are fine-grained labels of `pe_present_on_image` which are combinations of slice-level labels and exam-level labels (`pe_present_on_image` and `rv_lv_ratio_gte_1` or `pe_present_on_image` and `central pe` for example). We also tried to grain the label finer but no luck.\n\nPart two is sequential slice-level models. From each slice in part one, we get a 7-d embeddings vector, concatenating 31 consecutive slice's embeddings vector and we have a 31-by-7 image as embeddings feature map of the center slice (in other words, for each slice, we also look at 15 slices before it and 15 slices after it). For edge cases, padding is used. Part two model is a module combining a simple shallow CNN with no pooling and a sequential model with two bi-directional LSTM layers. We get the output of this model as the final prediction of each slice, and with reversing augmentation and two models (CNN and LSTM), we have 4 outputs. Concatenate them all and we have a final 32-d embeddings vector for each slice.\n\nPart three is the exam-level CNN models. We just stacking the 32-d embeddings vector of all the slice in an exam and chose 1024-by-32 as the common image size. For exams with less than 1024 slices, we zero padding and for exams with more than 1024 slices, we truncate it. This model will predict 9 exam-level labels.\n\nMy training source code:\nhttps://github.com/moewiee/RSNA2020-Team-VinBDI-MedicalImaging\n\nYou guys can also take a look at my inference kernel:\nhttps://www.kaggle.com/moewie94/rsna-2020-inference", "1061463": "@moewie94 wow, congratulations for being master😁", "1064484": "![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5168115%2F877e47111764ec535f1b872144061fe7%2Fsuper_reo.png?generation=1604043007238481&alt=media) \nCongrats, a gaggle for you!", "1062459": "This is a very interesting approach, well done on implementing it. 💪💪", "1061638": "Congratulations, good thinking of reversing augmentation!", "1061486": "congrats!!!", "1061459": "Congrats on results and becoming master @moewie94 bro Dat and Dung :D", "1062476": "Congrats. Great solution! Very elegant\n\nI love how you used 3 stages and made 7D embeddings. What CNN did you use in stage 1? Did you use sample weights or weighted loss for stage 1 CNN and/or stage 2?", "1061840": "Congratulations on becoming a master.", "1063063": "Thanks for sharing bro!!!" } }