{
  "id": 391676,
  "title": "2nd place solution",
  "url": "/competitions/rsna-breast-cancer-detection/discussion/391676",
  "author_name": "sakaku",
  "post_date": "2023-03-02T08:54:24.686000",
  "votes": 79,
  "comment_count": 25,
  "views": 0,
  "content": "<h1>2nd place solution</h1>\n<p>I would like to express my gratitude to Kaggle for hosting this meaningful competition, and to my teammates, particularly <a href=\"https://www.kaggle.com/kapenon\" target=\"_blank\">@kapenon</a>, who persevered alongside me throughout the entire competition.</p>\n<p>I would like to extend my gratitude to <a href=\"https://www.kaggle.com/theoviel\" target=\"_blank\">@theoviel</a> for providing the fast DALI inference notebook, which greatly aided in the completion of this competition. Additionally, I would like to thank <a href=\"https://www.kaggle.com/pourchot\" target=\"_blank\">@pourchot</a> for generously sharing the external data, which contained valuable positive case data that contributed to the success of our final solution.</p>\n<p>Fortunately, our team was able to get 2nd place, and I am excited to share our approach.</p>\n<h2>Summary of our approach</h2>\n<h3>Stages</h3>\n<ol>\n<li>Pretrain a single view model in 1280x1280 resolution with external dataset (Thanks to <a href=\"https://www.kaggle.com/pourchot\" target=\"_blank\">@pourchot</a>, <a href=\"https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/377790\" target=\"_blank\">Dataset</a>)</li>\n<li>Fine-tune the single view model in 1536x1536 resolution without external dataset</li>\n<li>Use the fine-tuned single view model to further fine-tune a dual view model and a four view model</li>\n</ol>\n<h3>Model</h3>\n<ul>\n<li>ConvnextV1 small (from mmclassification)</li>\n</ul>\n<h2>1. Data preparation</h2>\n<ul>\n<li>We performed manual annotation of the bounding box for the target breast in approximately 300 images. Subsequently, we trained a basic Faster R-CNN model to crop all the breast regions for the subsequent stages of training.<ul>\n<li>When annotating the bounding box, our aim was to refine the size of the box, with the intention of directing the focus of the subsequent stage model more precisely on the breast region. More specifically, our approach involved minimizing the bounding box to exclude the nipple and other extraneous body parts</li></ul></li>\n<li>Use trained Faster R-CNN to crop external dataset</li>\n</ul>\n<h2>2. Data augmentation</h2>\n<ol>\n<li><code>ShiftScaleRotate</code> from albumentation<ul>\n<li>We assumed that the model detects cancer based on the tissue or texture of the image. Therefore, it should be safe to rotate the image at any angle.</li></ul></li>\n<li>RandomFlip from mmcls</li>\n<li>RandAugment from mmcls</li>\n<li>RandomErasing from mmcls</li>\n</ol>\n<h2>3. Model</h2>\n<p>Backbone: ConvnextV1 small</p>\n<h3>Single view model</h3>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2369671%2Fbc8e8c9d3b6bb90135c6319d97115fbf%2Fsingle_view.png?generation=1677747094568388&amp;alt=media\" alt=\"singleview\"></p>\n<h4>Loss</h4>\n<ol>\n<li>Cancer: EQL loss <a href=\"https://github.com/Ezra-Yu/ACCV2022_FGIA_1st\" target=\"_blank\">link</a></li>\n<li>Aux Loss with weight 0.1:<ol>\n<li>BIRADS: EQL loss</li>\n<li>Density: EQL loss</li>\n<li>Difficult_negative_case: EQL loss, only for negative case</li>\n<li>View: CE loss</li>\n<li>Invasive: CE loss, only for positive case</li></ol></li>\n</ol>\n<h3>Dual view model</h3>\n<p>Meta Info: age, implant, machine_id<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2369671%2Fb134019ffa72690bb6136e4e393d22ed%2Fdual_view.png?generation=1677747161738296&amp;alt=media\" alt=\"dualview\">)</p>\n<h3>Multi laterality dual view model</h3>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2369671%2F38ea4a585c5f3e5954aa60c9867a60e1%2Fmulti_laterality_dual_view.png?generation=1677747196689304&amp;alt=media\" alt=\"fourview\"></p>\n<h2>4. Train</h2>\n<ul>\n<li>Optimizer: AdamW</li>\n<li>lr: 0.00015</li>\n<li>Scheduler: CosineAnnealingLR</li>\n<li>Epochs: 24</li>\n<li>Batchsize: 192 (gradient acclumulation)<ul>\n<li>Large batchsize is important in our experiments to get better performance and stable training results</li></ul></li>\n<li>EMA</li>\n</ul>\n<h2>5. Submission</h2>\n<h3>Ensemble (Not used)</h3>\n<p>During our discussion on final submission methods, two approaches were considered:</p>\n<ol>\n<li>Utilizing a single model with high resolution, without the use of an ensemble.</li>\n<li>Using ensemble with lower resolution.</li>\n</ol>\n<p>Ultimately, our team concluded that resolution plays a crucial role in detecting cancer, leading us to choose the first option of using a single model with high resolution.</p>\n<h3>About cropping</h3>\n<p>After fine-tuning the model using the original images without cropping, there was a slight improvement in the score and it remained stable. This suggests that the size of the breast may play a role in cancer detection. Taking this into account, our team decided not to crop the images for final submission.</p>\n<h3>TTA</h3>\n<ul>\n<li>Diagonal flip</li>\n</ul>\n<h3>Best submission results of each model</h3>\n<p>Our team has limited computational resources, and we only trained with five folds at the beginning of the competition. However, upon discovering that the scores obtained from fold 0 exhibit a strong correlation with both cross-validation and leaderboard results, we decided to solely focus our experimentation on this particular fold. Therefore, we only have pf1 score for fold 0 locally.</p>\n<table>\n<thead>\n<tr>\n<th>Model</th>\n<th>External Data</th>\n<th>Fold 0 score</th>\n<th>Public LB</th>\n<th>Private LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>SingleView(2nd stage fine-tuned)</td>\n<td>No</td>\n<td>0.42~44(calculated by each image)</td>\n<td>0.57</td>\n<td>0.51</td>\n</tr>\n<tr>\n<td>SingleView(2nd stage fine-tuned)</td>\n<td>Yes</td>\n<td>0.44~0.45(calculated by each image)</td>\n<td>0.58</td>\n<td>0.53</td>\n</tr>\n<tr>\n<td>DualView</td>\n<td>Yes</td>\n<td>0.55~0.56(calculated by each prediction_id)</td>\n<td>0.57</td>\n<td>0.52</td>\n</tr>\n<tr>\n<td>MultiLateralityDualView</td>\n<td>Yes</td>\n<td>-</td>\n<td>0.52</td>\n<td>0.53</td>\n</tr>\n</tbody>\n</table>\n<p>After careful evaluation, our team has selected the second SingleView model and the DualView model due to their higher Public LB score and Fold 0 score.</p>\n<p>The MultiLateralityDualView model, on the other hand, aims to compare the left and right breasts when detecting cancer. We noticed that the image style varies significantly with machine IDs, but for a particular patient, the image style remains the same. Additionally, most patients only have cancer on one side of the breast. Therefore, we believed it would be more logical to enable the model to compare the left and right sides to predict cancer.</p>\n<p>Unfortunately, we came up with this idea towards the end of the competition, and there was insufficient time to optimize and tune the model. As a result, we trained a model with the complete dataset and submitted it. However, we believe that the MultiLateralityDualView model still holds potential for future research and development.</p>\n<h2>What works</h2>\n<ul>\n<li>ConvnextV1</li>\n<li>EQL loss</li>\n<li>High resolution</li>\n<li>Large batchsize</li>\n<li>Auxiliary loss</li>\n<li>More training epochs</li>\n<li>External dataset for 1st stage pretraining</li>\n<li>MultiLateralityDualView?(Holds potential in our opinion)</li>\n</ul>\n<h2>Not work for us</h2>\n<ul>\n<li>Train models with external dataset at 2nd stage</li>\n<li>Concat cropped image horizontally</li>\n<li>Rule-based crop with cv2</li>\n<li>Upsample dataset</li>\n<li>Effcientnet, SE-ResNext, SwinTransformer</li>\n<li>Mixup augmentation</li>\n<li>Max pooling</li>\n<li>Lion optimizer</li>\n<li>BIRADS, density pseudo label</li>\n<li>Train models by machine id</li>\n<li>SWA? (Neither a reduction in the local score nor any improvement)</li>\n<li>DualView? (Minor reduction in the Public LB score)</li>\n</ul>\n<h2>Solution code</h2>\n<p>We have made our code publicly available.</p>\n<ul>\n<li><a href=\"https://github.com/ShuzhiLiu/RSNABreast2ndPlace\" target=\"_blank\">Code</a></li>\n<li><a href=\"https://www.kaggle.com/code/liushuzhi/2ndplacesolutionsingleviewinfer\" target=\"_blank\">Inference notebook</a></li>\n<li><a href=\"https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/377790\" target=\"_blank\">External data</a></li>\n</ul>\n<p>The code includes a single view model, which has demonstrated strong performance with a Private Leaderboard score of 0.53. Furthermore, the results obtained with this model can be reliably reproduced using a single RTX 3090.</p>\n<h2>Acknowledgement</h2>\n<p>We would like to express our gratitude to the Kaggle support system and the emotional support of Rist inc. </p>",
  "messages": [
    {
      "id": 2165523,
      "postDate": "2023-03-02T08:54:24.687Z",
      "content": "<h1>2nd place solution</h1>\n<p>I would like to express my gratitude to Kaggle for hosting this meaningful competition, and to my teammates, particularly <a href=\"https://www.kaggle.com/kapenon\" target=\"_blank\">@kapenon</a>, who persevered alongside me throughout the entire competition.</p>\n<p>I would like to extend my gratitude to <a href=\"https://www.kaggle.com/theoviel\" target=\"_blank\">@theoviel</a> for providing the fast DALI inference notebook, which greatly aided in the completion of this competition. Additionally, I would like to thank <a href=\"https://www.kaggle.com/pourchot\" target=\"_blank\">@pourchot</a> for generously sharing the external data, which contained valuable positive case data that contributed to the success of our final solution.</p>\n<p>Fortunately, our team was able to get 2nd place, and I am excited to share our approach.</p>\n<h2>Summary of our approach</h2>\n<h3>Stages</h3>\n<ol>\n<li>Pretrain a single view model in 1280x1280 resolution with external dataset (Thanks to <a href=\"https://www.kaggle.com/pourchot\" target=\"_blank\">@pourchot</a>, <a href=\"https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/377790\" target=\"_blank\">Dataset</a>)</li>\n<li>Fine-tune the single view model in 1536x1536 resolution without external dataset</li>\n<li>Use the fine-tuned single view model to further fine-tune a dual view model and a four view model</li>\n</ol>\n<h3>Model</h3>\n<ul>\n<li>ConvnextV1 small (from mmclassification)</li>\n</ul>\n<h2>1. Data preparation</h2>\n<ul>\n<li>We performed manual annotation of the bounding box for the target breast in approximately 300 images. Subsequently, we trained a basic Faster R-CNN model to crop all the breast regions for the subsequent stages of training.<ul>\n<li>When annotating the bounding box, our aim was to refine the size of the box, with the intention of directing the focus of the subsequent stage model more precisely on the breast region. More specifically, our approach involved minimizing the bounding box to exclude the nipple and other extraneous body parts</li></ul></li>\n<li>Use trained Faster R-CNN to crop external dataset</li>\n</ul>\n<h2>2. Data augmentation</h2>\n<ol>\n<li><code>ShiftScaleRotate</code> from albumentation<ul>\n<li>We assumed that the model detects cancer based on the tissue or texture of the image. Therefore, it should be safe to rotate the image at any angle.</li></ul></li>\n<li>RandomFlip from mmcls</li>\n<li>RandAugment from mmcls</li>\n<li>RandomErasing from mmcls</li>\n</ol>\n<h2>3. Model</h2>\n<p>Backbone: ConvnextV1 small</p>\n<h3>Single view model</h3>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2369671%2Fbc8e8c9d3b6bb90135c6319d97115fbf%2Fsingle_view.png?generation=1677747094568388&amp;alt=media\" alt=\"singleview\"></p>\n<h4>Loss</h4>\n<ol>\n<li>Cancer: EQL loss <a href=\"https://github.com/Ezra-Yu/ACCV2022_FGIA_1st\" target=\"_blank\">link</a></li>\n<li>Aux Loss with weight 0.1:<ol>\n<li>BIRADS: EQL loss</li>\n<li>Density: EQL loss</li>\n<li>Difficult_negative_case: EQL loss, only for negative case</li>\n<li>View: CE loss</li>\n<li>Invasive: CE loss, only for positive case</li></ol></li>\n</ol>\n<h3>Dual view model</h3>\n<p>Meta Info: age, implant, machine_id<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2369671%2Fb134019ffa72690bb6136e4e393d22ed%2Fdual_view.png?generation=1677747161738296&amp;alt=media\" alt=\"dualview\">)</p>\n<h3>Multi laterality dual view model</h3>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2369671%2F38ea4a585c5f3e5954aa60c9867a60e1%2Fmulti_laterality_dual_view.png?generation=1677747196689304&amp;alt=media\" alt=\"fourview\"></p>\n<h2>4. Train</h2>\n<ul>\n<li>Optimizer: AdamW</li>\n<li>lr: 0.00015</li>\n<li>Scheduler: CosineAnnealingLR</li>\n<li>Epochs: 24</li>\n<li>Batchsize: 192 (gradient acclumulation)<ul>\n<li>Large batchsize is important in our experiments to get better performance and stable training results</li></ul></li>\n<li>EMA</li>\n</ul>\n<h2>5. Submission</h2>\n<h3>Ensemble (Not used)</h3>\n<p>During our discussion on final submission methods, two approaches were considered:</p>\n<ol>\n<li>Utilizing a single model with high resolution, without the use of an ensemble.</li>\n<li>Using ensemble with lower resolution.</li>\n</ol>\n<p>Ultimately, our team concluded that resolution plays a crucial role in detecting cancer, leading us to choose the first option of using a single model with high resolution.</p>\n<h3>About cropping</h3>\n<p>After fine-tuning the model using the original images without cropping, there was a slight improvement in the score and it remained stable. This suggests that the size of the breast may play a role in cancer detection. Taking this into account, our team decided not to crop the images for final submission.</p>\n<h3>TTA</h3>\n<ul>\n<li>Diagonal flip</li>\n</ul>\n<h3>Best submission results of each model</h3>\n<p>Our team has limited computational resources, and we only trained with five folds at the beginning of the competition. However, upon discovering that the scores obtained from fold 0 exhibit a strong correlation with both cross-validation and leaderboard results, we decided to solely focus our experimentation on this particular fold. Therefore, we only have pf1 score for fold 0 locally.</p>\n<table>\n<thead>\n<tr>\n<th>Model</th>\n<th>External Data</th>\n<th>Fold 0 score</th>\n<th>Public LB</th>\n<th>Private LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>SingleView(2nd stage fine-tuned)</td>\n<td>No</td>\n<td>0.42~44(calculated by each image)</td>\n<td>0.57</td>\n<td>0.51</td>\n</tr>\n<tr>\n<td>SingleView(2nd stage fine-tuned)</td>\n<td>Yes</td>\n<td>0.44~0.45(calculated by each image)</td>\n<td>0.58</td>\n<td>0.53</td>\n</tr>\n<tr>\n<td>DualView</td>\n<td>Yes</td>\n<td>0.55~0.56(calculated by each prediction_id)</td>\n<td>0.57</td>\n<td>0.52</td>\n</tr>\n<tr>\n<td>MultiLateralityDualView</td>\n<td>Yes</td>\n<td>-</td>\n<td>0.52</td>\n<td>0.53</td>\n</tr>\n</tbody>\n</table>\n<p>After careful evaluation, our team has selected the second SingleView model and the DualView model due to their higher Public LB score and Fold 0 score.</p>\n<p>The MultiLateralityDualView model, on the other hand, aims to compare the left and right breasts when detecting cancer. We noticed that the image style varies significantly with machine IDs, but for a particular patient, the image style remains the same. Additionally, most patients only have cancer on one side of the breast. Therefore, we believed it would be more logical to enable the model to compare the left and right sides to predict cancer.</p>\n<p>Unfortunately, we came up with this idea towards the end of the competition, and there was insufficient time to optimize and tune the model. As a result, we trained a model with the complete dataset and submitted it. However, we believe that the MultiLateralityDualView model still holds potential for future research and development.</p>\n<h2>What works</h2>\n<ul>\n<li>ConvnextV1</li>\n<li>EQL loss</li>\n<li>High resolution</li>\n<li>Large batchsize</li>\n<li>Auxiliary loss</li>\n<li>More training epochs</li>\n<li>External dataset for 1st stage pretraining</li>\n<li>MultiLateralityDualView?(Holds potential in our opinion)</li>\n</ul>\n<h2>Not work for us</h2>\n<ul>\n<li>Train models with external dataset at 2nd stage</li>\n<li>Concat cropped image horizontally</li>\n<li>Rule-based crop with cv2</li>\n<li>Upsample dataset</li>\n<li>Effcientnet, SE-ResNext, SwinTransformer</li>\n<li>Mixup augmentation</li>\n<li>Max pooling</li>\n<li>Lion optimizer</li>\n<li>BIRADS, density pseudo label</li>\n<li>Train models by machine id</li>\n<li>SWA? (Neither a reduction in the local score nor any improvement)</li>\n<li>DualView? (Minor reduction in the Public LB score)</li>\n</ul>\n<h2>Solution code</h2>\n<p>We have made our code publicly available.</p>\n<ul>\n<li><a href=\"https://github.com/ShuzhiLiu/RSNABreast2ndPlace\" target=\"_blank\">Code</a></li>\n<li><a href=\"https://www.kaggle.com/code/liushuzhi/2ndplacesolutionsingleviewinfer\" target=\"_blank\">Inference notebook</a></li>\n<li><a href=\"https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/377790\" target=\"_blank\">External data</a></li>\n</ul>\n<p>The code includes a single view model, which has demonstrated strong performance with a Private Leaderboard score of 0.53. Furthermore, the results obtained with this model can be reliably reproduced using a single RTX 3090.</p>\n<h2>Acknowledgement</h2>\n<p>We would like to express our gratitude to the Kaggle support system and the emotional support of Rist inc. </p>",
      "rawMarkdown": "# 2nd place solution\n\nI would like to express my gratitude to Kaggle for hosting this meaningful competition, and to my teammates, particularly @kapenon, who persevered alongside me throughout the entire competition.\n\nI would like to extend my gratitude to @theoviel for providing the fast DALI inference notebook, which greatly aided in the completion of this competition. Additionally, I would like to thank @pourchot for generously sharing the external data, which contained valuable positive case data that contributed to the success of our final solution.\n\nFortunately, our team was able to get 2nd place, and I am excited to share our approach.\n\n## Summary of our approach\n\n### Stages\n\n1. Pretrain a single view model in 1280x1280 resolution with external dataset (Thanks to @pourchot, [Dataset](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/377790))\n2. Fine-tune the single view model in 1536x1536 resolution without external dataset\n3. Use the fine-tuned single view model to further fine-tune a dual view model and a four view model\n\n### Model\n\n* ConvnextV1 small (from mmclassification)\n\n## 1. Data preparation\n\n* We performed manual annotation of the bounding box for the target breast in approximately 300 images. Subsequently, we trained a basic Faster R-CNN model to crop all the breast regions for the subsequent stages of training.\n    * When annotating the bounding box, our aim was to refine the size of the box, with the intention of directing the focus of the subsequent stage model more precisely on the breast region. More specifically, our approach involved minimizing the bounding box to exclude the nipple and other extraneous body parts\n* Use trained Faster R-CNN to crop external dataset\n\n## 2. Data augmentation\n\n1. `ShiftScaleRotate` from albumentation\n    * We assumed that the model detects cancer based on the tissue or texture of the image. Therefore, it should be safe to rotate the image at any angle.\n2. RandomFlip from mmcls\n3. RandAugment from mmcls\n4. RandomErasing from mmcls\n\n## 3. Model\n\nBackbone: ConvnextV1 small\n\n### Single view model\n\n![singleview](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2369671%2Fbc8e8c9d3b6bb90135c6319d97115fbf%2Fsingle_view.png?generation=1677747094568388&alt=media)\n\n#### Loss\n\n1. Cancer: EQL loss [link](https://github.com/Ezra-Yu/ACCV2022_FGIA_1st)\n2. Aux Loss with weight 0.1:\n    1. BIRADS: EQL loss\n    2. Density: EQL loss\n    3. Difficult_negative_case: EQL loss, only for negative case\n    4. View: CE loss\n    5. Invasive: CE loss, only for positive case\n\n### Dual view model\n\nMeta Info: age, implant, machine_id\n![dualview](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2369671%2Fb134019ffa72690bb6136e4e393d22ed%2Fdual_view.png?generation=1677747161738296&alt=media))\n\n### Multi laterality dual view model\n\n![fourview](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2369671%2F38ea4a585c5f3e5954aa60c9867a60e1%2Fmulti_laterality_dual_view.png?generation=1677747196689304&alt=media)\n\n## 4. Train\n\n* Optimizer: AdamW\n* lr: 0.00015\n* Scheduler: CosineAnnealingLR\n* Epochs: 24\n* Batchsize: 192 (gradient acclumulation)\n    * Large batchsize is important in our experiments to get better performance and stable training results\n* EMA\n\n## 5. Submission\n\n### Ensemble (Not used)\n\nDuring our discussion on final submission methods, two approaches were considered:\n\n1. Utilizing a single model with high resolution, without the use of an ensemble.\n2. Using ensemble with lower resolution.\n\nUltimately, our team concluded that resolution plays a crucial role in detecting cancer, leading us to choose the first option of using a single model with high resolution.\n\n### About cropping\n\nAfter fine-tuning the model using the original images without cropping, there was a slight improvement in the score and it remained stable. This suggests that the size of the breast may play a role in cancer detection. Taking this into account, our team decided not to crop the images for final submission.\n\n### TTA\n* Diagonal flip\n\n### Best submission results of each model\n\nOur team has limited computational resources, and we only trained with five folds at the beginning of the competition. However, upon discovering that the scores obtained from fold 0 exhibit a strong correlation with both cross-validation and leaderboard results, we decided to solely focus our experimentation on this particular fold. Therefore, we only have pf1 score for fold 0 locally.\n\n| Model                            | External Data | Fold 0 score                                | Public LB | Private LB |\n|----------------------------------|---------------|---------------------------------------------|-----------|------------|\n| SingleView(2nd stage fine-tuned) | No            | 0.42~44(calculated by each image)           | 0.57      | 0.51       |\n| SingleView(2nd stage fine-tuned) | Yes           | 0.44~0.45(calculated by each image)         | 0.58      | 0.53       |\n| DualView                         | Yes           | 0.55~0.56(calculated by each prediction_id) | 0.57      | 0.52       |\n| MultiLateralityDualView          | Yes           | -                                           | 0.52      | 0.53       |\n\nAfter careful evaluation, our team has selected the second SingleView model and the DualView model due to their higher Public LB score and Fold 0 score.\n\nThe MultiLateralityDualView model, on the other hand, aims to compare the left and right breasts when detecting cancer. We noticed that the image style varies significantly with machine IDs, but for a particular patient, the image style remains the same. Additionally, most patients only have cancer on one side of the breast. Therefore, we believed it would be more logical to enable the model to compare the left and right sides to predict cancer.\n\nUnfortunately, we came up with this idea towards the end of the competition, and there was insufficient time to optimize and tune the model. As a result, we trained a model with the complete dataset and submitted it. However, we believe that the MultiLateralityDualView model still holds potential for future research and development.\n\n## What works\n\n* ConvnextV1\n* EQL loss\n* High resolution\n* Large batchsize\n* Auxiliary loss\n* More training epochs\n* External dataset for 1st stage pretraining\n* MultiLateralityDualView?(Holds potential in our opinion)\n\n## Not work for us\n* Train models with external dataset at 2nd stage\n* Concat cropped image horizontally\n* Rule-based crop with cv2\n* Upsample dataset\n* Effcientnet, SE-ResNext, SwinTransformer\n* Mixup augmentation\n* Max pooling\n* Lion optimizer\n* BIRADS, density pseudo label\n* Train models by machine id\n* SWA? (Neither a reduction in the local score nor any improvement)\n* DualView? (Minor reduction in the Public LB score)\n\n## Solution code\n\nWe have made our code publicly available.\n\n* [Code](https://github.com/ShuzhiLiu/RSNABreast2ndPlace)\n* [Inference notebook](https://www.kaggle.com/code/liushuzhi/2ndplacesolutionsingleviewinfer)\n* [External data](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/377790)\n\nThe code includes a single view model, which has demonstrated strong performance with a Private Leaderboard score of 0.53. Furthermore, the results obtained with this model can be reliably reproduced using a single RTX 3090.\n\n## Acknowledgement\n\nWe would like to express our gratitude to the Kaggle support system and the emotional support of Rist inc. ",
      "votes": 79
    },
    {
      "id": 2165635,
      "postDate": "2023-03-02T10:21:31.177Z",
      "content": "<p>Relevance of EQL looks particularly interesting. Thanks for sharing!</p>",
      "rawMarkdown": "Relevance of EQL looks particularly interesting. Thanks for sharing!",
      "votes": 3
    },
    {
      "id": 2236079,
      "postDate": "2023-04-26T14:11:49.130Z",
      "content": "<p>Thanks for sharing, I think MultiLateralityDualView is a great idea, it's worth exploring.</p>",
      "rawMarkdown": "Thanks for sharing, I think MultiLateralityDualView is a great idea, it's worth exploring.",
      "votes": 1
    },
    {
      "id": 2215401,
      "postDate": "2023-04-09T09:16:01.427Z",
      "content": "<p>Thank you for sharing 👍. But for the Multi laterality dual view model, what did you do with this model, can you describe in more detail? </p>",
      "rawMarkdown": "Thank you for sharing 👍. But for the Multi laterality dual view model, what did you do with this model, can you describe in more detail? ",
      "votes": 1
    },
    {
      "id": 2181190,
      "postDate": "2023-03-14T11:30:11.197Z",
      "content": "<p>MultiLateralityDualView model was great idea I think. Very professional approach! </p>",
      "rawMarkdown": "MultiLateralityDualView model was great idea I think. Very professional approach! ",
      "votes": 1
    },
    {
      "id": 2166903,
      "postDate": "2023-03-03T05:36:22.163Z",
      "content": "<p>Well deserved win👌 Thank you for sharing the approach 👍 </p>",
      "rawMarkdown": "Well deserved win👌 Thank you for sharing the approach 👍 ",
      "votes": 1,
      "replies": [
        {
          "id": 2167399,
          "postDate": "2023-03-03T13:13:56.990Z",
          "content": "<p>Hope it helps!</p>",
          "rawMarkdown": "Hope it helps!",
          "votes": 1
        }
      ]
    },
    {
      "id": 2166068,
      "postDate": "2023-03-02T15:50:42.030Z",
      "content": "<blockquote>\n  <p>Additionally, most patients only have cancer on one side of the breast.<br>\n  Interesting find👍</p>\n</blockquote>",
      "rawMarkdown": ">Additionally, most patients only have cancer on one side of the breast.\nInteresting find👍",
      "votes": 1,
      "replies": [
        {
          "id": 2167396,
          "postDate": "2023-03-03T13:11:52.633Z",
          "content": "<p>Yes, we were also looking forward to it very much, but we ran out of time.</p>",
          "rawMarkdown": "Yes, we were also looking forward to it very much, but we ran out of time."
        }
      ]
    },
    {
      "id": 2165908,
      "postDate": "2023-03-02T14:33:47.243Z",
      "content": "<p>Tidy codebase. 👍 Thanks for sharing and congratulations.</p>",
      "rawMarkdown": "Tidy codebase. 👍 Thanks for sharing and congratulations.",
      "votes": 1
    },
    {
      "id": 2165628,
      "postDate": "2023-03-02T10:06:58.210Z",
      "content": "<p><a href=\"https://www.kaggle.com/liushuzhi\" target=\"_blank\">@liushuzhi</a> , Congratulations !!!</p>",
      "rawMarkdown": "@liushuzhi , Congratulations !!!",
      "votes": 1,
      "replies": [
        {
          "id": 2167382,
          "postDate": "2023-03-03T13:04:42.880Z",
          "content": "<p>Thank you! The dataset helps a lot!</p>",
          "rawMarkdown": "Thank you! The dataset helps a lot!"
        }
      ]
    },
    {
      "id": 2165602,
      "postDate": "2023-03-02T09:49:49.187Z",
      "content": "<p>Congratulations. This is esence of ML solution - as simple as possible. Great!</p>",
      "rawMarkdown": "Congratulations. This is esence of ML solution - as simple as possible. Great!",
      "votes": 1,
      "replies": [
        {
          "id": 2167381,
          "postDate": "2023-03-03T13:03:50.703Z",
          "content": "<p>Thank you!</p>",
          "rawMarkdown": "Thank you!"
        }
      ]
    },
    {
      "id": 2467211,
      "postDate": "2023-10-04T11:01:03.770Z",
      "content": "<p>Very clean solution. However, I've a question about the dual / multiview classification.<br>\nDid you use your single view finetuned backbone to process each image individually and then concat the features or did you concat the images before inputting in the backbone?</p>\n<p>And if you did the first one, the features of which layer did you extract?</p>",
      "rawMarkdown": "Very clean solution. However, I've a question about the dual / multiview classification.\nDid you use your single view finetuned backbone to process each image individually and then concat the features or did you concat the images before inputting in the backbone?\n\nAnd if you did the first one, the features of which layer did you extract?"
    },
    {
      "id": 2177755,
      "postDate": "2023-03-11T18:36:20.973Z",
      "content": "<p>Congratulations And thank you for sharing!</p>",
      "rawMarkdown": "Congratulations And thank you for sharing!"
    },
    {
      "id": 2177556,
      "postDate": "2023-03-11T15:11:38.120Z",
      "content": "<p>simple and effective, congratulations!</p>",
      "rawMarkdown": "simple and effective, congratulations!"
    },
    {
      "id": 2173804,
      "postDate": "2023-03-08T16:47:24.647Z",
      "content": "<p>Congratulations and thanks for sharing your solution. I have one question. How did you handle cases in the for the multilateral dual view when there were images missing?</p>",
      "rawMarkdown": "Congratulations and thanks for sharing your solution. I have one question. How did you handle cases in the for the multilateral dual view when there were images missing?"
    },
    {
      "id": 2168450,
      "postDate": "2023-03-04T08:47:59.927Z",
      "content": "<p>Congratulations &gt;&lt;</p>",
      "rawMarkdown": "Congratulations ><"
    },
    {
      "id": 2166064,
      "postDate": "2023-03-02T15:49:42.903Z",
      "content": "<p>Thanks for inspiring usage of mmclasification. </p>",
      "rawMarkdown": "Thanks for inspiring usage of mmclasification. ",
      "replies": [
        {
          "id": 2166100,
          "postDate": "2023-03-02T16:01:45.143Z",
          "content": "<p>Also, how did you enable gpu with mmclasification in offline submission? Installing wheel and downloading it take almost half hour in my experiments.</p>",
          "rawMarkdown": "Also, how did you enable gpu with mmclasification in offline submission? Installing wheel and downloading it take almost half hour in my experiments.",
          "votes": 1,
          "replies": [
            {
              "id": 2167388,
              "postDate": "2023-03-03T13:07:25.907Z",
              "content": "<p>I wheeled the mmclasification related packages with another notebook and added it to inference notebook to install them. Wheel them takes time but installation is fast in my case.</p>",
              "rawMarkdown": "I wheeled the mmclasification related packages with another notebook and added it to inference notebook to install them. Wheel them takes time but installation is fast in my case.",
              "votes": 2
            },
            {
              "id": 2167471,
              "postDate": "2023-03-03T14:09:23.650Z",
              "content": "<p>Would you share that notebook or the code to install them?</p>",
              "rawMarkdown": "Would you share that notebook or the code to install them?",
              "votes": 1
            }
          ]
        }
      ]
    },
    {
      "id": 2236076,
      "postDate": "2023-04-26T14:11:20.060Z",
      "rawMarkdown": "",
      "isDeleted": true
    },
    {
      "id": 2236015,
      "postDate": "2023-04-26T13:41:12.030Z",
      "rawMarkdown": "",
      "isDeleted": true
    },
    {
      "id": 2165646,
      "postDate": "2023-03-02T10:35:17.147Z",
      "content": "<p>thanks for sharing! Congratulations !!!</p>",
      "rawMarkdown": "thanks for sharing! Congratulations !!!",
      "votes": 1
    }
  ],
  "comments": [
    {
      "id": 2165635,
      "author_name": "raddar",
      "author_url": "",
      "post_date": "2023-03-02T10:21:31.177000",
      "content": "<p>Relevance of EQL looks particularly interesting. Thanks for sharing!</p>",
      "votes": 3,
      "replies": []
    },
    {
      "id": 2236079,
      "author_name": "CJLUer",
      "author_url": "",
      "post_date": "2023-04-26T14:11:49.130000",
      "content": "<p>Thanks for sharing, I think MultiLateralityDualView is a great idea, it's worth exploring.</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 2215401,
      "author_name": "LYLY123",
      "author_url": "",
      "post_date": "2023-04-09T09:16:01.427000",
      "content": "<p>Thank you for sharing 👍. But for the Multi laterality dual view model, what did you do with this model, can you describe in more detail? </p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 2181190,
      "author_name": "Nihat Abdullayev",
      "author_url": "",
      "post_date": "2023-03-14T11:30:11.197000",
      "content": "<p>MultiLateralityDualView model was great idea I think. Very professional approach! </p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 2166903,
      "author_name": "Dheeraj Pandey",
      "author_url": "",
      "post_date": "2023-03-03T05:36:22.163000",
      "content": "<p>Well deserved win👌 Thank you for sharing the approach 👍 </p>",
      "votes": 1,
      "replies": [
        {
          "id": 2167399,
          "author_name": "sakaku",
          "author_url": "",
          "post_date": "2023-03-03T13:13:56.990000",
          "content": "<p>Hope it helps!</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 2166068,
      "author_name": "RickyLu",
      "author_url": "",
      "post_date": "2023-03-02T15:50:42.030000",
      "content": "<blockquote>\n  <p>Additionally, most patients only have cancer on one side of the breast.<br>\n  Interesting find👍</p>\n</blockquote>",
      "votes": 1,
      "replies": [
        {
          "id": 2167396,
          "author_name": "sakaku",
          "author_url": "",
          "post_date": "2023-03-03T13:11:52.633000",
          "content": "<p>Yes, we were also looking forward to it very much, but we ran out of time.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 2165908,
      "author_name": "Antti Isosalo",
      "author_url": "",
      "post_date": "2023-03-02T14:33:47.243000",
      "content": "<p>Tidy codebase. 👍 Thanks for sharing and congratulations.</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 2165628,
      "author_name": "Laurent Pourchot",
      "author_url": "",
      "post_date": "2023-03-02T10:06:58.210000",
      "content": "<p><a href=\"https://www.kaggle.com/liushuzhi\" target=\"_blank\">@liushuzhi</a> , Congratulations !!!</p>",
      "votes": 1,
      "replies": [
        {
          "id": 2167382,
          "author_name": "sakaku",
          "author_url": "",
          "post_date": "2023-03-03T13:04:42.880000",
          "content": "<p>Thank you! The dataset helps a lot!</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 2165602,
      "author_name": "Remek Kinas",
      "author_url": "",
      "post_date": "2023-03-02T09:49:49.187000",
      "content": "<p>Congratulations. This is esence of ML solution - as simple as possible. Great!</p>",
      "votes": 1,
      "replies": [
        {
          "id": 2167381,
          "author_name": "sakaku",
          "author_url": "",
          "post_date": "2023-03-03T13:03:50.703000",
          "content": "<p>Thank you!</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 2467211,
      "author_name": "Kaschi14",
      "author_url": "",
      "post_date": "2023-10-04T11:01:03.770000",
      "content": "<p>Very clean solution. However, I've a question about the dual / multiview classification.<br>\nDid you use your single view finetuned backbone to process each image individually and then concat the features or did you concat the images before inputting in the backbone?</p>\n<p>And if you did the first one, the features of which layer did you extract?</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2177755,
      "author_name": "Tsvetan Rankov",
      "author_url": "",
      "post_date": "2023-03-11T18:36:20.973000",
      "content": "<p>Congratulations And thank you for sharing!</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2177556,
      "author_name": "daeek14",
      "author_url": "",
      "post_date": "2023-03-11T15:11:38.120000",
      "content": "<p>simple and effective, congratulations!</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2173804,
      "author_name": "Fernando Cossio",
      "author_url": "",
      "post_date": "2023-03-08T16:47:24.647000",
      "content": "<p>Congratulations and thanks for sharing your solution. I have one question. How did you handle cases in the for the multilateral dual view when there were images missing?</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2168450,
      "author_name": "Nam Van Nguyen",
      "author_url": "",
      "post_date": "2023-03-04T08:47:59.927000",
      "content": "<p>Congratulations &gt;&lt;</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2166064,
      "author_name": "Erdi Kılıç",
      "author_url": "",
      "post_date": "2023-03-02T15:49:42.903000",
      "content": "<p>Thanks for inspiring usage of mmclasification. </p>",
      "votes": 0,
      "replies": [
        {
          "id": 2166100,
          "author_name": "Erdi Kılıç",
          "author_url": "",
          "post_date": "2023-03-02T16:01:45.143000",
          "content": "<p>Also, how did you enable gpu with mmclasification in offline submission? Installing wheel and downloading it take almost half hour in my experiments.</p>",
          "votes": 1,
          "replies": [
            {
              "id": 2167388,
              "author_name": "sakaku",
              "author_url": "",
              "post_date": "2023-03-03T13:07:25.907000",
              "content": "<p>I wheeled the mmclasification related packages with another notebook and added it to inference notebook to install them. Wheel them takes time but installation is fast in my case.</p>",
              "votes": 2,
              "replies": []
            },
            {
              "id": 2167471,
              "author_name": "Erdi Kılıç",
              "author_url": "",
              "post_date": "2023-03-03T14:09:23.650000",
              "content": "<p>Would you share that notebook or the code to install them?</p>",
              "votes": 1,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 2236076,
      "author_name": "",
      "author_url": "",
      "post_date": "2023-04-26T14:11:20.060000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2236015,
      "author_name": "",
      "author_url": "",
      "post_date": "2023-04-26T13:41:12.030000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2165646,
      "author_name": "Bá Huy Nguyễn",
      "author_url": "",
      "post_date": "2023-03-02T10:35:17.147000",
      "content": "<p>thanks for sharing! Congratulations !!!</p>",
      "votes": 1,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2165523": "# 2nd place solution\n\nI would like to express my gratitude to Kaggle for hosting this meaningful competition, and to my teammates, particularly @kapenon, who persevered alongside me throughout the entire competition.\n\nI would like to extend my gratitude to @theoviel for providing the fast DALI inference notebook, which greatly aided in the completion of this competition. Additionally, I would like to thank @pourchot for generously sharing the external data, which contained valuable positive case data that contributed to the success of our final solution.\n\nFortunately, our team was able to get 2nd place, and I am excited to share our approach.\n\n## Summary of our approach\n\n### Stages\n\n1. Pretrain a single view model in 1280x1280 resolution with external dataset (Thanks to @pourchot, [Dataset](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/377790))\n2. Fine-tune the single view model in 1536x1536 resolution without external dataset\n3. Use the fine-tuned single view model to further fine-tune a dual view model and a four view model\n\n### Model\n\n* ConvnextV1 small (from mmclassification)\n\n## 1. Data preparation\n\n* We performed manual annotation of the bounding box for the target breast in approximately 300 images. Subsequently, we trained a basic Faster R-CNN model to crop all the breast regions for the subsequent stages of training.\n    * When annotating the bounding box, our aim was to refine the size of the box, with the intention of directing the focus of the subsequent stage model more precisely on the breast region. More specifically, our approach involved minimizing the bounding box to exclude the nipple and other extraneous body parts\n* Use trained Faster R-CNN to crop external dataset\n\n## 2. Data augmentation\n\n1. `ShiftScaleRotate` from albumentation\n    * We assumed that the model detects cancer based on the tissue or texture of the image. Therefore, it should be safe to rotate the image at any angle.\n2. RandomFlip from mmcls\n3. RandAugment from mmcls\n4. RandomErasing from mmcls\n\n## 3. Model\n\nBackbone: ConvnextV1 small\n\n### Single view model\n\n![singleview](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2369671%2Fbc8e8c9d3b6bb90135c6319d97115fbf%2Fsingle_view.png?generation=1677747094568388&alt=media)\n\n#### Loss\n\n1. Cancer: EQL loss [link](https://github.com/Ezra-Yu/ACCV2022_FGIA_1st)\n2. Aux Loss with weight 0.1:\n    1. BIRADS: EQL loss\n    2. Density: EQL loss\n    3. Difficult_negative_case: EQL loss, only for negative case\n    4. View: CE loss\n    5. Invasive: CE loss, only for positive case\n\n### Dual view model\n\nMeta Info: age, implant, machine_id\n![dualview](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2369671%2Fb134019ffa72690bb6136e4e393d22ed%2Fdual_view.png?generation=1677747161738296&alt=media))\n\n### Multi laterality dual view model\n\n![fourview](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2369671%2F38ea4a585c5f3e5954aa60c9867a60e1%2Fmulti_laterality_dual_view.png?generation=1677747196689304&alt=media)\n\n## 4. Train\n\n* Optimizer: AdamW\n* lr: 0.00015\n* Scheduler: CosineAnnealingLR\n* Epochs: 24\n* Batchsize: 192 (gradient acclumulation)\n    * Large batchsize is important in our experiments to get better performance and stable training results\n* EMA\n\n## 5. Submission\n\n### Ensemble (Not used)\n\nDuring our discussion on final submission methods, two approaches were considered:\n\n1. Utilizing a single model with high resolution, without the use of an ensemble.\n2. Using ensemble with lower resolution.\n\nUltimately, our team concluded that resolution plays a crucial role in detecting cancer, leading us to choose the first option of using a single model with high resolution.\n\n### About cropping\n\nAfter fine-tuning the model using the original images without cropping, there was a slight improvement in the score and it remained stable. This suggests that the size of the breast may play a role in cancer detection. Taking this into account, our team decided not to crop the images for final submission.\n\n### TTA\n* Diagonal flip\n\n### Best submission results of each model\n\nOur team has limited computational resources, and we only trained with five folds at the beginning of the competition. However, upon discovering that the scores obtained from fold 0 exhibit a strong correlation with both cross-validation and leaderboard results, we decided to solely focus our experimentation on this particular fold. Therefore, we only have pf1 score for fold 0 locally.\n\n| Model                            | External Data | Fold 0 score                                | Public LB | Private LB |\n|----------------------------------|---------------|---------------------------------------------|-----------|------------|\n| SingleView(2nd stage fine-tuned) | No            | 0.42~44(calculated by each image)           | 0.57      | 0.51       |\n| SingleView(2nd stage fine-tuned) | Yes           | 0.44~0.45(calculated by each image)         | 0.58      | 0.53       |\n| DualView                         | Yes           | 0.55~0.56(calculated by each prediction_id) | 0.57      | 0.52       |\n| MultiLateralityDualView          | Yes           | -                                           | 0.52      | 0.53       |\n\nAfter careful evaluation, our team has selected the second SingleView model and the DualView model due to their higher Public LB score and Fold 0 score.\n\nThe MultiLateralityDualView model, on the other hand, aims to compare the left and right breasts when detecting cancer. We noticed that the image style varies significantly with machine IDs, but for a particular patient, the image style remains the same. Additionally, most patients only have cancer on one side of the breast. Therefore, we believed it would be more logical to enable the model to compare the left and right sides to predict cancer.\n\nUnfortunately, we came up with this idea towards the end of the competition, and there was insufficient time to optimize and tune the model. As a result, we trained a model with the complete dataset and submitted it. However, we believe that the MultiLateralityDualView model still holds potential for future research and development.\n\n## What works\n\n* ConvnextV1\n* EQL loss\n* High resolution\n* Large batchsize\n* Auxiliary loss\n* More training epochs\n* External dataset for 1st stage pretraining\n* MultiLateralityDualView?(Holds potential in our opinion)\n\n## Not work for us\n* Train models with external dataset at 2nd stage\n* Concat cropped image horizontally\n* Rule-based crop with cv2\n* Upsample dataset\n* Effcientnet, SE-ResNext, SwinTransformer\n* Mixup augmentation\n* Max pooling\n* Lion optimizer\n* BIRADS, density pseudo label\n* Train models by machine id\n* SWA? (Neither a reduction in the local score nor any improvement)\n* DualView? (Minor reduction in the Public LB score)\n\n## Solution code\n\nWe have made our code publicly available.\n\n* [Code](https://github.com/ShuzhiLiu/RSNABreast2ndPlace)\n* [Inference notebook](https://www.kaggle.com/code/liushuzhi/2ndplacesolutionsingleviewinfer)\n* [External data](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/377790)\n\nThe code includes a single view model, which has demonstrated strong performance with a Private Leaderboard score of 0.53. Furthermore, the results obtained with this model can be reliably reproduced using a single RTX 3090.\n\n## Acknowledgement\n\nWe would like to express our gratitude to the Kaggle support system and the emotional support of Rist inc. ",
    "2165635": "Relevance of EQL looks particularly interesting. Thanks for sharing!",
    "2236079": "Thanks for sharing, I think MultiLateralityDualView is a great idea, it's worth exploring.",
    "2215401": "Thank you for sharing 👍. But for the Multi laterality dual view model, what did you do with this model, can you describe in more detail? ",
    "2181190": "MultiLateralityDualView model was great idea I think. Very professional approach! ",
    "2166903": "Well deserved win👌 Thank you for sharing the approach 👍 ",
    "2166068": ">Additionally, most patients only have cancer on one side of the breast.\nInteresting find👍",
    "2165908": "Tidy codebase. 👍 Thanks for sharing and congratulations.",
    "2165628": "@liushuzhi , Congratulations !!!",
    "2165602": "Congratulations. This is esence of ML solution - as simple as possible. Great!",
    "2467211": "Very clean solution. However, I've a question about the dual / multiview classification.\nDid you use your single view finetuned backbone to process each image individually and then concat the features or did you concat the images before inputting in the backbone?\n\nAnd if you did the first one, the features of which layer did you extract?",
    "2177755": "Congratulations And thank you for sharing!",
    "2177556": "simple and effective, congratulations!",
    "2173804": "Congratulations and thanks for sharing your solution. I have one question. How did you handle cases in the for the multilateral dual view when there were images missing?",
    "2168450": "Congratulations ><",
    "2166064": "Thanks for inspiring usage of mmclasification. ",
    "2236076": "",
    "2236015": "",
    "2165646": "thanks for sharing! Congratulations !!!"
  }
}