We would like to thank the competition host(s) and Kaggle for organizing the competition and congratulate all the winners, and anyone who benefited in some way from the competition. Special thanks to my teammates @erniechiew, @css919, @zehuigong and @stephkua
\nPre-processing of the bounding box annotations
\nI believe that most participants used wbf to merge the original boxes, as this notebook(https://www.kaggle.com/sreevishnudamodaran/vinbigdata-fusing-bboxes-coco-dataset). I apply the same idea that averaging the coordinate of boxes if their IOUs are higher than a threshold, e.g., 0.2. Then using this pre-processed boxes as my annotations in this competition.
\nOur YOLO-V4 Detector
\n• Optimizer: SGD, BS = 32, LR=0.001, CosineAnnealing.
\n• Image size 1280 for P6, 1536 for P7.
\n• Augmentations:
\n(1) RandomFlipping (p=0.5)
\n(2) RandomScaling (0.2-1.8)
\n(3) Mosaic
\n(4) Mixup(p=0.2)
\n(5) Random translation (0.5)
We notice that the processing method of annotation between training and test set are different (see this notebook https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/211035). So we analyze how many images each radiologist annotates, and found that rad 8, 9, and 10 annotate the majority of bounding boxes. So we train three detectors for rad 8, 9, and 10, separately. The score of the separate detector are showed as follow: (CV/LB/Private)
\nRad 8: 0.498/0.252/0.233
\nRad9: 0.376/0.213/0.234
\nRad10: 0.384/0.228/0.226
And then I think, why not try to merge them all? Then after I merge them, I got 0.274 on LB, and a private score of 0.248.
\nThen, I try to train the detectors on all radiologists. For this part, YOLO-V4 P6 and P7 version are employed. (LB/Private)
\nP6: 0.278/0.261
\nP7: 0.264/0.263
\nAfter I got these models, I merge them all with the step one submission, and we achieve 0.280 on the public leaderboard, and 0.272 on the private.
Things not work:
\n(1) Albu augmentation, e.g., CLAHE, RandomBrigheness
\n(2) Class aware sampling
\n(3) Training with larger image 1536 for P6
\n(4) Focal loss + BCE
\n(5) Pretraining on RSNA dataset and finetune on training set of this competition
\n(6) Pseudo labeling on the test set
\n(7) Train specialized detector on class ILD only
Things that we haven’t tried for details:
\n(1) Pseudo labeling the RSNA or other dataset, denoted as PseData, and training on PseData + TrainSet, this improve the CV from 0.498 to 0.55+, but not improve the LB, however, it did make a improvement on the private score, 0.250 vs. 0.255.
\n(2) Crop the boxes from abnormal images and paste them on normal images, with different sampling ratios for different classes.
As highlighted in the discussion forums during the competition, there is a big discrepancy between CV and LB scores. To understand this discrepancy better, we decided to compare LB and local CV scores for each individual class. The AP scores of two classes -- aortic enlargement (class 0) and cardiomegaly (class 3) stood out to us. Below are the individual class AP scores from one of our earlier detectors:
\n- Aortic enlargement: 0.712 CV / 0.24 LB
\n- Cardiomegaly: 0.772 CV / 0.81 LB
Intuitively, we felt that LB performance on these classes should follow the same trends since they are i) “easy” (based on their high local CV scores), ii) relatively popular assuming the test set follows a similar disease distribution as the training set. The observation that aortic enlargement performs drastically worse on LB warranted a further investigation.
\nWe found that CV scores for aortic enlargement on images annotated by non-R8/R9/R10 radiologists were unusually low (similar to our LB score for this class). Upon further analysis, we found that:
\nAnd so we decided to fine-tune one of our detectors on non-R8/R9/R10 rad_ids only. We also trained this detector only on aortic enlargement and calcification classes in an attempt to make the model focus on better distinguishing these two classes.
\nWe then replaced our final predictions for aortic enlargement from our main detector with predictions from this specialized detector. This yielded an increase in overall mAP of approximately 0.020 on public LB, and 0.010 on private LB from our original score.
\nAfter reading the great solutions posted by other teams, it is comforting to know that several teams also leveraged this data peculiarity in similar ways!
\nA detailed breakdown of this step has been posted in a separate discussion(https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229636).
\ncode is https://github.com/Scu-sen/VinBigData-Chest-X-ray-Abnormalities-Detection
Ensemble of 11 detection models with 2-class classifier Public: 0.290 Private:0.284
\n +Specialized Aortic detector Public:0.310 Private: 0.294
\n + Multi-label Classifier-based Post-processing Public: 0.348 Private: 0.305
We would like to thank the competition host(s) and Kaggle for organizing the competition and congratulate all the winners, and anyone who benefited in some way from the competition. Special thanks to my teammates @erniechiew, @css919, @zehuigong and @stephkua
\nPre-processing of the bounding box annotations
\nI believe that most participants used wbf to merge the original boxes, as this notebook(https://www.kaggle.com/sreevishnudamodaran/vinbigdata-fusing-bboxes-coco-dataset). I apply the same idea that averaging the coordinate of boxes if their IOUs are higher than a threshold, e.g., 0.2. Then using this pre-processed boxes as my annotations in this competition.
\nOur YOLO-V4 Detector
\n• Optimizer: SGD, BS = 32, LR=0.001, CosineAnnealing.
\n• Image size 1280 for P6, 1536 for P7.
\n• Augmentations:
\n(1) RandomFlipping (p=0.5)
\n(2) RandomScaling (0.2-1.8)
\n(3) Mosaic
\n(4) Mixup(p=0.2)
\n(5) Random translation (0.5)
We notice that the processing method of annotation between training and test set are different (see this notebook https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/211035). So we analyze how many images each radiologist annotates, and found that rad 8, 9, and 10 annotate the majority of bounding boxes. So we train three detectors for rad 8, 9, and 10, separately. The score of the separate detector are showed as follow: (CV/LB/Private)
\nRad 8: 0.498/0.252/0.233
\nRad9: 0.376/0.213/0.234
\nRad10: 0.384/0.228/0.226
And then I think, why not try to merge them all? Then after I merge them, I got 0.274 on LB, and a private score of 0.248.
\nThen, I try to train the detectors on all radiologists. For this part, YOLO-V4 P6 and P7 version are employed. (LB/Private)
\nP6: 0.278/0.261
\nP7: 0.264/0.263
\nAfter I got these models, I merge them all with the step one submission, and we achieve 0.280 on the public leaderboard, and 0.272 on the private.
Things not work:
\n(1) Albu augmentation, e.g., CLAHE, RandomBrigheness
\n(2) Class aware sampling
\n(3) Training with larger image 1536 for P6
\n(4) Focal loss + BCE
\n(5) Pretraining on RSNA dataset and finetune on training set of this competition
\n(6) Pseudo labeling on the test set
\n(7) Train specialized detector on class ILD only
Things that we haven’t tried for details:
\n(1) Pseudo labeling the RSNA or other dataset, denoted as PseData, and training on PseData + TrainSet, this improve the CV from 0.498 to 0.55+, but not improve the LB, however, it did make a improvement on the private score, 0.250 vs. 0.255.
\n(2) Crop the boxes from abnormal images and paste them on normal images, with different sampling ratios for different classes.
As highlighted in the discussion forums during the competition, there is a big discrepancy between CV and LB scores. To understand this discrepancy better, we decided to compare LB and local CV scores for each individual class. The AP scores of two classes -- aortic enlargement (class 0) and cardiomegaly (class 3) stood out to us. Below are the individual class AP scores from one of our earlier detectors:
\n- Aortic enlargement: 0.712 CV / 0.24 LB
\n- Cardiomegaly: 0.772 CV / 0.81 LB
Intuitively, we felt that LB performance on these classes should follow the same trends since they are i) “easy” (based on their high local CV scores), ii) relatively popular assuming the test set follows a similar disease distribution as the training set. The observation that aortic enlargement performs drastically worse on LB warranted a further investigation.
\nWe found that CV scores for aortic enlargement on images annotated by non-R8/R9/R10 radiologists were unusually low (similar to our LB score for this class). Upon further analysis, we found that:
\nAnd so we decided to fine-tune one of our detectors on non-R8/R9/R10 rad_ids only. We also trained this detector only on aortic enlargement and calcification classes in an attempt to make the model focus on better distinguishing these two classes.
\nWe then replaced our final predictions for aortic enlargement from our main detector with predictions from this specialized detector. This yielded an increase in overall mAP of approximately 0.020 on public LB, and 0.010 on private LB from our original score.
\nAfter reading the great solutions posted by other teams, it is comforting to know that several teams also leveraged this data peculiarity in similar ways!
\nA detailed breakdown of this step has been posted in a separate discussion(https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229636).
\ncode is https://github.com/Scu-sen/VinBigData-Chest-X-ray-Abnormalities-Detection
Ensemble of 11 detection models with 2-class classifier Public: 0.290 Private:0.284
\n +Specialized Aortic detector Public:0.310 Private: 0.294
\n + Multi-label Classifier-based Post-processing Public: 0.348 Private: 0.305
Congratulations!
", "rawMarkdown": "Congratulations!", "votes": 1 }, { "id": 1259743, "postDate": "2021-04-01T16:20:11.617Z", "content": "Good job! Congrats and thanks for the detailed writeup @scusywxy
", "rawMarkdown": "Good job! Congrats and thanks for the detailed writeup @scusywxy", "votes": 1 }, { "id": 1259695, "postDate": "2021-04-01T15:37:24.083Z", "content": "Congratulations! Well done team! Are all your detection models Yolo?
", "rawMarkdown": "Congratulations! Well done team! Are all your detection models Yolo? ", "votes": 1, "replies": [ { "id": 1259752, "postDate": "2021-04-01T16:27:13.877Z", "content": "Yes, we only use single model YOLOv4
", "rawMarkdown": "Yes, we only use single model YOLOv4", "replies": [ { "id": 3307972, "postDate": "2025-10-28T09:13:55.117Z", "content": "An Internship opportunity has just arrived to your Port!!!!!!!! If interested please DM , Join me into this Relation,\nAbout BNP Paribas India Solutions
\nEstablished in 2005, BNP Paribas India Solutions is a wholly owned subsidiary of BNP Paribas SA, European Union’s leading bank with an international reach. With delivery centers located in Bengaluru, Chennai and Mumbai, we are a 24x7 global delivery center. India Solutions services three business lines: Corporate and Institutional Banking, Investment Solutions and Retail Banking for BNP Paribas across the Group. Driving innovation and growth, we are harnessing the potential of over 10000 employees, to provide support and develop best-in-class solutions.\n IYKYK
", "rawMarkdown": "An Internship opportunity has just arrived to your Port!!!!!!!! If interested please DM , Join me into this Relation,\nAbout BNP Paribas India Solutions\n\nEstablished in 2005, BNP Paribas India Solutions is a wholly owned subsidiary of BNP Paribas SA, European Union’s leading bank with an international reach. With delivery centers located in Bengaluru, Chennai and Mumbai, we are a 24x7 global delivery center. India Solutions services three business lines: Corporate and Institutional Banking, Investment Solutions and Retail Banking for BNP Paribas across the Group. Driving innovation and growth, we are harnessing the potential of over 10000 employees, to provide support and develop best-in-class solutions.\n IYKYK" } ] } ] } ], "comments": [ { "id": 1270211, "author_name": "Wonjun Park", "author_url": "", "post_date": "2021-04-11T12:29:39.603000", "content": "Congratulations!
", "votes": 1, "replies": [] }, { "id": 1259743, "author_name": "KhanhVD", "author_url": "", "post_date": "2021-04-01T16:20:11.617000", "content": "Good job! Congrats and thanks for the detailed writeup @scusywxy
", "votes": 1, "replies": [] }, { "id": 1259695, "author_name": "Chris Deotte", "author_url": "", "post_date": "2021-04-01T15:37:24.083000", "content": "Congratulations! Well done team! Are all your detection models Yolo?
", "votes": 1, "replies": [ { "id": 1259752, "author_name": "Sen Yang", "author_url": "", "post_date": "2021-04-01T16:27:13.877000", "content": "Yes, we only use single model YOLOv4
", "votes": 0, "replies": [ { "id": 3307972, "author_name": "ANIKET NARAYAN NIKALJE", "author_url": "", "post_date": "2025-10-28T09:13:55.117000", "content": "An Internship opportunity has just arrived to your Port!!!!!!!! If interested please DM , Join me into this Relation,\nAbout BNP Paribas India Solutions
\nEstablished in 2005, BNP Paribas India Solutions is a wholly owned subsidiary of BNP Paribas SA, European Union’s leading bank with an international reach. With delivery centers located in Bengaluru, Chennai and Mumbai, we are a 24x7 global delivery center. India Solutions services three business lines: Corporate and Institutional Banking, Investment Solutions and Retail Banking for BNP Paribas across the Group. Driving innovation and growth, we are harnessing the potential of over 10000 employees, to provide support and develop best-in-class solutions.\n IYKYK
", "votes": 0, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "1258432": "We would like to thank the competition host(s) and Kaggle for organizing the competition and congratulate all the winners, and anyone who benefited in some way from the competition. Special thanks to my teammates @erniechiew, @css919, @zehuigong and @stephkua \n\n# Solution Components:\n\n1. Detection models\n2. Specialized detector for aortic enlargement\n3. Multi-label classifier-based post-processing\n\n## 1.Detection models\nPre-processing of the bounding box annotations\nI believe that most participants used wbf to merge the original boxes, as this notebook(https://www.kaggle.com/sreevishnudamodaran/vinbigdata-fusing-bboxes-coco-dataset). I apply the same idea that averaging the coordinate of boxes if their IOUs are higher than a threshold, e.g., 0.2. Then using this pre-processed boxes as my annotations in this competition.\nOur YOLO-V4 Detector\n•\tOptimizer: SGD, BS = 32, LR=0.001, CosineAnnealing.\n•\tImage size 1280 for P6, 1536 for P7.\n•\tAugmentations: \n(1)\tRandomFlipping (p=0.5)\n(2)\tRandomScaling (0.2-1.8)\n(3)\tMosaic\n(4)\tMixup(p=0.2)\n(5)\tRandom translation (0.5)\n\n### 1.1\tTraining on the boxes of a single radiologist (LB 0.274)\nWe notice that the processing method of annotation between training and test set are different (see this notebook https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/211035). So we analyze how many images each radiologist annotates, and found that rad 8, 9, and 10 annotate the majority of bounding boxes. So we train three detectors for rad 8, 9, and 10, separately. The score of the separate detector are showed as follow: (CV/LB/Private)\nRad 8: 0.498/0.252/0.233 \nRad9: 0.376/0.213/0.234\nRad10: 0.384/0.228/0.226\n\nAnd then I think, why not try to merge them all? Then after I merge them, I got 0.274 on LB, and a private score of 0.248. \n### 1.2 To the LB 0.280.\nThen, I try to train the detectors on all radiologists. For this part, YOLO-V4 P6 and P7 version are employed. (LB/Private)\nP6: 0.278/0.261\nP7: 0.264/0.263\nAfter I got these models, I merge them all with the step one submission, and we achieve 0.280 on the public leaderboard, and 0.272 on the private.\n\nThings not work:\n(1)\tAlbu augmentation, e.g., CLAHE, RandomBrigheness\n(2)\tClass aware sampling\n(3)\tTraining with larger image 1536 for P6\n(4)\tFocal loss + BCE\n(5)\tPretraining on RSNA dataset and finetune on training set of this competition\n(6)\tPseudo labeling on the test set\n(7)\tTrain specialized detector on class ILD only\n\nThings that we haven’t tried for details:\n(1)\tPseudo labeling the RSNA or other dataset, denoted as PseData, and training on PseData + TrainSet, this improve the CV from 0.498 to 0.55+, but not improve the LB, however, it did make a improvement on the private score, 0.250 vs. 0.255.\n(2)\tCrop the boxes from abnormal images and paste them on normal images, with different sampling ratios for different classes.\n## 2. Specialized Detector for Aortic Enlargement\n\nAs highlighted in the discussion forums during the competition, there is a big discrepancy between CV and LB scores. To understand this discrepancy better, we decided to compare LB and local CV scores for each individual class. The AP scores of two classes -- aortic enlargement (class 0) and cardiomegaly (class 3) stood out to us. Below are the individual class AP scores from one of our earlier detectors:\n\n**- Aortic enlargement: 0.712 CV / 0.24 LB**\n**- Cardiomegaly: 0.772 CV / 0.81 LB**\n\nIntuitively, we felt that LB performance on these classes should follow the same trends since they are i) “easy” (based on their high local CV scores), ii) relatively popular assuming the test set follows a similar disease distribution as the training set. The observation that aortic enlargement performs drastically worse on LB warranted a further investigation.\n\nWe found that CV scores for aortic enlargement on images annotated by non-R8/R9/R10 radiologists were unusually low (similar to our LB score for this class). Upon further analysis, we found that:\n1. Aortic enlargement bboxes annotated by non-R8/R9/R10 rad_ids tend to be larger than those annotated by R8, R9, R10.\n2. Our model has a non-negligible tendency to mixup Aortic enlargement and calcification for images annotated by non-R8/R9/R10 rad_ids.\n\nAnd so we decided to fine-tune one of our detectors on non-R8/R9/R10 rad_ids only. We also trained this detector only on aortic enlargement and calcification classes in an attempt to make the model focus on better distinguishing these two classes.\n\nWe then replaced our final predictions for aortic enlargement from our main detector with predictions from this specialized detector. **This yielded an increase in overall mAP of approximately 0.020 on public LB, and 0.010 on private LB from our original score.**\n\nAfter reading the great solutions posted by other teams, it is comforting to know that several teams also leveraged this data peculiarity in similar ways!\n\n## 3. Multi-label Classifier-based Post-processing\nA detailed breakdown of this step has been posted in a separate discussion(https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229636).\ncode is https://github.com/Scu-sen/VinBigData-Chest-X-ray-Abnormalities-Detection\n### Final Results\n\n \nEnsemble of 11 detection models with 2-class classifier Public: 0.290 Private:0.284\n +Specialized Aortic detector Public:0.310 Private: 0.294 \n + Multi-label Classifier-based Post-processing Public: 0.348 Private: 0.305 \n\n\n\n\n\n\n\n\n\n", "1270211": "Congratulations!", "1259743": "Good job! Congrats and thanks for the detailed writeup @scusywxy", "1259695": "Congratulations! Well done team! Are all your detection models Yolo? " } }