![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3514793%2Fe0de1918d70b0981b93d9195e7475a32%2F__results___36_1.png?generation=1610628973015921&alt=media\")
Hi there,
\nThe VinDr team at Vingroup Big Data Institute (VinBigdata) is very excited to kick off the challenge on localization and classification of abnormalities from chest X-ray images. Thank you all for participating.
\nIn this competition, you’re given a set of training X-ray images in DICOM format, each of which was blindly annotated with bounding boxes of 14 classes by 3 radiologists from a pool of 17, encoded with Rad IDs from R1 to R17. The task is to automatically correctly predict boxes around abnormalities and classify them for the test images, whose ground-truth labels are hidden. Unlike the labels of the training set, those of the test set were already a consensus of 5 radiologists per image.
\nWe have attempted to solve this problem on our own and even deployed the models in our products, but we’re aware that this is still a challenging problem and there is a lot of room for improvement. Through this competition, we would like to see novel ideas in processing DICOM images, harmonizing different opinions of radiologists, training predictive models, and so on.
\nThe dataset (VinDr-CXR) used in this competition was created through the collaboration between VinBigdata and two hospitals in Vietnam, the Hospital 108 and the Hanoi Medical University Hospital. We would like to make this dataset public since we believe that data sharing is the best way to accelerate the development of machine learning algorithms for medical applications. We encourage you to look at our data descriptor paper for more details.
\nWe have two Kaggle Masters in the host team, Nhan T. Nguyen and Dung B. Nguyen, who will serve as our technical contact points for this competition.
\nGood luck!
\nHa Q. Nguyen
\nHead of Medical Imaging Department, VinBigdata
Hi there,
\nThe VinDr team at Vingroup Big Data Institute (VinBigdata) is very excited to kick off the challenge on localization and classification of abnormalities from chest X-ray images. Thank you all for participating.
\nIn this competition, you’re given a set of training X-ray images in DICOM format, each of which was blindly annotated with bounding boxes of 14 classes by 3 radiologists from a pool of 17, encoded with Rad IDs from R1 to R17. The task is to automatically correctly predict boxes around abnormalities and classify them for the test images, whose ground-truth labels are hidden. Unlike the labels of the training set, those of the test set were already a consensus of 5 radiologists per image.
\nWe have attempted to solve this problem on our own and even deployed the models in our products, but we’re aware that this is still a challenging problem and there is a lot of room for improvement. Through this competition, we would like to see novel ideas in processing DICOM images, harmonizing different opinions of radiologists, training predictive models, and so on.
\nThe dataset (VinDr-CXR) used in this competition was created through the collaboration between VinBigdata and two hospitals in Vietnam, the Hospital 108 and the Hanoi Medical University Hospital. We would like to make this dataset public since we believe that data sharing is the best way to accelerate the development of machine learning algorithms for medical applications. We encourage you to look at our data descriptor paper for more details.
\nWe have two Kaggle Masters in the host team, Nhan T. Nguyen and Dung B. Nguyen, who will serve as our technical contact points for this competition.
\nGood luck!
\nHa Q. Nguyen
\nHead of Medical Imaging Department, VinBigdata
I am very proud of VinBigData when organizing this competition. A competition is of great significance not only in terms of medical image processing but also its practical significance of helping doctors a lot not only in Vietnam. Hope to learn more from this competition. Thank you VinBigData for organizing a competition with this large scale and wish the contest will be held successfully and successfully! 💯💕🔥
", "rawMarkdown": "I am very proud of VinBigData when organizing this competition. A competition is of great significance not only in terms of medical image processing but also its practical significance of helping doctors a lot not only in Vietnam. Hope to learn more from this competition. Thank you VinBigData for organizing a competition with this large scale and wish the contest will be held successfully and successfully! 💯💕🔥", "votes": 12 }, { "id": 1152799, "postDate": "2021-01-14T12:58:14.793Z", "content": "I would like to get clarity on the Competition metric.
\nSince the training set and test set is annotated by Human, I have a confusion
\n
You can see here that multiple boxes were linked to same disease and disease space.
\nBut my question is will your competition metric calculates True Positive for each of the overlapping boxes
Thanks in Advance. Just for clarification
", "rawMarkdown": "I would like to get clarity on the Competition metric.\n\nSince the training set and test set is annotated by Human, I have a confusion\n\n\n\nYou can see here that multiple boxes were linked to same disease and disease space.\nBut my question is will your competition metric calculates True Positive for each of the overlapping boxes\n\nThanks in Advance. Just for clarification", "votes": 7, "replies": [ { "id": 1157498, "postDate": "2021-01-18T00:38:08.883Z", "content": "Great question @morizin . I'm also interested as well. So for the bottom orange bound box predictions, are there any False Positive penalty for predicting multiple bounding boxes? So is the orange bounding boxes example in the bottom right going to be counted as a True Positive, or 2 False Positives and 1 True positive? @nguyenbadung @andy2709
\nThanks for the help,
", "rawMarkdown": "Great question @morizin . I'm also interested as well. So for the bottom orange bound box predictions, are there any False Positive penalty for predicting multiple bounding boxes? So is the orange bounding boxes example in the bottom right going to be counted as a True Positive, or 2 False Positives and 1 True positive? @nguyenbadung @andy2709\n\nThanks for the help," }, { "id": 1158212, "postDate": "2021-01-18T13:02:54.697Z", "content": "We are expecting the host responsive please @nguyenbadung @andy2709 @nguyenquyha
", "rawMarkdown": "We are expecting the host responsive please @nguyenbadung @andy2709 @nguyenquyha " }, { "id": 1159126, "postDate": "2021-01-19T04:14:23.063Z", "content": "@morizin The reason why multiple boxes were linked to same disease was that each image in the train data set was independently annotated by 3 different rads. Such cases won't exist in the test data set since we asked 2 more rads to review and 'unify' all the bounding boxes. In my opinion, you need to think of creative ways to combine our provided bounding boxes in the train data set before conducting any experiment.
\nThe competition metric is a variant of the PascalVOC, you can refer to this document http://host.robots.ox.ac.uk/pascal/VOC/voc2010/devkit_doc_08-May-2010.pdf for further details. In section 4.4 on page 11, it says:
\nExample code for computing this overlap measure is provided in the development kit. Multiple detections of the same object in an image are considered\nfalse detections e.g. 5 detections of a single object is counted as 1 correct detection and 4 false detections – it is the responsibility of the participant’s system\nto filter multiple detections from its output.\nFor predicting the test set, you have to apply NMS to filter multiple detections of the same lesion region.
\nAlso, these two discussion threads by Peter and Phalanx are very helpful
\nhttps://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/212287
\nhttps://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/208837
Thank you for great clarification @andy2709
", "rawMarkdown": "Thank you for great clarification @andy2709 " }, { "id": 1163896, "postDate": "2021-01-22T03:11:57.323Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1170601, "postDate": "2021-01-26T10:08:54.370Z", "content": "If there are multiple detections of the same object, which one your implementation selects as correct? The one with largest IoU or confidence? @andy2709
", "rawMarkdown": "If there are multiple detections of the same object, which one your implementation selects as correct? The one with largest IoU or confidence? @andy2709 " }, { "id": 1175808, "postDate": "2021-01-29T11:05:53.930Z", "content": "In original Pascal VOC implementation predictions are sorted by confidence before computing the average precision, so in case of multiple detections only the one with largest confidence is considered as TP. I think it's the same here, but a clarification from the host would be appreciated.
", "rawMarkdown": "In original Pascal VOC implementation predictions are sorted by confidence before computing the average precision, so in case of multiple detections only the one with largest confidence is considered as TP. I think it's the same here, but a clarification from the host would be appreciated.", "votes": 2 }, { "id": 1216738, "postDate": "2021-02-24T13:02:52.277Z", "content": "\n\nSuch cases won't exist in the test data set since we asked 2 more rads to review and 'unify' all the bounding boxes.
\n
@andy2709 did they add or subtract any bounding boxes while 'unify' the annotations
\nor they will just merge the annotations identified by the previous 3 rads
@nguyenquyha thank you for organizing this competition!
\nThe general rules indicate:
\n\n\nB. Data Security. You agree to use reasonable and suitable measures to prevent persons who have not formally agreed to these Rules from gaining access to the Competition Data. You agree not to transmit, duplicate, publish, redistribute or otherwise provide or make available the Competition Data to any party not participating in the Competition. You agree to notify Kaggle immediately upon learning of any possible unauthorized transmission of or unauthorized access to the Competition Data and agree to work with Kaggle to rectify any unauthorized transmission or access.
\n
I'd like to create a preprocessing dataset to help participants more easily load the data, and I will of course link to this competition and its rules. Is that be allowed?
", "rawMarkdown": "@nguyenquyha thank you for organizing this competition!\n\nThe general rules indicate:\n> B. Data Security. You agree to use reasonable and suitable measures to prevent persons who have not formally agreed to these Rules from gaining access to the Competition Data. You agree not to transmit, duplicate, publish, redistribute or otherwise provide or make available the Competition Data to any party not participating in the Competition. You agree to notify Kaggle immediately upon learning of any possible unauthorized transmission of or unauthorized access to the Competition Data and agree to work with Kaggle to rectify any unauthorized transmission or access.\n\nI'd like to create a preprocessing **dataset** to help participants more easily load the data, and I will of course link to this competition and its rules. **Is that be allowed?**", "votes": 5, "replies": [ { "id": 1135238, "postDate": "2021-01-02T03:56:38.230Z", "content": "We're totally fine with that, as long as you point the derived dataset to this competition.
", "rawMarkdown": "We're totally fine with that, as long as you point the derived dataset to this competition.", "votes": 2 } ] }, { "id": 1153188, "postDate": "2021-01-14T17:01:14.693Z", "content": "@nguyenquyha Thank you for hosting this competition.
\nI'm interested in this task.
I have a question about labels.
\nradiologist cannot see another rad_id labeling?
\nif the radiologist check rad0, the annotated image in rad1,2 can not be seen by rad0 radiologist
Maybe sometimes the rad0 maybe could miss it. they are humans
", "rawMarkdown": "Maybe sometimes the rad0 maybe could miss it. they are humans" }, { "id": 1153490, "postDate": "2021-01-14T23:50:52.043Z", "content": "I agree. the labeling process is very important for us.
\nI want to clearly
Hi @tereka
\nIn trainset, the radiologists didn't see each other's labels
Thank you for the clarification
", "rawMarkdown": "Thank you for the clarification" }, { "id": 1167100, "postDate": "2021-01-24T03:47:48.533Z", "content": "@nguyenbadung
\nthank you for the clarification!
Quick question regarding the public test set distribution.
\nShould we expect the label distribution ( various abnormalities & normal ) to be similar to the private set ? Or it could be quite different ?
", "rawMarkdown": "@nguyenquyha \n\nQuick question regarding the public test set distribution.\n\nShould we expect the label distribution ( various abnormalities & normal ) to be similar to the private set ? Or it could be quite different ? ", "votes": 1 }, { "id": 1209998, "postDate": "2021-02-19T06:37:28.517Z", "content": "Hi @nguyenbadung @andy2709 @nguyenquyha
\nCan we use external dataset
\nI mean of Chexpert , MIMIC ,NIH chest14.
\nCan we use the data of ongoing competition Ranzcr.
@nguyenbadung @andy2709 @nguyenquyha
\nto get access to Chexpert and MIMIC data, we need to be a PhysioNet credentialing user.
\nseems to require some training to become one.
\nIs such data still considered as public data?
Yes, we consider CheXpert, MIMIC as public datasets.
", "rawMarkdown": "Yes, we consider CheXpert, MIMIC as public datasets.", "votes": 1 }, { "id": 1230765, "postDate": "2021-03-08T12:50:01.847Z", "content": "Thanks for clarification
", "rawMarkdown": "Thanks for clarification" } ] }, { "id": 1157848, "postDate": "2021-01-18T07:52:34.077Z", "content": "![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F385905%2F8bafe485431584320ede6c5df26062c9%2F__results___61_0.png?generation=1610956184045040&alt=media\")
Hi @nguyenbadung @andy2709,Nhan and Dung, can you please help me with the bottom left image. There are big and small overlapping bounding boxes for the class Nodule/Mass for the image in the lower left. How do we make sense of this? Can you give some insight, because it seems like maybe one annotator maybe just annotated two big boxes on the left and right lung for 'Nodule/Mass' while another annotated many small boxes.
Thanks for the help,
", "rawMarkdown": "\n\nHi @nguyenbadung @andy2709,Nhan and Dung, can you please help me with the bottom left image. There are big and small overlapping bounding boxes for the class `Nodule/Mass` for the image in the lower left. How do we make sense of this? Can you give some insight, because it seems like maybe one annotator maybe just annotated two big boxes on the left and right lung for 'Nodule/Mass' while another annotated many small boxes.\n\nThanks for the help, ", "votes": 1, "replies": [ { "id": 1157987, "postDate": "2021-01-18T10:13:50.770Z", "content": "Yes, you're right, It seems like the annotator just draws one big box for a group of abnormalities.
\nIt's human problem and we can't control it.
@nguyenquyha
\nThanks for your work!
\nMay I ask you about the algorithm of consensus for the test data? Was it just a result of the discussion of 5 radiologists in one room or you received predictions from each radiologist and went this data through some consensus algorithm?
It was mentioned in our paper:
\n\"For the test set, 5 radiologists involved into a two-stage labeling process. During the first stage, each image was independently annotated by 3 radiologists. In the second stage, 2 other radiologists, who have a higher level of experience, reviewed the annotations of the 3 previous annotators and communicated with each other in order to decide the final labels. The disagreements among initial annotators were carefully discussed and resolved by the 2 reviewers. Finally, the consensus of their opinions will serve as reference ground-truth.\"
", "rawMarkdown": "It was mentioned in [our paper](https://arxiv.org/pdf/2012.15029.pdf):\n\n\"For the test set, 5 radiologists involved into a two-stage labeling process. During the first stage, each image was independently annotated by 3 radiologists. In the second stage, 2 other radiologists, who have a higher level of experience, reviewed the annotations of the 3 previous annotators and communicated with each other in order to decide the final labels. The disagreements among initial annotators were carefully discussed and resolved by the 2 reviewers. Finally, the consensus of their opinions will serve as reference ground-truth.\"", "votes": 3 }, { "id": 1191687, "postDate": "2021-02-08T15:55:05.900Z", "content": "Thanks for sharing this amazing dataset. Can you tell us a little bit how multiple correctly labeled bounding boxes (size, location) were average by the 2 experienced annotators ? Let say all 3 initial annotators correctly labeled a nodule with small difference in size and location. Did the 2 experienced radiologists just selected 1 of the 3 bounding boxes as a final label or there was some kind of averaging ? Thx a lot !
", "rawMarkdown": "Thanks for sharing this amazing dataset. Can you tell us a little bit how multiple correctly labeled bounding boxes (size, location) were average by the 2 experienced annotators ? Let say all 3 initial annotators correctly labeled a nodule with small difference in size and location. Did the 2 experienced radiologists just selected 1 of the 3 bounding boxes as a final label or there was some kind of averaging ? Thx a lot ! " } ] }, { "id": 1135443, "postDate": "2021-01-02T08:56:54.577Z", "content": "Thanks for organizing this competition! @nguyenquyha . I have few questions below:
\nThe test set contains 3000 images. The public score will be calculated on approx. 10% of the test set and will be visible to you during the competition. Final rankings will be determined on the remaining 90% i.e. private LB
Yes, pre-trained models and external data are allowed provided that they meet the rules stated in Section 7.C. In general, repositories like detectron2, mmdet, torchvision, timm, efficientdet with pre-trained ImageNet/COCO/OpenImages models and datasets like Chexpert, MIMIC, PadChest are allowed
YOLO v5 which is GPL-3.0 Licensed; can we use that or only MIT licensed ones? I am getting confused
", "rawMarkdown": "YOLO v5 which is GPL-3.0 Licensed; can we use that or only MIT licensed ones? I am getting confused" }, { "id": 1142016, "postDate": "2021-01-07T04:40:29.463Z", "content": "https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/207679
", "rawMarkdown": "https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/207679", "votes": 1 } ] }, { "id": 1133631, "postDate": "2020-12-31T11:46:12.930Z", "content": "Thanks for the host and for the amazing dataset.
\nKnowing some great kagglers are among organizers will surely help for quick and accurate feedback :)
", "rawMarkdown": "Thanks for the host and for the amazing dataset. \n\nKnowing some great kagglers are among organizers will surely help for quick and accurate feedback :)", "votes": 1 }, { "id": 1233551, "postDate": "2021-03-10T13:57:08.557Z", "content": "@nguyenbadung @andy2709 @nguyenquyha
\nThanks for hosting this interesting competition.
\nI am wondering if semi-supervised learning methods such as pseudo-labeling is allowed in this competition?
\nI believe it should be allowed, but I still want a confirmation from host.
This is really an interesting problem and thanks for hosting this competition.
", "rawMarkdown": "This is really an interesting problem and thanks for hosting this competition. ", "votes": 2 }, { "id": 1563614, "postDate": "2021-10-28T13:06:17.040Z", "content": "Hello dear
\nWhy MRI images are not used instead of X-rays to diagnose lung disease?
\nAren't MRI images more accurate for this?
Hi ! I am new to Kaggle and trying to explore this competition and i am little confused about the values mentioned under Score within Leaderboard.
\nSo, are these MAP (Mean average precision) values of each model? or is this something else?
\nthank you.
i agree with you, kind sir
", "rawMarkdown": "i agree with you, kind sir\n" }, { "id": 1256347, "postDate": "2021-03-29T19:04:38.817Z", "content": "@nguyenquyha , @andy2709 , @nguyenbadung ,
\nnow, that the end of the competition is near, i would like to ask you something technical.
\nI found some attenuation patterns on many of the images: namely hyperlucent hemithoraces, mainly on the left side like here :
\n![\"att\"](\"https://i.postimg.cc/RhDdzZJ8/attenuation.jpg\")
\nThese are not oblique chest radiographs - that could explain this phenomenon -, we can furthermore exclude the other obvious causes (absence of a breast after mastectomy, absence of a pectoralis muscle, unilateral or asymmetric bullous disease/emphysema, or air trapping, or an endobronchial foreign body). It made me really hard to evaluate the lung parenchyma!
Is that a general technical issue? I must admit i do not encounter such images in our daily routine.
\nThank you in advance.
", "rawMarkdown": "@nguyenquyha , @andy2709 , @nguyenbadung ,\nnow, that the end of the competition is near, i would like to ask you something technical.\nI found some attenuation patterns on many of the images: namely hyperlucent hemithoraces, mainly on the left side like here :\n\nThese are not oblique chest radiographs - that could explain this phenomenon -, we can furthermore exclude the other obvious causes (absence of a breast after mastectomy, absence of a pectoralis muscle, unilateral or asymmetric bullous disease/emphysema, or air trapping, or an endobronchial foreign body). It made me really hard to evaluate the lung parenchyma!\n\nIs that a general technical issue? I must admit i do not encounter such images in our daily routine.\n\nThank you in advance.\n\n\n" }, { "id": 1192268, "postDate": "2021-02-09T04:14:00.177Z", "content": "Hi Ha,
\nI'm wondering how can I access the age of the cases?
\nThank you.
@nguyenquyha Thanks for organizing interesting challenge!
\nI may miss to read some description, but may I ask how train/test set were split?
Please refer to Figure 1 in our paper.
", "rawMarkdown": "Please refer to Figure 1 in [our paper](https://arxiv.org/pdf/2012.15029.pdf)." } ] }, { "id": 1732595, "postDate": "2022-03-23T14:43:25.777Z", "content": "Thanks for clarification.
", "rawMarkdown": "Thanks for clarification." }, { "id": 1143144, "postDate": "2021-01-07T19:15:14.033Z", "content": "Thanks for organizing this competition!
", "rawMarkdown": "Thanks for organizing this competition! " } ], "comments": [ { "id": 1133286, "author_name": "KhanhVD", "author_url": "", "post_date": "2020-12-31T04:59:11.897000", "content": "I am very proud of VinBigData when organizing this competition. A competition is of great significance not only in terms of medical image processing but also its practical significance of helping doctors a lot not only in Vietnam. Hope to learn more from this competition. Thank you VinBigData for organizing a competition with this large scale and wish the contest will be held successfully and successfully! 💯💕🔥
", "votes": 12, "replies": [] }, { "id": 1152799, "author_name": "Mohammed Rizin V K", "author_url": "", "post_date": "2021-01-14T12:58:14.793000", "content": "I would like to get clarity on the Competition metric.
\nSince the training set and test set is annotated by Human, I have a confusion
\n
You can see here that multiple boxes were linked to same disease and disease space.
\nBut my question is will your competition metric calculates True Positive for each of the overlapping boxes
Thanks in Advance. Just for clarification
", "votes": 7, "replies": [ { "id": 1157498, "author_name": "Bo Peng", "author_url": "", "post_date": "2021-01-18T00:38:08.883000", "content": "Great question @morizin . I'm also interested as well. So for the bottom orange bound box predictions, are there any False Positive penalty for predicting multiple bounding boxes? So is the orange bounding boxes example in the bottom right going to be counted as a True Positive, or 2 False Positives and 1 True positive? @nguyenbadung @andy2709
\nThanks for the help,
", "votes": 0, "replies": [] }, { "id": 1158212, "author_name": "Mohammed Rizin V K", "author_url": "", "post_date": "2021-01-18T13:02:54.697000", "content": "We are expecting the host responsive please @nguyenbadung @andy2709 @nguyenquyha
", "votes": 0, "replies": [] }, { "id": 1159126, "author_name": "NguyenThanhNhan", "author_url": "", "post_date": "2021-01-19T04:14:23.063000", "content": "@morizin The reason why multiple boxes were linked to same disease was that each image in the train data set was independently annotated by 3 different rads. Such cases won't exist in the test data set since we asked 2 more rads to review and 'unify' all the bounding boxes. In my opinion, you need to think of creative ways to combine our provided bounding boxes in the train data set before conducting any experiment.
\nThe competition metric is a variant of the PascalVOC, you can refer to this document http://host.robots.ox.ac.uk/pascal/VOC/voc2010/devkit_doc_08-May-2010.pdf for further details. In section 4.4 on page 11, it says:
\nExample code for computing this overlap measure is provided in the development kit. Multiple detections of the same object in an image are considered\nfalse detections e.g. 5 detections of a single object is counted as 1 correct detection and 4 false detections – it is the responsibility of the participant’s system\nto filter multiple detections from its output.\nFor predicting the test set, you have to apply NMS to filter multiple detections of the same lesion region.
\nAlso, these two discussion threads by Peter and Phalanx are very helpful
\nhttps://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/212287
\nhttps://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/208837
Thank you for great clarification @andy2709
", "votes": 0, "replies": [] }, { "id": 1163896, "author_name": "", "author_url": "", "post_date": "2021-01-22T03:11:57.323000", "content": "", "votes": 0, "replies": [] }, { "id": 1170601, "author_name": "Ahmet Erdem", "author_url": "", "post_date": "2021-01-26T10:08:54.370000", "content": "If there are multiple detections of the same object, which one your implementation selects as correct? The one with largest IoU or confidence? @andy2709
", "votes": 0, "replies": [] }, { "id": 1175808, "author_name": "LAZCoder", "author_url": "", "post_date": "2021-01-29T11:05:53.930000", "content": "In original Pascal VOC implementation predictions are sorted by confidence before computing the average precision, so in case of multiple detections only the one with largest confidence is considered as TP. I think it's the same here, but a clarification from the host would be appreciated.
", "votes": 2, "replies": [] }, { "id": 1216738, "author_name": "Shrijeet16", "author_url": "", "post_date": "2021-02-24T13:02:52.277000", "content": "\n\nSuch cases won't exist in the test data set since we asked 2 more rads to review and 'unify' all the bounding boxes.
\n
@andy2709 did they add or subtract any bounding boxes while 'unify' the annotations
\nor they will just merge the annotations identified by the previous 3 rads
@nguyenquyha thank you for organizing this competition!
\nThe general rules indicate:
\n\n\nB. Data Security. You agree to use reasonable and suitable measures to prevent persons who have not formally agreed to these Rules from gaining access to the Competition Data. You agree not to transmit, duplicate, publish, redistribute or otherwise provide or make available the Competition Data to any party not participating in the Competition. You agree to notify Kaggle immediately upon learning of any possible unauthorized transmission of or unauthorized access to the Competition Data and agree to work with Kaggle to rectify any unauthorized transmission or access.
\n
I'd like to create a preprocessing dataset to help participants more easily load the data, and I will of course link to this competition and its rules. Is that be allowed?
", "votes": 5, "replies": [ { "id": 1135238, "author_name": "Ha Q. Nguyen", "author_url": "", "post_date": "2021-01-02T03:56:38.230000", "content": "We're totally fine with that, as long as you point the derived dataset to this competition.
", "votes": 2, "replies": [] } ] }, { "id": 1153188, "author_name": "tereka", "author_url": "", "post_date": "2021-01-14T17:01:14.693000", "content": "@nguyenquyha Thank you for hosting this competition.
\nI'm interested in this task.
I have a question about labels.
\nradiologist cannot see another rad_id labeling?
\nif the radiologist check rad0, the annotated image in rad1,2 can not be seen by rad0 radiologist
Maybe sometimes the rad0 maybe could miss it. they are humans
", "votes": 0, "replies": [] }, { "id": 1153490, "author_name": "tereka", "author_url": "", "post_date": "2021-01-14T23:50:52.043000", "content": "I agree. the labeling process is very important for us.
\nI want to clearly
Hi @tereka
\nIn trainset, the radiologists didn't see each other's labels
Thank you for the clarification
", "votes": 0, "replies": [] }, { "id": 1167100, "author_name": "tereka", "author_url": "", "post_date": "2021-01-24T03:47:48.533000", "content": "@nguyenbadung
\nthank you for the clarification!
Quick question regarding the public test set distribution.
\nShould we expect the label distribution ( various abnormalities & normal ) to be similar to the private set ? Or it could be quite different ?
", "votes": 1, "replies": [] }, { "id": 1209998, "author_name": "Mohammed Rizin V K", "author_url": "", "post_date": "2021-02-19T06:37:28.517000", "content": "Hi @nguyenbadung @andy2709 @nguyenquyha
\nCan we use external dataset
\nI mean of Chexpert , MIMIC ,NIH chest14.
\nCan we use the data of ongoing competition Ranzcr.
@nguyenbadung @andy2709 @nguyenquyha
\nto get access to Chexpert and MIMIC data, we need to be a PhysioNet credentialing user.
\nseems to require some training to become one.
\nIs such data still considered as public data?
Yes, we consider CheXpert, MIMIC as public datasets.
", "votes": 1, "replies": [] }, { "id": 1230765, "author_name": "Mohammed Rizin V K", "author_url": "", "post_date": "2021-03-08T12:50:01.847000", "content": "Thanks for clarification
", "votes": 0, "replies": [] } ] }, { "id": 1157848, "author_name": "Bo Peng", "author_url": "", "post_date": "2021-01-18T07:52:34.077000", "content": "
Hi @nguyenbadung @andy2709,Nhan and Dung, can you please help me with the bottom left image. There are big and small overlapping bounding boxes for the class Nodule/Mass for the image in the lower left. How do we make sense of this? Can you give some insight, because it seems like maybe one annotator maybe just annotated two big boxes on the left and right lung for 'Nodule/Mass' while another annotated many small boxes.
Thanks for the help,
", "votes": 1, "replies": [ { "id": 1157987, "author_name": "Phat Tran", "author_url": "", "post_date": "2021-01-18T10:13:50.770000", "content": "Yes, you're right, It seems like the annotator just draws one big box for a group of abnormalities.
\nIt's human problem and we can't control it.
@nguyenquyha
\nThanks for your work!
\nMay I ask you about the algorithm of consensus for the test data? Was it just a result of the discussion of 5 radiologists in one room or you received predictions from each radiologist and went this data through some consensus algorithm?
It was mentioned in our paper:
\n\"For the test set, 5 radiologists involved into a two-stage labeling process. During the first stage, each image was independently annotated by 3 radiologists. In the second stage, 2 other radiologists, who have a higher level of experience, reviewed the annotations of the 3 previous annotators and communicated with each other in order to decide the final labels. The disagreements among initial annotators were carefully discussed and resolved by the 2 reviewers. Finally, the consensus of their opinions will serve as reference ground-truth.\"
", "votes": 3, "replies": [] }, { "id": 1191687, "author_name": "Alexandre Cadrin-Chênevert", "author_url": "", "post_date": "2021-02-08T15:55:05.900000", "content": "Thanks for sharing this amazing dataset. Can you tell us a little bit how multiple correctly labeled bounding boxes (size, location) were average by the 2 experienced annotators ? Let say all 3 initial annotators correctly labeled a nodule with small difference in size and location. Did the 2 experienced radiologists just selected 1 of the 3 bounding boxes as a final label or there was some kind of averaging ? Thx a lot !
", "votes": 0, "replies": [] } ] }, { "id": 1135443, "author_name": "The fearless", "author_url": "", "post_date": "2021-01-02T08:56:54.577000", "content": "Thanks for organizing this competition! @nguyenquyha . I have few questions below:
\nThe test set contains 3000 images. The public score will be calculated on approx. 10% of the test set and will be visible to you during the competition. Final rankings will be determined on the remaining 90% i.e. private LB
Yes, pre-trained models and external data are allowed provided that they meet the rules stated in Section 7.C. In general, repositories like detectron2, mmdet, torchvision, timm, efficientdet with pre-trained ImageNet/COCO/OpenImages models and datasets like Chexpert, MIMIC, PadChest are allowed
YOLO v5 which is GPL-3.0 Licensed; can we use that or only MIT licensed ones? I am getting confused
", "votes": 0, "replies": [] }, { "id": 1142016, "author_name": "Awsaf", "author_url": "", "post_date": "2021-01-07T04:40:29.463000", "content": "https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/207679
", "votes": 1, "replies": [] } ] }, { "id": 1133631, "author_name": "Serigne ", "author_url": "", "post_date": "2020-12-31T11:46:12.930000", "content": "Thanks for the host and for the amazing dataset.
\nKnowing some great kagglers are among organizers will surely help for quick and accurate feedback :)
", "votes": 1, "replies": [] }, { "id": 1233551, "author_name": "nvnn", "author_url": "", "post_date": "2021-03-10T13:57:08.557000", "content": "@nguyenbadung @andy2709 @nguyenquyha
\nThanks for hosting this interesting competition.
\nI am wondering if semi-supervised learning methods such as pseudo-labeling is allowed in this competition?
\nI believe it should be allowed, but I still want a confirmation from host.
This is really an interesting problem and thanks for hosting this competition.
", "votes": 2, "replies": [] }, { "id": 1563614, "author_name": "HamidReza69GH", "author_url": "", "post_date": "2021-10-28T13:06:17.040000", "content": "Hello dear
\nWhy MRI images are not used instead of X-rays to diagnose lung disease?
\nAren't MRI images more accurate for this?
Hi ! I am new to Kaggle and trying to explore this competition and i am little confused about the values mentioned under Score within Leaderboard.
\nSo, are these MAP (Mean average precision) values of each model? or is this something else?
\nthank you.
i agree with you, kind sir
", "votes": 0, "replies": [] }, { "id": 1256347, "author_name": "dr. Konya", "author_url": "", "post_date": "2021-03-29T19:04:38.817000", "content": "@nguyenquyha , @andy2709 , @nguyenbadung ,
\nnow, that the end of the competition is near, i would like to ask you something technical.
\nI found some attenuation patterns on many of the images: namely hyperlucent hemithoraces, mainly on the left side like here :
\n
\nThese are not oblique chest radiographs - that could explain this phenomenon -, we can furthermore exclude the other obvious causes (absence of a breast after mastectomy, absence of a pectoralis muscle, unilateral or asymmetric bullous disease/emphysema, or air trapping, or an endobronchial foreign body). It made me really hard to evaluate the lung parenchyma!
Is that a general technical issue? I must admit i do not encounter such images in our daily routine.
\nThank you in advance.
", "votes": 0, "replies": [] }, { "id": 1192268, "author_name": "ServehPadash", "author_url": "", "post_date": "2021-02-09T04:14:00.177000", "content": "Hi Ha,
\nI'm wondering how can I access the age of the cases?
\nThank you.
@nguyenquyha Thanks for organizing interesting challenge!
\nI may miss to read some description, but may I ask how train/test set were split?
Please refer to Figure 1 in our paper.
", "votes": 0, "replies": [] } ] }, { "id": 1732595, "author_name": "Ruixuan ALEX", "author_url": "", "post_date": "2022-03-23T14:43:25.777000", "content": "Thanks for clarification.
", "votes": 0, "replies": [] }, { "id": 1143144, "author_name": "Tilek", "author_url": "", "post_date": "2021-01-07T19:15:14.033000", "content": "Thanks for organizing this competition!
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1133281": "Hi there,\n\nThe [VinDr](https://vindr.ai/) team at Vingroup Big Data Institute ([VinBigdata](https://vinbigdata.org/en/)) is very excited to kick off the challenge on localization and classification of abnormalities from chest X-ray images. Thank you all for participating.\n\nIn this competition, you’re given a set of training X-ray images in DICOM format, each of which was blindly annotated with bounding boxes of 14 classes by 3 radiologists from a pool of 17, encoded with Rad IDs from R1 to R17. The task is to automatically correctly predict boxes around abnormalities and classify them for the test images, whose ground-truth labels are hidden. Unlike the labels of the training set, those of the test set were already a consensus of 5 radiologists per image. \n\nWe have attempted to solve this problem on our own and even deployed the models in our products, but we’re aware that this is still a challenging problem and there is a lot of room for improvement. Through this competition, we would like to see novel ideas in processing DICOM images, harmonizing different opinions of radiologists, training predictive models, and so on.\n\nThe dataset (VinDr-CXR) used in this competition was created through the collaboration between VinBigdata and two hospitals in Vietnam, the Hospital 108 and the Hanoi Medical University Hospital. We would like to make this dataset public since we believe that data sharing is the best way to accelerate the development of machine learning algorithms for medical applications. We encourage you to look at our data descriptor paper for more details. \n\nWe have two Kaggle Masters in the host team, [Nhan T. Nguyen](https://www.kaggle.com/andy2709) and [Dung B. Nguyen](https://www.kaggle.com/nguyenbadung), who will serve as our technical contact points for this competition. \n\nGood luck!\n\nHa Q. Nguyen\nHead of Medical Imaging Department, VinBigdata\n", "1133286": "I am very proud of VinBigData when organizing this competition. A competition is of great significance not only in terms of medical image processing but also its practical significance of helping doctors a lot not only in Vietnam. Hope to learn more from this competition. Thank you VinBigData for organizing a competition with this large scale and wish the contest will be held successfully and successfully! 💯💕🔥", "1152799": "I would like to get clarity on the Competition metric.\n\nSince the training set and test set is annotated by Human, I have a confusion\n\n\n\nYou can see here that multiple boxes were linked to same disease and disease space.\nBut my question is will your competition metric calculates True Positive for each of the overlapping boxes\n\nThanks in Advance. Just for clarification", "1133975": "@nguyenquyha thank you for organizing this competition!\n\nThe general rules indicate:\n> B. Data Security. You agree to use reasonable and suitable measures to prevent persons who have not formally agreed to these Rules from gaining access to the Competition Data. You agree not to transmit, duplicate, publish, redistribute or otherwise provide or make available the Competition Data to any party not participating in the Competition. You agree to notify Kaggle immediately upon learning of any possible unauthorized transmission of or unauthorized access to the Competition Data and agree to work with Kaggle to rectify any unauthorized transmission or access.\n\nI'd like to create a preprocessing **dataset** to help participants more easily load the data, and I will of course link to this competition and its rules. **Is that be allowed?**", "1153188": "@nguyenquyha Thank you for hosting this competition.\nI'm interested in this task.\n\nI have a question about labels.\n\nradiologist cannot see another rad_id labeling?\nif the radiologist check rad0, the annotated image in rad1,2 can not be seen by rad0 radiologist", "1232780": "@nguyenquyha \n\nQuick question regarding the public test set distribution.\n\nShould we expect the label distribution ( various abnormalities & normal ) to be similar to the private set ? Or it could be quite different ? ", "1209998": "Hi @nguyenbadung @andy2709 @nguyenquyha\nCan we use external dataset\nI mean of Chexpert , MIMIC ,NIH chest14.\nCan we use the data of ongoing competition Ranzcr.", "1157848": "\n\nHi @nguyenbadung @andy2709,Nhan and Dung, can you please help me with the bottom left image. There are big and small overlapping bounding boxes for the class `Nodule/Mass` for the image in the lower left. How do we make sense of this? Can you give some insight, because it seems like maybe one annotator maybe just annotated two big boxes on the left and right lung for 'Nodule/Mass' while another annotated many small boxes.\n\nThanks for the help, ", "1147819": "@nguyenquyha \nThanks for your work!\nMay I ask you about the algorithm of consensus for the test data? Was it just a result of the discussion of 5 radiologists in one room or you received predictions from each radiologist and went this data through some consensus algorithm?", "1135443": "Thanks for organizing this competition! @nguyenquyha . I have few questions below:\n1. Is the full (public + private) test just containing 3000 images? Or 3000 images are just used for public LB? (IMO, to prevent the issue of just forking and blending, the private data should be hidden).\n2. I haven't seen the rule of this competition, e.g. Are we allowed to use pre-trained models or external data?\n", "1133631": "Thanks for the host and for the amazing dataset. \n\nKnowing some great kagglers are among organizers will surely help for quick and accurate feedback :)", "1233551": "@nguyenbadung @andy2709 @nguyenquyha \nThanks for hosting this interesting competition.\nI am wondering if semi-supervised learning methods such as pseudo-labeling is allowed in this competition? \nI believe it should be allowed, but I still want a confirmation from host. ", "1134118": "This is really an interesting problem and thanks for hosting this competition. ", "1563614": "Hello dear\nWhy MRI images are not used instead of X-rays to diagnose lung disease? \nAren't MRI images more accurate for this?", "1462434": "Hi ! I am new to Kaggle and trying to explore this competition and i am little confused about the values mentioned under Score within Leaderboard.\nSo, are these MAP (Mean average precision) values of each model? or is this something else?\nthank you.", "1457267": "i agree with you, kind sir\n", "1256347": "@nguyenquyha , @andy2709 , @nguyenbadung ,\nnow, that the end of the competition is near, i would like to ask you something technical.\nI found some attenuation patterns on many of the images: namely hyperlucent hemithoraces, mainly on the left side like here :\n\nThese are not oblique chest radiographs - that could explain this phenomenon -, we can furthermore exclude the other obvious causes (absence of a breast after mastectomy, absence of a pectoralis muscle, unilateral or asymmetric bullous disease/emphysema, or air trapping, or an endobronchial foreign body). It made me really hard to evaluate the lung parenchyma!\n\nIs that a general technical issue? I must admit i do not encounter such images in our daily routine.\n\nThank you in advance.\n\n\n", "1192268": "Hi Ha,\nI'm wondering how can I access the age of the cases?\nThank you.", "1145500": "@nguyenquyha Thanks for organizing interesting challenge!\nI may miss to read some description, but may I ask how train/test set were split? ", "1732595": "Thanks for clarification.", "1143144": "Thanks for organizing this competition! " } }