I have a question. We might have few bboxes from few doctors for one finding. It's personal deal how to treat it during training process. But if test set have the same situation, how these cases will be estimated, if we give only one bbox for finding (but test case can have few ones)
", "messages": [ { "id": 1159150, "postDate": "2021-01-19T04:33:38.943Z", "content": "Hi @jwegas
\nEach image in the test set of 3,000 images was labeled by the consensus of 5 radiologists so only one box were linked to same disease and disease space.
\nIf the highest IoU is greater than our threshold (0.4), we have a true positive TP. If you give only one bbox for finding (iou>0.4) you will have 1 TP, if you give 3 bboxes (iou>0.4) for finding you will have 1 TP and 2 FP.
https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/207741#1159126
", "rawMarkdown": "Hi @jwegas \nEach image in the test set of 3,000 images was labeled by the consensus of 5 radiologists so only one box were linked to same disease and disease space.\nIf the highest IoU is greater than our threshold (0.4), we have a true positive TP. If you give only one bbox for finding (iou>0.4) you will have 1 TP, if you give 3 bboxes (iou>0.4) for finding you will have 1 TP and 2 FP.\n\nhttps://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/207741#1159126", "votes": 4 }, { "id": 1158503, "postDate": "2021-01-18T15:45:02.377Z", "content": "The process is described on the paper linked on the competition overview page. This EDA notebook gives a summary and there's further discussion on this in this discussion thread (which also ends up discussing some data/labeling issues). In short: it's 3 radiologists independently assess the images and then two more experienced ones decide what to make of the disagreements (and can discuss with each other while doing so).
", "rawMarkdown": "The process is described on the [paper linked on the competition overview page](https://arxiv.org/pdf/2012.15029.pdf). This [EDA notebook](https://www.kaggle.com/bjoernholzhauer/eda-dicom-reading-vinbigdata-chest-x-ray) gives a summary and there's further discussion on this in [this discussion thread](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/211035) (which also ends up discussing some data/labeling issues). In short: it's 3 radiologists independently assess the images and then two more experienced ones decide what to make of the disagreements (and can discuss with each other while doing so).", "votes": 2 }, { "id": 1158068, "postDate": "2021-01-18T11:11:00.973Z", "content": "I have a question. We might have few bboxes from few doctors for one finding. It's personal deal how to treat it during training process. But if test set have the same situation, how these cases will be estimated, if we give only one bbox for finding (but test case can have few ones)
", "rawMarkdown": "I have a question. We might have few bboxes from few doctors for one finding. It's personal deal how to treat it during training process. But if test set have the same situation, how these cases will be estimated, if we give only one bbox for finding (but test case can have few ones)", "votes": 2 } ], "comments": [ { "id": 1159150, "author_name": "DungNB", "author_url": "", "post_date": "2021-01-19T04:33:38.943000", "content": "Hi @jwegas
\nEach image in the test set of 3,000 images was labeled by the consensus of 5 radiologists so only one box were linked to same disease and disease space.
\nIf the highest IoU is greater than our threshold (0.4), we have a true positive TP. If you give only one bbox for finding (iou>0.4) you will have 1 TP, if you give 3 bboxes (iou>0.4) for finding you will have 1 TP and 2 FP.
https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/207741#1159126
", "votes": 4, "replies": [] }, { "id": 1158503, "author_name": "Björn", "author_url": "", "post_date": "2021-01-18T15:45:02.377000", "content": "The process is described on the paper linked on the competition overview page. This EDA notebook gives a summary and there's further discussion on this in this discussion thread (which also ends up discussing some data/labeling issues). In short: it's 3 radiologists independently assess the images and then two more experienced ones decide what to make of the disagreements (and can discuss with each other while doing so).
", "votes": 2, "replies": [] } ], "raw_markdown_by_id": { "1159150": "Hi @jwegas \nEach image in the test set of 3,000 images was labeled by the consensus of 5 radiologists so only one box were linked to same disease and disease space.\nIf the highest IoU is greater than our threshold (0.4), we have a true positive TP. If you give only one bbox for finding (iou>0.4) you will have 1 TP, if you give 3 bboxes (iou>0.4) for finding you will have 1 TP and 2 FP.\n\nhttps://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/207741#1159126", "1158503": "The process is described on the [paper linked on the competition overview page](https://arxiv.org/pdf/2012.15029.pdf). This [EDA notebook](https://www.kaggle.com/bjoernholzhauer/eda-dicom-reading-vinbigdata-chest-x-ray) gives a summary and there's further discussion on this in [this discussion thread](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/211035) (which also ends up discussing some data/labeling issues). In short: it's 3 radiologists independently assess the images and then two more experienced ones decide what to make of the disagreements (and can discuss with each other while doing so).", "1158068": "I have a question. We might have few bboxes from few doctors for one finding. It's personal deal how to treat it during training process. But if test set have the same situation, how these cases will be estimated, if we give only one bbox for finding (but test case can have few ones)" } }