I made a notebook exploring the merging of bounding boxes based on the annotation process and particulars of the possible findings. Essentially, we are asked to predict the output of a consensus of radiologist for each image but we are given 3 radiologists' labels for each image.
\nPlease check it out and let me know what you think!
\n", "messages": [ { "id": 1155704, "postDate": "2021-01-16T15:36:03.693Z", "content": "\nI made a notebook exploring the merging of bounding boxes based on the annotation process and particulars of the possible findings. Essentially, we are asked to predict the output of a consensus of radiologist for each image but we are given 3 radiologists' labels for each image.
\nPlease check it out and let me know what you think!
\n", "rawMarkdown": "[📦Bounding Box Fusion📦](https://www.kaggle.com/quillio/vindr-bounding-box-fusion)\n\nI made a notebook exploring the merging of bounding boxes based on the annotation process and particulars of the possible findings. Essentially, we are asked to predict the output of a consensus of radiologist for each image but we are given 3 radiologists' labels for each image.\n\nPlease check it out and let me know what you think!\n\n[📦Bounding Box Fusion📦](https://www.kaggle.com/quillio/vindr-bounding-box-fusion)", "votes": 5 }, { "id": 1157921, "postDate": "2021-01-18T08:54:35.177Z", "content": "Hi @quillio, does box fusion improves your score?
\nI've seen that you only fuse boxes for Cardiomegaly and Aortic Enlargement, are your CV and LB scores correlated after that?
\nI mean, in my experiments when applying WBF (to all classes) my local mAP score improves substantially (both in general and for each class, especially for cardiomegaly and aortic enlargement), while my LB scores decreases.
", "rawMarkdown": "Hi @quillio, does box fusion improves your score? \n\nI've seen that you only fuse boxes for Cardiomegaly and Aortic Enlargement, are your CV and LB scores correlated after that?\n\nI mean, in my experiments when applying WBF (to all classes) my local mAP score improves substantially (both in general and for each class, especially for cardiomegaly and aortic enlargement), while my LB scores decreases.", "votes": 1, "replies": [ { "id": 1158094, "postDate": "2021-01-18T11:29:26.890Z", "content": "In the notebook code, I apply WBF to all data points. For Cardiomegaly and Aortic Enlargement, I apply WBF such that only one box comes from each finding (iou variable set to 0). Below is relevant code (note \"only_one\" variable true for classes 0,3:
\nfor image_id in train_w_finding_image_ids:\n records = copy.deepcopy(train_w_finding.loc[train_w_finding.image_id == image_id,:])\n # make only one each of cardiomegaly and aortic enlargement boxes\n for class_id in [0,3]:\n idx = records.loc[records.class_id==class_id,:].index\n if len(idx) > 1:\n finding = records.loc[idx,:]\n boxes = get_fused_boxes(image_id, finding, only_one=True)\n train_fused.append(boxes)\n records = records.drop(idx)\n\n boxes = get_fused_boxes(image_id, records)\n train_fused.append(boxes)\nAlso, it's important to note that LB is based on 300 scored samples from the 3000 in the test set. Overfitting will probably bump a lot of scores down when the private LB is scored.
", "rawMarkdown": "Also, it's important to note that LB is based on 300 scored samples from the 3000 in the test set. Overfitting will probably bump a lot of scores down when the private LB is scored.", "votes": 2 }, { "id": 1158127, "postDate": "2021-01-18T11:48:22.187Z", "content": "Thanks for the clarification, very interesting approach! I used a very small threshold to achieve something similar, but your approach is definitely more interesting.
", "rawMarkdown": "Thanks for the clarification, very interesting approach! I used a very small threshold to achieve something similar, but your approach is definitely more interesting.", "votes": 1 }, { "id": 1158174, "postDate": "2021-01-18T12:28:44.497Z", "content": "No problem. Yes, I think a lot of improvement can be found here. Lots of simple improvements like the fact that in some few images, cardiomegaly and aortic enlargement locations seem swapped or double counted as noted by @bjoernholzhauer in their notebook here.
", "rawMarkdown": "No problem. Yes, I think a lot of improvement can be found here. Lots of simple improvements like the fact that in some few images, cardiomegaly and aortic enlargement locations seem swapped or double counted as noted by @bjoernholzhauer in their notebook [here](https://www.kaggle.com/bjoernholzhauer/eda-dicom-reading-vinbigdata-chest-x-ray).", "votes": 2 } ] }, { "id": 1156701, "postDate": "2021-01-17T10:54:57.923Z", "content": "liked ur approach @quillio
", "rawMarkdown": "liked ur approach @quillio ", "votes": 1, "replies": [ { "id": 1156748, "postDate": "2021-01-17T11:37:03.290Z", "content": "Thanks! I think this competition is very sensitive to the methods used to label the data.
", "rawMarkdown": "Thanks! I think this competition is very sensitive to the methods used to label the data.", "votes": 1 }, { "id": 1156763, "postDate": "2021-01-17T11:50:00.033Z", "content": "That is true!
", "rawMarkdown": "That is true!", "votes": 1 } ] } ], "comments": [ { "id": 1157921, "author_name": "LAZCoder", "author_url": "", "post_date": "2021-01-18T08:54:35.177000", "content": "Hi @quillio, does box fusion improves your score?
\nI've seen that you only fuse boxes for Cardiomegaly and Aortic Enlargement, are your CV and LB scores correlated after that?
\nI mean, in my experiments when applying WBF (to all classes) my local mAP score improves substantially (both in general and for each class, especially for cardiomegaly and aortic enlargement), while my LB scores decreases.
", "votes": 1, "replies": [ { "id": 1158094, "author_name": "quillio", "author_url": "", "post_date": "2021-01-18T11:29:26.890000", "content": "In the notebook code, I apply WBF to all data points. For Cardiomegaly and Aortic Enlargement, I apply WBF such that only one box comes from each finding (iou variable set to 0). Below is relevant code (note \"only_one\" variable true for classes 0,3:
\nfor image_id in train_w_finding_image_ids:\n records = copy.deepcopy(train_w_finding.loc[train_w_finding.image_id == image_id,:])\n # make only one each of cardiomegaly and aortic enlargement boxes\n for class_id in [0,3]:\n idx = records.loc[records.class_id==class_id,:].index\n if len(idx) > 1:\n finding = records.loc[idx,:]\n boxes = get_fused_boxes(image_id, finding, only_one=True)\n train_fused.append(boxes)\n records = records.drop(idx)\n\n boxes = get_fused_boxes(image_id, records)\n train_fused.append(boxes)\nAlso, it's important to note that LB is based on 300 scored samples from the 3000 in the test set. Overfitting will probably bump a lot of scores down when the private LB is scored.
", "votes": 2, "replies": [] }, { "id": 1158127, "author_name": "LAZCoder", "author_url": "", "post_date": "2021-01-18T11:48:22.187000", "content": "Thanks for the clarification, very interesting approach! I used a very small threshold to achieve something similar, but your approach is definitely more interesting.
", "votes": 1, "replies": [] }, { "id": 1158174, "author_name": "quillio", "author_url": "", "post_date": "2021-01-18T12:28:44.497000", "content": "No problem. Yes, I think a lot of improvement can be found here. Lots of simple improvements like the fact that in some few images, cardiomegaly and aortic enlargement locations seem swapped or double counted as noted by @bjoernholzhauer in their notebook here.
", "votes": 2, "replies": [] } ] }, { "id": 1156701, "author_name": "Saurabh Shahane", "author_url": "", "post_date": "2021-01-17T10:54:57.923000", "content": "liked ur approach @quillio
", "votes": 1, "replies": [ { "id": 1156748, "author_name": "quillio", "author_url": "", "post_date": "2021-01-17T11:37:03.290000", "content": "Thanks! I think this competition is very sensitive to the methods used to label the data.
", "votes": 1, "replies": [] }, { "id": 1156763, "author_name": "Saurabh Shahane", "author_url": "", "post_date": "2021-01-17T11:50:00.033000", "content": "That is true!
", "votes": 1, "replies": [] } ] } ], "raw_markdown_by_id": { "1155704": "[📦Bounding Box Fusion📦](https://www.kaggle.com/quillio/vindr-bounding-box-fusion)\n\nI made a notebook exploring the merging of bounding boxes based on the annotation process and particulars of the possible findings. Essentially, we are asked to predict the output of a consensus of radiologist for each image but we are given 3 radiologists' labels for each image.\n\nPlease check it out and let me know what you think!\n\n[📦Bounding Box Fusion📦](https://www.kaggle.com/quillio/vindr-bounding-box-fusion)", "1157921": "Hi @quillio, does box fusion improves your score? \n\nI've seen that you only fuse boxes for Cardiomegaly and Aortic Enlargement, are your CV and LB scores correlated after that?\n\nI mean, in my experiments when applying WBF (to all classes) my local mAP score improves substantially (both in general and for each class, especially for cardiomegaly and aortic enlargement), while my LB scores decreases.", "1156701": "liked ur approach @quillio " } }