{
  "id": 210770,
  "title": "Handling Multiple annotations by multiple radiologists ",
  "url": "/competitions/vinbigdata-chest-xray-abnormalities-detection/discussion/210770",
  "author_name": "Aseem Kannal",
  "post_date": "2021-01-12T08:50:52.059000",
  "votes": 7,
  "comment_count": 1,
  "views": 0,
  "content": "<p>In cases where a particular pathy occurs only once but has multiple annotations by different radiologits, an intercept can be taken easily. (see the example below)<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1377382%2Fca7d1c6daf41399ece408387f7d630fc%2FScreenshot%20from%202021-01-12%2014-11-30.png?generation=1610441110509854&amp;alt=media\" alt=\"\"></p>\n<p>But it starts getting complicated when there are multiple instances of the same pathology in the same image, each having different annotations by radiologists. (see example below)<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1377382%2F65830e06f247d1dcdee936508d52c1c8%2FScreenshot%20from%202021-01-12%2014-03-32.png?generation=1610441189145939&amp;alt=media\" alt=\"\"></p>\n<p>And then there's this monstrosity:<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1377382%2F7b9b5edbfa7b9b3b571ec96b8512a269%2FScreenshot%20from%202021-01-12%2001-16-11.png?generation=1610441371938145&amp;alt=media\" alt=\"\"></p>\n<p>What's the best way to handle multiple overlaps with multiple occurrences in the same image?  </p>",
  "messages": [
    {
      "id": 1149925,
      "postDate": "2021-01-12T08:50:52.060Z",
      "content": "<p>In cases where a particular pathy occurs only once but has multiple annotations by different radiologits, an intercept can be taken easily. (see the example below)<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1377382%2Fca7d1c6daf41399ece408387f7d630fc%2FScreenshot%20from%202021-01-12%2014-11-30.png?generation=1610441110509854&amp;alt=media\" alt=\"\"></p>\n<p>But it starts getting complicated when there are multiple instances of the same pathology in the same image, each having different annotations by radiologists. (see example below)<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1377382%2F65830e06f247d1dcdee936508d52c1c8%2FScreenshot%20from%202021-01-12%2014-03-32.png?generation=1610441189145939&amp;alt=media\" alt=\"\"></p>\n<p>And then there's this monstrosity:<br>\n<img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1377382%2F7b9b5edbfa7b9b3b571ec96b8512a269%2FScreenshot%20from%202021-01-12%2001-16-11.png?generation=1610441371938145&amp;alt=media\" alt=\"\"></p>\n<p>What's the best way to handle multiple overlaps with multiple occurrences in the same image?  </p>",
      "rawMarkdown": "In cases where a particular pathy occurs only once but has multiple annotations by different radiologits, an intercept can be taken easily. (see the example below)\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1377382%2Fca7d1c6daf41399ece408387f7d630fc%2FScreenshot%20from%202021-01-12%2014-11-30.png?generation=1610441110509854&alt=media)\n\nBut it starts getting complicated when there are multiple instances of the same pathology in the same image, each having different annotations by radiologists. (see example below)\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1377382%2F65830e06f247d1dcdee936508d52c1c8%2FScreenshot%20from%202021-01-12%2014-03-32.png?generation=1610441189145939&alt=media)\n\nAnd then there's this monstrosity:\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1377382%2F7b9b5edbfa7b9b3b571ec96b8512a269%2FScreenshot%20from%202021-01-12%2001-16-11.png?generation=1610441371938145&alt=media)\n\nWhat's the best way to handle multiple overlaps with multiple occurrences in the same image?  \n",
      "votes": 7
    },
    {
      "id": 1153156,
      "postDate": "2021-01-14T16:44:19.863Z",
      "content": "<p>This <a href=\"https://www.kaggle.com/sreevishnudamodaran/vinbigdata-fusing-bboxes-coco-dataset\" target=\"_blank\">notebook</a> explores some of the methods for combining bounding boxes such as Non-maximum Suppression (NMS), Soft-NMS, Non-maximum Weighted (NMW) and Weighted Bounding Box Fusion. </p>\n<p>There's also the <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/211035\" target=\"_blank\">question</a> on whether for some <code>class_id</code>values you should only ever allow 1 bounding box (that may be relevant both for combining bounding boxes before training, as well as as a possible post-processing for predictions). </p>\n<p>Finally, I've wondered to what extent just randomly selecting the annotations by a different radiologist in each epoch would be a sensible augmentation strategy. <a href=\"https://www.kaggle.com/bjoernholzhauer/vinbigdata-chest-x-ray-comparing-dataloader-speed\" target=\"_blank\">Here</a> is a prototype dataloader that does that when reading the data from a <code>shelve</code> file I created in <a href=\"https://www.kaggle.com/bjoernholzhauer/eda-dicom-reading-vinbigdata-chest-x-ray\" target=\"_blank\">another notebook</a> <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/211010\" target=\"_blank\">the idea</a> with using shelve is to speed things up vs. reading the .dicom files again, and again for each batch.</p>",
      "rawMarkdown": "This [notebook](https://www.kaggle.com/sreevishnudamodaran/vinbigdata-fusing-bboxes-coco-dataset) explores some of the methods for combining bounding boxes such as Non-maximum Suppression (NMS), Soft-NMS, Non-maximum Weighted (NMW) and Weighted Bounding Box Fusion. \n\nThere's also the [question](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/211035) on whether for some `class_id`values you should only ever allow 1 bounding box (that may be relevant both for combining bounding boxes before training, as well as as a possible post-processing for predictions). \n\nFinally, I've wondered to what extent just randomly selecting the annotations by a different radiologist in each epoch would be a sensible augmentation strategy. [Here](https://www.kaggle.com/bjoernholzhauer/vinbigdata-chest-x-ray-comparing-dataloader-speed) is a prototype dataloader that does that when reading the data from a `shelve` file I created in [another notebook](https://www.kaggle.com/bjoernholzhauer/eda-dicom-reading-vinbigdata-chest-x-ray) [the idea](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/211010) with using shelve is to speed things up vs. reading the .dicom files again, and again for each batch.",
      "votes": 2
    }
  ],
  "comments": [
    {
      "id": 1153156,
      "author_name": "Björn",
      "author_url": "",
      "post_date": "2021-01-14T16:44:19.863000",
      "content": "<p>This <a href=\"https://www.kaggle.com/sreevishnudamodaran/vinbigdata-fusing-bboxes-coco-dataset\" target=\"_blank\">notebook</a> explores some of the methods for combining bounding boxes such as Non-maximum Suppression (NMS), Soft-NMS, Non-maximum Weighted (NMW) and Weighted Bounding Box Fusion. </p>\n<p>There's also the <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/211035\" target=\"_blank\">question</a> on whether for some <code>class_id</code>values you should only ever allow 1 bounding box (that may be relevant both for combining bounding boxes before training, as well as as a possible post-processing for predictions). </p>\n<p>Finally, I've wondered to what extent just randomly selecting the annotations by a different radiologist in each epoch would be a sensible augmentation strategy. <a href=\"https://www.kaggle.com/bjoernholzhauer/vinbigdata-chest-x-ray-comparing-dataloader-speed\" target=\"_blank\">Here</a> is a prototype dataloader that does that when reading the data from a <code>shelve</code> file I created in <a href=\"https://www.kaggle.com/bjoernholzhauer/eda-dicom-reading-vinbigdata-chest-x-ray\" target=\"_blank\">another notebook</a> <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/211010\" target=\"_blank\">the idea</a> with using shelve is to speed things up vs. reading the .dicom files again, and again for each batch.</p>",
      "votes": 2,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1149925": "In cases where a particular pathy occurs only once but has multiple annotations by different radiologits, an intercept can be taken easily. (see the example below)\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1377382%2Fca7d1c6daf41399ece408387f7d630fc%2FScreenshot%20from%202021-01-12%2014-11-30.png?generation=1610441110509854&alt=media)\n\nBut it starts getting complicated when there are multiple instances of the same pathology in the same image, each having different annotations by radiologists. (see example below)\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1377382%2F65830e06f247d1dcdee936508d52c1c8%2FScreenshot%20from%202021-01-12%2014-03-32.png?generation=1610441189145939&alt=media)\n\nAnd then there's this monstrosity:\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1377382%2F7b9b5edbfa7b9b3b571ec96b8512a269%2FScreenshot%20from%202021-01-12%2001-16-11.png?generation=1610441371938145&alt=media)\n\nWhat's the best way to handle multiple overlaps with multiple occurrences in the same image?  \n",
    "1153156": "This [notebook](https://www.kaggle.com/sreevishnudamodaran/vinbigdata-fusing-bboxes-coco-dataset) explores some of the methods for combining bounding boxes such as Non-maximum Suppression (NMS), Soft-NMS, Non-maximum Weighted (NMW) and Weighted Bounding Box Fusion. \n\nThere's also the [question](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/211035) on whether for some `class_id`values you should only ever allow 1 bounding box (that may be relevant both for combining bounding boxes before training, as well as as a possible post-processing for predictions). \n\nFinally, I've wondered to what extent just randomly selecting the annotations by a different radiologist in each epoch would be a sensible augmentation strategy. [Here](https://www.kaggle.com/bjoernholzhauer/vinbigdata-chest-x-ray-comparing-dataloader-speed) is a prototype dataloader that does that when reading the data from a `shelve` file I created in [another notebook](https://www.kaggle.com/bjoernholzhauer/eda-dicom-reading-vinbigdata-chest-x-ray) [the idea](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/211010) with using shelve is to speed things up vs. reading the .dicom files again, and again for each batch."
  }
}