{
  "id": 207969,
  "title": "Test dataset has annotation from multiple radiologists??",
  "url": "/competitions/vinbigdata-chest-xray-abnormalities-detection/discussion/207969",
  "author_name": "Awsaf",
  "post_date": "2021-01-01T08:30:17.815000",
  "votes": 4,
  "comment_count": 5,
  "views": 0,
  "content": "<p>As it's mentioned that <code>Note that a key part of this competition is working with ground truth from multiple radiologists</code>. I'm wondering if the test dataset has annotation from multiple radiologists like the train dataset? <a href=\"https://www.kaggle.com/nguyenquyha\" target=\"_blank\">@nguyenquyha</a> <a href=\"https://www.kaggle.com/juliaelliott\" target=\"_blank\">@juliaelliott</a> </p>",
  "messages": [
    {
      "id": 1134645,
      "postDate": "2021-01-01T13:08:50.300Z",
      "content": "<p>Details on building the dataset can be found in our recent paper “VinDr-CXR: An open dataset of chest X-rays with radiologist's annotations” <a href=\"url\" target=\"_blank\">https://storage.googleapis.com/kaggle-media/competitions/VinBigData/VinDr_CXR_data_paper.pdf</a></p>\n<p>Each image in the test set of 3,000 images was labeled by the <strong>consensus</strong> of 5 radiologists. That means there are no overlapping boxes of multiple radiologists in each image.</p>",
      "rawMarkdown": "Details on building the dataset can be found in our recent paper “VinDr-CXR: An open dataset of chest X-rays with radiologist's annotations” [https://storage.googleapis.com/kaggle-media/competitions/VinBigData/VinDr_CXR_data_paper.pdf](url)\n\nEach image in the test set of 3,000 images was labeled by the **consensus** of 5 radiologists. That means there are no overlapping boxes of multiple radiologists in each image.",
      "votes": 11,
      "replies": [
        {
          "id": 1161551,
          "postDate": "2021-01-20T16:01:29.967Z",
          "content": "<p><a href=\"https://www.kaggle.com/nguyenquyha\" target=\"_blank\">@nguyenquyha</a> <a href=\"https://www.kaggle.com/nguyenbadung\" target=\"_blank\">@nguyenbadung</a> according to the paper there are <code>22</code> local labels and <code>6</code> global labels but for the competition, we have <code>14</code> labels(without <code>No Finding</code>). I'm wondering how <code>22</code> labels were merged to <code>14</code> labels? </p>",
          "rawMarkdown": "@nguyenquyha @nguyenbadung according to the paper there are `22` local labels and `6` global labels but for the competition, we have `14` labels(without `No Finding`). I'm wondering how `22` labels were merged to `14` labels? "
        },
        {
          "id": 1162941,
          "postDate": "2021-01-21T12:15:10.717Z",
          "content": "<p><a href=\"https://www.kaggle.com/juliaelliott\" target=\"_blank\">@juliaelliott</a> <a href=\"https://www.kaggle.com/nguyenquyha\" target=\"_blank\">@nguyenquyha</a> <a href=\"https://www.kaggle.com/nguyenbadung\" target=\"_blank\">@nguyenbadung</a> could you please clear this matter. It would be really helpful</p>",
          "rawMarkdown": "@juliaelliott @nguyenquyha @nguyenbadung could you please clear this matter. It would be really helpful"
        },
        {
          "id": 1163641,
          "postDate": "2021-01-21T20:35:49.803Z",
          "content": "<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5438562%2Fd8cbf7a49c2fdf2d08320f2ca49fcb37%2Fpaper.png?generation=1611261035087159&amp;alt=media\" alt=\"Training Set\"></p>\n<p>I dont think there is merge.</p>\n<p>I think host just removed the labels don't have enough samples. If you check the original dataset paper , see attached snapshot. For instance, Clavicle fracture has only 1 sample, Edema 1, Emphysema 14, they were removed.</p>",
          "rawMarkdown": "![Training Set](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5438562%2Fd8cbf7a49c2fdf2d08320f2ca49fcb37%2Fpaper.png?generation=1611261035087159&alt=media)\n\nI dont think there is merge.\n\nI think host just removed the labels don't have enough samples. If you check the original dataset paper , see attached snapshot. For instance, Clavicle fracture has only 1 sample, Edema 1, Emphysema 14, they were removed."
        },
        {
          "id": 1163692,
          "postDate": "2021-01-21T21:23:45.840Z",
          "content": "<p><a href=\"https://www.kaggle.com/dennywangdev\" target=\"_blank\">@dennywangdev</a> but the stat doesn't seem to agree with each other.<br>\n<a href=\"https://ibb.co/PjyFgVg\"><img src=\"https://i.ibb.co/9g0cWdW/vbd-problem.png\" alt=\"vbd-problem\"></a></p>",
          "rawMarkdown": "@dennywangdev but the stat doesn't seem to agree with each other.\n<a href=\"https://ibb.co/PjyFgVg\"><img src=\"https://i.ibb.co/9g0cWdW/vbd-problem.png\" alt=\"vbd-problem\" border=\"0\"></a>"
        }
      ]
    },
    {
      "id": 1134389,
      "postDate": "2021-01-01T08:30:17.817Z",
      "content": "<p>As it's mentioned that <code>Note that a key part of this competition is working with ground truth from multiple radiologists</code>. I'm wondering if the test dataset has annotation from multiple radiologists like the train dataset? <a href=\"https://www.kaggle.com/nguyenquyha\" target=\"_blank\">@nguyenquyha</a> <a href=\"https://www.kaggle.com/juliaelliott\" target=\"_blank\">@juliaelliott</a> </p>",
      "rawMarkdown": "As it's mentioned that `Note that a key part of this competition is working with ground truth from multiple radiologists`. I'm wondering if the test dataset has annotation from multiple radiologists like the train dataset? @nguyenquyha @juliaelliott ",
      "votes": 4
    }
  ],
  "comments": [
    {
      "id": 1134645,
      "author_name": "DungNB",
      "author_url": "",
      "post_date": "2021-01-01T13:08:50.300000",
      "content": "<p>Details on building the dataset can be found in our recent paper “VinDr-CXR: An open dataset of chest X-rays with radiologist's annotations” <a href=\"url\" target=\"_blank\">https://storage.googleapis.com/kaggle-media/competitions/VinBigData/VinDr_CXR_data_paper.pdf</a></p>\n<p>Each image in the test set of 3,000 images was labeled by the <strong>consensus</strong> of 5 radiologists. That means there are no overlapping boxes of multiple radiologists in each image.</p>",
      "votes": 11,
      "replies": [
        {
          "id": 1161551,
          "author_name": "Awsaf",
          "author_url": "",
          "post_date": "2021-01-20T16:01:29.967000",
          "content": "<p><a href=\"https://www.kaggle.com/nguyenquyha\" target=\"_blank\">@nguyenquyha</a> <a href=\"https://www.kaggle.com/nguyenbadung\" target=\"_blank\">@nguyenbadung</a> according to the paper there are <code>22</code> local labels and <code>6</code> global labels but for the competition, we have <code>14</code> labels(without <code>No Finding</code>). I'm wondering how <code>22</code> labels were merged to <code>14</code> labels? </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1162941,
          "author_name": "Awsaf",
          "author_url": "",
          "post_date": "2021-01-21T12:15:10.717000",
          "content": "<p><a href=\"https://www.kaggle.com/juliaelliott\" target=\"_blank\">@juliaelliott</a> <a href=\"https://www.kaggle.com/nguyenquyha\" target=\"_blank\">@nguyenquyha</a> <a href=\"https://www.kaggle.com/nguyenbadung\" target=\"_blank\">@nguyenbadung</a> could you please clear this matter. It would be really helpful</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1163641,
          "author_name": "Denny Wang",
          "author_url": "",
          "post_date": "2021-01-21T20:35:49.803000",
          "content": "<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5438562%2Fd8cbf7a49c2fdf2d08320f2ca49fcb37%2Fpaper.png?generation=1611261035087159&amp;alt=media\" alt=\"Training Set\"></p>\n<p>I dont think there is merge.</p>\n<p>I think host just removed the labels don't have enough samples. If you check the original dataset paper , see attached snapshot. For instance, Clavicle fracture has only 1 sample, Edema 1, Emphysema 14, they were removed.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1163692,
          "author_name": "Awsaf",
          "author_url": "",
          "post_date": "2021-01-21T21:23:45.840000",
          "content": "<p><a href=\"https://www.kaggle.com/dennywangdev\" target=\"_blank\">@dennywangdev</a> but the stat doesn't seem to agree with each other.<br>\n<a href=\"https://ibb.co/PjyFgVg\"><img src=\"https://i.ibb.co/9g0cWdW/vbd-problem.png\" alt=\"vbd-problem\"></a></p>",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1134645": "Details on building the dataset can be found in our recent paper “VinDr-CXR: An open dataset of chest X-rays with radiologist's annotations” [https://storage.googleapis.com/kaggle-media/competitions/VinBigData/VinDr_CXR_data_paper.pdf](url)\n\nEach image in the test set of 3,000 images was labeled by the **consensus** of 5 radiologists. That means there are no overlapping boxes of multiple radiologists in each image.",
    "1134389": "As it's mentioned that `Note that a key part of this competition is working with ground truth from multiple radiologists`. I'm wondering if the test dataset has annotation from multiple radiologists like the train dataset? @nguyenquyha @juliaelliott "
  }
}