{
  "id": 223315,
  "title": "Unbalanced Dataset",
  "url": "/competitions/vinbigdata-chest-xray-abnormalities-detection/discussion/223315",
  "author_name": "Giuseppe",
  "post_date": "2021-03-03T10:26:35.706000",
  "votes": 2,
  "comment_count": 0,
  "views": 0,
  "content": "<p>The train set is highly unbalanced as to the categories of the deseases. I am wondering whether the test set contains similar imbalances as well. They can be meaningful, if the train set were well sampled, and therefore representative of the  \"chest X-ray reality\", which should be (more or less) known to the organizers of the challenge, but not (necessarily) to ML scientists. If there were a large data mismatch between the train and test sets, as I guess, finding a good model which works well on a particular test set becomes more a matter of chance than science.</p>\n<p>Does anyone have an idea about the relationship between train and test sets of this challenge? Are you just trying different category ratios until one gets a good test score?</p>",
  "messages": [
    {
      "id": 1225112,
      "postDate": "2021-03-03T10:26:35.707Z",
      "content": "<p>The train set is highly unbalanced as to the categories of the deseases. I am wondering whether the test set contains similar imbalances as well. They can be meaningful, if the train set were well sampled, and therefore representative of the  \"chest X-ray reality\", which should be (more or less) known to the organizers of the challenge, but not (necessarily) to ML scientists. If there were a large data mismatch between the train and test sets, as I guess, finding a good model which works well on a particular test set becomes more a matter of chance than science.</p>\n<p>Does anyone have an idea about the relationship between train and test sets of this challenge? Are you just trying different category ratios until one gets a good test score?</p>",
      "rawMarkdown": "The train set is highly unbalanced as to the categories of the deseases. I am wondering whether the test set contains similar imbalances as well. They can be meaningful, if the train set were well sampled, and therefore representative of the  \"chest X-ray reality\", which should be (more or less) known to the organizers of the challenge, but not (necessarily) to ML scientists. If there were a large data mismatch between the train and test sets, as I guess, finding a good model which works well on a particular test set becomes more a matter of chance than science.\n\nDoes anyone have an idea about the relationship between train and test sets of this challenge? Are you just trying different category ratios until one gets a good test score?\n\n",
      "votes": 2
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "1225112": "The train set is highly unbalanced as to the categories of the deseases. I am wondering whether the test set contains similar imbalances as well. They can be meaningful, if the train set were well sampled, and therefore representative of the  \"chest X-ray reality\", which should be (more or less) known to the organizers of the challenge, but not (necessarily) to ML scientists. If there were a large data mismatch between the train and test sets, as I guess, finding a good model which works well on a particular test set becomes more a matter of chance than science.\n\nDoes anyone have an idea about the relationship between train and test sets of this challenge? Are you just trying different category ratios until one gets a good test score?\n\n"
  }
}