{
  "id": 357591,
  "title": "Submission Score",
  "url": "/competitions/mayo-clinic-strip-ai/discussion/357591",
  "author_name": "Fatma Hussein",
  "post_date": "2022-10-04T21:15:02.190000",
  "votes": 2,
  "comment_count": 1,
  "views": 0,
  "content": "<p>I used this notebook to implement the loss function <a href=\"https://www.kaggle.com/code/towhidultonmoy/implementing-custom-weighted-multi-class-log-loss/notebook\" target=\"_blank\">https://www.kaggle.com/code/towhidultonmoy/implementing-custom-weighted-multi-class-log-loss/notebook</a><br>\nMy model gives a good loss value on test data (less than 0.4). But, when I do a submission I got a very bad score (0.9 or greater). Anyone have an idea why the loss is always higher after submission? Thanks.</p>",
  "messages": [
    {
      "id": 1971972,
      "postDate": "2022-10-04T21:15:02.190Z",
      "content": "<p>I used this notebook to implement the loss function <a href=\"https://www.kaggle.com/code/towhidultonmoy/implementing-custom-weighted-multi-class-log-loss/notebook\" target=\"_blank\">https://www.kaggle.com/code/towhidultonmoy/implementing-custom-weighted-multi-class-log-loss/notebook</a><br>\nMy model gives a good loss value on test data (less than 0.4). But, when I do a submission I got a very bad score (0.9 or greater). Anyone have an idea why the loss is always higher after submission? Thanks.</p>",
      "rawMarkdown": "I used this notebook to implement the loss function https://www.kaggle.com/code/towhidultonmoy/implementing-custom-weighted-multi-class-log-loss/notebook\nMy model gives a good loss value on test data (less than 0.4). But, when I do a submission I got a very bad score (0.9 or greater). Anyone have an idea why the loss is always higher after submission? Thanks.",
      "votes": 2
    },
    {
      "id": 1973809,
      "postDate": "2022-10-05T21:14:07.390Z",
      "content": "<p>Two reasons for that, discounting any reasons that involve loss definition:</p>\n<p>1) The 4 images in the test data are actually a part of the training data. So you have already trained on those while modelling, they are provided to just let you prepare a successful submission notebook.</p>\n<p>2) The hidden test images are different and contain 280 images, out of which the public LB score is calculated on 20 or so images. So that is a portion of the actual test result. </p>\n<p>All the best.</p>",
      "rawMarkdown": "Two reasons for that, discounting any reasons that involve loss definition:\n\n1) The 4 images in the test data are actually a part of the training data. So you have already trained on those while modelling, they are provided to just let you prepare a successful submission notebook.\n\n2) The hidden test images are different and contain 280 images, out of which the public LB score is calculated on 20 or so images. So that is a portion of the actual test result. \n\nAll the best."
    }
  ],
  "comments": [
    {
      "id": 1973809,
      "author_name": "tdiceman",
      "author_url": "",
      "post_date": "2022-10-05T21:14:07.390000",
      "content": "<p>Two reasons for that, discounting any reasons that involve loss definition:</p>\n<p>1) The 4 images in the test data are actually a part of the training data. So you have already trained on those while modelling, they are provided to just let you prepare a successful submission notebook.</p>\n<p>2) The hidden test images are different and contain 280 images, out of which the public LB score is calculated on 20 or so images. So that is a portion of the actual test result. </p>\n<p>All the best.</p>",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1971972": "I used this notebook to implement the loss function https://www.kaggle.com/code/towhidultonmoy/implementing-custom-weighted-multi-class-log-loss/notebook\nMy model gives a good loss value on test data (less than 0.4). But, when I do a submission I got a very bad score (0.9 or greater). Anyone have an idea why the loss is always higher after submission? Thanks.",
    "1973809": "Two reasons for that, discounting any reasons that involve loss definition:\n\n1) The 4 images in the test data are actually a part of the training data. So you have already trained on those while modelling, they are provided to just let you prepare a successful submission notebook.\n\n2) The hidden test images are different and contain 280 images, out of which the public LB score is calculated on 20 or so images. So that is a portion of the actual test result. \n\nAll the best."
  }
}