{
  "id": 511043,
  "title": "root mean squared vs standard deviation",
  "url": "/competitions/leap-atmospheric-physics-ai-climsim/discussion/511043",
  "author_name": "snehal",
  "post_date": "2024-06-08T19:45:49.489000",
  "votes": 1,
  "comment_count": 3,
  "views": 0,
  "content": "<pre><code> = y.mean(axis=)\n = np.maximum(np.sqrt((y*y).mean(axis=)), min_std)\n = (y - my.reshape(,-)) / sy.reshape(,-)\n</code></pre>\n<p>I've seen a lot of public code use this to scale y. Why are people using rms rather than the normal standard deviation?</p>",
  "messages": [
    {
      "id": 2862679,
      "postDate": "2024-06-08T22:07:06.770Z",
      "content": "<p>People using it because the best published notebook at some point (quite early) in this competition used it. He had some reason which I don't remember, the rest is probably mostly copy-paste. In any case just try and see what works for you the best.</p>",
      "rawMarkdown": "People using it because the best published notebook at some point (quite early) in this competition used it. He had some reason which I don't remember, the rest is probably mostly copy-paste. In any case just try and see what works for you the best.",
      "votes": 3
    },
    {
      "id": 2862541,
      "postDate": "2024-06-08T19:45:49.490Z",
      "content": "<pre><code> = y.mean(axis=)\n = np.maximum(np.sqrt((y*y).mean(axis=)), min_std)\n = (y - my.reshape(,-)) / sy.reshape(,-)\n</code></pre>\n<p>I've seen a lot of public code use this to scale y. Why are people using rms rather than the normal standard deviation?</p>",
      "rawMarkdown": "```\nmy = y.mean(axis=0)\nsy = np.maximum(np.sqrt((y*y).mean(axis=0)), min_std)\ny = (y - my.reshape(1,-1)) / sy.reshape(1,-1)\n```\nI've seen a lot of public code use this to scale y. Why are people using rms rather than the normal standard deviation?",
      "votes": 1
    },
    {
      "id": 2869845,
      "postDate": "2024-06-13T10:04:13.970Z",
      "content": "<p>Since the weights in sample_submission already scale the targets by the inverse of the standard deviation, I don't know if yet another transformation could help.</p>\n<p><a href=\"https://www.kaggle.com/competitions/leap-atmospheric-physics-ai-climsim/data\" target=\"_blank\">https://www.kaggle.com/competitions/leap-atmospheric-physics-ai-climsim/data</a></p>",
      "rawMarkdown": "Since the weights in sample_submission already scale the targets by the inverse of the standard deviation, I don't know if yet another transformation could help.\n\nhttps://www.kaggle.com/competitions/leap-atmospheric-physics-ai-climsim/data"
    },
    {
      "id": 2867730,
      "postDate": "2024-06-12T04:06:23.177Z",
      "content": "<p>Mooers et al 2021 scale inputs using the range and claim to observe benefits. Lots of things to try!</p>\n<p><a href=\"https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2020MS002385\" target=\"_blank\">https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2020MS002385</a></p>",
      "rawMarkdown": "Mooers et al 2021 scale inputs using the range and claim to observe benefits. Lots of things to try!\n\nhttps://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2020MS002385"
    }
  ],
  "comments": [
    {
      "id": 2862679,
      "author_name": "greySnow",
      "author_url": "",
      "post_date": "2024-06-08T22:07:06.770000",
      "content": "<p>People using it because the best published notebook at some point (quite early) in this competition used it. He had some reason which I don't remember, the rest is probably mostly copy-paste. In any case just try and see what works for you the best.</p>",
      "votes": 3,
      "replies": []
    },
    {
      "id": 2869845,
      "author_name": "steubk",
      "author_url": "",
      "post_date": "2024-06-13T10:04:13.970000",
      "content": "<p>Since the weights in sample_submission already scale the targets by the inverse of the standard deviation, I don't know if yet another transformation could help.</p>\n<p><a href=\"https://www.kaggle.com/competitions/leap-atmospheric-physics-ai-climsim/data\" target=\"_blank\">https://www.kaggle.com/competitions/leap-atmospheric-physics-ai-climsim/data</a></p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2867730,
      "author_name": "Jerry Lin",
      "author_url": "",
      "post_date": "2024-06-12T04:06:23.177000",
      "content": "<p>Mooers et al 2021 scale inputs using the range and claim to observe benefits. Lots of things to try!</p>\n<p><a href=\"https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2020MS002385\" target=\"_blank\">https://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2020MS002385</a></p>",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2862679": "People using it because the best published notebook at some point (quite early) in this competition used it. He had some reason which I don't remember, the rest is probably mostly copy-paste. In any case just try and see what works for you the best.",
    "2862541": "```\nmy = y.mean(axis=0)\nsy = np.maximum(np.sqrt((y*y).mean(axis=0)), min_std)\ny = (y - my.reshape(1,-1)) / sy.reshape(1,-1)\n```\nI've seen a lot of public code use this to scale y. Why are people using rms rather than the normal standard deviation?",
    "2869845": "Since the weights in sample_submission already scale the targets by the inverse of the standard deviation, I don't know if yet another transformation could help.\n\nhttps://www.kaggle.com/competitions/leap-atmospheric-physics-ai-climsim/data",
    "2867730": "Mooers et al 2021 scale inputs using the range and claim to observe benefits. Lots of things to try!\n\nhttps://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2020MS002385"
  }
}