{
  "id": 542921,
  "title": "synthetic or any other public dataset",
  "url": "/competitions/ariel-data-challenge-2024/discussion/542921",
  "author_name": "highDopamine",
  "post_date": "2024-10-27T15:23:25.392000",
  "votes": 4,
  "comment_count": 5,
  "views": 0,
  "content": "<p>I tried a lot of methods for fitting the transit curve but my model would  always overfit the training data. At this point it seems the only plausible way to move up is to get more data. Did somebody try using other datasets? </p>",
  "messages": [
    {
      "id": 3029668,
      "postDate": "2024-10-27T15:23:25.393Z",
      "content": "<p>I tried a lot of methods for fitting the transit curve but my model would  always overfit the training data. At this point it seems the only plausible way to move up is to get more data. Did somebody try using other datasets? </p>",
      "rawMarkdown": "I tried a lot of methods for fitting the transit curve but my model would  always overfit the training data. At this point it seems the only plausible way to move up is to get more data. Did somebody try using other datasets? ",
      "votes": 4
    },
    {
      "id": 3029673,
      "postDate": "2024-10-27T15:35:07.603Z",
      "content": "<p>the training distribution is heavily biased for star 0 and 1.<br>\nMy model would work badly on the other stars as it seems.<br>\nTo my naive understanding the star only affects the mean intensity of the transit curve. But still i was unable to remove the bias due to the star</p>",
      "rawMarkdown": "the training distribution is heavily biased for star 0 and 1.\nMy model would work badly on the other stars as it seems.\nTo my naive understanding the star only affects the mean intensity of the transit curve. But still i was unable to remove the bias due to the star",
      "votes": 1,
      "replies": [
        {
          "id": 3029845,
          "postDate": "2024-10-27T20:11:53.437Z",
          "content": "<p>I think a significant dependence on the star is reduced when normalizing data by the out-of-transit flux. This way, the out-of-transit part of time-dependent curves should be around the unity. Other effects arising from differences between stars, such as a typical R_planet/R_star ratio or anything else, should be taken into account carefully during model training.</p>",
          "rawMarkdown": "I think a significant dependence on the star is reduced when normalizing data by the out-of-transit flux. This way, the out-of-transit part of time-dependent curves should be around the unity. Other effects arising from differences between stars, such as a typical R_planet/R_star ratio or anything else, should be taken into account carefully during model training.",
          "votes": 1,
          "replies": [
            {
              "id": 3029894,
              "postDate": "2024-10-27T21:58:48.887Z",
              "content": "<p>I dont understand why should i only normalize using out of transit flux, even if i normalize the white curve by the mean intensity, the effect of star must me removed?</p>",
              "rawMarkdown": "I dont understand why should i only normalize using out of transit flux, even if i normalize the white curve by the mean intensity, the effect of star must me removed?",
              "votes": 1
            },
            {
              "id": 3029896,
              "postDate": "2024-10-27T22:02:31.523Z",
              "content": "<p>i modeled a polynomial regression, which essentially takes input of the mean intensity of the transit curve (every star is associated with different mean intensity), however as the mean intensity was always around 2000, 4000 my model would behave bad when it would see any other mean intensity, hence it cannot generalize. </p>",
              "rawMarkdown": "i modeled a polynomial regression, which essentially takes input of the mean intensity of the transit curve (every star is associated with different mean intensity), however as the mean intensity was always around 2000, 4000 my model would behave bad when it would see any other mean intensity, hence it cannot generalize. \n\n",
              "votes": 2
            },
            {
              "id": 3029900,
              "postDate": "2024-10-27T22:11:24.093Z",
              "content": "<p>oh alright i got it.<br>\nThank you so much Oleh !</p>",
              "rawMarkdown": "oh alright i got it.\nThank you so much Oleh !",
              "votes": 1
            }
          ]
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 3029673,
      "author_name": "highDopamine",
      "author_url": "",
      "post_date": "2024-10-27T15:35:07.603000",
      "content": "<p>the training distribution is heavily biased for star 0 and 1.<br>\nMy model would work badly on the other stars as it seems.<br>\nTo my naive understanding the star only affects the mean intensity of the transit curve. But still i was unable to remove the bias due to the star</p>",
      "votes": 1,
      "replies": [
        {
          "id": 3029845,
          "author_name": "Oleh Kivernyk",
          "author_url": "",
          "post_date": "2024-10-27T20:11:53.437000",
          "content": "<p>I think a significant dependence on the star is reduced when normalizing data by the out-of-transit flux. This way, the out-of-transit part of time-dependent curves should be around the unity. Other effects arising from differences between stars, such as a typical R_planet/R_star ratio or anything else, should be taken into account carefully during model training.</p>",
          "votes": 1,
          "replies": [
            {
              "id": 3029894,
              "author_name": "highDopamine",
              "author_url": "",
              "post_date": "2024-10-27T21:58:48.887000",
              "content": "<p>I dont understand why should i only normalize using out of transit flux, even if i normalize the white curve by the mean intensity, the effect of star must me removed?</p>",
              "votes": 1,
              "replies": []
            },
            {
              "id": 3029896,
              "author_name": "highDopamine",
              "author_url": "",
              "post_date": "2024-10-27T22:02:31.523000",
              "content": "<p>i modeled a polynomial regression, which essentially takes input of the mean intensity of the transit curve (every star is associated with different mean intensity), however as the mean intensity was always around 2000, 4000 my model would behave bad when it would see any other mean intensity, hence it cannot generalize. </p>",
              "votes": 2,
              "replies": []
            },
            {
              "id": 3029900,
              "author_name": "highDopamine",
              "author_url": "",
              "post_date": "2024-10-27T22:11:24.093000",
              "content": "<p>oh alright i got it.<br>\nThank you so much Oleh !</p>",
              "votes": 1,
              "replies": []
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "3029668": "I tried a lot of methods for fitting the transit curve but my model would  always overfit the training data. At this point it seems the only plausible way to move up is to get more data. Did somebody try using other datasets? ",
    "3029673": "the training distribution is heavily biased for star 0 and 1.\nMy model would work badly on the other stars as it seems.\nTo my naive understanding the star only affects the mean intensity of the transit curve. But still i was unable to remove the bias due to the star"
  }
}