{
  "id": 589905,
  "title": "Questions for us?",
  "url": "/competitions/ariel-data-challenge-2025/discussion/589905",
  "author_name": "Gordon Yip",
  "post_date": "2025-07-16T09:39:32.931000",
  "votes": 15,
  "comment_count": 54,
  "views": 0,
  "content": "<p>You’re right. The competition is really hard. The difficulty is actually the reason we set these data challenges to begin with - our problem is so hard that we need your help! </p>\n<p>While we cannot disclose the solution to you, we wanted you to have a way to ask questions to us - to really make a breakthrough, all of us need to work together -</p>\n<p>Here is what we will do:<br>\n1.⁠ ⁠You can ask your questions in the forum and we will address them there (we are monitoring the discussion)<br>\n2.⁠ ⁠You can leave your question below and we will reply to you.<br>\n3.⁠ ⁠You can leave your questions in the <a href=\"https://forms.gle/KtdD3gVX5eA64hpx9\" target=\"_blank\">questionnaire</a>, we will leave a reply in this thread. </p>\n<p>Any appropriate questions submitted in the questionnaire will also be added and responded to here for the benefit of the whole community.</p>\n<p>Here is the catch, we cannot promise to answer every question. We might not answer your question if we think it will give away too much details about the setup of the game/ground truth, sorry!</p>\n<p>Side note - if you have a few minutes to spare, please fill in the <a href=\"https://forms.gle/m52D74GATX8hm8vV9\" target=\"_blank\">survey</a>!</p>",
  "messages": [
    {
      "id": 3249350,
      "postDate": "2025-07-16T09:39:32.933Z",
      "content": "<p>You’re right. The competition is really hard. The difficulty is actually the reason we set these data challenges to begin with - our problem is so hard that we need your help! </p>\n<p>While we cannot disclose the solution to you, we wanted you to have a way to ask questions to us - to really make a breakthrough, all of us need to work together -</p>\n<p>Here is what we will do:<br>\n1.⁠ ⁠You can ask your questions in the forum and we will address them there (we are monitoring the discussion)<br>\n2.⁠ ⁠You can leave your question below and we will reply to you.<br>\n3.⁠ ⁠You can leave your questions in the <a href=\"https://forms.gle/KtdD3gVX5eA64hpx9\" target=\"_blank\">questionnaire</a>, we will leave a reply in this thread. </p>\n<p>Any appropriate questions submitted in the questionnaire will also be added and responded to here for the benefit of the whole community.</p>\n<p>Here is the catch, we cannot promise to answer every question. We might not answer your question if we think it will give away too much details about the setup of the game/ground truth, sorry!</p>\n<p>Side note - if you have a few minutes to spare, please fill in the <a href=\"https://forms.gle/m52D74GATX8hm8vV9\" target=\"_blank\">survey</a>!</p>",
      "rawMarkdown": "You’re right. The competition is really hard. The difficulty is actually the reason we set these data challenges to begin with - our problem is so hard that we need your help! \n\nWhile we cannot disclose the solution to you, we wanted you to have a way to ask questions to us - to really make a breakthrough, all of us need to work together -\n\nHere is what we will do:\n1.⁠ ⁠You can ask your questions in the forum and we will address them there (we are monitoring the discussion)\n2.⁠ ⁠You can leave your question below and we will reply to you.\n3.⁠ ⁠You can leave your questions in the [questionnaire](https://forms.gle/KtdD3gVX5eA64hpx9), we will leave a reply in this thread. \n\nAny appropriate questions submitted in the questionnaire will also be added and responded to here for the benefit of the whole community.\n\nHere is the catch, we cannot promise to answer every question. We might not answer your question if we think it will give away too much details about the setup of the game/ground truth, sorry!\n\nSide note - if you have a few minutes to spare, please fill in the [survey](https://forms.gle/m52D74GATX8hm8vV9)!",
      "votes": 15
    },
    {
      "id": 3252421,
      "postDate": "2025-07-22T17:35:59.937Z",
      "content": "<p>Can you tell us what physical parameter we are actually trying to estimate?  Transit depth is obvious in the absence of limb darkening, but with limb darkening, it is not clear what we are being asked to predict.  It could be one of the following:</p>\n<ol>\n<li>The effective (Rp/Rs)^2.  This would be the values in the EXOSIM2 planetary radius file:  <a href=\"https://exosim2-public.readthedocs.io/en/latest/api/exosim/tasks/astrosignal/estimatePlanetarySignal/index.html#module-exosim.tasks.astrosignal.estimatePlanetarySignal\" target=\"_blank\">load_rp</a></li>\n<li>The light reduction at mid-transit</li>\n<li>The maximum reduction in light intensity.  The paper <em>Analytic solutions to the maximum and average exoplanet\ntransit depth for common stellar limb darkening laws</em> by <a href=\"https://arxiv.org/abs/1901.01730\" target=\"_blank\">Heller 2019</a>, Fig 1, shows delta (transit depth) as the maximum and relates that to the (Rp/Rs)^2 for various limb darkening laws.</li>\n<li>Just the ground truth you give us and don't ask too many questions 😀</li>\n</ol>",
      "rawMarkdown": "Can you tell us what physical parameter we are actually trying to estimate?  Transit depth is obvious in the absence of limb darkening, but with limb darkening, it is not clear what we are being asked to predict.  It could be one of the following:\n1. The effective (Rp/Rs)^2.  This would be the values in the EXOSIM2 planetary radius file:  [load_rp](https://exosim2-public.readthedocs.io/en/latest/api/exosim/tasks/astrosignal/estimatePlanetarySignal/index.html#module-exosim.tasks.astrosignal.estimatePlanetarySignal)\n2. The light reduction at mid-transit\n3.  The maximum reduction in light intensity.  The paper *Analytic solutions to the maximum and average exoplanet\ntransit depth for common stellar limb darkening laws* by [Heller 2019](https://arxiv.org/abs/1901.01730), Fig 1, shows delta (transit depth) as the maximum and relates that to the (Rp/Rs)^2 for various limb darkening laws.\n4. Just the ground truth you give us and don't ask too many questions 😀",
      "votes": 5,
      "replies": [
        {
          "id": 3252461,
          "postDate": "2025-07-22T19:22:00.803Z",
          "rawMarkdown": "",
          "isDeleted": true
        },
        {
          "id": 3252559,
          "postDate": "2025-07-23T02:12:54.503Z",
          "content": "<p>I think it's 1.</p>",
          "rawMarkdown": "I think it's 1.",
          "votes": 1,
          "replies": [
            {
              "id": 3252570,
              "postDate": "2025-07-23T02:52:50.153Z",
              "content": "<p>same<br>\nMy analysis is based on the assumption that the situation is 1.</p>",
              "rawMarkdown": "same\nMy analysis is based on the assumption that the situation is 1.",
              "votes": 1
            },
            {
              "id": 3252576,
              "postDate": "2025-07-23T03:09:15.020Z",
              "rawMarkdown": "",
              "isDeleted": true
            }
          ]
        },
        {
          "id": 3252754,
          "postDate": "2025-07-23T10:58:07.760Z",
          "content": "<p>The goal of the challenge is to recover the ground truth, the intrinsic transmission spectrum, namely the <em>(Rp/Rs)^2</em> at each wavelength. This fundamental transit measurement directly reveals how the planet’s atmosphere absorbs starlight.</p>\n<p>In practice, you must build an estimator that disentangles the atmospheric signal in the data from instrumental and astrophysical noise, such as limb darkening, to yield the most precise and unbiased transmission profile possible. That is your ground truth 😄</p>",
          "rawMarkdown": "The goal of the challenge is to recover the ground truth, the intrinsic transmission spectrum, namely the *(Rp/Rs)^2* at each wavelength. This fundamental transit measurement directly reveals how the planet’s atmosphere absorbs starlight.\n\nIn practice, you must build an estimator that disentangles the atmospheric signal in the data from instrumental and astrophysical noise, such as limb darkening, to yield the most precise and unbiased transmission profile possible. That is your ground truth 😄",
          "votes": 5
        },
        {
          "id": 3252798,
          "postDate": "2025-07-23T12:51:12.533Z",
          "content": "<p>The physical parameter  is the effective (Rp/Rs)^2, where:</p>\n<ul>\n<li>Rp is the planetary radius</li>\n<li>Rs is the stellar radius</li>\n</ul>\n<p>This parameter represents the ratio of the planetary radius to the stellar radius squared and is a key factor in determining the transit depth of an exoplanet.</p>\n<p>In the context of transit observations, the effective (Rp/Rs)^2 is often used to describe the decrease in stellar flux caused by the planet passing in front of the star. Limb darkening can affect this measurement, but the effective (Rp/Rs)^2 remains a fundamental parameter for characterizing exoplanet properties.</p>",
          "rawMarkdown": "The physical parameter  is the effective (Rp/Rs)^2, where:\n\n- Rp is the planetary radius\n- Rs is the stellar radius\n\nThis parameter represents the ratio of the planetary radius to the stellar radius squared and is a key factor in determining the transit depth of an exoplanet.\n\nIn the context of transit observations, the effective (Rp/Rs)^2 is often used to describe the decrease in stellar flux caused by the planet passing in front of the star. Limb darkening can affect this measurement, but the effective (Rp/Rs)^2 remains a fundamental parameter for characterizing exoplanet properties.",
          "votes": -1
        }
      ]
    },
    {
      "id": 3253630,
      "postDate": "2025-07-25T02:48:50.707Z",
      "content": "<p>Is there any <strong>intentional</strong> data drift between public test (for public LB) and private test (for private LB) ?<br>\nLast year, I've seen some teams dropping their rank by several hundreds assumably because of the distruibution difference, and I'm curious whether we should be prepared for that this year also.</p>\n<p>By <strong>intentional</strong>, I mean different types of molecules in the atmosphere, different limb darkening functions, different transit position etc.</p>",
      "rawMarkdown": "Is there any **intentional** data drift between public test (for public LB) and private test (for private LB) ?\nLast year, I've seen some teams dropping their rank by several hundreds assumably because of the distruibution difference, and I'm curious whether we should be prepared for that this year also.\n\nBy **intentional**, I mean different types of molecules in the atmosphere, different limb darkening functions, different transit position etc.",
      "votes": 4,
      "replies": [
        {
          "id": 3253966,
          "postDate": "2025-07-25T15:20:24.437Z",
          "content": "<p>No , the public LB and private LB should be homogeneous in terms of test distribution. </p>",
          "rawMarkdown": "No , the public LB and private LB should be homogeneous in terms of test distribution. ",
          "votes": 6,
          "replies": [
            {
              "id": 3254095,
              "postDate": "2025-07-25T19:38:36.277Z",
              "content": "<p>This was also the case last year, right? I always assumed the teams that dropped a lot simply had a solution that wasn't robust and made a very poor predictions on a single (or a few) elements of the private test set.</p>",
              "rawMarkdown": "This was also the case last year, right? I always assumed the teams that dropped a lot simply had a solution that wasn't robust and made a very poor predictions on a single (or a few) elements of the private test set.",
              "votes": 4
            }
          ]
        }
      ]
    },
    {
      "id": 3281524,
      "postDate": "2025-09-04T13:27:14.100Z",
      "content": "<p>Hello, I've noticed that when there are double measurements, the star intensity can vary considerably (by at least 10% for some exoplanets). I was wondering what the physical explanations for this phenomenon might be, since I assumed the two measurements were taken under the same conditions.</p>",
      "rawMarkdown": "Hello, I've noticed that when there are double measurements, the star intensity can vary considerably (by at least 10% for some exoplanets). I was wondering what the physical explanations for this phenomenon might be, since I assumed the two measurements were taken under the same conditions.",
      "votes": 1,
      "replies": [
        {
          "id": 3281598,
          "postDate": "2025-09-04T14:51:23.163Z",
          "content": "<p>Yes you are right - the stellar intensity can change between observations. While we are observing the same star but there could be variability on the stellar flux. We are simulating this situation here, but, in some cases we might have overdone it !</p>",
          "rawMarkdown": "Yes you are right - the stellar intensity can change between observations. While we are observing the same star but there could be variability on the stellar flux. We are simulating this situation here, but, in some cases we might have overdone it !"
        },
        {
          "id": 3281676,
          "postDate": "2025-09-04T17:02:06.723Z",
          "content": "<p>I don't know what's actually covered in the simulations, but there are some truly wild <a href=\"https://en.wikipedia.org/wiki/Variable_star#Pulsating_variable_stars\" target=\"_blank\">variable stars</a> that you might enjoy reading about. </p>",
          "rawMarkdown": "I don't know what's actually covered in the simulations, but there are some truly wild [variable stars](https://en.wikipedia.org/wiki/Variable_star#Pulsating_variable_stars) that you might enjoy reading about. ",
          "votes": 2
        }
      ]
    },
    {
      "id": 3249875,
      "postDate": "2025-07-17T09:19:58.110Z",
      "content": "<p>Can you say something about the constant background/foreground light? Last year, I could see obvious offset far from the star, and that caused systematic offset in the target values that even some top players could not understand. This year I don't see that offset. Is it subtracted? Or, it is a non-trivial problem that we should think about?</p>",
      "rawMarkdown": "Can you say something about the constant background/foreground light? Last year, I could see obvious offset far from the star, and that caused systematic offset in the target values that even some top players could not understand. This year I don't see that offset. Is it subtracted? Or, it is a non-trivial problem that we should think about?",
      "votes": 4,
      "replies": [
        {
          "id": 3250978,
          "postDate": "2025-07-19T15:58:05.223Z",
          "content": "<p>Sure thing. We can confirm that the background/foreground light has been implemented in this edition of the data challenge. The seeming absence of the effect is due to the way Analog-to-Digital conversion has been implemented this year. We moved from dynamic scaling (each observation having its own gain and offset). This year, instead, we are using fixed gain and fixed offset to be more representative of the real instrument. This change means that any value below the \"offset\" is lost, contributing to the apparent disappearance of the background/foreground light.</p>\n<p>hope that makes sense - let me know if you have more questions. </p>",
          "rawMarkdown": "Sure thing. We can confirm that the background/foreground light has been implemented in this edition of the data challenge. The seeming absence of the effect is due to the way Analog-to-Digital conversion has been implemented this year. We moved from dynamic scaling (each observation having its own gain and offset). This year, instead, we are using fixed gain and fixed offset to be more representative of the real instrument. This change means that any value below the \"offset\" is lost, contributing to the apparent disappearance of the background/foreground light.\n\nhope that makes sense - let me know if you have more questions. \n\n",
          "votes": 4
        }
      ]
    },
    {
      "id": 3269906,
      "postDate": "2025-08-15T09:51:51.333Z",
      "content": "<p><a href=\"https://www.kaggle.com/gordonyip\" target=\"_blank\">@gordonyip</a> see this discussion: <a href=\"https://www.kaggle.com/competitions/ariel-data-challenge-2025/discussion/598324\" target=\"_blank\">https://www.kaggle.com/competitions/ariel-data-challenge-2025/discussion/598324</a></p>\n<p>Wouldn't it make sense to discard these samples where ingress and/or egress are not fully contained in the data we have? I am speaking of the test set here of course.</p>\n<p>I spend time fixing these but I doubt this is useful, nor do I think I did a perfect job at it. I am pretty sure many others spent time on these samples too. If we knew for sure that nothing similar occurs in the test set then we can all save time an focus on what is useful to you.</p>",
      "rawMarkdown": "@gordonyip see this discussion: https://www.kaggle.com/competitions/ariel-data-challenge-2025/discussion/598324\n\nWouldn't it make sense to discard these samples where ingress and/or egress are not fully contained in the data we have? I am speaking of the test set here of course.\n\nI spend time fixing these but I doubt this is useful, nor do I think I did a perfect job at it. I am pretty sure many others spent time on these samples too. If we knew for sure that nothing similar occurs in the test set then we can all save time an focus on what is useful to you.",
      "votes": 1,
      "replies": [
        {
          "id": 3274824,
          "postDate": "2025-08-25T13:44:03.627Z",
          "content": "<p>Hi <a href=\"https://www.kaggle.com/cpmpml\" target=\"_blank\">@cpmpml</a> , also tagging <a href=\"https://www.kaggle.com/jeroencottaar\" target=\"_blank\">@jeroencottaar</a> here as you might be interested. discarding/changing the metric would make sense but at the same time will be hard on the competition organisation side since the competition is entering its final month, and it will inevitably change the workflow of many others. We are inclined to not change that, but what we could do is calculate the score if those examples were discarded, after the competition has ended. </p>",
          "rawMarkdown": "Hi @cpmpml , also tagging @jeroencottaar here as you might be interested. discarding/changing the metric would make sense but at the same time will be hard on the competition organisation side since the competition is entering its final month, and it will inevitably change the workflow of many others. We are inclined to not change that, but what we could do is calculate the score if those examples were discarded, after the competition has ended. ",
          "replies": [
            {
              "id": 3274850,
              "postDate": "2025-08-25T14:16:50.620Z",
              "content": "<p>Please don't change the way the metric is calculated between the public and private leaderboard.  That would kind of defeat the purpose of the public leaderboard.  I would think you would also have to make an (official) public announcement  and it would seem unfair at this late stage in the competition.</p>",
              "rawMarkdown": "Please don't change the way the metric is calculated between the public and private leaderboard.  That would kind of defeat the purpose of the public leaderboard.  I would think you would also have to make an (official) public announcement  and it would seem unfair at this late stage in the competition.",
              "votes": 3
            },
            {
              "id": 3274864,
              "postDate": "2025-08-25T14:29:57.527Z",
              "content": "<p>Yep we fully understand the implication so we are not planning to do it, but we would like to communicate our current thoughts here just to maintain transparency. </p>",
              "rawMarkdown": "Yep we fully understand the implication so we are not planning to do it, but we would like to communicate our current thoughts here just to maintain transparency. "
            },
            {
              "id": 3275021,
              "postDate": "2025-08-25T19:03:35.550Z",
              "content": "<p><a href=\"https://www.kaggle.com/gordonyip\" target=\"_blank\">@gordonyip</a> </p>\n<blockquote>\n  <p>what we could do is calculate the score if those examples were discarded, after the competition has ended. </p>\n</blockquote>\n<p>I don't get what you mean by the above. </p>",
              "rawMarkdown": "@gordonyip \n\n> what we could do is calculate the score if those examples were discarded, after the competition has ended. \n\nI don't get what you mean by the above. "
            },
            {
              "id": 3277569,
              "postDate": "2025-08-28T09:04:59.513Z",
              "content": "<p>As in , after the competition has ended, we can release <code>unofficial score</code>, i.e. the overall score if those lightcruves without full ingress and egress are removed , that will not change the leaderboard, nor the prize ranking unfortunately, and will only purely for our own interest. </p>",
              "rawMarkdown": "As in , after the competition has ended, we can release `unofficial score`, i.e. the overall score if those lightcruves without full ingress and egress are removed , that will not change the leaderboard, nor the prize ranking unfortunately, and will only purely for our own interest. ",
              "votes": 1
            },
            {
              "id": 3277580,
              "postDate": "2025-08-28T09:32:39.453Z",
              "content": "<p>Thank you, this is clearer.</p>",
              "rawMarkdown": "Thank you, this is clearer.",
              "votes": 1
            }
          ]
        }
      ]
    },
    {
      "id": 3283156,
      "postDate": "2025-09-07T10:36:57.130Z",
      "content": "<p>Hello! I’ve noticed something unusual in the linear correction of the FGS1 channel for the exoplanet 34983 (and, since the correlation files are duplicated, it also affects many others). The degree-0 polynomial coefficient is 91.5597 for spatial pixel 572, which is a very large offset. This looks incorrect when applied to the FGS1 observation, especially compared with the adjacent pixels.</p>\n<p>While this offset is removed once we compute the CDS, it raises concerns about the validity of the rest of the linear correction — especially the higher-order terms. This may well be intentional, but I wanted to check whether such behaviour is expected.</p>",
      "rawMarkdown": "Hello! I’ve noticed something unusual in the linear correction of the FGS1 channel for the exoplanet 34983 (and, since the correlation files are duplicated, it also affects many others). The degree-0 polynomial coefficient is 91.5597 for spatial pixel 572, which is a very large offset. This looks incorrect when applied to the FGS1 observation, especially compared with the adjacent pixels.\n\nWhile this offset is removed once we compute the CDS, it raises concerns about the validity of the rest of the linear correction — especially the higher-order terms. This may well be intentional, but I wanted to check whether such behaviour is expected.",
      "replies": [
        {
          "id": 3289027,
          "postDate": "2025-09-15T09:50:33.317Z",
          "content": "<p>Sorry to insist, but I’d like to know if you’ve noticed the same thing as I did, or if I’m doing something wrong in the preprocessing.</p>",
          "rawMarkdown": "Sorry to insist, but I’d like to know if you’ve noticed the same thing as I did, or if I’m doing something wrong in the preprocessing.",
          "replies": [
            {
              "id": 3289309,
              "postDate": "2025-09-15T20:20:05.063Z",
              "content": "<p>no worries  you are absolutely right to pursue - i am going to check with our instrument team and get back to you on this, sorry for the delay</p>",
              "rawMarkdown": "no worries  you are absolutely right to pursue - i am going to check with our instrument team and get back to you on this, sorry for the delay\n"
            },
            {
              "id": 3290468,
              "postDate": "2025-09-17T21:02:08.560Z",
              "content": "<p><a href=\"https://www.kaggle.com/mgerard\" target=\"_blank\">@mgerard</a> <br>\nI asked <a href=\"https://www.kaggle.com/lorenzomugnai\" target=\"_blank\">@lorenzomugnai</a> about this and here is the reply:<br>\nThe calibration data we use originate from JWST and include, for the same detector, dark current, read noise, bad pixels, hot pixels, and linearity correction. We use these data to generate the simulations and then provide them to the participants with an associated uncertainty.</p>\n<p>This means that the values for individual pixels are consistent across the different calibration products, but they are outside of our control. There are several reasons why a given pixel may appear unusual in one of the calibration files. A clearer picture is often obtained by comparing different calibrators to see whether the same pixel shows anomalous behaviour in multiple cases. For instance, a non-zero degree offset in the linearity correction often indicates a hot pixel or one with a strong residual initial charge.</p>\n<p>In any case, these are genuine calibration data from a real and very complex instrument, and they are representative of what will be encountered in Ariel, since it will use detectors of the same type.</p>",
              "rawMarkdown": "@mgerard \nI asked @lorenzomugnai about this and here is the reply:\nThe calibration data we use originate from JWST and include, for the same detector, dark current, read noise, bad pixels, hot pixels, and linearity correction. We use these data to generate the simulations and then provide them to the participants with an associated uncertainty.\n\nThis means that the values for individual pixels are consistent across the different calibration products, but they are outside of our control. There are several reasons why a given pixel may appear unusual in one of the calibration files. A clearer picture is often obtained by comparing different calibrators to see whether the same pixel shows anomalous behaviour in multiple cases. For instance, a non-zero degree offset in the linearity correction often indicates a hot pixel or one with a strong residual initial charge.\n\nIn any case, these are genuine calibration data from a real and very complex instrument, and they are representative of what will be encountered in Ariel, since it will use detectors of the same type.",
              "votes": 1
            }
          ]
        }
      ]
    },
    {
      "id": 3281941,
      "postDate": "2025-09-05T09:48:32.777Z",
      "content": "<p>Another one nooby question. How shall I submit my solution? I've run my notebook from start to the end without problems. I am predicting generalized test. In logs I can see, that test (1 planet from example) was predicted and submission created. But after this happend I know nothing. And after hours of waiting submission scorring failed with an error. What should I submit and how.<br>\nI've heared about substetution of test data, but I do not see it in the submission.</p>",
      "rawMarkdown": "Another one nooby question. How shall I submit my solution? I've run my notebook from start to the end without problems. I am predicting generalized test. In logs I can see, that test (1 planet from example) was predicted and submission created. But after this happend I know nothing. And after hours of waiting submission scorring failed with an error. What should I submit and how.\nI've heared about substetution of test data, but I do not see it in the submission.",
      "replies": [
        {
          "id": 3284356,
          "postDate": "2025-09-08T07:17:28.653Z",
          "content": "<p>hmm , did you see a score in the public leaderboard?</p>",
          "rawMarkdown": "hmm , did you see a score in the public leaderboard?"
        }
      ]
    },
    {
      "id": 3281872,
      "postDate": "2025-09-05T06:16:16.333Z",
      "content": "<p>The airs and fgs shapes are (11250, 32, 356) and (135000, 32, 32)<br>\nDoes the airs spatial dimension match the first or second spatial dimension of fgs1?</p>",
      "rawMarkdown": "The airs and fgs shapes are (11250, 32, 356) and (135000, 32, 32)\nDoes the airs spatial dimension match the first or second spatial dimension of fgs1?",
      "replies": [
        {
          "id": 3281935,
          "postDate": "2025-09-05T09:23:35.557Z",
          "content": "<p>It does not really match - as they are both spatial channel of an instrument, basically just two different kinds of images</p>",
          "rawMarkdown": "It does not really match - as they are both spatial channel of an instrument, basically just two different kinds of images",
          "replies": [
            {
              "id": 3281943,
              "postDate": "2025-09-05T09:49:33.677Z",
              "content": "<p>Interesting. So just to confirm, they have no common axis spatially?</p>",
              "rawMarkdown": "Interesting. So just to confirm, they have no common axis spatially?"
            },
            {
              "id": 3282178,
              "postDate": "2025-09-05T20:34:25.770Z",
              "content": "<p>in terms of instrument, FGS 1 is a photometer , and AIRS-Ch0 is a spectrometer, and they are pointing at different wavelength. you can see that FGS1 is looking at a point source, while AIRS-Ch0 is looking at a spectrum (with light spreading across the spatial channel)</p>",
              "rawMarkdown": "in terms of instrument, FGS 1 is a photometer , and AIRS-Ch0 is a spectrometer, and they are pointing at different wavelength. you can see that FGS1 is looking at a point source, while AIRS-Ch0 is looking at a spectrum (with light spreading across the spatial channel)\n",
              "votes": 1
            },
            {
              "id": 3282280,
              "postDate": "2025-09-06T02:35:21.053Z",
              "content": "<p>Perhaps common is the wrong word. Parallel?<br>\nI was under the assumption that the sensors are aligned on the same optical axis and focal plane so the AIRS cross dispersion dimension would be parallel to one of FGS's spatial dimension.</p>",
              "rawMarkdown": "Perhaps common is the wrong word. Parallel?\nI was under the assumption that the sensors are aligned on the same optical axis and focal plane so the AIRS cross dispersion dimension would be parallel to one of FGS's spatial dimension."
            },
            {
              "id": 3286174,
              "postDate": "2025-09-09T11:06:55.027Z",
              "content": "<p>Hi <a href=\"https://www.kaggle.com/sroger\" target=\"_blank\">@sroger</a> , sorry for the delay - I went to ask our expert on the instrumentation <a href=\"https://www.kaggle.com/lorenzomugnai\" target=\"_blank\">@lorenzomugnai</a> before coming back to you - so regarding your question:</p>\n<p>Both channels are collecting light from the same source as the light goes through the same telescope down to the two detectors. However only part of the optical path is in common, as the light is split multiple times in the optical to deliver to every channel only the part of the wavelength spectrum that is needed for that channel. In this regard, FGS1 only receives light between around 0.6 and 0.8 and AIRS-ch0 only light between 1.95 and 3.90. The alignment of the images is not guaranteed nor is needed as we are looking at point sources, so just a point in the sky with no spatial features, and we are only collecting (and dispersing for the spectrometer) all the light coming from it. Ariel is not meant to do imaging but just collect photons as a bucket. </p>\n<p>Hope it helps!</p>",
              "rawMarkdown": "Hi @sroger , sorry for the delay - I went to ask our expert on the instrumentation @lorenzomugnai before coming back to you - so regarding your question:\n\nBoth channels are collecting light from the same source as the light goes through the same telescope down to the two detectors. However only part of the optical path is in common, as the light is split multiple times in the optical to deliver to every channel only the part of the wavelength spectrum that is needed for that channel. In this regard, FGS1 only receives light between around 0.6 and 0.8 and AIRS-ch0 only light between 1.95 and 3.90. The alignment of the images is not guaranteed nor is needed as we are looking at point sources, so just a point in the sky with no spatial features, and we are only collecting (and dispersing for the spectrometer) all the light coming from it. Ariel is not meant to do imaging but just collect photons as a bucket. \n\nHope it helps!",
              "votes": 1
            },
            {
              "id": 3286271,
              "postDate": "2025-09-09T14:02:32.453Z",
              "content": "<p>I was hoping I could do something useful with the \"spatial\" dimensions… but thanks for the response!</p>",
              "rawMarkdown": "I was hoping I could do something useful with the \"spatial\" dimensions... but thanks for the response!"
            }
          ]
        }
      ]
    },
    {
      "id": 3278069,
      "postDate": "2025-08-29T07:02:33.070Z",
      "content": "<p>why are there 2 signal files per instrument in test directory? Also, is that the case for all of the test files?</p>",
      "rawMarkdown": "why are there 2 signal files per instrument in test directory? Also, is that the case for all of the test files?",
      "replies": [
        {
          "id": 3281600,
          "postDate": "2025-09-04T14:51:50.057Z",
          "content": "<p>yes , they represent repeated observations , hence 2 signal files. it is not the case for all the test files though!</p>",
          "rawMarkdown": "yes , they represent repeated observations , hence 2 signal files. it is not the case for all the test files though!",
          "replies": [
            {
              "id": 3281740,
              "postDate": "2025-09-04T20:03:19.357Z",
              "content": "<p>can you confirm that there are never more than 2 observations per planet?</p>",
              "rawMarkdown": "can you confirm that there are never more than 2 observations per planet?",
              "votes": 1
            },
            {
              "id": 3281932,
              "postDate": "2025-09-05T09:17:31.570Z",
              "content": "<p>I checked this through probing - this never occurs in train or test.</p>",
              "rawMarkdown": "I checked this through probing - this never occurs in train or test.",
              "votes": 3
            },
            {
              "id": 3281936,
              "postDate": "2025-09-05T09:25:06.670Z",
              "content": "<p>yeah i can confirm no more than 2 observations per planet</p>",
              "rawMarkdown": "yeah i can confirm no more than 2 observations per planet",
              "votes": 1
            },
            {
              "id": 3282130,
              "postDate": "2025-09-05T18:30:20.497Z",
              "content": "<p>How thoughtful of you  😀</p>",
              "rawMarkdown": "How thoughtful of you  😀"
            }
          ]
        }
      ]
    },
    {
      "id": 3272892,
      "postDate": "2025-08-21T17:56:09.503Z",
      "content": "<p>Hello. I'm nooby in this competition and last few days trying to create a working pipeline.<br>\nAnd now I've got a question without answer in overviews and data description.<br>\nWe have AIRS data with wavelength along one axis, and it has bounds = [1.95, 3.90]. But nowhere pointed what is the boundaries of target spectrum. It's length along wavelength axis is not equal to AIRS one.<br>\nCan you tell what's the bounds?</p>",
      "rawMarkdown": "Hello. I'm nooby in this competition and last few days trying to create a working pipeline.\nAnd now I've got a question without answer in overviews and data description.\nWe have AIRS data with wavelength along one axis, and it has bounds = [1.95, 3.90]. But nowhere pointed what is the boundaries of target spectrum. It's length along wavelength axis is not equal to AIRS one.\nCan you tell what's the bounds?",
      "replies": [
        {
          "id": 3274829,
          "postDate": "2025-08-25T13:47:45.050Z",
          "content": "<p>Hi welcome! yeah the length over wavelength axis is not equal to the AIRS. This is due to the fact that the actual spectrograph itself has a wavelength coverage that is a bit longer than that (you might have realised). The boundary of the target spectrum is recorded in the wavelength.csv file. The reason behind the cut is due to the fact that some of the channels will be too noisy (meaning the SNR is too low to be useful), so for simplicity we cut it, but dont get me wrong, they still contain some signal, if you managed to denoise them!</p>",
          "rawMarkdown": "Hi welcome! yeah the length over wavelength axis is not equal to the AIRS. This is due to the fact that the actual spectrograph itself has a wavelength coverage that is a bit longer than that (you might have realised). The boundary of the target spectrum is recorded in the wavelength.csv file. The reason behind the cut is due to the fact that some of the channels will be too noisy (meaning the SNR is too low to be useful), so for simplicity we cut it, but dont get me wrong, they still contain some signal, if you managed to denoise them!"
        }
      ]
    },
    {
      "id": 3259486,
      "postDate": "2025-08-01T14:58:54.990Z",
      "content": "<p>\"Orbital inclination in degrees\" was mentioned. Is it relative to something? Possibly to the horisontal axis of FGS-1?</p>",
      "rawMarkdown": "\"Orbital inclination in degrees\" was mentioned. Is it relative to something? Possibly to the horisontal axis of FGS-1?",
      "replies": [
        {
          "id": 3259507,
          "postDate": "2025-08-01T15:41:10.477Z",
          "content": "<p>I believe it is relative to a line perpendicular to a line from the star to the viewer (the sensors).  More succinctly, p=cos(i)*orbital_radius gives the offset from the center of the star to where the planet is passing closest to the star center on its transit.  With the units given in star_info, p is the impact parameter from 0 to 1.0 where 0 means the planet passes though the star center and p=1 means it just kisses the edge of the star.  Note that because of noise added to the reported values, you will find some p&gt;1.</p>",
          "rawMarkdown": "I believe it is relative to a line perpendicular to a line from the star to the viewer (the sensors).  More succinctly, p=cos(i)*orbital_radius gives the offset from the center of the star to where the planet is passing closest to the star center on its transit.  With the units given in star_info, p is the impact parameter from 0 to 1.0 where 0 means the planet passes though the star center and p=1 means it just kisses the edge of the star.  Note that because of noise added to the reported values, you will find some p>1.",
          "votes": 5,
          "replies": [
            {
              "id": 3259544,
              "postDate": "2025-08-01T17:01:36.400Z",
              "content": "<p>Didn't get how it could be &gt;1 since it's cosine and angle between 86 and 90. Seems more like its between 0 and ≈0.07. And 0.07 is probably the edge case when planet kisses the star. Maybe you said the same and I got it wrong</p>",
              "rawMarkdown": "Didn't get how it could be >1 since it's cosine and angle between 86 and 90. Seems more like its between 0 and ≈0.07. And 0.07 is probably the edge case when planet kisses the star. Maybe you said the same and I got it wrong"
            },
            {
              "id": 3259570,
              "postDate": "2025-08-01T18:31:39.700Z",
              "content": "<p>You have to multiply by the orbital radius (in stellar radius units) to get the impact parameter p, which can be &gt;1.</p>",
              "rawMarkdown": "You have to multiply by the orbital radius (in stellar radius units) to get the impact parameter p, which can be >1.",
              "votes": 1
            },
            {
              "id": 3259574,
              "postDate": "2025-08-01T18:43:04.650Z",
              "content": "<p>I see now, appreciate it</p>",
              "rawMarkdown": "I see now, appreciate it"
            }
          ]
        }
      ]
    },
    {
      "id": 3254098,
      "postDate": "2025-07-25T19:40:48.997Z",
      "content": "<p>Are you willing to elaborate on the population shift between the train and test sets? Last year I believe it was revealed that the only difference was in the elements used to generate the ground truth spectra and the stellar spectra, while all other physics followed the same distributions.</p>",
      "rawMarkdown": "Are you willing to elaborate on the population shift between the train and test sets? Last year I believe it was revealed that the only difference was in the elements used to generate the ground truth spectra and the stellar spectra, while all other physics followed the same distributions.",
      "replies": [
        {
          "id": 3269311,
          "postDate": "2025-08-14T08:48:50.320Z",
          "content": "<p><a href=\"https://www.kaggle.com/jeroencottaar\" target=\"_blank\">@jeroencottaar</a> - I must have missed this one. Let me try to elaborate without revealing too much. We follow the following philosophy when constructing the dataset: Ariel will observe both known and unknown exoplanet, meaning those we have learnt something about the atmosphere, and those we dont and is completely unknown to us. They may have new physics (unknown atmospheric processes) as well as known physics (like water molecules). Now what are the known and unknown exoplanets? There are planets who are known to us (discovered) and have known atmospheres (characterised), but also planets where they have been discovered, but never characterised. In the test dataset, you will find both cases! <br>\nHope it helps :) </p>",
          "rawMarkdown": "@jeroencottaar - I must have missed this one. Let me try to elaborate without revealing too much. We follow the following philosophy when constructing the dataset: Ariel will observe both known and unknown exoplanet, meaning those we have learnt something about the atmosphere, and those we dont and is completely unknown to us. They may have new physics (unknown atmospheric processes) as well as known physics (like water molecules). Now what are the known and unknown exoplanets? There are planets who are known to us (discovered) and have known atmospheres (characterised), but also planets where they have been discovered, but never characterised. In the test dataset, you will find both cases! \nHope it helps :) ",
          "votes": 1,
          "replies": [
            {
              "id": 3269316,
              "postDate": "2025-08-14T09:01:06.083Z",
              "content": "<p>Thanks for the response. Can you confirm that the characteristics of the sensor do not vary between train and test set? For example the defocused shape on the FGS sensor, the nature of the jitter, distribution of dead and hot pixels, etc.?</p>",
              "rawMarkdown": "Thanks for the response. Can you confirm that the characteristics of the sensor do not vary between train and test set? For example the defocused shape on the FGS sensor, the nature of the jitter, distribution of dead and hot pixels, etc.?"
            },
            {
              "id": 3269326,
              "postDate": "2025-08-14T09:39:13.237Z",
              "content": "<p>I am afraid the characteristic of the sensor will also vary - which means distribution of dead/hot pixels , cosmic ray etc. The jitter time series distribution will not change. will come back on the shape of FGS. </p>",
              "rawMarkdown": "I am afraid the characteristic of the sensor will also vary - which means distribution of dead/hot pixels , cosmic ray etc. The jitter time series distribution will not change. will come back on the shape of FGS. "
            }
          ]
        }
      ]
    },
    {
      "id": 3262711,
      "postDate": "2025-08-04T06:49:42.180Z",
      "rawMarkdown": "",
      "isDeleted": true
    },
    {
      "id": 3258880,
      "postDate": "2025-07-31T11:48:27.443Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 3252421,
      "author_name": "SolverWorld",
      "author_url": "",
      "post_date": "2025-07-22T17:35:59.937000",
      "content": "<p>Can you tell us what physical parameter we are actually trying to estimate?  Transit depth is obvious in the absence of limb darkening, but with limb darkening, it is not clear what we are being asked to predict.  It could be one of the following:</p>\n<ol>\n<li>The effective (Rp/Rs)^2.  This would be the values in the EXOSIM2 planetary radius file:  <a href=\"https://exosim2-public.readthedocs.io/en/latest/api/exosim/tasks/astrosignal/estimatePlanetarySignal/index.html#module-exosim.tasks.astrosignal.estimatePlanetarySignal\" target=\"_blank\">load_rp</a></li>\n<li>The light reduction at mid-transit</li>\n<li>The maximum reduction in light intensity.  The paper <em>Analytic solutions to the maximum and average exoplanet\ntransit depth for common stellar limb darkening laws</em> by <a href=\"https://arxiv.org/abs/1901.01730\" target=\"_blank\">Heller 2019</a>, Fig 1, shows delta (transit depth) as the maximum and relates that to the (Rp/Rs)^2 for various limb darkening laws.</li>\n<li>Just the ground truth you give us and don't ask too many questions 😀</li>\n</ol>",
      "votes": 5,
      "replies": [
        {
          "id": 3252461,
          "author_name": "",
          "author_url": "",
          "post_date": "2025-07-22T19:22:00.803000",
          "content": "",
          "votes": 0,
          "replies": []
        },
        {
          "id": 3252559,
          "author_name": "c-number",
          "author_url": "",
          "post_date": "2025-07-23T02:12:54.503000",
          "content": "<p>I think it's 1.</p>",
          "votes": 1,
          "replies": [
            {
              "id": 3252570,
              "author_name": "Horikita Saku",
              "author_url": "",
              "post_date": "2025-07-23T02:52:50.153000",
              "content": "<p>same<br>\nMy analysis is based on the assumption that the situation is 1.</p>",
              "votes": 1,
              "replies": []
            },
            {
              "id": 3252576,
              "author_name": "",
              "author_url": "",
              "post_date": "2025-07-23T03:09:15.020000",
              "content": "",
              "votes": 0,
              "replies": []
            }
          ]
        },
        {
          "id": 3252754,
          "author_name": "Lorenzo Mugnai",
          "author_url": "",
          "post_date": "2025-07-23T10:58:07.760000",
          "content": "<p>The goal of the challenge is to recover the ground truth, the intrinsic transmission spectrum, namely the <em>(Rp/Rs)^2</em> at each wavelength. This fundamental transit measurement directly reveals how the planet’s atmosphere absorbs starlight.</p>\n<p>In practice, you must build an estimator that disentangles the atmospheric signal in the data from instrumental and astrophysical noise, such as limb darkening, to yield the most precise and unbiased transmission profile possible. That is your ground truth 😄</p>",
          "votes": 5,
          "replies": []
        },
        {
          "id": 3252798,
          "author_name": "malinzi jeremiah",
          "author_url": "",
          "post_date": "2025-07-23T12:51:12.533000",
          "content": "<p>The physical parameter  is the effective (Rp/Rs)^2, where:</p>\n<ul>\n<li>Rp is the planetary radius</li>\n<li>Rs is the stellar radius</li>\n</ul>\n<p>This parameter represents the ratio of the planetary radius to the stellar radius squared and is a key factor in determining the transit depth of an exoplanet.</p>\n<p>In the context of transit observations, the effective (Rp/Rs)^2 is often used to describe the decrease in stellar flux caused by the planet passing in front of the star. Limb darkening can affect this measurement, but the effective (Rp/Rs)^2 remains a fundamental parameter for characterizing exoplanet properties.</p>",
          "votes": -1,
          "replies": []
        }
      ]
    },
    {
      "id": 3253630,
      "author_name": "c-number",
      "author_url": "",
      "post_date": "2025-07-25T02:48:50.707000",
      "content": "<p>Is there any <strong>intentional</strong> data drift between public test (for public LB) and private test (for private LB) ?<br>\nLast year, I've seen some teams dropping their rank by several hundreds assumably because of the distruibution difference, and I'm curious whether we should be prepared for that this year also.</p>\n<p>By <strong>intentional</strong>, I mean different types of molecules in the atmosphere, different limb darkening functions, different transit position etc.</p>",
      "votes": 4,
      "replies": [
        {
          "id": 3253966,
          "author_name": "Gordon Yip",
          "author_url": "",
          "post_date": "2025-07-25T15:20:24.437000",
          "content": "<p>No , the public LB and private LB should be homogeneous in terms of test distribution. </p>",
          "votes": 6,
          "replies": [
            {
              "id": 3254095,
              "author_name": "Jeroen Cottaar",
              "author_url": "",
              "post_date": "2025-07-25T19:38:36.277000",
              "content": "<p>This was also the case last year, right? I always assumed the teams that dropped a lot simply had a solution that wasn't robust and made a very poor predictions on a single (or a few) elements of the private test set.</p>",
              "votes": 4,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 3281524,
      "author_name": "Maxence Gérard",
      "author_url": "",
      "post_date": "2025-09-04T13:27:14.100000",
      "content": "<p>Hello, I've noticed that when there are double measurements, the star intensity can vary considerably (by at least 10% for some exoplanets). I was wondering what the physical explanations for this phenomenon might be, since I assumed the two measurements were taken under the same conditions.</p>",
      "votes": 1,
      "replies": [
        {
          "id": 3281598,
          "author_name": "Gordon Yip",
          "author_url": "",
          "post_date": "2025-09-04T14:51:23.163000",
          "content": "<p>Yes you are right - the stellar intensity can change between observations. While we are observing the same star but there could be variability on the stellar flux. We are simulating this situation here, but, in some cases we might have overdone it !</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 3281676,
          "author_name": "Sohier Dane",
          "author_url": "",
          "post_date": "2025-09-04T17:02:06.723000",
          "content": "<p>I don't know what's actually covered in the simulations, but there are some truly wild <a href=\"https://en.wikipedia.org/wiki/Variable_star#Pulsating_variable_stars\" target=\"_blank\">variable stars</a> that you might enjoy reading about. </p>",
          "votes": 2,
          "replies": []
        }
      ]
    },
    {
      "id": 3249875,
      "author_name": "🐢 Jun Koda",
      "author_url": "",
      "post_date": "2025-07-17T09:19:58.110000",
      "content": "<p>Can you say something about the constant background/foreground light? Last year, I could see obvious offset far from the star, and that caused systematic offset in the target values that even some top players could not understand. This year I don't see that offset. Is it subtracted? Or, it is a non-trivial problem that we should think about?</p>",
      "votes": 4,
      "replies": [
        {
          "id": 3250978,
          "author_name": "Gordon Yip",
          "author_url": "",
          "post_date": "2025-07-19T15:58:05.223000",
          "content": "<p>Sure thing. We can confirm that the background/foreground light has been implemented in this edition of the data challenge. The seeming absence of the effect is due to the way Analog-to-Digital conversion has been implemented this year. We moved from dynamic scaling (each observation having its own gain and offset). This year, instead, we are using fixed gain and fixed offset to be more representative of the real instrument. This change means that any value below the \"offset\" is lost, contributing to the apparent disappearance of the background/foreground light.</p>\n<p>hope that makes sense - let me know if you have more questions. </p>",
          "votes": 4,
          "replies": []
        }
      ]
    },
    {
      "id": 3269906,
      "author_name": "CPMP",
      "author_url": "",
      "post_date": "2025-08-15T09:51:51.333000",
      "content": "<p><a href=\"https://www.kaggle.com/gordonyip\" target=\"_blank\">@gordonyip</a> see this discussion: <a href=\"https://www.kaggle.com/competitions/ariel-data-challenge-2025/discussion/598324\" target=\"_blank\">https://www.kaggle.com/competitions/ariel-data-challenge-2025/discussion/598324</a></p>\n<p>Wouldn't it make sense to discard these samples where ingress and/or egress are not fully contained in the data we have? I am speaking of the test set here of course.</p>\n<p>I spend time fixing these but I doubt this is useful, nor do I think I did a perfect job at it. I am pretty sure many others spent time on these samples too. If we knew for sure that nothing similar occurs in the test set then we can all save time an focus on what is useful to you.</p>",
      "votes": 1,
      "replies": [
        {
          "id": 3274824,
          "author_name": "Gordon Yip",
          "author_url": "",
          "post_date": "2025-08-25T13:44:03.627000",
          "content": "<p>Hi <a href=\"https://www.kaggle.com/cpmpml\" target=\"_blank\">@cpmpml</a> , also tagging <a href=\"https://www.kaggle.com/jeroencottaar\" target=\"_blank\">@jeroencottaar</a> here as you might be interested. discarding/changing the metric would make sense but at the same time will be hard on the competition organisation side since the competition is entering its final month, and it will inevitably change the workflow of many others. We are inclined to not change that, but what we could do is calculate the score if those examples were discarded, after the competition has ended. </p>",
          "votes": 0,
          "replies": [
            {
              "id": 3274850,
              "author_name": "SolverWorld",
              "author_url": "",
              "post_date": "2025-08-25T14:16:50.620000",
              "content": "<p>Please don't change the way the metric is calculated between the public and private leaderboard.  That would kind of defeat the purpose of the public leaderboard.  I would think you would also have to make an (official) public announcement  and it would seem unfair at this late stage in the competition.</p>",
              "votes": 3,
              "replies": []
            },
            {
              "id": 3274864,
              "author_name": "Gordon Yip",
              "author_url": "",
              "post_date": "2025-08-25T14:29:57.527000",
              "content": "<p>Yep we fully understand the implication so we are not planning to do it, but we would like to communicate our current thoughts here just to maintain transparency. </p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 3275021,
              "author_name": "CPMP",
              "author_url": "",
              "post_date": "2025-08-25T19:03:35.550000",
              "content": "<p><a href=\"https://www.kaggle.com/gordonyip\" target=\"_blank\">@gordonyip</a> </p>\n<blockquote>\n  <p>what we could do is calculate the score if those examples were discarded, after the competition has ended. </p>\n</blockquote>\n<p>I don't get what you mean by the above. </p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 3277569,
              "author_name": "Gordon Yip",
              "author_url": "",
              "post_date": "2025-08-28T09:04:59.513000",
              "content": "<p>As in , after the competition has ended, we can release <code>unofficial score</code>, i.e. the overall score if those lightcruves without full ingress and egress are removed , that will not change the leaderboard, nor the prize ranking unfortunately, and will only purely for our own interest. </p>",
              "votes": 1,
              "replies": []
            },
            {
              "id": 3277580,
              "author_name": "CPMP",
              "author_url": "",
              "post_date": "2025-08-28T09:32:39.453000",
              "content": "<p>Thank you, this is clearer.</p>",
              "votes": 1,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 3283156,
      "author_name": "Maxence Gérard",
      "author_url": "",
      "post_date": "2025-09-07T10:36:57.130000",
      "content": "<p>Hello! I’ve noticed something unusual in the linear correction of the FGS1 channel for the exoplanet 34983 (and, since the correlation files are duplicated, it also affects many others). The degree-0 polynomial coefficient is 91.5597 for spatial pixel 572, which is a very large offset. This looks incorrect when applied to the FGS1 observation, especially compared with the adjacent pixels.</p>\n<p>While this offset is removed once we compute the CDS, it raises concerns about the validity of the rest of the linear correction — especially the higher-order terms. This may well be intentional, but I wanted to check whether such behaviour is expected.</p>",
      "votes": 0,
      "replies": [
        {
          "id": 3289027,
          "author_name": "Maxence Gérard",
          "author_url": "",
          "post_date": "2025-09-15T09:50:33.317000",
          "content": "<p>Sorry to insist, but I’d like to know if you’ve noticed the same thing as I did, or if I’m doing something wrong in the preprocessing.</p>",
          "votes": 0,
          "replies": [
            {
              "id": 3289309,
              "author_name": "Gordon Yip",
              "author_url": "",
              "post_date": "2025-09-15T20:20:05.063000",
              "content": "<p>no worries  you are absolutely right to pursue - i am going to check with our instrument team and get back to you on this, sorry for the delay</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 3290468,
              "author_name": "Gordon Yip",
              "author_url": "",
              "post_date": "2025-09-17T21:02:08.560000",
              "content": "<p><a href=\"https://www.kaggle.com/mgerard\" target=\"_blank\">@mgerard</a> <br>\nI asked <a href=\"https://www.kaggle.com/lorenzomugnai\" target=\"_blank\">@lorenzomugnai</a> about this and here is the reply:<br>\nThe calibration data we use originate from JWST and include, for the same detector, dark current, read noise, bad pixels, hot pixels, and linearity correction. We use these data to generate the simulations and then provide them to the participants with an associated uncertainty.</p>\n<p>This means that the values for individual pixels are consistent across the different calibration products, but they are outside of our control. There are several reasons why a given pixel may appear unusual in one of the calibration files. A clearer picture is often obtained by comparing different calibrators to see whether the same pixel shows anomalous behaviour in multiple cases. For instance, a non-zero degree offset in the linearity correction often indicates a hot pixel or one with a strong residual initial charge.</p>\n<p>In any case, these are genuine calibration data from a real and very complex instrument, and they are representative of what will be encountered in Ariel, since it will use detectors of the same type.</p>",
              "votes": 1,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 3281941,
      "author_name": "Stephan Kafanov",
      "author_url": "",
      "post_date": "2025-09-05T09:48:32.777000",
      "content": "<p>Another one nooby question. How shall I submit my solution? I've run my notebook from start to the end without problems. I am predicting generalized test. In logs I can see, that test (1 planet from example) was predicted and submission created. But after this happend I know nothing. And after hours of waiting submission scorring failed with an error. What should I submit and how.<br>\nI've heared about substetution of test data, but I do not see it in the submission.</p>",
      "votes": 0,
      "replies": [
        {
          "id": 3284356,
          "author_name": "Gordon Yip",
          "author_url": "",
          "post_date": "2025-09-08T07:17:28.653000",
          "content": "<p>hmm , did you see a score in the public leaderboard?</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 3281872,
      "author_name": "sroger",
      "author_url": "",
      "post_date": "2025-09-05T06:16:16.333000",
      "content": "<p>The airs and fgs shapes are (11250, 32, 356) and (135000, 32, 32)<br>\nDoes the airs spatial dimension match the first or second spatial dimension of fgs1?</p>",
      "votes": 0,
      "replies": [
        {
          "id": 3281935,
          "author_name": "Gordon Yip",
          "author_url": "",
          "post_date": "2025-09-05T09:23:35.557000",
          "content": "<p>It does not really match - as they are both spatial channel of an instrument, basically just two different kinds of images</p>",
          "votes": 0,
          "replies": [
            {
              "id": 3281943,
              "author_name": "sroger",
              "author_url": "",
              "post_date": "2025-09-05T09:49:33.677000",
              "content": "<p>Interesting. So just to confirm, they have no common axis spatially?</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 3282178,
              "author_name": "Gordon Yip",
              "author_url": "",
              "post_date": "2025-09-05T20:34:25.770000",
              "content": "<p>in terms of instrument, FGS 1 is a photometer , and AIRS-Ch0 is a spectrometer, and they are pointing at different wavelength. you can see that FGS1 is looking at a point source, while AIRS-Ch0 is looking at a spectrum (with light spreading across the spatial channel)</p>",
              "votes": 1,
              "replies": []
            },
            {
              "id": 3282280,
              "author_name": "sroger",
              "author_url": "",
              "post_date": "2025-09-06T02:35:21.053000",
              "content": "<p>Perhaps common is the wrong word. Parallel?<br>\nI was under the assumption that the sensors are aligned on the same optical axis and focal plane so the AIRS cross dispersion dimension would be parallel to one of FGS's spatial dimension.</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 3286174,
              "author_name": "Gordon Yip",
              "author_url": "",
              "post_date": "2025-09-09T11:06:55.027000",
              "content": "<p>Hi <a href=\"https://www.kaggle.com/sroger\" target=\"_blank\">@sroger</a> , sorry for the delay - I went to ask our expert on the instrumentation <a href=\"https://www.kaggle.com/lorenzomugnai\" target=\"_blank\">@lorenzomugnai</a> before coming back to you - so regarding your question:</p>\n<p>Both channels are collecting light from the same source as the light goes through the same telescope down to the two detectors. However only part of the optical path is in common, as the light is split multiple times in the optical to deliver to every channel only the part of the wavelength spectrum that is needed for that channel. In this regard, FGS1 only receives light between around 0.6 and 0.8 and AIRS-ch0 only light between 1.95 and 3.90. The alignment of the images is not guaranteed nor is needed as we are looking at point sources, so just a point in the sky with no spatial features, and we are only collecting (and dispersing for the spectrometer) all the light coming from it. Ariel is not meant to do imaging but just collect photons as a bucket. </p>\n<p>Hope it helps!</p>",
              "votes": 1,
              "replies": []
            },
            {
              "id": 3286271,
              "author_name": "sroger",
              "author_url": "",
              "post_date": "2025-09-09T14:02:32.453000",
              "content": "<p>I was hoping I could do something useful with the \"spatial\" dimensions… but thanks for the response!</p>",
              "votes": 0,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 3278069,
      "author_name": "A S SHIVAM",
      "author_url": "",
      "post_date": "2025-08-29T07:02:33.070000",
      "content": "<p>why are there 2 signal files per instrument in test directory? Also, is that the case for all of the test files?</p>",
      "votes": 0,
      "replies": [
        {
          "id": 3281600,
          "author_name": "Gordon Yip",
          "author_url": "",
          "post_date": "2025-09-04T14:51:50.057000",
          "content": "<p>yes , they represent repeated observations , hence 2 signal files. it is not the case for all the test files though!</p>",
          "votes": 0,
          "replies": [
            {
              "id": 3281740,
              "author_name": "SolverWorld",
              "author_url": "",
              "post_date": "2025-09-04T20:03:19.357000",
              "content": "<p>can you confirm that there are never more than 2 observations per planet?</p>",
              "votes": 1,
              "replies": []
            },
            {
              "id": 3281932,
              "author_name": "Jeroen Cottaar",
              "author_url": "",
              "post_date": "2025-09-05T09:17:31.570000",
              "content": "<p>I checked this through probing - this never occurs in train or test.</p>",
              "votes": 3,
              "replies": []
            },
            {
              "id": 3281936,
              "author_name": "Gordon Yip",
              "author_url": "",
              "post_date": "2025-09-05T09:25:06.670000",
              "content": "<p>yeah i can confirm no more than 2 observations per planet</p>",
              "votes": 1,
              "replies": []
            },
            {
              "id": 3282130,
              "author_name": "SolverWorld",
              "author_url": "",
              "post_date": "2025-09-05T18:30:20.497000",
              "content": "<p>How thoughtful of you  😀</p>",
              "votes": 0,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 3272892,
      "author_name": "Stephan Kafanov",
      "author_url": "",
      "post_date": "2025-08-21T17:56:09.503000",
      "content": "<p>Hello. I'm nooby in this competition and last few days trying to create a working pipeline.<br>\nAnd now I've got a question without answer in overviews and data description.<br>\nWe have AIRS data with wavelength along one axis, and it has bounds = [1.95, 3.90]. But nowhere pointed what is the boundaries of target spectrum. It's length along wavelength axis is not equal to AIRS one.<br>\nCan you tell what's the bounds?</p>",
      "votes": 0,
      "replies": [
        {
          "id": 3274829,
          "author_name": "Gordon Yip",
          "author_url": "",
          "post_date": "2025-08-25T13:47:45.050000",
          "content": "<p>Hi welcome! yeah the length over wavelength axis is not equal to the AIRS. This is due to the fact that the actual spectrograph itself has a wavelength coverage that is a bit longer than that (you might have realised). The boundary of the target spectrum is recorded in the wavelength.csv file. The reason behind the cut is due to the fact that some of the channels will be too noisy (meaning the SNR is too low to be useful), so for simplicity we cut it, but dont get me wrong, they still contain some signal, if you managed to denoise them!</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 3259486,
      "author_name": "Артём Хапилов",
      "author_url": "",
      "post_date": "2025-08-01T14:58:54.990000",
      "content": "<p>\"Orbital inclination in degrees\" was mentioned. Is it relative to something? Possibly to the horisontal axis of FGS-1?</p>",
      "votes": 0,
      "replies": [
        {
          "id": 3259507,
          "author_name": "SolverWorld",
          "author_url": "",
          "post_date": "2025-08-01T15:41:10.477000",
          "content": "<p>I believe it is relative to a line perpendicular to a line from the star to the viewer (the sensors).  More succinctly, p=cos(i)*orbital_radius gives the offset from the center of the star to where the planet is passing closest to the star center on its transit.  With the units given in star_info, p is the impact parameter from 0 to 1.0 where 0 means the planet passes though the star center and p=1 means it just kisses the edge of the star.  Note that because of noise added to the reported values, you will find some p&gt;1.</p>",
          "votes": 5,
          "replies": [
            {
              "id": 3259544,
              "author_name": "Артём Хапилов",
              "author_url": "",
              "post_date": "2025-08-01T17:01:36.400000",
              "content": "<p>Didn't get how it could be &gt;1 since it's cosine and angle between 86 and 90. Seems more like its between 0 and ≈0.07. And 0.07 is probably the edge case when planet kisses the star. Maybe you said the same and I got it wrong</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 3259570,
              "author_name": "SolverWorld",
              "author_url": "",
              "post_date": "2025-08-01T18:31:39.700000",
              "content": "<p>You have to multiply by the orbital radius (in stellar radius units) to get the impact parameter p, which can be &gt;1.</p>",
              "votes": 1,
              "replies": []
            },
            {
              "id": 3259574,
              "author_name": "Артём Хапилов",
              "author_url": "",
              "post_date": "2025-08-01T18:43:04.650000",
              "content": "<p>I see now, appreciate it</p>",
              "votes": 0,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 3254098,
      "author_name": "Jeroen Cottaar",
      "author_url": "",
      "post_date": "2025-07-25T19:40:48.997000",
      "content": "<p>Are you willing to elaborate on the population shift between the train and test sets? Last year I believe it was revealed that the only difference was in the elements used to generate the ground truth spectra and the stellar spectra, while all other physics followed the same distributions.</p>",
      "votes": 0,
      "replies": [
        {
          "id": 3269311,
          "author_name": "Gordon Yip",
          "author_url": "",
          "post_date": "2025-08-14T08:48:50.320000",
          "content": "<p><a href=\"https://www.kaggle.com/jeroencottaar\" target=\"_blank\">@jeroencottaar</a> - I must have missed this one. Let me try to elaborate without revealing too much. We follow the following philosophy when constructing the dataset: Ariel will observe both known and unknown exoplanet, meaning those we have learnt something about the atmosphere, and those we dont and is completely unknown to us. They may have new physics (unknown atmospheric processes) as well as known physics (like water molecules). Now what are the known and unknown exoplanets? There are planets who are known to us (discovered) and have known atmospheres (characterised), but also planets where they have been discovered, but never characterised. In the test dataset, you will find both cases! <br>\nHope it helps :) </p>",
          "votes": 1,
          "replies": [
            {
              "id": 3269316,
              "author_name": "Jeroen Cottaar",
              "author_url": "",
              "post_date": "2025-08-14T09:01:06.083000",
              "content": "<p>Thanks for the response. Can you confirm that the characteristics of the sensor do not vary between train and test set? For example the defocused shape on the FGS sensor, the nature of the jitter, distribution of dead and hot pixels, etc.?</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 3269326,
              "author_name": "Gordon Yip",
              "author_url": "",
              "post_date": "2025-08-14T09:39:13.237000",
              "content": "<p>I am afraid the characteristic of the sensor will also vary - which means distribution of dead/hot pixels , cosmic ray etc. The jitter time series distribution will not change. will come back on the shape of FGS. </p>",
              "votes": 0,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 3262711,
      "author_name": "",
      "author_url": "",
      "post_date": "2025-08-04T06:49:42.180000",
      "content": "",
      "votes": 0,
      "replies": []
    },
    {
      "id": 3258880,
      "author_name": "",
      "author_url": "",
      "post_date": "2025-07-31T11:48:27.443000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3249350": "You’re right. The competition is really hard. The difficulty is actually the reason we set these data challenges to begin with - our problem is so hard that we need your help! \n\nWhile we cannot disclose the solution to you, we wanted you to have a way to ask questions to us - to really make a breakthrough, all of us need to work together -\n\nHere is what we will do:\n1.⁠ ⁠You can ask your questions in the forum and we will address them there (we are monitoring the discussion)\n2.⁠ ⁠You can leave your question below and we will reply to you.\n3.⁠ ⁠You can leave your questions in the [questionnaire](https://forms.gle/KtdD3gVX5eA64hpx9), we will leave a reply in this thread. \n\nAny appropriate questions submitted in the questionnaire will also be added and responded to here for the benefit of the whole community.\n\nHere is the catch, we cannot promise to answer every question. We might not answer your question if we think it will give away too much details about the setup of the game/ground truth, sorry!\n\nSide note - if you have a few minutes to spare, please fill in the [survey](https://forms.gle/m52D74GATX8hm8vV9)!",
    "3252421": "Can you tell us what physical parameter we are actually trying to estimate?  Transit depth is obvious in the absence of limb darkening, but with limb darkening, it is not clear what we are being asked to predict.  It could be one of the following:\n1. The effective (Rp/Rs)^2.  This would be the values in the EXOSIM2 planetary radius file:  [load_rp](https://exosim2-public.readthedocs.io/en/latest/api/exosim/tasks/astrosignal/estimatePlanetarySignal/index.html#module-exosim.tasks.astrosignal.estimatePlanetarySignal)\n2. The light reduction at mid-transit\n3.  The maximum reduction in light intensity.  The paper *Analytic solutions to the maximum and average exoplanet\ntransit depth for common stellar limb darkening laws* by [Heller 2019](https://arxiv.org/abs/1901.01730), Fig 1, shows delta (transit depth) as the maximum and relates that to the (Rp/Rs)^2 for various limb darkening laws.\n4. Just the ground truth you give us and don't ask too many questions 😀",
    "3253630": "Is there any **intentional** data drift between public test (for public LB) and private test (for private LB) ?\nLast year, I've seen some teams dropping their rank by several hundreds assumably because of the distruibution difference, and I'm curious whether we should be prepared for that this year also.\n\nBy **intentional**, I mean different types of molecules in the atmosphere, different limb darkening functions, different transit position etc.",
    "3281524": "Hello, I've noticed that when there are double measurements, the star intensity can vary considerably (by at least 10% for some exoplanets). I was wondering what the physical explanations for this phenomenon might be, since I assumed the two measurements were taken under the same conditions.",
    "3249875": "Can you say something about the constant background/foreground light? Last year, I could see obvious offset far from the star, and that caused systematic offset in the target values that even some top players could not understand. This year I don't see that offset. Is it subtracted? Or, it is a non-trivial problem that we should think about?",
    "3269906": "@gordonyip see this discussion: https://www.kaggle.com/competitions/ariel-data-challenge-2025/discussion/598324\n\nWouldn't it make sense to discard these samples where ingress and/or egress are not fully contained in the data we have? I am speaking of the test set here of course.\n\nI spend time fixing these but I doubt this is useful, nor do I think I did a perfect job at it. I am pretty sure many others spent time on these samples too. If we knew for sure that nothing similar occurs in the test set then we can all save time an focus on what is useful to you.",
    "3283156": "Hello! I’ve noticed something unusual in the linear correction of the FGS1 channel for the exoplanet 34983 (and, since the correlation files are duplicated, it also affects many others). The degree-0 polynomial coefficient is 91.5597 for spatial pixel 572, which is a very large offset. This looks incorrect when applied to the FGS1 observation, especially compared with the adjacent pixels.\n\nWhile this offset is removed once we compute the CDS, it raises concerns about the validity of the rest of the linear correction — especially the higher-order terms. This may well be intentional, but I wanted to check whether such behaviour is expected.",
    "3281941": "Another one nooby question. How shall I submit my solution? I've run my notebook from start to the end without problems. I am predicting generalized test. In logs I can see, that test (1 planet from example) was predicted and submission created. But after this happend I know nothing. And after hours of waiting submission scorring failed with an error. What should I submit and how.\nI've heared about substetution of test data, but I do not see it in the submission.",
    "3281872": "The airs and fgs shapes are (11250, 32, 356) and (135000, 32, 32)\nDoes the airs spatial dimension match the first or second spatial dimension of fgs1?",
    "3278069": "why are there 2 signal files per instrument in test directory? Also, is that the case for all of the test files?",
    "3272892": "Hello. I'm nooby in this competition and last few days trying to create a working pipeline.\nAnd now I've got a question without answer in overviews and data description.\nWe have AIRS data with wavelength along one axis, and it has bounds = [1.95, 3.90]. But nowhere pointed what is the boundaries of target spectrum. It's length along wavelength axis is not equal to AIRS one.\nCan you tell what's the bounds?",
    "3259486": "\"Orbital inclination in degrees\" was mentioned. Is it relative to something? Possibly to the horisontal axis of FGS-1?",
    "3254098": "Are you willing to elaborate on the population shift between the train and test sets? Last year I believe it was revealed that the only difference was in the elements used to generate the ground truth spectra and the stellar spectra, while all other physics followed the same distributions.",
    "3262711": "",
    "3258880": ""
  }
}