{ "id": 532639, "title": "Details about jitter noise assumptions", "url": "/competitions/ariel-data-challenge-2024/discussion/532639", "author_name": "Simon Dupourque", "post_date": "2024-09-07T09:32:23.694000", "votes": 6, "comment_count": 10, "views": 0, "content": "

Hi, thank you very much for hosting this competition, this is very fun to do!

\n

I wondered if you could share any assumption about the jitter noise you model in these observations, in particular the expected rms or equivalent, is it the same accross all the observed transits etc. I would like to test some denoising models that require a bit of a priori knowledge on the noise sources in the data, and having more details on this would be great!

", "messages": [ { "id": 2983874, "postDate": "2024-09-09T07:31:23.677Z", "content": "

Hello, that’s an excellent question!

\n

Pointing jitter on a space telescope arises from a variety of sources. To put it simply, it’s the result of a noninear interplay between solar wind, vibrations from the instruments and the efforts of the pointing system to stay locked onto the target. Therefore the noise’s power sprectrum is not flat but significantly non-Gaussian. Simulating this is quite challenging, as it largely depends on how the payload is constructed. If you're interested in exploring this topic from an academic perspective, there is some literature we can point you to. However, you’ll find that the impact and spectral characteristics of jitter varies significantly between space missions due to the unique design and components of each mission.

\n

For this exercise, we used realistic jitter models for Ariel space mission provided by the manufacturer for the specific observed targets in question.

", "rawMarkdown": "Hello, that’s an excellent question!\n\nPointing jitter on a space telescope arises from a variety of sources. To put it simply, it’s the result of a noninear interplay between solar wind, vibrations from the instruments and the efforts of the pointing system to stay locked onto the target. Therefore the noise’s power sprectrum is not flat but significantly non-Gaussian. Simulating this is quite challenging, as it largely depends on how the payload is constructed. If you're interested in exploring this topic from an academic perspective, there is some literature we can point you to. However, you’ll find that the impact and spectral characteristics of jitter varies significantly between space missions due to the unique design and components of each mission.\n\nFor this exercise, we used realistic jitter models for Ariel space mission provided by the manufacturer for the specific observed targets in question.", "votes": 5, "replies": [ { "id": 2985616, "postDate": "2024-09-10T19:40:55.387Z", "content": "

Lorenzo thanks for your detailed answer- can you please clarify if the (pointing) jitter is of the same origin in the 2 instruments in our data? From what I saw they seem to be independent- is this an effect of the used simulator or is this the expected behavior?

", "rawMarkdown": "Lorenzo thanks for your detailed answer- can you please clarify if the (pointing) jitter is of the same origin in the 2 instruments in our data? From what I saw they seem to be independent- is this an effect of the used simulator or is this the expected behavior?", "replies": [ { "id": 2985943, "postDate": "2024-09-11T07:09:55.537Z", "content": "

Thanks for your question! The pointing jitter originates from the same source for both instruments, as they are observing the same target simultaneously. However, the apparent independence you've noticed could be due to differences between the two detectors. While the physical pointing jitter is identical, each detector has its own characteristics, such as PSF shape, pixel quantum efficiency, and integration time, which influence how the jitter manifests in the final data.

\n

So, the fact that the jitter seems different across the two instruments is not an effect of the simulator, but rather an expected behaviour due to the distinct properties of the detectors. The movement of the point source during integration is convolved with the different characteristics for each detector, leading to results that may appear independent.

", "rawMarkdown": "Thanks for your question! The pointing jitter originates from the same source for both instruments, as they are observing the same target simultaneously. However, the apparent independence you've noticed could be due to differences between the two detectors. While the physical pointing jitter is identical, each detector has its own characteristics, such as PSF shape, pixel quantum efficiency, and integration time, which influence how the jitter manifests in the final data.\n\nSo, the fact that the jitter seems different across the two instruments is not an effect of the simulator, but rather an expected behaviour due to the distinct properties of the detectors. The movement of the point source during integration is convolved with the different characteristics for each detector, leading to results that may appear independent.", "votes": 1 } ] } ] }, { "id": 2981928, "postDate": "2024-09-07T09:32:23.693Z", "content": "

Hi, thank you very much for hosting this competition, this is very fun to do!

\n

I wondered if you could share any assumption about the jitter noise you model in these observations, in particular the expected rms or equivalent, is it the same accross all the observed transits etc. I would like to test some denoising models that require a bit of a priori knowledge on the noise sources in the data, and having more details on this would be great!

", "rawMarkdown": "Hi, thank you very much for hosting this competition, this is very fun to do! \n\nI wondered if you could share any assumption about the jitter noise you model in these observations, in particular the expected rms or equivalent, is it the same accross all the observed transits etc. I would like to test some denoising models that require a bit of *a priori* knowledge on the noise sources in the data, and having more details on this would be great!", "votes": 6 }, { "id": 2983986, "postDate": "2024-09-09T09:24:35.157Z", "content": "

I find these \"bumps\", one per planet, at different times. Are these the \"jitter\" noise?
\n\"\"

", "rawMarkdown": "I find these \"bumps\", one per planet, at different times. Are these the \"jitter\" noise?\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F199197%2F9e945beb1f4df0d72a588ff4f1dd394c%2FBump.PNG?generation=1725873829656356&alt=media)", "votes": 2, "replies": [ { "id": 2984005, "postDate": "2024-09-09T09:50:43.947Z", "content": "

My data does not have these bumps. Maybe they are due to calibration error?

", "rawMarkdown": "My data does not have these bumps. Maybe they are due to calibration error?", "replies": [ { "id": 2984011, "postDate": "2024-09-09T09:55:03.993Z", "content": "

They could be an artifact of my data cleaning, but they are similar between planets, but at different times, so they look like something injected.
\nCalibrated, before \"cleaning\"
\n\"\"

", "rawMarkdown": "They could be an artifact of my data cleaning, but they are similar between planets, but at different times, so they look like something injected.\nCalibrated, before \"cleaning\"\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F199197%2F2902bd7a8dc66e9c329fec0ad276a6a0%2FCalibrated.PNG?generation=1725876082186160&alt=media)" } ] }, { "id": 2984216, "postDate": "2024-09-09T13:25:59.010Z", "content": "

Yep, this is the jitter blows up your timeline like bandits blowing up railways. They can come in different forms too, just like debris being shattered in different patterns and distances.

", "rawMarkdown": "Yep, this is the jitter blows up your timeline like bandits blowing up railways. They can come in different forms too, just like debris being shattered in different patterns and distances.", "votes": 4 } ] }, { "id": 2982339, "postDate": "2024-09-07T17:43:34.197Z", "content": "

I'm new here, but I imagine it shouldn't be available. After the competition starts. I started the challenge today, after they set the start and end date. Go until the end with the available resources.

", "rawMarkdown": "I'm new here, but I imagine it shouldn't be available. After the competition starts. I started the challenge today, after they set the start and end date. Go until the end with the available resources.", "replies": [ { "id": 2983069, "postDate": "2024-09-08T13:36:58.933Z", "content": "

The whole point of this competition is to predict the expected variance on the reconstructed spectrum, which is hard when this is not included in the training labels… Jitter noise comes in many flavours depending on if it is vibration, pointing issues or else, and a bit of details around this point would only help the hosts to get better solutions for this specific problem 👌

", "rawMarkdown": "The whole point of this competition is to predict the expected variance on the reconstructed spectrum, which is hard when this is not included in the training labels... Jitter noise comes in many flavours depending on if it is vibration, pointing issues or else, and a bit of details around this point would only help the hosts to get better solutions for this specific problem 👌" } ] }, { "id": 2982569, "postDate": "2024-09-08T00:34:37.007Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 2983874, "author_name": "Lorenzo Mugnai", "author_url": "", "post_date": "2024-09-09T07:31:23.677000", "content": "

Hello, that’s an excellent question!

\n

Pointing jitter on a space telescope arises from a variety of sources. To put it simply, it’s the result of a noninear interplay between solar wind, vibrations from the instruments and the efforts of the pointing system to stay locked onto the target. Therefore the noise’s power sprectrum is not flat but significantly non-Gaussian. Simulating this is quite challenging, as it largely depends on how the payload is constructed. If you're interested in exploring this topic from an academic perspective, there is some literature we can point you to. However, you’ll find that the impact and spectral characteristics of jitter varies significantly between space missions due to the unique design and components of each mission.

\n

For this exercise, we used realistic jitter models for Ariel space mission provided by the manufacturer for the specific observed targets in question.

", "votes": 5, "replies": [ { "id": 2985616, "author_name": "Yahalom", "author_url": "", "post_date": "2024-09-10T19:40:55.387000", "content": "

Lorenzo thanks for your detailed answer- can you please clarify if the (pointing) jitter is of the same origin in the 2 instruments in our data? From what I saw they seem to be independent- is this an effect of the used simulator or is this the expected behavior?

", "votes": 0, "replies": [ { "id": 2985943, "author_name": "Lorenzo Mugnai", "author_url": "", "post_date": "2024-09-11T07:09:55.537000", "content": "

Thanks for your question! The pointing jitter originates from the same source for both instruments, as they are observing the same target simultaneously. However, the apparent independence you've noticed could be due to differences between the two detectors. While the physical pointing jitter is identical, each detector has its own characteristics, such as PSF shape, pixel quantum efficiency, and integration time, which influence how the jitter manifests in the final data.

\n

So, the fact that the jitter seems different across the two instruments is not an effect of the simulator, but rather an expected behaviour due to the distinct properties of the detectors. The movement of the point source during integration is convolved with the different characteristics for each detector, leading to results that may appear independent.

", "votes": 1, "replies": [] } ] } ] }, { "id": 2983986, "author_name": "Tord Malmgren", "author_url": "", "post_date": "2024-09-09T09:24:35.157000", "content": "

I find these \"bumps\", one per planet, at different times. Are these the \"jitter\" noise?
\n\"\"

", "votes": 2, "replies": [ { "id": 2984005, "author_name": "ChingYinNg", "author_url": "", "post_date": "2024-09-09T09:50:43.947000", "content": "

My data does not have these bumps. Maybe they are due to calibration error?

", "votes": 0, "replies": [ { "id": 2984011, "author_name": "Tord Malmgren", "author_url": "", "post_date": "2024-09-09T09:55:03.993000", "content": "

They could be an artifact of my data cleaning, but they are similar between planets, but at different times, so they look like something injected.
\nCalibrated, before \"cleaning\"
\n\"\"

", "votes": 0, "replies": [] } ] }, { "id": 2984216, "author_name": "Gordon Yip", "author_url": "", "post_date": "2024-09-09T13:25:59.010000", "content": "

Yep, this is the jitter blows up your timeline like bandits blowing up railways. They can come in different forms too, just like debris being shattered in different patterns and distances.

", "votes": 4, "replies": [] } ] }, { "id": 2982339, "author_name": "sc0v0ne", "author_url": "", "post_date": "2024-09-07T17:43:34.197000", "content": "

I'm new here, but I imagine it shouldn't be available. After the competition starts. I started the challenge today, after they set the start and end date. Go until the end with the available resources.

", "votes": 0, "replies": [ { "id": 2983069, "author_name": "Simon Dupourque", "author_url": "", "post_date": "2024-09-08T13:36:58.933000", "content": "

The whole point of this competition is to predict the expected variance on the reconstructed spectrum, which is hard when this is not included in the training labels… Jitter noise comes in many flavours depending on if it is vibration, pointing issues or else, and a bit of details around this point would only help the hosts to get better solutions for this specific problem 👌

", "votes": 0, "replies": [] } ] }, { "id": 2982569, "author_name": "", "author_url": "", "post_date": "2024-09-08T00:34:37.007000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2983874": "Hello, that’s an excellent question!\n\nPointing jitter on a space telescope arises from a variety of sources. To put it simply, it’s the result of a noninear interplay between solar wind, vibrations from the instruments and the efforts of the pointing system to stay locked onto the target. Therefore the noise’s power sprectrum is not flat but significantly non-Gaussian. Simulating this is quite challenging, as it largely depends on how the payload is constructed. If you're interested in exploring this topic from an academic perspective, there is some literature we can point you to. However, you’ll find that the impact and spectral characteristics of jitter varies significantly between space missions due to the unique design and components of each mission.\n\nFor this exercise, we used realistic jitter models for Ariel space mission provided by the manufacturer for the specific observed targets in question.", "2981928": "Hi, thank you very much for hosting this competition, this is very fun to do! \n\nI wondered if you could share any assumption about the jitter noise you model in these observations, in particular the expected rms or equivalent, is it the same accross all the observed transits etc. I would like to test some denoising models that require a bit of *a priori* knowledge on the noise sources in the data, and having more details on this would be great!", "2983986": "I find these \"bumps\", one per planet, at different times. Are these the \"jitter\" noise?\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F199197%2F9e945beb1f4df0d72a588ff4f1dd394c%2FBump.PNG?generation=1725873829656356&alt=media)", "2982339": "I'm new here, but I imagine it shouldn't be available. After the competition starts. I started the challenge today, after they set the start and end date. Go until the end with the available resources.", "2982569": "" } }