\na couple of examples:
\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1944043%2F61d573aba0228bb735a623f31b04d406%2F1.jpg?generation=1726161919557282&alt=media\")
Hello, can someone explain the nature of the change in \"white curves\" over time outside the transit zone (like a trend)?
\na couple of examples:
\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1944043%2F5a1f04b09d2e0a55db0612b0f4431161%2F3.jpg?generation=1726163032622184&alt=media\")
(More information i'm using pre-processed data from input of host solution (data_train.npy))
", "messages": [ { "id": 2988832, "postDate": "2024-09-14T09:39:09.650Z", "content": "Hello,
\nThank you for the great question and for highlighting some significant examples!
\nThe trends observed in the light curve can be attributed to gain noise, a multiplicative effect on the signal. This noise may exhibit both temporal and chromatic correlations and can have various origins. Since this is a relative measurement, essentially anything that alters the measured flux can manifest as a multiplicative effect. Similar phenomena have been observed in various space missions, with amplitudes and patterns ranging from almost random to more structured. The JWST also experiences these effects, though generally with lower amplitudes compared to the simulations presented here.
\nThis type of noise can include both a random component, such as electronic fluctuations, and a part that can be modelled using instrumental models. A notable example is the HST's scanning mode, where such effects are identified and corrected through dedicated models.
", "rawMarkdown": "Hello, \n\nThank you for the great question and for highlighting some significant examples! \n\nThe trends observed in the light curve can be attributed to gain noise, a multiplicative effect on the signal. This noise may exhibit both temporal and chromatic correlations and can have various origins. Since this is a relative measurement, essentially anything that alters the measured flux can manifest as a multiplicative effect. Similar phenomena have been observed in various space missions, with amplitudes and patterns ranging from almost random to more structured. The JWST also experiences these effects, though generally with lower amplitudes compared to the simulations presented here.\n\nThis type of noise can include both a random component, such as electronic fluctuations, and a part that can be modelled using instrumental models. A notable example is the HST's scanning mode, where such effects are identified and corrected through dedicated models.", "votes": 7, "replies": [ { "id": 2988952, "postDate": "2024-09-14T12:15:56.433Z", "content": "Many thanks, Lorenzo.
", "rawMarkdown": "Many thanks, Lorenzo. " }, { "id": 2992174, "postDate": "2024-09-18T09:19:23.723Z", "content": "Do I understand it correctly that the star drifts across the prism, thus creating the \"trend\" and then Ariel slews back, thus creating the \"jitter\"?
\n ![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F199197%2F6bf0ed7223ec4101be2aaf4b3bb82ed0%2FPrism.png?generation=1726651060064292&alt=media\")
Thank you for your question! It seems there might be some confusion between the concepts of \"trend\" and \"jitter.\"
\nThe drift of the source across the focal plane can indeed cause a trend similar to what is observed in the light curve. This type of drift typically results from a gradual pointing shift in the telescope over several hours, leading to a slow change in the measured signal. When the star moves across the detector, it passes over different pixels (or parts of them), altering the quantum efficiency that converts light into electrons. This introduces a multiplicative effect, which indeed falls under the definition of \"gain noise\" or \"gain drift.\"
\nHowever, jitter is a different phenomenon. It is a high-frequency effect caused by vibrations in the telescope and the corrective actions of the pointing system as it tries to recenter the object. Jitter manifests as rapid, small-scale variations, unlike the slower \"trend\" caused by drift, as highlighted in these examples.
\nI hope this clarifies the distinction between the two!
", "rawMarkdown": "Thank you for your question! It seems there might be some confusion between the concepts of \"trend\" and \"jitter.\"\n\nThe drift of the source across the focal plane can indeed cause a trend similar to what is observed in the light curve. This type of drift typically results from a gradual pointing shift in the telescope over several hours, leading to a slow change in the measured signal. When the star moves across the detector, it passes over different pixels (or parts of them), altering the quantum efficiency that converts light into electrons. This introduces a multiplicative effect, which indeed falls under the definition of \"gain noise\" or \"gain drift.\"\n\nHowever, jitter is a different phenomenon. It is a high-frequency effect caused by vibrations in the telescope and the corrective actions of the pointing system as it tries to recenter the object. Jitter manifests as rapid, small-scale variations, unlike the slower \"trend\" caused by drift, as highlighted in these examples.\n\nI hope this clarifies the distinction between the two!", "votes": 4 }, { "id": 2997847, "postDate": "2024-09-25T00:58:48.183Z", "content": "@lorenzomugnai thanks - sorry for the beginner questions:
\n1) differences in quantum efficiency: wont these be corrected by the flat field corrections applied?
\n2) jitter vibrations: do these show up on the spectrogram in such a way that same wavelength lights are being resolved as different wavelengths? (perhaps due to the entering light rays not being normal to the prism?)
Thank you for your questions, and no need to apologise!
\n1 ) Regarding the differences in quantum efficiency:
\nYes, flat field corrections are applied to account for variations in quantum efficiency across the detector. However, these corrections are never perfect—in real-world scenarios, we can only achieve the best possible estimate. Additionally, the structure of each pixel plays a role. Specifically, the quantum response is not uniformly distributed across the pixel's surface; it typically decreases towards the edges. As a result, when the signal drifts by fractions of a pixel, the measurements vary depending on which areas of the pixel are more or less illuminated. It's important to note that this internal pixel effect is not corrected by the flat field; the flat field only accounts for relative variations in quantum efficiency between different pixels.
2) Regarding jitter vibrations:
\nYou're absolutely right. The \"blurring\" effect introduced by the PSF (Point Spread Function) due to these vibrations means that the collected light during an integration is spread over more pixels (or fractions of pixels) than if the source were stationary. In a spectrometer, while in the spatial direction this results in a broader PSF, in the spectral direction it means that when we extract the spectrum based on the wavelength solution, there’s contamination from neighbouring wavelengths in each spectral bin. The extent of this contamination depends on the amplitude of the jitter. Part of the challenge is decorrelating this effect, which is difficult to trace because the jitter isn’t necessarily the same in both directions.
", "rawMarkdown": "Thank you for your questions, and no need to apologise!\n\n1 ) Regarding the differences in quantum efficiency:\nYes, flat field corrections are applied to account for variations in quantum efficiency across the detector. However, these corrections are never perfect—in real-world scenarios, we can only achieve the best possible estimate. Additionally, the structure of each pixel plays a role. Specifically, the quantum response is not uniformly distributed across the pixel's surface; it typically decreases towards the edges. As a result, when the signal drifts by fractions of a pixel, the measurements vary depending on which areas of the pixel are more or less illuminated. It's important to note that this internal pixel effect is not corrected by the flat field; the flat field only accounts for relative variations in quantum efficiency between different pixels.\n\n2) Regarding jitter vibrations:\n\nYou're absolutely right. The \"blurring\" effect introduced by the PSF (Point Spread Function) due to these vibrations means that the collected light during an integration is spread over more pixels (or fractions of pixels) than if the source were stationary. In a spectrometer, while in the spatial direction this results in a broader PSF, in the spectral direction it means that when we extract the spectrum based on the wavelength solution, there’s contamination from neighbouring wavelengths in each spectral bin. The extent of this contamination depends on the amplitude of the jitter. Part of the challenge is decorrelating this effect, which is difficult to trace because the jitter isn’t necessarily the same in both directions.", "votes": 1 } ] } ] }, { "id": 2987474, "postDate": "2024-09-12T17:25:56.500Z", "content": "Hello, can someone explain the nature of the change in \"white curves\" over time outside the transit zone (like a trend)?
\na couple of examples:
\n
(More information i'm using pre-processed data from input of host solution (data_train.npy))
", "rawMarkdown": "Hello, can someone explain the nature of the change in \"white curves\" over time outside the transit zone (like a trend)?\na couple of examples:\n\n\n\n\n\n\n(More information i'm using pre-processed data from input of [host solution](https://www.kaggle.com/code/gordonyip/host-starter-solution) (data_train.npy))", "votes": 7 }, { "id": 2987964, "postDate": "2024-09-13T08:23:23.847Z", "rawMarkdown": "", "votes": -6, "replies": [ { "id": 2987978, "postDate": "2024-09-13T08:52:50.813Z", "content": "ChatGPT much?
", "rawMarkdown": "ChatGPT much?", "votes": 1 } ] }, { "id": 2988072, "postDate": "2024-09-13T11:01:17.073Z", "content": "I found that too. I had no idea of astronomy, and I thought it as clockwise or counterclockwise at that time. Don't laugh at me🥲
", "rawMarkdown": "I found that too. I had no idea of astronomy, and I thought it as clockwise or counterclockwise at that time. Don't laugh at me🥲" }, { "id": 2988018, "postDate": "2024-09-13T09:43:48.183Z", "content": "What are the planet ids ?
", "rawMarkdown": "What are the planet ids ?", "replies": [ { "id": 2988056, "postDate": "2024-09-13T10:36:25.747Z", "content": "There are quite a lot of planets in the data with this type of \"white curves\". We can look at the first planet in the training dataset with id 785834 like exmpl
", "rawMarkdown": "There are quite a lot of planets in the data with this type of \"white curves\". We can look at the first planet in the training dataset with id 785834 like exmpl", "replies": [ { "id": 2988196, "postDate": "2024-09-13T14:01:25.420Z", "content": "Just curious. I have only looked at 777145011 and 1420180584.
\n785834 was interesting.
\nWhatever, praxis is to model the background and divide it away.
\nLeft: orange = modelled background. Right : after dividing with background model.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F199197%2Ffecc4acf1966a7278ae82697b65332ff%2FBackground.PNG?generation=1726235994912588&alt=media\")
Perhaps they are variable stars? I will ask my astrophysics professor about this next week.
", "rawMarkdown": "Perhaps they are variable stars? I will ask my astrophysics professor about this next week." } ], "comments": [ { "id": 2988832, "author_name": "Lorenzo Mugnai", "author_url": "", "post_date": "2024-09-14T09:39:09.650000", "content": "Hello,
\nThank you for the great question and for highlighting some significant examples!
\nThe trends observed in the light curve can be attributed to gain noise, a multiplicative effect on the signal. This noise may exhibit both temporal and chromatic correlations and can have various origins. Since this is a relative measurement, essentially anything that alters the measured flux can manifest as a multiplicative effect. Similar phenomena have been observed in various space missions, with amplitudes and patterns ranging from almost random to more structured. The JWST also experiences these effects, though generally with lower amplitudes compared to the simulations presented here.
\nThis type of noise can include both a random component, such as electronic fluctuations, and a part that can be modelled using instrumental models. A notable example is the HST's scanning mode, where such effects are identified and corrected through dedicated models.
", "votes": 7, "replies": [ { "id": 2988952, "author_name": "Timmy Juicehouse", "author_url": "", "post_date": "2024-09-14T12:15:56.433000", "content": "Many thanks, Lorenzo.
", "votes": 0, "replies": [] }, { "id": 2992174, "author_name": "Tord Malmgren", "author_url": "", "post_date": "2024-09-18T09:19:23.723000", "content": "Do I understand it correctly that the star drifts across the prism, thus creating the \"trend\" and then Ariel slews back, thus creating the \"jitter\"?
\n
Thank you for your question! It seems there might be some confusion between the concepts of \"trend\" and \"jitter.\"
\nThe drift of the source across the focal plane can indeed cause a trend similar to what is observed in the light curve. This type of drift typically results from a gradual pointing shift in the telescope over several hours, leading to a slow change in the measured signal. When the star moves across the detector, it passes over different pixels (or parts of them), altering the quantum efficiency that converts light into electrons. This introduces a multiplicative effect, which indeed falls under the definition of \"gain noise\" or \"gain drift.\"
\nHowever, jitter is a different phenomenon. It is a high-frequency effect caused by vibrations in the telescope and the corrective actions of the pointing system as it tries to recenter the object. Jitter manifests as rapid, small-scale variations, unlike the slower \"trend\" caused by drift, as highlighted in these examples.
\nI hope this clarifies the distinction between the two!
", "votes": 4, "replies": [] }, { "id": 2997847, "author_name": "Heisenger", "author_url": "", "post_date": "2024-09-25T00:58:48.183000", "content": "@lorenzomugnai thanks - sorry for the beginner questions:
\n1) differences in quantum efficiency: wont these be corrected by the flat field corrections applied?
\n2) jitter vibrations: do these show up on the spectrogram in such a way that same wavelength lights are being resolved as different wavelengths? (perhaps due to the entering light rays not being normal to the prism?)
Thank you for your questions, and no need to apologise!
\n1 ) Regarding the differences in quantum efficiency:
\nYes, flat field corrections are applied to account for variations in quantum efficiency across the detector. However, these corrections are never perfect—in real-world scenarios, we can only achieve the best possible estimate. Additionally, the structure of each pixel plays a role. Specifically, the quantum response is not uniformly distributed across the pixel's surface; it typically decreases towards the edges. As a result, when the signal drifts by fractions of a pixel, the measurements vary depending on which areas of the pixel are more or less illuminated. It's important to note that this internal pixel effect is not corrected by the flat field; the flat field only accounts for relative variations in quantum efficiency between different pixels.
2) Regarding jitter vibrations:
\nYou're absolutely right. The \"blurring\" effect introduced by the PSF (Point Spread Function) due to these vibrations means that the collected light during an integration is spread over more pixels (or fractions of pixels) than if the source were stationary. In a spectrometer, while in the spatial direction this results in a broader PSF, in the spectral direction it means that when we extract the spectrum based on the wavelength solution, there’s contamination from neighbouring wavelengths in each spectral bin. The extent of this contamination depends on the amplitude of the jitter. Part of the challenge is decorrelating this effect, which is difficult to trace because the jitter isn’t necessarily the same in both directions.
", "votes": 1, "replies": [] } ] } ] }, { "id": 2987964, "author_name": "", "author_url": "", "post_date": "2024-09-13T08:23:23.847000", "content": "", "votes": -6, "replies": [ { "id": 2987978, "author_name": "DennisSakva", "author_url": "", "post_date": "2024-09-13T08:52:50.813000", "content": "ChatGPT much?
", "votes": 1, "replies": [] } ] }, { "id": 2988072, "author_name": "Timmy Juicehouse", "author_url": "", "post_date": "2024-09-13T11:01:17.073000", "content": "I found that too. I had no idea of astronomy, and I thought it as clockwise or counterclockwise at that time. Don't laugh at me🥲
", "votes": 0, "replies": [] }, { "id": 2988018, "author_name": "Tord Malmgren", "author_url": "", "post_date": "2024-09-13T09:43:48.183000", "content": "What are the planet ids ?
", "votes": 0, "replies": [ { "id": 2988056, "author_name": "Taksants Maksim", "author_url": "", "post_date": "2024-09-13T10:36:25.747000", "content": "There are quite a lot of planets in the data with this type of \"white curves\". We can look at the first planet in the training dataset with id 785834 like exmpl
", "votes": 0, "replies": [ { "id": 2988196, "author_name": "Tord Malmgren", "author_url": "", "post_date": "2024-09-13T14:01:25.420000", "content": "Just curious. I have only looked at 777145011 and 1420180584.
\n785834 was interesting.
\nWhatever, praxis is to model the background and divide it away.
\nLeft: orange = modelled background. Right : after dividing with background model.
\n
Perhaps they are variable stars? I will ask my astrophysics professor about this next week.
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2988832": "Hello, \n\nThank you for the great question and for highlighting some significant examples! \n\nThe trends observed in the light curve can be attributed to gain noise, a multiplicative effect on the signal. This noise may exhibit both temporal and chromatic correlations and can have various origins. Since this is a relative measurement, essentially anything that alters the measured flux can manifest as a multiplicative effect. Similar phenomena have been observed in various space missions, with amplitudes and patterns ranging from almost random to more structured. The JWST also experiences these effects, though generally with lower amplitudes compared to the simulations presented here.\n\nThis type of noise can include both a random component, such as electronic fluctuations, and a part that can be modelled using instrumental models. A notable example is the HST's scanning mode, where such effects are identified and corrected through dedicated models.", "2987474": "Hello, can someone explain the nature of the change in \"white curves\" over time outside the transit zone (like a trend)?\na couple of examples:\n\n\n\n\n\n\n(More information i'm using pre-processed data from input of [host solution](https://www.kaggle.com/code/gordonyip/host-starter-solution) (data_train.npy))", "2987964": "", "2988072": "I found that too. I had no idea of astronomy, and I thought it as clockwise or counterclockwise at that time. Don't laugh at me🥲", "2988018": "What are the planet ids ?", "2987976": "Perhaps they are variable stars? I will ask my astrophysics professor about this next week." } }