![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3365741%2F4b0ff55f9390b5a8a283b5ea587d30c6%2F505.png?generation=1730313397330938&alt=media\")
Hi all!
\nI wonder whether somebody experienced the same problem. I decided to fit several polynomials in order to get several estimations of s (I remove the first 4 points from both sides of the mean signal with shape (187,) to get s_1, 8 points from both sides to get s_2, etc.). In this way my MSE equals to ~3.7e-9 (between the predicted mean and the true mean, so we have 673 points here to compute this MSE). However, this submission scores 0 on the public LB (even with the constant sigma 1.6e-4).
\nI took my submission which scores ~0.500 on the public LB, computed MSE between true and predicted mean planet size, and got ~8e-9 for MSE.
\nI was quite surprised to get this result, and have no idea why it happens. Has anyone faced the same issue?
\nThanks in advance, and good luck everyone with the private LB :)
", "messages": [ { "id": 3032219, "postDate": "2024-10-30T17:21:42.063Z", "content": "Hi all!
\nI wonder whether somebody experienced the same problem. I decided to fit several polynomials in order to get several estimations of s (I remove the first 4 points from both sides of the mean signal with shape (187,) to get s_1, 8 points from both sides to get s_2, etc.). In this way my MSE equals to ~3.7e-9 (between the predicted mean and the true mean, so we have 673 points here to compute this MSE). However, this submission scores 0 on the public LB (even with the constant sigma 1.6e-4).
\nI took my submission which scores ~0.500 on the public LB, computed MSE between true and predicted mean planet size, and got ~8e-9 for MSE.
\nI was quite surprised to get this result, and have no idea why it happens. Has anyone faced the same issue?
\nThanks in advance, and good luck everyone with the private LB :)
", "rawMarkdown": "Hi all!\n\nI wonder whether somebody experienced the same problem. I decided to fit several polynomials in order to get several estimations of **s** (I remove the first 4 points from both sides of the mean signal with shape (187,) to get s_1, 8 points from both sides to get s_2, etc.). In this way my MSE equals to ~3.7e-9 (between the predicted mean and the true mean, so we have 673 points here to compute this MSE). However, this submission scores 0 on the public LB (even with the constant sigma 1.6e-4).\n\nI took my submission which scores ~0.500 on the public LB, computed MSE between true and predicted mean planet size, and got ~8e-9 for MSE.\n\nI was quite surprised to get this result, and have no idea why it happens. Has anyone faced the same issue?\n\nThanks in advance, and good luck everyone with the private LB :)", "votes": 3 }, { "id": 3032276, "postDate": "2024-10-30T18:26:29.610Z", "content": "It's an interesting idea. But the less points you use to calculate mean signal, the more noisy it is? In my experiments it was the problem.
", "rawMarkdown": "It's an interesting idea. But the less points you use to calculate mean signal, the more noisy it is? In my experiments it was the problem.", "replies": [ { "id": 3032282, "postDate": "2024-10-30T18:37:40.397Z", "content": "Technically, yes. But I use not less than 140 points out of 187. Moreover, it can help to tackle noisy sides of your signal like in the following example:
\n
I see. Well, 140 points should be enough for a good mean signal. You still predict the mean s, right?
", "rawMarkdown": "I see. Well, 140 points should be enough for a good mean signal. You still predict the mean s, right?" }, { "id": 3032285, "postDate": "2024-10-30T18:43:24.423Z", "content": "Yes, I predict several s values for different margins (planet_model.s_), then I compute the mean value across these values, and use it as a prediction for all wavelengths for this planet (with sigma=1.6e-4). In this particular example the more points you cut from the sides, the more accurate estimate of s you get.
", "rawMarkdown": "Yes, I predict several **s** values for different margins (*planet_model.s_*), then I compute the mean value across these values, and use it as a prediction for all wavelengths for this planet (with sigma=1.6e-4). In this particular example the more points you cut from the sides, the more accurate estimate of **s** you get.", "votes": 1 }, { "id": 3032287, "postDate": "2024-10-30T18:47:19.867Z", "content": "I'm also still on predicting mean_s. I can't get good fluctuation model. I tried to use Ridge to predict fluctuation, but results are worse or the same compared to using only mean_s. Also I'm getting inconsistent results when using savgol_filter - it seems you need to choose a window from a quite narrow range to get it to improve your result.
", "rawMarkdown": "I'm also still on predicting mean_s. I can't get good fluctuation model. I tried to use Ridge to predict fluctuation, but results are worse or the same compared to using only mean_s. Also I'm getting inconsistent results when using savgol_filter - it seems you need to choose a window from a quite narrow range to get it to improve your result.", "votes": 1 }, { "id": 3032291, "postDate": "2024-10-30T18:51:06.420Z", "content": "What do you mean by inconsistent results using savgol_filter?
\nIt is about finding breakpoints in a signal?
Oh, and I think I know why your method improves the result. It's probably because of presence of the trend in our signals. You reduce it by taking out these points. I tried to detrend signals before estimating mean_s, and they looks quite nice, but estimation of mean_s becomes less accurate.
", "rawMarkdown": "Oh, and I think I know why your method improves the result. It's probably because of presence of the trend in our signals. You reduce it by taking out these points. I tried to detrend signals before estimating mean_s, and they looks quite nice, but estimation of mean_s becomes less accurate." }, { "id": 3032293, "postDate": "2024-10-30T18:52:57.190Z", "content": "It helps very much with breakpoints, but can hurt during mean_s estimation. One of the problems in this competition is choosing a method to denoise the signal in order to increase accuracy of s estimation.
", "rawMarkdown": "It helps very much with breakpoints, but can hurt during mean_s estimation. One of the problems in this competition is choosing a method to denoise the signal in order to increase accuracy of s estimation." }, { "id": 3032688, "postDate": "2024-10-31T08:25:32.017Z", "content": "If you don't mind can I know how you get the pred value with the fluctuations and is the way you get the s is（pred-in/in）?
", "rawMarkdown": "If you don't mind can I know how you get the pred value with the fluctuations and is the way you get the s is（pred-in/in）?" }, { "id": 3032734, "postDate": "2024-10-31T09:53:48.927Z", "content": "I used Ridge to predict fluctuations s - mean(s). As for the s - I used (pred-in)/in.
", "rawMarkdown": "I used Ridge to predict fluctuations s - mean(s). As for the s - I used (pred-in)/in." } ] } ] } ], "comments": [ { "id": 3032276, "author_name": "Araik Tamazian", "author_url": "", "post_date": "2024-10-30T18:26:29.610000", "content": "It's an interesting idea. But the less points you use to calculate mean signal, the more noisy it is? In my experiments it was the problem.
", "votes": 0, "replies": [ { "id": 3032282, "author_name": "gromml", "author_url": "", "post_date": "2024-10-30T18:37:40.397000", "content": "Technically, yes. But I use not less than 140 points out of 187. Moreover, it can help to tackle noisy sides of your signal like in the following example:
\n
I see. Well, 140 points should be enough for a good mean signal. You still predict the mean s, right?
", "votes": 0, "replies": [] }, { "id": 3032285, "author_name": "gromml", "author_url": "", "post_date": "2024-10-30T18:43:24.423000", "content": "Yes, I predict several s values for different margins (planet_model.s_), then I compute the mean value across these values, and use it as a prediction for all wavelengths for this planet (with sigma=1.6e-4). In this particular example the more points you cut from the sides, the more accurate estimate of s you get.
", "votes": 1, "replies": [] }, { "id": 3032287, "author_name": "Araik Tamazian", "author_url": "", "post_date": "2024-10-30T18:47:19.867000", "content": "I'm also still on predicting mean_s. I can't get good fluctuation model. I tried to use Ridge to predict fluctuation, but results are worse or the same compared to using only mean_s. Also I'm getting inconsistent results when using savgol_filter - it seems you need to choose a window from a quite narrow range to get it to improve your result.
", "votes": 1, "replies": [] }, { "id": 3032291, "author_name": "gromml", "author_url": "", "post_date": "2024-10-30T18:51:06.420000", "content": "What do you mean by inconsistent results using savgol_filter?
\nIt is about finding breakpoints in a signal?
Oh, and I think I know why your method improves the result. It's probably because of presence of the trend in our signals. You reduce it by taking out these points. I tried to detrend signals before estimating mean_s, and they looks quite nice, but estimation of mean_s becomes less accurate.
", "votes": 0, "replies": [] }, { "id": 3032293, "author_name": "Araik Tamazian", "author_url": "", "post_date": "2024-10-30T18:52:57.190000", "content": "It helps very much with breakpoints, but can hurt during mean_s estimation. One of the problems in this competition is choosing a method to denoise the signal in order to increase accuracy of s estimation.
", "votes": 0, "replies": [] }, { "id": 3032688, "author_name": "lllleeeo", "author_url": "", "post_date": "2024-10-31T08:25:32.017000", "content": "If you don't mind can I know how you get the pred value with the fluctuations and is the way you get the s is（pred-in/in）?
", "votes": 0, "replies": [] }, { "id": 3032734, "author_name": "Araik Tamazian", "author_url": "", "post_date": "2024-10-31T09:53:48.927000", "content": "I used Ridge to predict fluctuations s - mean(s). As for the s - I used (pred-in)/in.
", "votes": 0, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "3032219": "Hi all!\n\nI wonder whether somebody experienced the same problem. I decided to fit several polynomials in order to get several estimations of **s** (I remove the first 4 points from both sides of the mean signal with shape (187,) to get s_1, 8 points from both sides to get s_2, etc.). In this way my MSE equals to ~3.7e-9 (between the predicted mean and the true mean, so we have 673 points here to compute this MSE). However, this submission scores 0 on the public LB (even with the constant sigma 1.6e-4).\n\nI took my submission which scores ~0.500 on the public LB, computed MSE between true and predicted mean planet size, and got ~8e-9 for MSE.\n\nI was quite surprised to get this result, and have no idea why it happens. Has anyone faced the same issue?\n\nThanks in advance, and good luck everyone with the private LB :)", "3032276": "It's an interesting idea. But the less points you use to calculate mean signal, the more noisy it is? In my experiments it was the problem." } }