I notice there is a big range is (public LB) gold medal scores. First place first gold (public LB) is currently GLL = 0.630 while 11th place last gold (public LB) is GLL = 0.538. In many Kaggle competitions, the top (public LB) gold position is only +0.001 better than the bottom gold position.
Does this large (public LB) difference indicate that there is a magic feature in this competition? What does everyone guess causes this +0.100 difference between top (public LB) gold and bottom (public LB) gold?
Also how much shakeup do we predict? Will private LB be more than +0.100 different than public LB? or did the top public LB teams discover something that will not help on private LB?
I notice there is a big range is (public LB) gold medal scores. First place first gold (public LB) is currently GLL = 0.630 while 11th place last gold (public LB) is GLL = 0.538. In many Kaggle competitions, the top (public LB) gold position is only +0.001 better than the bottom gold position.
Does this large (public LB) difference indicate that there is a magic feature in this competition? What does everyone guess causes this +0.100 difference between top (public LB) gold and bottom (public LB) gold?
Also how much shakeup do we predict? Will private LB be more than +0.100 different than public LB? or did the top public LB teams discover something that will not help on private LB?
The LB score is clipped between reference and ideal scores, which inflates differences in scores compared regular log-likelihood.
\nAlthough there is 263.4 GB of training data, there are only 1100 planets for which 283 predictions need to be made. Moreover, the base level for a particular planet is much more important for the score than the spectrum, so in reality we are asked to make 1100 important predictions. That means that we face the challenges associated with working with a small dataset, e.g., handling of outliers becomes very important.
", "rawMarkdown": "The LB score is clipped between reference and ideal scores, which inflates differences in scores compared regular log-likelihood.\n\nAlthough there is 263.4 GB of training data, there are only 1100 planets for which 283 predictions need to be made. Moreover, the base level for a particular planet is much more important for the score than the spectrum, so in reality we are asked to make 1100 important predictions. That means that we face the challenges associated with working with a small dataset, e.g., handling of outliers becomes very important.", "votes": 5 }, { "id": 3289498, "postDate": "2025-09-16T06:16:53.237Z", "content": "Dealing with incomplete/malformed transits would unfortunately (as it doesn't further the scientific goals) be a factor.
", "rawMarkdown": "Dealing with incomplete/malformed transits would unfortunately (as it doesn't further the scientific goals) be a factor.", "votes": 3, "replies": [ { "id": 3289692, "postDate": "2025-09-16T11:24:39.063Z", "content": "Yes, I think I'll have to bite that bullet and look into these. I didn't so far as it is a total waste of our time given the purpose of the competition.
", "rawMarkdown": "Yes, I think I'll have to bite that bullet and look into these. I didn't so far as it is a total waste of our time given the purpose of the competition.", "votes": 2 } ] }, { "id": 3289382, "postDate": "2025-09-16T01:24:27.530Z", "content": "If you're curious, last year's competition was very similar in terms of objective and score (albeit with slight additions that make the problem much much harder).
", "rawMarkdown": "1. Almost no shakeup. The only shakeup that will occur is between people with similar scores.\n2. This is much more complex than the usual kaggle machine learning competition (and this is more model building rather than feature selection). So people's solutions are quite different and a lot more opportunity for differentiation between competitors.\n3. The metric is quite sensitive, having a slightly better model can result in quite large gains in score. Even some leaderboard probing can lead to significant gains.\n\n\nIf you're curious, last year's competition was very similar in terms of objective and score (albeit with slight additions that make the problem much much harder).", "votes": 3 } ], "comments": [ { "id": 3289554, "author_name": "Thomas Dueholm Hansen", "author_url": "", "post_date": "2025-09-16T08:19:57.120000", "content": "The LB score is clipped between reference and ideal scores, which inflates differences in scores compared regular log-likelihood.
\nAlthough there is 263.4 GB of training data, there are only 1100 planets for which 283 predictions need to be made. Moreover, the base level for a particular planet is much more important for the score than the spectrum, so in reality we are asked to make 1100 important predictions. That means that we face the challenges associated with working with a small dataset, e.g., handling of outliers becomes very important.
", "votes": 5, "replies": [] }, { "id": 3289498, "author_name": "sroger", "author_url": "", "post_date": "2025-09-16T06:16:53.237000", "content": "Dealing with incomplete/malformed transits would unfortunately (as it doesn't further the scientific goals) be a factor.
", "votes": 3, "replies": [ { "id": 3289692, "author_name": "CPMP", "author_url": "", "post_date": "2025-09-16T11:24:39.063000", "content": "Yes, I think I'll have to bite that bullet and look into these. I didn't so far as it is a total waste of our time given the purpose of the competition.
", "votes": 2, "replies": [] } ] }, { "id": 3289382, "author_name": "JungleBeastDS", "author_url": "", "post_date": "2025-09-16T01:24:27.530000", "content": "If you're curious, last year's competition was very similar in terms of objective and score (albeit with slight additions that make the problem much much harder).
", "votes": 3, "replies": [] } ], "raw_markdown_by_id": { "3289342": "I notice there is a big range is (public LB) gold medal scores. First place first gold (public LB) is currently `GLL = 0.630` while 11th place last gold (public LB) is `GLL = 0.538`. In many Kaggle competitions, the top (public LB) gold position is only `+0.001` better than the bottom gold position.\n\nDoes this large (public LB) difference indicate that there is a magic feature in this competition? What does everyone guess causes this `+0.100` difference between top (public LB) gold and bottom (public LB) gold?\n\nAlso how much shakeup do we predict? Will private LB be more than `+0.100` different than public LB? or did the top public LB teams discover something that will not help on private LB?", "3289554": "The LB score is clipped between reference and ideal scores, which inflates differences in scores compared regular log-likelihood.\n\nAlthough there is 263.4 GB of training data, there are only 1100 planets for which 283 predictions need to be made. Moreover, the base level for a particular planet is much more important for the score than the spectrum, so in reality we are asked to make 1100 important predictions. That means that we face the challenges associated with working with a small dataset, e.g., handling of outliers becomes very important.", "3289498": "Dealing with incomplete/malformed transits would unfortunately (as it doesn't further the scientific goals) be a factor.", "3289382": "1. Almost no shakeup. The only shakeup that will occur is between people with similar scores.\n2. This is much more complex than the usual kaggle machine learning competition (and this is more model building rather than feature selection). So people's solutions are quite different and a lot more opportunity for differentiation between competitors.\n3. The metric is quite sensitive, having a slightly better model can result in quite large gains in score. Even some leaderboard probing can lead to significant gains.\n\n\nIf you're curious, last year's competition was very similar in terms of objective and score (albeit with slight additions that make the problem much much harder)." } }