This competition has a pretty unusual shakeup pattern. The first 10 or so places have a rather minimal shakeup (cudos to them and their validation skills), while after that you see pretty wild changes in public/private LB scores and places. Like +70-100 places are not that rare.
\nAnd this guy \"269 Jungwoo Park\" must be some kind of Master Yoda of self-control and believing in himself to pick submission that was super low on Public LB but propelled him to almost gold in the end.
\nDuring the competition, I was pretty sure that the shakeup will be small due to randomly sampled public/private dataset split. But this assumption turned out wrong. So, what do you think people were overfitting to?
In my case, I took a risky assumption. I believed that the uncertainty could be quantified using Isaac Horowitz's Quatitative Feedback Theory (https://en.wikipedia.org/wiki/Quantitative_feedback_theory). I tried to bring this theory in practice here with the detectors like if they will be an \"industrial plant\". But I failed in my premise and I fell in a great Bias. I had to try it. I'm so sorry for my mistakes, but from mistakes we learn wise lessons too.
", "rawMarkdown": "In my case, I took a risky assumption. I believed that the uncertainty could be quantified using Isaac Horowitz's Quatitative Feedback Theory (https://en.wikipedia.org/wiki/Quantitative_feedback_theory). I tried to bring this theory in practice here with the detectors like if they will be an \"industrial plant\". But I failed in my premise and I fell in a great Bias. I had to try it. I'm so sorry for my mistakes, but from mistakes we learn wise lessons too.", "votes": 3 }, { "id": 2085423, "postDate": "2023-01-04T06:51:03.930Z", "content": "This competition has a pretty unusual shakeup pattern. The first 10 or so places have a rather minimal shakeup (cudos to them and their validation skills), while after that you see pretty wild changes in public/private LB scores and places. Like +70-100 places are not that rare.
\nAnd this guy \"269 Jungwoo Park\" must be some kind of Master Yoda of self-control and believing in himself to pick submission that was super low on Public LB but propelled him to almost gold in the end.
\nDuring the competition, I was pretty sure that the shakeup will be small due to randomly sampled public/private dataset split. But this assumption turned out wrong. So, what do you think people were overfitting to?
The results below were quite densely packed mostly due available high-scoring public notebooks. And a good split between private/public test dataset makes it less overfitting.
", "rawMarkdown": "The results below were quite densely packed mostly due available high-scoring public notebooks. And a good split between private/public test dataset makes it less overfitting. ", "votes": 1 }, { "id": 2086456, "postDate": "2023-01-04T19:33:34.053Z", "content": "But there's always something bigger, better, totally unhinged
\nhttps://www.kaggle.com/competitions/novozymes-enzyme-stability-prediction/leaderboard
yeah, that was a little out of control. These things happen tho, check out some history https://www.kaggle.com/code/jtrotman/meta-kaggle-scatter-plot-competition-shake-up/notebook
", "rawMarkdown": "yeah, that was a little out of control. These things happen tho, check out some history https://www.kaggle.com/code/jtrotman/meta-kaggle-scatter-plot-competition-shake-up/notebook" } ] }, { "id": 2086387, "postDate": "2023-01-04T18:32:17.367Z", "content": "I think that such shake-up pattern was caused by the score distribution in the public LB. There results in gold/silver zone were quite spread out. The results below were quite densely packed, probably mostly due available high-scoring public notebooks.
\nThe other factor was a good split between private/public test dataset, so there was less overfitting.
", "rawMarkdown": "I think that such shake-up pattern was caused by the score distribution in the public LB. There results in gold/silver zone were quite spread out. The results below were quite densely packed, probably mostly due available high-scoring public notebooks. \n\nThe other factor was a good split between private/public test dataset, so there was less overfitting. ", "votes": 1 }, { "id": 2085465, "postDate": "2023-01-04T07:16:47.607Z", "content": "Haha, thank you 😄😄 I am writing up my solution, but there's nothing special. I believed there should be shake-ups because of the noisy data and a small portion of the public set, but the miracles only happened to me 🤔🤔
", "rawMarkdown": "Haha, thank you 😄😄 I am writing up my solution, but there's nothing special. I believed there should be shake-ups because of the noisy data and a small portion of the public set, but the miracles only happened to me 🤔🤔", "votes": 1, "replies": [ { "id": 2085488, "postDate": "2023-01-04T07:36:53.857Z", "content": "Congrats! Anyway, I was expecting more regular shakup. Like +/- some random number of places roughly randomly distributed. But in this competition, participants have pretty impressive gains, especially in the silver-bronze medal zone. Something like 80-90% of those who received medals were up by many tens or even a few hundred places.
", "rawMarkdown": "Congrats! Anyway, I was expecting more regular shakup. Like +/- some random number of places roughly randomly distributed. But in this competition, participants have pretty impressive gains, especially in the silver-bronze medal zone. Something like 80-90% of those who received medals were up by many tens or even a few hundred places. ", "replies": [ { "id": 2085491, "postDate": "2023-01-04T07:38:29.700Z", "content": "That's right. Very lucky 😮😮
", "rawMarkdown": "That's right. Very lucky 😮😮", "votes": 1 } ] } ] } ], "comments": [ { "id": 2085435, "author_name": "Zollkron", "author_url": "", "post_date": "2023-01-04T06:59:11.163000", "content": "In my case, I took a risky assumption. I believed that the uncertainty could be quantified using Isaac Horowitz's Quatitative Feedback Theory (https://en.wikipedia.org/wiki/Quantitative_feedback_theory). I tried to bring this theory in practice here with the detectors like if they will be an \"industrial plant\". But I failed in my premise and I fell in a great Bias. I had to try it. I'm so sorry for my mistakes, but from mistakes we learn wise lessons too.
", "votes": 3, "replies": [] }, { "id": 2087487, "author_name": "BarryZhou", "author_url": "", "post_date": "2023-01-05T16:14:05.880000", "content": "The results below were quite densely packed mostly due available high-scoring public notebooks. And a good split between private/public test dataset makes it less overfitting.
", "votes": 1, "replies": [] }, { "id": 2086456, "author_name": "DennisSakva", "author_url": "", "post_date": "2023-01-04T19:33:34.053000", "content": "But there's always something bigger, better, totally unhinged
\nhttps://www.kaggle.com/competitions/novozymes-enzyme-stability-prediction/leaderboard
yeah, that was a little out of control. These things happen tho, check out some history https://www.kaggle.com/code/jtrotman/meta-kaggle-scatter-plot-competition-shake-up/notebook
", "votes": 0, "replies": [] } ] }, { "id": 2086387, "author_name": "Alex Z", "author_url": "", "post_date": "2023-01-04T18:32:17.367000", "content": "I think that such shake-up pattern was caused by the score distribution in the public LB. There results in gold/silver zone were quite spread out. The results below were quite densely packed, probably mostly due available high-scoring public notebooks.
\nThe other factor was a good split between private/public test dataset, so there was less overfitting.
", "votes": 1, "replies": [] }, { "id": 2085465, "author_name": "Jungwoo Park", "author_url": "", "post_date": "2023-01-04T07:16:47.607000", "content": "Haha, thank you 😄😄 I am writing up my solution, but there's nothing special. I believed there should be shake-ups because of the noisy data and a small portion of the public set, but the miracles only happened to me 🤔🤔
", "votes": 1, "replies": [ { "id": 2085488, "author_name": "DennisSakva", "author_url": "", "post_date": "2023-01-04T07:36:53.857000", "content": "Congrats! Anyway, I was expecting more regular shakup. Like +/- some random number of places roughly randomly distributed. But in this competition, participants have pretty impressive gains, especially in the silver-bronze medal zone. Something like 80-90% of those who received medals were up by many tens or even a few hundred places.
", "votes": 0, "replies": [ { "id": 2085491, "author_name": "Jungwoo Park", "author_url": "", "post_date": "2023-01-04T07:38:29.700000", "content": "That's right. Very lucky 😮😮
", "votes": 1, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2085435": "In my case, I took a risky assumption. I believed that the uncertainty could be quantified using Isaac Horowitz's Quatitative Feedback Theory (https://en.wikipedia.org/wiki/Quantitative_feedback_theory). I tried to bring this theory in practice here with the detectors like if they will be an \"industrial plant\". But I failed in my premise and I fell in a great Bias. I had to try it. I'm so sorry for my mistakes, but from mistakes we learn wise lessons too.", "2085423": "This competition has a pretty unusual shakeup pattern. The first 10 or so places have a rather minimal shakeup (cudos to them and their validation skills), while after that you see pretty wild changes in public/private LB scores and places. Like +70-100 places are not that rare.\nAnd this guy \"269 Jungwoo Park\" must be some kind of Master Yoda of self-control and believing in himself to pick submission that was super low on Public LB but propelled him to almost gold in the end.\nDuring the competition, I was pretty sure that the shakeup will be small due to randomly sampled public/private dataset split. But this assumption turned out wrong. So, what do you think people were overfitting to?", "2087487": "The results below were quite densely packed mostly due available high-scoring public notebooks. And a good split between private/public test dataset makes it less overfitting. ", "2086456": "But there's always something bigger, better, totally unhinged\nhttps://www.kaggle.com/competitions/novozymes-enzyme-stability-prediction/leaderboard", "2086387": "I think that such shake-up pattern was caused by the score distribution in the public LB. There results in gold/silver zone were quite spread out. The results below were quite densely packed, probably mostly due available high-scoring public notebooks. \n\nThe other factor was a good split between private/public test dataset, so there was less overfitting. ", "2085465": "Haha, thank you 😄😄 I am writing up my solution, but there's nothing special. I believed there should be shake-ups because of the noisy data and a small portion of the public set, but the miracles only happened to me 🤔🤔" } }