![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6735218%2F0120c68e49246b20b664029a2307a233%2Frank_shakeup.png?generation=1730455583010375&alt=media\")
I have done this a few times in the past and thought it would be interesting to see for this competition as well, so I created some plots to inspect the shakeup that happened from public LB -> private LB.
\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6735218%2Fe7cd74de397160885b5436fde63dd906%2Fscore_shakeup.png?generation=1730455590258415&alt=media\")
Overall we had a quite small shakeup in the medal zones, with some few outliers. I saw a few top teams got shaken down really bad, which is of course a very frustrating experience. I suspect that this competition can create such instances, due to edge-cases in preprocessing (e.g. wrong ingress egress detection for single sample?), which can predicting on new data very scary.
\nHere is the magnified rank shakeup for the top 200 places:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6735218%2F9e7e1d0af70cbffb6c17cf5f94cabb36%2Frank_shakeup_200.png?generation=1730455822694315&alt=media\")
There were not many big shake \"ups\" in this competition, the largest one I saw was by Maximilian Schnarr:
\nRank: 749 -> 106
\nScore: 0.541 -> 0.577
In this notebook you can find the interactive plots and some more stats:
\nhttps://www.kaggle.com/code/fritzcremer/ariel-shakeup
I have done this a few times in the past and thought it would be interesting to see for this competition as well, so I created some plots to inspect the shakeup that happened from public LB -> private LB.
\n
Overall we had a quite small shakeup in the medal zones, with some few outliers. I saw a few top teams got shaken down really bad, which is of course a very frustrating experience. I suspect that this competition can create such instances, due to edge-cases in preprocessing (e.g. wrong ingress egress detection for single sample?), which can predicting on new data very scary.
\nHere is the magnified rank shakeup for the top 200 places:
\n
There were not many big shake \"ups\" in this competition, the largest one I saw was by Maximilian Schnarr:
\nRank: 749 -> 106
\nScore: 0.541 -> 0.577
In this notebook you can find the interactive plots and some more stats:
\nhttps://www.kaggle.com/code/fritzcremer/ariel-shakeup
Nice visualizations. Emotionally it did feel like a big shakeup to me…though I was more surprised by how our winners seem to have tweaked their final model on the test set over the course of a single day. Really curious to see how they approached this. (And for those waiting for my solution: it should arrive later today).
", "rawMarkdown": "Nice visualizations. Emotionally it did feel like a big shakeup to me...though I was more surprised by how our winners seem to have tweaked their final model on the test set over the course of a single day. Really curious to see how they approached this. (And for those waiting for my solution: it should arrive later today).", "votes": 3, "replies": [ { "id": 3033651, "postDate": "2024-11-01T12:00:56.683Z", "content": "I'm very sorry for you. Just imagine what would have happened if they had started solving the competition not yesterday, but two days before…
", "rawMarkdown": "I'm very sorry for you. Just imagine what would have happened if they had started solving the competition not yesterday, but two days before... \n", "replies": [ { "id": 3033954, "postDate": "2024-11-01T16:41:28.667Z", "content": "Now that all decimals are shown, I can also see that I have 3 submissions that would have taken first place… (none of them were serious candidates for selection though)
", "rawMarkdown": "Now that all decimals are shown, I can also see that I have 3 submissions that would have taken first place... (none of them were serious candidates for selection though)" } ] }, { "id": 3033716, "postDate": "2024-11-01T12:54:52.190Z", "content": "we have also noticed that a few teams managed to break through the 0.7 barrier in the last few days, which is incredible.
", "rawMarkdown": "we have also noticed that a few teams managed to break through the 0.7 barrier in the last few days, which is incredible. ", "replies": [ { "id": 3033724, "postDate": "2024-11-01T13:03:05.070Z", "content": "This is a common story for competitions. First, they develop several approaches one at a time, and in the end they combine them in some way to get a strong breakthrough. in fact, people are training to make the final leap.
", "rawMarkdown": "This is a common story for competitions. First, they develop several approaches one at a time, and in the end they combine them in some way to get a strong breakthrough. in fact, people are training to make the final leap.", "votes": 1 } ] } ] } ], "comments": [ { "id": 3033644, "author_name": "Jeroen Cottaar", "author_url": "", "post_date": "2024-11-01T11:43:30.363000", "content": "Nice visualizations. Emotionally it did feel like a big shakeup to me…though I was more surprised by how our winners seem to have tweaked their final model on the test set over the course of a single day. Really curious to see how they approached this. (And for those waiting for my solution: it should arrive later today).
", "votes": 3, "replies": [ { "id": 3033651, "author_name": "Sergei Fironov", "author_url": "", "post_date": "2024-11-01T12:00:56.683000", "content": "I'm very sorry for you. Just imagine what would have happened if they had started solving the competition not yesterday, but two days before…
", "votes": 0, "replies": [ { "id": 3033954, "author_name": "Jeroen Cottaar", "author_url": "", "post_date": "2024-11-01T16:41:28.667000", "content": "Now that all decimals are shown, I can also see that I have 3 submissions that would have taken first place… (none of them were serious candidates for selection though)
", "votes": 0, "replies": [] } ] }, { "id": 3033716, "author_name": "Gordon Yip", "author_url": "", "post_date": "2024-11-01T12:54:52.190000", "content": "we have also noticed that a few teams managed to break through the 0.7 barrier in the last few days, which is incredible.
", "votes": 0, "replies": [ { "id": 3033724, "author_name": "Sergei Fironov", "author_url": "", "post_date": "2024-11-01T13:03:05.070000", "content": "This is a common story for competitions. First, they develop several approaches one at a time, and in the end they combine them in some way to get a strong breakthrough. in fact, people are training to make the final leap.
", "votes": 1, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "3033567": "I have done this a few times in the past and thought it would be interesting to see for this competition as well, so I created some plots to inspect the shakeup that happened from public LB -> private LB.\n\n\n\n\n\nOverall we had a quite small shakeup in the medal zones, with some few outliers. I saw a few top teams got shaken down really bad, which is of course a very frustrating experience. I suspect that this competition can create such instances, due to edge-cases in preprocessing (e.g. wrong ingress egress detection for single sample?), which can predicting on new data very scary.\n\nHere is the magnified rank shakeup for the top 200 places:\n\n\nThere were not many big shake \"ups\" in this competition, the largest one I saw was by **Maximilian Schnarr**:\nRank: 749 -> 106\nScore: 0.541 -> 0.577\n\nIn this notebook you can find the interactive plots and some more stats:\nhttps://www.kaggle.com/code/fritzcremer/ariel-shakeup", "3033644": "Nice visualizations. Emotionally it did feel like a big shakeup to me...though I was more surprised by how our winners seem to have tweaked their final model on the test set over the course of a single day. Really curious to see how they approached this. (And for those waiting for my solution: it should arrive later today)." } }