![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1794509%2F9468fad56fbe01d66d223f4a50c0304f%2Funcrop.png?generation=1760528693093368&alt=media\")
Sincere Thank You to RSNA and Kaggle for hosting this competition
\nI tackled the challenge as a speedrun as I had only 14 days left on the competition when I started, which is nowhere near enough to cover everything one can do with this dataset, the dataset is super cool and I have barely scratched the surface. I managed to get in the gold medal line in 8 days.
\nFinally a GM, yay!
\nI started with training a segmentation model… which was not really happy to learn beyond 0.6 Dice in the short time I tried which was nowhere near enough for my confidence in the approach.
\nI shifted to a ViT (dinov3) model that takes in input of equidistant 48 slices for each patient, input shape: 48x1x128x128 volume and predicts x1, x2, y1, y2 values of where the crop boundaries of segmetation mask should be at patient-level, this model was far more reliable and efficient for ROI cropping
\nOriginal:
\n
ROI Crop:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1794509%2F3cea5be1e8254117722f8b66541e9682%2Froi_crop.png?generation=1760528860832407&alt=media\")
I kept the ROI relatively large as I did not want to miss out on aneurisms, I tested with the coordinate data and doing it in this configuration let me keep 95%+ of aneurism locations inside the cropped region
\nI took ~2.2k samples in the train_localizer csv file and all samples from negative patients which gives me a dataset of:
\nI managed to find a combination of pipeline where model starts to learn even in this highly imbalance without weighted sampling (weighted sampling did not help).
\nMy entire solution is this classification model and here’s how I improve it’s score (very simple once baseline starts learning)
\nMy baseline hyperparameters:
\n| Approach | \nCV | \nPublic/Private Leaderboard | \n
|---|---|---|
| CoaT-Lite-Medium on ROI Crops | \n0.805 | \n0.78/0.78 | \n
| + Rotate (-25, 25) | \n0.83 | \n0.8/0.81 | \n
| + 2.5D (-2, 0, +2) | \n0.86 | \n0.86/0.83 | \n
| + Soft Pseudo-Distill on remaining 500k potentially positive samples | \n0.89 | \n0.86/0.84 | \n
| Ensembling with MaxViT + CoaT | \n0.896 | \n0.87/0.84 | \n
On inference, the model predicts on every image for each patient, and final predictions are simply max aggregated
\nThank you for reading!
\nInference Code here
\nTraining Code here
\nModel Weights here
Sincere Thank You to RSNA and Kaggle for hosting this competition
\nI tackled the challenge as a speedrun as I had only 14 days left on the competition when I started, which is nowhere near enough to cover everything one can do with this dataset, the dataset is super cool and I have barely scratched the surface. I managed to get in the gold medal line in 8 days.
\nFinally a GM, yay!
\nI started with training a segmentation model… which was not really happy to learn beyond 0.6 Dice in the short time I tried which was nowhere near enough for my confidence in the approach.
\nI shifted to a ViT (dinov3) model that takes in input of equidistant 48 slices for each patient, input shape: 48x1x128x128 volume and predicts x1, x2, y1, y2 values of where the crop boundaries of segmetation mask should be at patient-level, this model was far more reliable and efficient for ROI cropping
\nOriginal:
\n
ROI Crop:
\n
I kept the ROI relatively large as I did not want to miss out on aneurisms, I tested with the coordinate data and doing it in this configuration let me keep 95%+ of aneurism locations inside the cropped region
\nI took ~2.2k samples in the train_localizer csv file and all samples from negative patients which gives me a dataset of:
\nI managed to find a combination of pipeline where model starts to learn even in this highly imbalance without weighted sampling (weighted sampling did not help).
\nMy entire solution is this classification model and here’s how I improve it’s score (very simple once baseline starts learning)
\nMy baseline hyperparameters:
\n| Approach | \nCV | \nPublic/Private Leaderboard | \n
|---|---|---|
| CoaT-Lite-Medium on ROI Crops | \n0.805 | \n0.78/0.78 | \n
| + Rotate (-25, 25) | \n0.83 | \n0.8/0.81 | \n
| + 2.5D (-2, 0, +2) | \n0.86 | \n0.86/0.83 | \n
| + Soft Pseudo-Distill on remaining 500k potentially positive samples | \n0.89 | \n0.86/0.84 | \n
| Ensembling with MaxViT + CoaT | \n0.896 | \n0.87/0.84 | \n
On inference, the model predicts on every image for each patient, and final predictions are simply max aggregated
\nThank you for reading!
\nInference Code here
\nTraining Code here
\nModel Weights here
Amazing work! The stepwise improvements and final ensemble strategy are very insightful. Thanks for sharing your process and code!
", "rawMarkdown": "Amazing work! The stepwise improvements and final ensemble strategy are very insightful. Thanks for sharing your process and code!", "votes": 1 }, { "id": 3304486, "postDate": "2025-10-20T16:08:57.830Z", "content": "Congrats!!Thank you very much for providing such a simple solution!
\nI have a question about Training.
\nWhen I was training my model, I used BCELoss and FocalLoss, but the training loss was quite unstable no matter how much I adjusted the learning rate or scheduler.
\nDid you experience the same issue in your training pipeline?
\nOr did you confirm that the loss converged smoothly?
Hi, Glad you liked my solution
\nYour score is near 0.7 which indicates that the model is not learning to predict the aneurism, it is likely that model has been picking up on other clues like weight imbalance between several modalities or other clues in the image to produce a score near 0.7, once your model really starts learning the competition target (0.78+ in my experiments), the training/validation loss/score metrics gets stable
\nI did face heavy unstability issues when my score was lower than 0.7
\nYes, in my most recent work of scores 0.8+ the loss converges smoothly
", "rawMarkdown": "Hi, Glad you liked my solution\n\nYour score is near 0.7 which indicates that the model is not learning to predict the aneurism, it is likely that model has been picking up on other clues like weight imbalance between several modalities or other clues in the image to produce a score near 0.7, once your model really starts learning the competition target (0.78+ in my experiments), the training/validation loss/score metrics gets stable\n\nI did face heavy unstability issues when my score was lower than 0.7\n\nYes, in my most recent work of scores 0.8+ the loss converges smoothly", "votes": 1, "replies": [ { "id": 3304628, "postDate": "2025-10-21T00:57:32.513Z", "content": "Thank you for your reply! So, if the training is going well, the loss stabilizes and converges — that’s reassuring to hear.
\nBy the way, I have a question: when the loss became stable (from 0.7 to 0.78), did you do anything in particular to achieve that? For example, did you add class weights or something similar?
", "rawMarkdown": "Thank you for your reply! So, if the training is going well, the loss stabilizes and converges — that’s reassuring to hear.\n\nBy the way, I have a question: when the loss became stable (from 0.7 to 0.78), did you do anything in particular to achieve that? For example, did you add class weights or something similar?" }, { "id": 3304662, "postDate": "2025-10-21T03:42:04.980Z", "content": "For me, it was data cleaning, I take 2k positive samples and 500k negative samples, it was hard for the model to start learning immediately
\nwith ROI crop I think my score went from 0.7 to 0.8 (baseline)
", "rawMarkdown": "For me, it was data cleaning, I take 2k positive samples and 500k negative samples, it was hard for the model to start learning immediately\n\nwith ROI crop I think my score went from 0.7 to 0.8 (baseline)" }, { "id": 3304934, "postDate": "2025-10-21T15:19:54.337Z", "content": "It seems that increasing the resolution based on the ROI is the key point in this comp.
\nI understand it well now. Thank you very much!!
congrats. This is the simplest and most clean solution I've ever seen
", "rawMarkdown": "congrats. This is the simplest and most clean solution I've ever seen", "votes": 1 }, { "id": 3302606, "postDate": "2025-10-16T06:31:00.357Z", "content": "Congrats @harshitsheoran
", "rawMarkdown": "Congrats @harshitsheoran ", "votes": 1 }, { "id": 3302267, "postDate": "2025-10-15T12:35:33.097Z", "content": "Congratulations! This is remarkably simple!
\nAny chance you can share your training code so that your solution can be reproduced ?
Yes, Simple is what one gets in 8 days :D,
\nI actually did not plan to use only a 2.5D model but wanted to use it as a pretraining for a 2.5D+1D pipeline that takes in multiple images but that was not improving the score any further for me, and I managed to make a really good 2.5D model so I ran with it to pseudo-distillation
\nI am sure that you are really excited to experiment with my code so I am making a priliminary open (sorry it's unclean):
\nhttps://www.kaggle.com/datasets/harshitsheoran/rsna-prilim-code
Ask if you are missing any file that I am forgetting to share
", "rawMarkdown": "Yes, Simple is what one gets in 8 days :D, \n\nI actually did not plan to use only a 2.5D model but wanted to use it as a pretraining for a 2.5D+1D pipeline that takes in multiple images but that was not improving the score any further for me, and I managed to make a really good 2.5D model so I ran with it to pseudo-distillation\n\nI am sure that you are really excited to experiment with my code so I am making a priliminary open (sorry it's unclean):\nhttps://www.kaggle.com/datasets/harshitsheoran/rsna-prilim-code\n\nAsk if you are missing any file that I am forgetting to share", "votes": 2, "replies": [ { "id": 3302279, "postDate": "2025-10-15T13:04:16.543Z", "content": "Unclean is an understatement here ! 😅
", "rawMarkdown": "Unclean is an understatement here ! 😅" } ] } ] }, { "id": 3304692, "postDate": "2025-10-21T05:04:41.740Z", "content": "HI @harshitsheoran, congratulations on your 4th-place finish! Your approach is simple yet very effective. I’m really impressed that you managed to develop everything within just 14 days, it must have taken quite some effort to verify the effectiveness of each component.
\nI have two questions:
\nit seems the 2.5D contributed the most to your approach. Our team didn’t experiment with different sampling ranges like (-2, 0, 2) or wider. How did the sampling range affect your results? Have you tried other ranges?
Also, I’m curious, what was your motivation for choosing ROI classification instead of pursuing a more refined object detection approach? Our team observes doing localization + classification task have better result then only doing classification.
Thank you!
", "rawMarkdown": "HI @harshitsheoran, congratulations on your 4th-place finish! Your approach is simple yet very effective. I’m really impressed that you managed to develop everything within just 14 days, it must have taken quite some effort to verify the effectiveness of each component.\n\nI have two questions: \n* it seems the 2.5D contributed the most to your approach. Our team didn’t experiment with different sampling ranges like (-2, 0, 2) or wider. How did the sampling range affect your results? Have you tried other ranges?\n\n* Also, I’m curious, what was your motivation for choosing ROI classification instead of pursuing a more refined object detection approach? Our team observes doing localization + classification task have better result then only doing classification.\n\nThank you!", "votes": 2, "replies": [ { "id": 3304716, "postDate": "2025-10-21T06:43:55.047Z", "content": "Hi Tom,
\nThank you for appreciating my work
\n14 days meant limited experiments on my part and I wish that I could have refined every method more much
\nI had tried with [-1, 0, 1] earlier but at that time my sampling was positive-overweighted and it did not improve (neither did [-2,0,2]), I did not check on [-1,0,1] when the model reached 0.8 AUC
\nROI classification is simpler, yet effective, it's a tried and tested technique that I have developed many times, being short on time and thinking about complexity that an object detection approach would have both to train and infer, direct ROI coordinate prediction was just a couple hours of work from idea to finish.
", "rawMarkdown": "Hi Tom,\n\nThank you for appreciating my work\n\n14 days meant limited experiments on my part and I wish that I could have refined every method more much\n\nI had tried with [-1, 0, 1] earlier but at that time my sampling was positive-overweighted and it did not improve (neither did [-2,0,2]), I did not check on [-1,0,1] when the model reached 0.8 AUC\n\nROI classification is simpler, yet effective, it's a tried and tested technique that I have developed many times, being short on time and thinking about complexity that an object detection approach would have both to train and infer, direct ROI coordinate prediction was just a couple hours of work from idea to finish.", "votes": 2 } ] }, { "id": 3304499, "postDate": "2025-10-20T16:25:32.570Z", "content": "Hey @harshitsheoran congrats on GM!, Your solution is very simple and elegant. I am trying to replicate your solution by taking some insights from your prelims-code. But it doesn't seems to have the training pipeline for cropping (maybe I just missed it). I am curious to what IoU score you got when doing such cropping using dinoV3? I am getting around ~0.65 on val
", "rawMarkdown": "Hey @harshitsheoran congrats on GM!, Your solution is very simple and elegant. I am trying to replicate your solution by taking some insights from your [prelims-code](https://www.kaggle.com/datasets/harshitsheoran/rsna-prilim-code). But it doesn't seems to have the training pipeline for cropping (maybe I just missed it). I am curious to what IoU score you got when doing such cropping using dinoV3? I am getting around ~0.65 on val", "votes": 2, "replies": [ { "id": 3304511, "postDate": "2025-10-20T17:20:05.293Z", "content": "Hi @iamparadox,
\nCongratulations to you too, Happy that you liked my solution and are excited to dive in
\nFor cropping, I am training a ViT-Small-Plus DinoV3 pretrained model, you can use any model really…, to take input a volume (48, 1, 128, 128) and predict (x1, x2, y1, y2) for where the volume should be cropped
\nx1, x2, y1, y2 are calculated by taking xmin, xmax, ymin, ymax around the 3D mask volume
\nThe metric I am using is MAE, I get an MAE of 0.027 (xy values normalized to 0 and 1)
for cropping I take image[0.9*ymin:1.1*ymax, 0.9*xmin:1.1*xmax]
I am sharing just about all of my code to reproduce everything:
\nhttps://www.kaggle.com/datasets/harshitsheoran/rsna2025-training-code
\nAnd all the raw logs, OOF outputs, and model weights:
\nhttps://www.kaggle.com/datasets/harshitsheoran/rsna2025-raw-models/data
", "rawMarkdown": "Hi @iamparadox,\n\nCongratulations to you too, Happy that you liked my solution and are excited to dive in\n\nFor cropping, I am training a ViT-Small-Plus DinoV3 pretrained model, you can use any model really..., to take input a volume `(48, 1, 128, 128)` and predict `(x1, x2, y1, y2)` for where the volume should be cropped\nx1, x2, y1, y2 are calculated by taking xmin, xmax, ymin, ymax around the 3D mask volume\nThe metric I am using is MAE, I get an MAE of 0.027 (xy values normalized to 0 and 1)\n\nfor cropping I take `image[0.9*ymin:1.1*ymax, 0.9*xmin:1.1*xmax]`\n\nI am sharing just about all of my code to reproduce everything:\n\nhttps://www.kaggle.com/datasets/harshitsheoran/rsna2025-training-code\n\nAnd all the raw logs, OOF outputs, and model weights:\n\nhttps://www.kaggle.com/datasets/harshitsheoran/rsna2025-raw-models/data", "votes": 1 } ] }, { "id": 3303159, "postDate": "2025-10-17T09:00:15.577Z", "content": "Congratz on GM !
\nLooks like we went for quite similar approaches, yours is much simpler though, I'm impressed it scores that well.
", "rawMarkdown": "Congratz on GM !\n\nLooks like we went for quite similar approaches, yours is much simpler though, I'm impressed it scores that well.", "votes": 2, "replies": [ { "id": 3303183, "postDate": "2025-10-17T10:10:54.810Z", "content": "Thank you and congrats to you too on developing a very stable solution (public vs private)
\nYes, I think our approach aligns a lot, If I had time to work on each area, I may have developed to something as refined as yours
\nThe score difference is a bit confusing but it might just be coat-lite-medium being an exceptionally good model in my experiments (on CV, did not compare on LB)
", "rawMarkdown": "Thank you and congrats to you too on developing a very stable solution (public vs private)\n\nYes, I think our approach aligns a lot, If I had time to work on each area, I may have developed to something as refined as yours\n\nThe score difference is a bit confusing but it might just be coat-lite-medium being an exceptionally good model in my experiments (on CV, did not compare on LB)", "votes": 2, "replies": [ { "id": 3304106, "postDate": "2025-10-19T20:11:26.277Z", "content": "I tried coat-lite-medium as well, it's CV was not as good. Did not submit though, maybe it does well on LB.
", "rawMarkdown": "I tried coat-lite-medium as well, it's CV was not as good. Did not submit though, maybe it does well on LB.", "votes": 1 } ] } ] }, { "id": 3401660, "postDate": "2026-02-04T08:51:16.153Z", "content": "Nice solution @harshitsheoran .. may I know what gpu you used for training the model and the training time in approx ?
", "rawMarkdown": "Nice solution @harshitsheoran .. may I know what gpu you used for training the model and the training time in approx ?" }, { "id": 3302401, "postDate": "2025-10-15T18:29:06.170Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 3304973, "author_name": "Dimpi Mittal", "author_url": "", "post_date": "2025-10-21T17:41:09.837000", "content": "Amazing work! The stepwise improvements and final ensemble strategy are very insightful. Thanks for sharing your process and code!
", "votes": 1, "replies": [] }, { "id": 3304486, "author_name": "shiba-inu", "author_url": "", "post_date": "2025-10-20T16:08:57.830000", "content": "Congrats!!Thank you very much for providing such a simple solution!
\nI have a question about Training.
\nWhen I was training my model, I used BCELoss and FocalLoss, but the training loss was quite unstable no matter how much I adjusted the learning rate or scheduler.
\nDid you experience the same issue in your training pipeline?
\nOr did you confirm that the loss converged smoothly?
Hi, Glad you liked my solution
\nYour score is near 0.7 which indicates that the model is not learning to predict the aneurism, it is likely that model has been picking up on other clues like weight imbalance between several modalities or other clues in the image to produce a score near 0.7, once your model really starts learning the competition target (0.78+ in my experiments), the training/validation loss/score metrics gets stable
\nI did face heavy unstability issues when my score was lower than 0.7
\nYes, in my most recent work of scores 0.8+ the loss converges smoothly
", "votes": 1, "replies": [ { "id": 3304628, "author_name": "shiba-inu", "author_url": "", "post_date": "2025-10-21T00:57:32.513000", "content": "Thank you for your reply! So, if the training is going well, the loss stabilizes and converges — that’s reassuring to hear.
\nBy the way, I have a question: when the loss became stable (from 0.7 to 0.78), did you do anything in particular to achieve that? For example, did you add class weights or something similar?
", "votes": 0, "replies": [] }, { "id": 3304662, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2025-10-21T03:42:04.980000", "content": "For me, it was data cleaning, I take 2k positive samples and 500k negative samples, it was hard for the model to start learning immediately
\nwith ROI crop I think my score went from 0.7 to 0.8 (baseline)
", "votes": 0, "replies": [] }, { "id": 3304934, "author_name": "shiba-inu", "author_url": "", "post_date": "2025-10-21T15:19:54.337000", "content": "It seems that increasing the resolution based on the ROI is the key point in this comp.
\nI understand it well now. Thank you very much!!
congrats. This is the simplest and most clean solution I've ever seen
", "votes": 1, "replies": [] }, { "id": 3302606, "author_name": "Navneet", "author_url": "", "post_date": "2025-10-16T06:31:00.357000", "content": "Congrats @harshitsheoran
", "votes": 1, "replies": [] }, { "id": 3302267, "author_name": "Optimo", "author_url": "", "post_date": "2025-10-15T12:35:33.097000", "content": "Congratulations! This is remarkably simple!
\nAny chance you can share your training code so that your solution can be reproduced ?
Yes, Simple is what one gets in 8 days :D,
\nI actually did not plan to use only a 2.5D model but wanted to use it as a pretraining for a 2.5D+1D pipeline that takes in multiple images but that was not improving the score any further for me, and I managed to make a really good 2.5D model so I ran with it to pseudo-distillation
\nI am sure that you are really excited to experiment with my code so I am making a priliminary open (sorry it's unclean):
\nhttps://www.kaggle.com/datasets/harshitsheoran/rsna-prilim-code
Ask if you are missing any file that I am forgetting to share
", "votes": 2, "replies": [ { "id": 3302279, "author_name": "Optimo", "author_url": "", "post_date": "2025-10-15T13:04:16.543000", "content": "Unclean is an understatement here ! 😅
", "votes": 0, "replies": [] } ] } ] }, { "id": 3304692, "author_name": "Tom", "author_url": "", "post_date": "2025-10-21T05:04:41.740000", "content": "HI @harshitsheoran, congratulations on your 4th-place finish! Your approach is simple yet very effective. I’m really impressed that you managed to develop everything within just 14 days, it must have taken quite some effort to verify the effectiveness of each component.
\nI have two questions:
\nit seems the 2.5D contributed the most to your approach. Our team didn’t experiment with different sampling ranges like (-2, 0, 2) or wider. How did the sampling range affect your results? Have you tried other ranges?
Also, I’m curious, what was your motivation for choosing ROI classification instead of pursuing a more refined object detection approach? Our team observes doing localization + classification task have better result then only doing classification.
Thank you!
", "votes": 2, "replies": [ { "id": 3304716, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2025-10-21T06:43:55.047000", "content": "Hi Tom,
\nThank you for appreciating my work
\n14 days meant limited experiments on my part and I wish that I could have refined every method more much
\nI had tried with [-1, 0, 1] earlier but at that time my sampling was positive-overweighted and it did not improve (neither did [-2,0,2]), I did not check on [-1,0,1] when the model reached 0.8 AUC
\nROI classification is simpler, yet effective, it's a tried and tested technique that I have developed many times, being short on time and thinking about complexity that an object detection approach would have both to train and infer, direct ROI coordinate prediction was just a couple hours of work from idea to finish.
", "votes": 2, "replies": [] } ] }, { "id": 3304499, "author_name": "IAmParadox", "author_url": "", "post_date": "2025-10-20T16:25:32.570000", "content": "Hey @harshitsheoran congrats on GM!, Your solution is very simple and elegant. I am trying to replicate your solution by taking some insights from your prelims-code. But it doesn't seems to have the training pipeline for cropping (maybe I just missed it). I am curious to what IoU score you got when doing such cropping using dinoV3? I am getting around ~0.65 on val
", "votes": 2, "replies": [ { "id": 3304511, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2025-10-20T17:20:05.293000", "content": "Hi @iamparadox,
\nCongratulations to you too, Happy that you liked my solution and are excited to dive in
\nFor cropping, I am training a ViT-Small-Plus DinoV3 pretrained model, you can use any model really…, to take input a volume (48, 1, 128, 128) and predict (x1, x2, y1, y2) for where the volume should be cropped
\nx1, x2, y1, y2 are calculated by taking xmin, xmax, ymin, ymax around the 3D mask volume
\nThe metric I am using is MAE, I get an MAE of 0.027 (xy values normalized to 0 and 1)
for cropping I take image[0.9*ymin:1.1*ymax, 0.9*xmin:1.1*xmax]
I am sharing just about all of my code to reproduce everything:
\nhttps://www.kaggle.com/datasets/harshitsheoran/rsna2025-training-code
\nAnd all the raw logs, OOF outputs, and model weights:
\nhttps://www.kaggle.com/datasets/harshitsheoran/rsna2025-raw-models/data
", "votes": 1, "replies": [] } ] }, { "id": 3303159, "author_name": "Theo Viel", "author_url": "", "post_date": "2025-10-17T09:00:15.577000", "content": "Congratz on GM !
\nLooks like we went for quite similar approaches, yours is much simpler though, I'm impressed it scores that well.
", "votes": 2, "replies": [ { "id": 3303183, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2025-10-17T10:10:54.810000", "content": "Thank you and congrats to you too on developing a very stable solution (public vs private)
\nYes, I think our approach aligns a lot, If I had time to work on each area, I may have developed to something as refined as yours
\nThe score difference is a bit confusing but it might just be coat-lite-medium being an exceptionally good model in my experiments (on CV, did not compare on LB)
", "votes": 2, "replies": [ { "id": 3304106, "author_name": "Theo Viel", "author_url": "", "post_date": "2025-10-19T20:11:26.277000", "content": "I tried coat-lite-medium as well, it's CV was not as good. Did not submit though, maybe it does well on LB.
", "votes": 1, "replies": [] } ] } ] }, { "id": 3401660, "author_name": "AK", "author_url": "", "post_date": "2026-02-04T08:51:16.153000", "content": "Nice solution @harshitsheoran .. may I know what gpu you used for training the model and the training time in approx ?
", "votes": 0, "replies": [] }, { "id": 3302401, "author_name": "", "author_url": "", "post_date": "2025-10-15T18:29:06.170000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "3302248": "Sincere Thank You to RSNA and Kaggle for hosting this competition \n\nI tackled the challenge as a speedrun as I had only 14 days left on the competition when I started, which is nowhere near enough to cover everything one can do with this dataset, the dataset is super cool and I have barely scratched the surface. I managed to get in the gold medal line in 8 days.\n\nFinally a GM, yay!\n\n### Segmentation Model Failure\n\nI started with training a segmentation model… which was not really happy to learn beyond 0.6 Dice in the short time I tried which was nowhere near enough for my confidence in the approach.\n\n### ROI Crop coordinates Model\n\nI shifted to a ViT (dinov3) model that takes in input of equidistant 48 slices for each patient, input shape: 48x1x128x128 volume and predicts x1, x2, y1, y2 values of where the crop boundaries of segmetation mask should be at patient-level, this model was far more reliable and efficient for ROI cropping\n\nOriginal:\n\n\n\nROI Crop:\n\n\n\nI kept the ROI relatively large as I did not want to miss out on aneurisms, I tested with the coordinate data and doing it in this configuration let me keep 95%+ of aneurism locations inside the cropped region\n\n### Classification Model\n\nI took ~2.2k samples in the train_localizer csv file and all samples from negative patients which gives me a dataset of:\n- 545k samples with ~2.2K positive samples and rest negative samples (1 in 250 positive), \n- There are 14 labels in the dataset, which keeps it that model needs to predict 1/1700 values to be 1, rest 0s\n\nI managed to find a combination of pipeline where model starts to learn even in this highly imbalance without weighted sampling (weighted sampling did not help).\n\nMy entire solution is this classification model and here’s how I improve it’s score (very simple once baseline starts learning)\n\nMy baseline hyperparameters:\n- Model: CoaT-Lite-Medium\n- Optimizer: AdamW | Learning-Rate: 1e-4 cosine annealing down to ~3e-5\n- Augmentations: HFlip, VFlip, no TTA\n- Image size: 384x384\n- Global Batch-Size: 192\n\n| Approach | CV | Public/Private Leaderboard |\n|---|---|---|\n| CoaT-Lite-Medium on ROI Crops | 0.805 | 0.78/0.78 |\n| + Rotate (-25, 25) | 0.83 | 0.8/0.81 |\n| + 2.5D (-2, 0, +2) | 0.86 | 0.86/0.83 |\n| + Soft Pseudo-Distill on remaining 500k potentially positive samples | 0.89 | 0.86/0.84 |\n| Ensembling with MaxViT + CoaT | 0.896 | 0.87/0.84 |\n\n