I started with @corochann ‘s notebook: https://www.kaggle.com/corochann/vinbigdata-detectron2-train
\nIt was a great notebook to begin with because it was already including so many processes like validation, augmentations, inference, and more importantly clear explanations about what’s going on.
\nPart - 1: To the LB 301
\nI prepared my own notebook based on the notebook above and ran a model with all the images at 1024x1024 resolution and also included class 14 as a new class during training (By giving full image size as true boxes). My very first proper model gave me 0.221 LB / 0.233 Private.
I started to read past OD competitions’ winning solutions to find out what are the main tricks applied in these competitions and of course found out @zfturbo ’s ensembling repo. https://github.com/ZFTurbo/Weighted-Boxes-Fusion
\nI wanted to give a try with WBF method, and I ensembled my 0.221 submission with the public 0.230 submission of the same notebook I mentioned above. This gave me 0.236 LB / 0.251 Private.
I was wandering in both discussions and kernels and found out that there was a YOLO thing that I had to spend some time on. And there was already another great and clear notebook from @awsaf49 https://www.kaggle.com/awsaf49/vinbigdata-cxr-ad-yolov5-14-class-infer
\nWhy not to try ensembling YOLO and my detectron submission, right? I applied WBF again on my 0.236 and 0.154 YOLO and the magic happened: LB became 0.279 / Private 0.270 and I placed to #10 rank so quickly after joining the competition. If you still think somehow there was a magic in this comp, I strongly think that it was ensembling different models with YOLO preds. I was both surprised and worried about this jump because I was questioning why others did not try this already… And still don’t know why.
Seeing this jump, I realized that the same YOLO notebook had different subs for different training folds. So I ensembled all YOLO fold predictions and that came 0.237. So After ensembling my detectron2 submission with this, LB became 0.283/ Private 0.284.
After this point, in the same days, this another YOLO notebook from
\n@nxhong93 (https://www.kaggle.com/nxhong93/yolov5-chest-512) was published and of course, I wanted to check their ensemble again :D then LB became 0.299 / Private 0.305. This time after applying WBF I also applied NMS on top of it because I found out about the difference of test annotations and it was obviously not good to have lots of overlapping boxes. So LB became 0.301 / Private 0.304.
Part 2: Back to the Validation, and to the LB 324
\nAt this step, I already had 0.301 LB submission but it was worrying because I literally did not do anything with a proper validation step, which is something I used to do in all my previous competitions..s So I decided to build a new submission from scratch in a validated manner. We can have 2 final submissions, right?
\nStarted with detectron2 model again and separated a holdout with 3k images (KFold would take so much time and I had no enough patience). CV came 0.33 and LB came 0.226 / Private 0.235. Btw for cv calculation, I adapted @its7171 ’s this notebook. https://www.kaggle.com/its7171/map-understanding-with-code-and-its-tips
Started to think about how I can improve this single model submission and decided to play with class distributions. So I ran several models by augmenting rare classes, and after checking their single class performances I realized that some models were performing differently in different classes. I realized that I couldn't directly use @zfturbo's ensembling function anymore because some classes hurt a lot during ensembling while others improved. So I built my own function on top of his functions where I could specify which classes to ensemble. With this method, I even applied different fusion parameters for different classes. You can find the function here: https://www.kaggle.com/fatihozturk/class-based-fusing?scriptVersionId=58351361.
I improved my CV to 0.358 and LB became 0.236 / 0.245. Then I couldn’t help myself but check if this new model improves the previous lb 0.301 submission and it did indeed and LB became 0.312 / Private 0.308.
I kept focusing on my CV. So this time instead of augmenting, I started to remove some of the classes during modeling. For example, till this point, I was always including class 14. What I did was simply I ran one model with only abnormal classes, then I ran another model by removing class 0,3,11,13 and so on. This part helped me a lot. With these new diverse models, I improved my CV to 0.423 and LB to 0.265 / Private 0.271.
Part - 3: To LB 0.324
\nPart 4: To LB 0.354 with Teaming and Making lots of analysis
\n@socom20 and his team were at 0.331 and they asked me to merge teams. Tbh, I could have kept going solo but I was still confused about my score given the number of public images and the difference in test annotations. I thought it would be more robust to ensemble boxes with another diverse model and then I accepted.
Of course, we again ensembled our submissions (lots of ensembles…)and by giving more weight to their submission we jumped to 0.35X region / Private 0.31X. Giving more weight to my submission was giving 0.33X LB which was sad because this one had 0.321 private score which we obviously didn't select…
\nSome findings:
\nAfter teaming, for the last ten days, I spent analyzing our good and bad submissions both by submitting only individual classes within them and also visually inspecting predicted boxes. I found out that our improved subs were indeed improving for almost all classes including common and rare ones.
After spending more time on the calculation of the competition metric I also realized “No Penalty For Adding More Bbox” as @cdeotte explains clearly here: https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229637 This was key to big jumps I guess. Having more confident boxes is good but at the same time having many low confidence boxes can only improve this metric…
I’ve analyzed @morizin ‘s YOLO model (he reran his model with my validation split, huge thanks to him) and found out that it was indeed performing much differently than detectron2 models and ensembling with it was boosting both cv and lb. It boosted my 0.265 to 0.300 again but didn't affect the very final ensembles at all. YOLO never failed in this comp :)
I was literally a noob for this field before the competition and obviously, I could’ve done nothing without public sharings and notebooks! So huge thanks to the authors of the notebooks I mentioned above! And I want to give hope to those feeling that they are not good enough to join these competitions, because I felt the same before and wanted join this competition anyways because I had to start somewhere…
", "messages": [ { "id": 1258151, "postDate": "2021-03-31T11:48:26.283Z", "content": "I started with @corochann ‘s notebook: https://www.kaggle.com/corochann/vinbigdata-detectron2-train
\nIt was a great notebook to begin with because it was already including so many processes like validation, augmentations, inference, and more importantly clear explanations about what’s going on.
\nPart - 1: To the LB 301
\nI prepared my own notebook based on the notebook above and ran a model with all the images at 1024x1024 resolution and also included class 14 as a new class during training (By giving full image size as true boxes). My very first proper model gave me 0.221 LB / 0.233 Private.
I started to read past OD competitions’ winning solutions to find out what are the main tricks applied in these competitions and of course found out @zfturbo ’s ensembling repo. https://github.com/ZFTurbo/Weighted-Boxes-Fusion
\nI wanted to give a try with WBF method, and I ensembled my 0.221 submission with the public 0.230 submission of the same notebook I mentioned above. This gave me 0.236 LB / 0.251 Private.
I was wandering in both discussions and kernels and found out that there was a YOLO thing that I had to spend some time on. And there was already another great and clear notebook from @awsaf49 https://www.kaggle.com/awsaf49/vinbigdata-cxr-ad-yolov5-14-class-infer
\nWhy not to try ensembling YOLO and my detectron submission, right? I applied WBF again on my 0.236 and 0.154 YOLO and the magic happened: LB became 0.279 / Private 0.270 and I placed to #10 rank so quickly after joining the competition. If you still think somehow there was a magic in this comp, I strongly think that it was ensembling different models with YOLO preds. I was both surprised and worried about this jump because I was questioning why others did not try this already… And still don’t know why.
Seeing this jump, I realized that the same YOLO notebook had different subs for different training folds. So I ensembled all YOLO fold predictions and that came 0.237. So After ensembling my detectron2 submission with this, LB became 0.283/ Private 0.284.
After this point, in the same days, this another YOLO notebook from
\n@nxhong93 (https://www.kaggle.com/nxhong93/yolov5-chest-512) was published and of course, I wanted to check their ensemble again :D then LB became 0.299 / Private 0.305. This time after applying WBF I also applied NMS on top of it because I found out about the difference of test annotations and it was obviously not good to have lots of overlapping boxes. So LB became 0.301 / Private 0.304.
Part 2: Back to the Validation, and to the LB 324
\nAt this step, I already had 0.301 LB submission but it was worrying because I literally did not do anything with a proper validation step, which is something I used to do in all my previous competitions..s So I decided to build a new submission from scratch in a validated manner. We can have 2 final submissions, right?
\nStarted with detectron2 model again and separated a holdout with 3k images (KFold would take so much time and I had no enough patience). CV came 0.33 and LB came 0.226 / Private 0.235. Btw for cv calculation, I adapted @its7171 ’s this notebook. https://www.kaggle.com/its7171/map-understanding-with-code-and-its-tips
Started to think about how I can improve this single model submission and decided to play with class distributions. So I ran several models by augmenting rare classes, and after checking their single class performances I realized that some models were performing differently in different classes. I realized that I couldn't directly use @zfturbo's ensembling function anymore because some classes hurt a lot during ensembling while others improved. So I built my own function on top of his functions where I could specify which classes to ensemble. With this method, I even applied different fusion parameters for different classes. You can find the function here: https://www.kaggle.com/fatihozturk/class-based-fusing?scriptVersionId=58351361.
I improved my CV to 0.358 and LB became 0.236 / 0.245. Then I couldn’t help myself but check if this new model improves the previous lb 0.301 submission and it did indeed and LB became 0.312 / Private 0.308.
I kept focusing on my CV. So this time instead of augmenting, I started to remove some of the classes during modeling. For example, till this point, I was always including class 14. What I did was simply I ran one model with only abnormal classes, then I ran another model by removing class 0,3,11,13 and so on. This part helped me a lot. With these new diverse models, I improved my CV to 0.423 and LB to 0.265 / Private 0.271.
Part - 3: To LB 0.324
\nPart 4: To LB 0.354 with Teaming and Making lots of analysis
\n@socom20 and his team were at 0.331 and they asked me to merge teams. Tbh, I could have kept going solo but I was still confused about my score given the number of public images and the difference in test annotations. I thought it would be more robust to ensemble boxes with another diverse model and then I accepted.
Of course, we again ensembled our submissions (lots of ensembles…)and by giving more weight to their submission we jumped to 0.35X region / Private 0.31X. Giving more weight to my submission was giving 0.33X LB which was sad because this one had 0.321 private score which we obviously didn't select…
\nSome findings:
\nAfter teaming, for the last ten days, I spent analyzing our good and bad submissions both by submitting only individual classes within them and also visually inspecting predicted boxes. I found out that our improved subs were indeed improving for almost all classes including common and rare ones.
After spending more time on the calculation of the competition metric I also realized “No Penalty For Adding More Bbox” as @cdeotte explains clearly here: https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229637 This was key to big jumps I guess. Having more confident boxes is good but at the same time having many low confidence boxes can only improve this metric…
I’ve analyzed @morizin ‘s YOLO model (he reran his model with my validation split, huge thanks to him) and found out that it was indeed performing much differently than detectron2 models and ensembling with it was boosting both cv and lb. It boosted my 0.265 to 0.300 again but didn't affect the very final ensembles at all. YOLO never failed in this comp :)
I was literally a noob for this field before the competition and obviously, I could’ve done nothing without public sharings and notebooks! So huge thanks to the authors of the notebooks I mentioned above! And I want to give hope to those feeling that they are not good enough to join these competitions, because I felt the same before and wanted join this competition anyways because I had to start somewhere…
", "rawMarkdown": "I started with @corochann ‘s notebook: https://www.kaggle.com/corochann/vinbigdata-detectron2-train\n\nIt was a great notebook to begin with because it was already including so many processes like validation, augmentations, inference, and more importantly clear explanations about what’s going on. \n\n**Part - 1: To the LB 301**\n\n1. I prepared my own notebook based on the notebook above and ran a model with all the images at 1024x1024 resolution and also included class 14 as a new class during training (By giving full image size as true boxes). My very first proper model gave me 0.221 LB / 0.233 Private.\n\n2. I started to read past OD competitions’ winning solutions to find out what are the main tricks applied in these competitions and of course found out @zfturbo ’s ensembling repo. https://github.com/ZFTurbo/Weighted-Boxes-Fusion\nI wanted to give a try with WBF method, and I ensembled my 0.221 submission with the public 0.230 submission of the same notebook I mentioned above. This gave me 0.236 LB / 0.251 Private.\n\n3. I was wandering in both discussions and kernels and found out that there was a YOLO thing that I had to spend some time on. And there was already another great and clear notebook from @awsaf49 https://www.kaggle.com/awsaf49/vinbigdata-cxr-ad-yolov5-14-class-infer\nWhy not to try ensembling YOLO and my detectron submission, right? I applied WBF again on my 0.236 and 0.154 YOLO and the magic happened: LB became 0.279 / Private 0.270 and I placed to #10 rank so quickly after joining the competition. If you still think somehow there was a magic in this comp, I strongly think that it was ensembling different models with YOLO preds. I was both surprised and worried about this jump because I was questioning why others did not try this already… And still don’t know why.\n\n4. Seeing this jump, I realized that the same YOLO notebook had different subs for different training folds. So I ensembled all YOLO fold predictions and that came 0.237. So After ensembling my detectron2 submission with this, LB became 0.283/ Private 0.284.\n\n5. After this point, in the same days, this another YOLO notebook from \n@nxhong93 (https://www.kaggle.com/nxhong93/yolov5-chest-512) was published and of course, I wanted to check their ensemble again :D then LB became 0.299 / Private 0.305. This time after applying WBF I also applied NMS on top of it because I found out about the difference of test annotations and it was obviously not good to have lots of overlapping boxes. So LB became 0.301 / Private 0.304.\n\n**Part 2: Back to the Validation, and to the LB 324**\n\nAt this step, I already had 0.301 LB submission but it was worrying because I literally did not do anything with a proper validation step, which is something I used to do in all my previous competitions..s So I decided to build a new submission from scratch in a validated manner. We can have 2 final submissions, right?\n\n1. Started with detectron2 model again and separated a holdout with 3k images (KFold would take so much time and I had no enough patience). CV came 0.33 and LB came 0.226 / Private 0.235. Btw for cv calculation, I adapted @its7171 ’s this notebook. https://www.kaggle.com/its7171/map-understanding-with-code-and-its-tips\n\n2. Started to think about how I can improve this single model submission and decided to play with class distributions. So I ran several models by augmenting rare classes, and after checking their single class performances I realized that some models were performing differently in different classes. I realized that I couldn't directly use @zfturbo's ensembling function anymore because some classes hurt a lot during ensembling while others improved. So I built my own function on top of his functions where I could specify which classes to ensemble. With this method, I even applied different fusion parameters for different classes. You can find the function here: https://www.kaggle.com/fatihozturk/class-based-fusing?scriptVersionId=58351361.\n\n3. I improved my CV to 0.358 and LB became 0.236 / 0.245. Then I couldn’t help myself but check if this new model improves the previous lb 0.301 submission and it did indeed and LB became 0.312 / Private 0.308.\n\n4. I kept focusing on my CV. So this time instead of augmenting, I started to remove some of the classes during modeling. For example, till this point, I was always including class 14. What I did was simply I ran one model with only abnormal classes, then I ran another model by removing class 0,3,11,13 and so on. This part helped me a lot. With these new diverse models, I improved my CV to 0.423 and LB to 0.265 / Private 0.271.\n \n**Part - 3: To LB 0.324**\n1. As you can assume, I ensembled my previous 0.301 submission with this new validated and diverse 0.265 submission, and the LB reached 0.324 LB / Private 0.314.\n\n**Part 4: To LB 0.354 with Teaming and Making lots of analysis**\n@socom20 and his team were at 0.331 and they asked me to merge teams. Tbh, I could have kept going solo but I was still confused about my score given the number of public images and the difference in test annotations. I thought it would be more robust to ensemble boxes with another diverse model and then I accepted. \n\nOf course, we again ensembled our submissions (lots of ensembles...)and by giving more weight to their submission we jumped to 0.35X region / Private 0.31X. Giving more weight to my submission was giving 0.33X LB which was sad because this one had 0.321 private score which we obviously didn't select…\n\n**Some findings:**\n- After teaming, for the last ten days, I spent analyzing our good and bad submissions both by submitting only individual classes within them and also visually inspecting predicted boxes. I found out that our improved subs were indeed improving for almost all classes including common and rare ones. \n\n- After spending more time on the calculation of the competition metric I also realized “No Penalty For Adding More Bbox” as @cdeotte explains clearly here: https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/229637 This was key to big jumps I guess. Having more confident boxes is good but at the same time having many low confidence boxes can only improve this metric...\n\n- I’ve analyzed @morizin ‘s YOLO model (he reran his model with my validation split, huge thanks to him) and found out that it was indeed performing much differently than detectron2 models and ensembling with it was boosting both cv and lb. It boosted my 0.265 to 0.300 again but didn't affect the very final ensembles at all. YOLO never failed in this comp :) \n\nI was literally a noob for this field before the competition and obviously, I could’ve done nothing without public sharings and notebooks! So huge thanks to the authors of the notebooks I mentioned above! And I want to give hope to those feeling that they are not good enough to join these competitions, because I felt the same before and wanted join this competition anyways because I had to start somewhere…\n\n\n\n \n\n\n", "votes": 197 }, { "id": 1258214, "postDate": "2021-03-31T12:59:34.220Z", "content": "Congrats again @fatihozturk and team. Thanks for the detailed writeup
", "rawMarkdown": "Congrats again @fatihozturk and team. Thanks for the detailed writeup", "votes": 3 }, { "id": 1258713, "postDate": "2021-03-31T20:27:31.170Z", "content": "@fatihozturk @socom20 @morizin @avsanjay Congratulations for the first prize, thank you for writing up!
", "rawMarkdown": "@fatihozturk @socom20 @morizin @avsanjay Congratulations for the first prize, thank you for writing up!", "votes": 4 }, { "id": 1258556, "postDate": "2021-03-31T17:52:30.840Z", "content": "Congratz !
\nYour write-up makes it seem like what you achieved is no big deal, but it's actually really impressive !
Thank you! It might be because I've not mentioned the things that I've tried and failed :)
", "rawMarkdown": "Thank you! It might be because I've not mentioned the things that I've tried and failed :)", "votes": 2 }, { "id": 1325802, "postDate": "2021-05-28T02:46:05.383Z", "content": "Would love to see that part as well! Congrats on your win.
", "rawMarkdown": "Would love to see that part as well! Congrats on your win.", "votes": 1 } ] }, { "id": 1258176, "postDate": "2021-03-31T12:07:58.427Z", "content": "@fatihozturk this is amazing and encouraging for anyone. It also proved the power of the community on Kaggle.
", "rawMarkdown": "@fatihozturk this is amazing and encouraging for anyone. It also proved the power of the community on Kaggle. ", "votes": 4, "replies": [ { "id": 1258178, "postDate": "2021-03-31T12:09:21.723Z", "content": "Indeed it is! Congrats for your silver medal too!
", "rawMarkdown": "Indeed it is! Congrats for your silver medal too!" } ] }, { "id": 1370818, "postDate": "2021-06-30T12:26:57.477Z", "content": "Congrats @fatihozturk and team.
", "rawMarkdown": "Congrats @fatihozturk and team.", "votes": 1 }, { "id": 1312998, "postDate": "2021-05-18T11:11:09.743Z", "content": "Congratulations! Thx for sharing works
", "rawMarkdown": "Congratulations! Thx for sharing works", "votes": 1 }, { "id": 1270208, "postDate": "2021-04-11T12:29:11.763Z", "content": "Congratulations!
", "rawMarkdown": "Congratulations!", "votes": 1 }, { "id": 1268191, "postDate": "2021-04-09T07:42:42.087Z", "content": "Congratulations on your outstanding results.💯
", "rawMarkdown": "Congratulations on your outstanding results.💯", "votes": 1 }, { "id": 1267751, "postDate": "2021-04-08T18:41:43.863Z", "content": "Congratulations! and thanks for sharing your approach.
", "rawMarkdown": "Congratulations! and thanks for sharing your approach.", "votes": 1 }, { "id": 1267731, "postDate": "2021-04-08T18:17:00.950Z", "content": "We posted our whole pipeline out here
", "rawMarkdown": "We posted our whole pipeline out [here](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/231511) ", "votes": 1 }, { "id": 1262587, "postDate": "2021-04-04T13:29:23.280Z", "content": "Congratulations! Nice work
", "rawMarkdown": "Congratulations! Nice work", "votes": 1 }, { "id": 1261861, "postDate": "2021-04-03T14:21:47.487Z", "content": "Congratulations!
", "rawMarkdown": "Congratulations! ", "votes": 1 }, { "id": 1260994, "postDate": "2021-04-02T15:38:59.537Z", "content": "Congratulations, great work!! What kind of hardware did you use for computation (GPU,.. )?
\nAnd, thanks to many notebook editors here - they all keep us learning from each other.
I used 1 RTX 6000. But tbh the dataset of this competition was quite small and a collab pro account would be also enough.
", "rawMarkdown": "I used 1 RTX 6000. But tbh the dataset of this competition was quite small and a collab pro account would be also enough." } ] }, { "id": 1259140, "postDate": "2021-04-01T07:07:40.537Z", "content": "Congrats for 1st place holder and thanks for ur kind explanation! What a nice solution …
\nI have a question here, Are u using wbf preprocessed images or raw images?
\nThanks.
Thanks!
\nI used wbf only on predicted boxes. Didn't apply it before training.
", "rawMarkdown": "Thanks!\n\nI used wbf only on predicted boxes. Didn't apply it before training.", "votes": 2 } ] }, { "id": 1258902, "postDate": "2021-04-01T02:11:04.013Z", "content": "Hi Faith @fatihozturk, I would like to congratulate you and your teammates on your work. Besides, I have some questions and hope that you will answer them
\nHi congratulations,
\nI have some questions:
\n1) Did you apply 2nd class filter?
\n2) How can you assemble your model (I guess you have 2nd class filter already) with another public notebook with 2nd class filter also?
\nEx: Assume 5 images in your model [1-boxes, 2-boxes, 3-nofinding, 4-boxes, 5-boxes] and other notebook [1-nofinding, 2-boxes, 3-boxes, 4-boxes, 5-nofinding], how can you merge them? Or you merge them all before then apply your 2nd classifier?
\n3) Did you merge all your model once at the final? or step by step like above? what is the IOU WBF did you select?
Hi thanks!
\n1) Applied it only on top of yolo 0.154 and it became 0.214. Then their ensemble with detection 236 submission became 279.
\n2) This was not a problem because until reaching 0.301 submission, all individual submissions already had class 14 preds in them. After switching to my validated models, since I defined my own function, I could be able to ensemble based on sub class groups. No need to ensemble all classes from the individual submissions.
\n3) It was a step by step process. Each ensemble had their own iou and weights but most of the time was giving 5 weight to better submission and 1 to the new one. IOU was also around 0.40-0.60 most of the time.
So detailed explanation, thankyou
", "rawMarkdown": "So detailed explanation, thankyou", "votes": 1 } ] }, { "id": 1258223, "postDate": "2021-03-31T13:07:53.517Z", "content": "Congratulations on your outstanding results. In the treatment of multiple doctors' annotations, do you use wbf fusion or another way to deal with it?
", "rawMarkdown": "Congratulations on your outstanding results. In the treatment of multiple doctors' annotations, do you use wbf fusion or another way to deal with it?", "votes": 1, "replies": [ { "id": 1258225, "postDate": "2021-03-31T13:09:38.237Z", "content": "Thanks! I didn't do anything for it and kept all the boxes. During my WBF ensembling, they have been handled I guess. It was better to have lots of boxes to blend further.
", "rawMarkdown": "Thanks! I didn't do anything for it and kept all the boxes. During my WBF ensembling, they have been handled I guess. It was better to have lots of boxes to blend further." } ] }, { "id": 1260637, "postDate": "2021-04-02T09:32:37.927Z", "content": "Congrats on 1st place! @fatihozturk @socom20 @morizin @avsanjay
\nAnd thank you for your kind explanation. Well done!
Yesterday I learned what WBF is, and today I have learned better useful an approach : Class Based Fusing. All very useful informations in total. Thank you for sharing your experience.
", "rawMarkdown": "Yesterday I learned what WBF is, and today I have learned better useful an approach : **Class Based Fusing**. All very useful informations in total. Thank you for sharing your experience.", "votes": 2, "replies": [ { "id": 1259352, "postDate": "2021-04-01T10:25:12.003Z", "content": "Thanks, Selman! Happy to hear that you find class-based fusing useful too :)
", "rawMarkdown": "Thanks, Selman! Happy to hear that you find class-based fusing useful too :) ", "votes": 2 } ] }, { "id": 1259252, "postDate": "2021-04-01T09:03:02.463Z", "content": "Congrats for 1st!
", "rawMarkdown": "Congrats for 1st!", "votes": 2 }, { "id": 1258878, "postDate": "2021-04-01T01:07:43.200Z", "content": "Congratz !
", "rawMarkdown": "Congratz !", "votes": 2 }, { "id": 1258803, "postDate": "2021-03-31T22:40:22.600Z", "content": "Congratulations. Great story. It must have been exicting watching your public LB climb so much. It looks like Yolo ensembles well with WBF.
\nOur situation was similar but more scary. As we ensembled more models with WBF (Yolo, EffDet, VFNet) our CV kept increasing but our LB did not increase and many times it decreased. We wanted to break LB 0.300 on public but couldn't. Thankfully, we selected our best CV for our final sub. It turns out that as we ensembled and increased CV that our private LB was climbing each time but we don't know for sure during the competition.
", "rawMarkdown": "Congratulations. Great story. It must have been exicting watching your public LB climb so much. It looks like Yolo ensembles well with WBF. \n\nOur situation was similar but more scary. As we ensembled more models with WBF (Yolo, EffDet, VFNet) our CV kept increasing but our LB did not increase and many times it decreased. We wanted to break LB 0.300 on public but couldn't. Thankfully, we selected our best CV for our final sub. It turns out that as we ensembled and increased CV that our private LB was climbing each time but we don't know for sure during the competition.", "votes": 2, "replies": [ { "id": 1259314, "postDate": "2021-04-01T10:00:47.923Z", "content": "Thanks, Chris, and congrats on your team too! It was indeed exciting :)
\nFor my side, I can say that I was a bit lucky at the beginning because my first ensembles with WBF were intuitive and they did work well and reached lb 0.301 so quickly. During WBF I used iou=0.4-0.5 and always gave higher weight to ensembles subs compared to single model subs.
\nAfter switching to validated subs, I also experienced inconsistency between my CV and LB from time to time and only kept changes if both increased at the same time. However, I was also aware that the validation set up itself is not a good representative of the test set itself and relying on LB still can be useful because it's the only sample we can see the results from the test set. So my final subs were relying on both validated sub and LB based sub :)
", "rawMarkdown": "Thanks, Chris, and congrats on your team too! It was indeed exciting :) \n\nFor my side, I can say that I was a bit lucky at the beginning because my first ensembles with WBF were intuitive and they did work well and reached lb 0.301 so quickly. During WBF I used iou=0.4-0.5 and always gave higher weight to ensembles subs compared to single model subs. \n\nAfter switching to validated subs, I also experienced inconsistency between my CV and LB from time to time and only kept changes if both increased at the same time. However, I was also aware that the validation set up itself is not a good representative of the test set itself and relying on LB still can be useful because it's the only sample we can see the results from the test set. So my final subs were relying on both validated sub and LB based sub :)", "votes": 1 } ] }, { "id": 1258722, "postDate": "2021-03-31T20:35:54.737Z", "content": "Congratulations and thank you very much for sharing your solution in such detail. It will help us to keep learning. Great job !
", "rawMarkdown": "Congratulations and thank you very much for sharing your solution in such detail. It will help us to keep learning. Great job !", "votes": 2 }, { "id": 1258240, "postDate": "2021-03-31T13:26:54.680Z", "content": "Congrats and thanks for your details. It is absolutely amazing!
", "rawMarkdown": "Congrats and thanks for your details. It is absolutely amazing! ", "votes": 2 }, { "id": 1258232, "postDate": "2021-03-31T13:17:08.080Z", "content": "Congrats on the First Place!!!
\nThanks for your detailed explanation, Congrats Excellent teammates @fatihozturk @avsanjay @morizin @socom20
Thank you,
\nCongratz on your solo silver
Huge congratulations and thanks for sharing your path, very helpful👍
", "rawMarkdown": "Huge congratulations and thanks for sharing your path, very helpful👍", "votes": 2 }, { "id": 1258159, "postDate": "2021-03-31T11:54:14.767Z", "content": "congrats 😄
", "rawMarkdown": "congrats 😄", "votes": 2 }, { "id": 2591601, "postDate": "2024-01-08T05:21:53.897Z", "content": "Thanks for your writing. It was clear and concise, helped me learned many new things. Congrats, on winning first price
", "rawMarkdown": "Thanks for your writing. It was clear and concise, helped me learned many new things. Congrats, on winning first price" }, { "id": 1597681, "postDate": "2021-11-27T19:16:21.997Z", "content": "I'm new here but i know this is a huge success.
\ncongratulations @faithozturk
Congratulations and thank you for sharing
", "rawMarkdown": "Congratulations and thank you for sharing" }, { "id": 1593909, "postDate": "2021-11-24T11:28:18.680Z", "content": "Congratulations! Thank you for sharing your work.
", "rawMarkdown": "Congratulations! Thank you for sharing your work." }, { "id": 1467874, "postDate": "2021-08-12T07:01:01.737Z", "content": "Hi Team @fatihozturk @socom20 @morizin @avsanjay. Congratulations.
\nI am new to Kaggle and trying to explore this competition and i am little confused about the values mentioned under Score within Leaderboard Tab.
\nSo, are these MAP (Mean average precision) values of each model? or is this something else?
\nthank you.
what does CV mean in \"CV and LB(for LeaderBoard)\"
", "rawMarkdown": "what does CV mean in \"CV and LB(for LeaderBoard)\"", "replies": [ { "id": 1370695, "postDate": "2021-06-30T10:40:18.183Z", "content": "cross validation
", "rawMarkdown": "cross validation" } ] }, { "id": 1270223, "postDate": "2021-04-11T12:43:21.090Z", "content": "What is NBS?
", "rawMarkdown": "What is NBS?", "replies": [ { "id": 1270268, "postDate": "2021-04-11T13:34:25.933Z", "content": "It was a typo. It's NMS, thanks for letting me know.
", "rawMarkdown": "It was a typo. It's NMS, thanks for letting me know.", "votes": 1 } ] }, { "id": 1258414, "postDate": "2021-03-31T15:29:03.470Z", "content": "Can you tell us what weights was used during the ensembling detectron2 0.236 and yolo 0.154 submissions?
", "rawMarkdown": "Can you tell us what weights was used during the ensembling detectron2 0.236 and yolo 0.154 submissions?", "replies": [ { "id": 1258441, "postDate": "2021-03-31T15:58:12.593Z", "content": "Just checked and it seems I gave 2 to detection and 1 to yolo. Other thing I've noticed now is I applied 2nd class filter on top of yolo before ensembling. So YOLO itself improved from 0.154 to 0.214
", "rawMarkdown": "Just checked and it seems I gave 2 to detection and 1 to yolo. Other thing I've noticed now is I applied 2nd class filter on top of yolo before ensembling. So YOLO itself improved from 0.154 to 0.214" }, { "id": 1258459, "postDate": "2021-03-31T16:14:36.097Z", "content": "1, 2-3 were most common weights? I had thoughts to set 10 for my weak on lb model and 1 for more robust one, but had no time to check this out
", "rawMarkdown": "1, 2-3 were most common weights? I had thoughts to set 10 for my weak on lb model and 1 for more robust one, but had no time to check this out " }, { "id": 1258463, "postDate": "2021-03-31T16:18:16.260Z", "content": "Since two models were (almost) single models, why not give similar weights? I personally gave weights like 5-7 to only really improved submissions (not to disturb good boxes in them), but here both yolo and detectron2 were performing quite similar.
", "rawMarkdown": "Since two models were (almost) single models, why not give similar weights? I personally gave weights like 5-7 to only really improved submissions (not to disturb good boxes in them), but here both yolo and detectron2 were performing quite similar." }, { "id": 1258466, "postDate": "2021-03-31T16:21:52.633Z", "content": "Thx for your responses and showing your great solution. Congrats!
", "rawMarkdown": "Thx for your responses and showing your great solution. Congrats!", "votes": 2 } ] }, { "id": 1260319, "postDate": "2021-04-02T02:42:43.260Z", "rawMarkdown": "", "votes": 2, "isDeleted": true }, { "id": 1259335, "postDate": "2021-04-01T10:15:06.840Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 1258214, "author_name": "KhanhVD", "author_url": "", "post_date": "2021-03-31T12:59:34.220000", "content": "Congrats again @fatihozturk and team. Thanks for the detailed writeup
", "votes": 3, "replies": [] }, { "id": 1258713, "author_name": "corochann", "author_url": "", "post_date": "2021-03-31T20:27:31.170000", "content": "@fatihozturk @socom20 @morizin @avsanjay Congratulations for the first prize, thank you for writing up!
", "votes": 4, "replies": [] }, { "id": 1258556, "author_name": "Theo Viel", "author_url": "", "post_date": "2021-03-31T17:52:30.840000", "content": "Congratz !
\nYour write-up makes it seem like what you achieved is no big deal, but it's actually really impressive !
Thank you! It might be because I've not mentioned the things that I've tried and failed :)
", "votes": 2, "replies": [] }, { "id": 1325802, "author_name": "zsun01", "author_url": "", "post_date": "2021-05-28T02:46:05.383000", "content": "Would love to see that part as well! Congrats on your win.
", "votes": 1, "replies": [] } ] }, { "id": 1258176, "author_name": "Charlie Craine", "author_url": "", "post_date": "2021-03-31T12:07:58.427000", "content": "@fatihozturk this is amazing and encouraging for anyone. It also proved the power of the community on Kaggle.
", "votes": 4, "replies": [ { "id": 1258178, "author_name": "Fatih Öztürk", "author_url": "", "post_date": "2021-03-31T12:09:21.723000", "content": "Indeed it is! Congrats for your silver medal too!
", "votes": 0, "replies": [] } ] }, { "id": 1370818, "author_name": "Sani Kamal", "author_url": "", "post_date": "2021-06-30T12:26:57.477000", "content": "Congrats @fatihozturk and team.
", "votes": 1, "replies": [] }, { "id": 1312998, "author_name": "Byungjun Yoon", "author_url": "", "post_date": "2021-05-18T11:11:09.743000", "content": "Congratulations! Thx for sharing works
", "votes": 1, "replies": [] }, { "id": 1270208, "author_name": "Wonjun Park", "author_url": "", "post_date": "2021-04-11T12:29:11.763000", "content": "Congratulations!
", "votes": 1, "replies": [] }, { "id": 1268191, "author_name": "Sani Kamal", "author_url": "", "post_date": "2021-04-09T07:42:42.087000", "content": "Congratulations on your outstanding results.💯
", "votes": 1, "replies": [] }, { "id": 1267751, "author_name": "Sheldon", "author_url": "", "post_date": "2021-04-08T18:41:43.863000", "content": "Congratulations! and thanks for sharing your approach.
", "votes": 1, "replies": [] }, { "id": 1267731, "author_name": "Mohammed Rizin V K", "author_url": "", "post_date": "2021-04-08T18:17:00.950000", "content": "We posted our whole pipeline out here
", "votes": 1, "replies": [] }, { "id": 1262587, "author_name": "Son Nguyen", "author_url": "", "post_date": "2021-04-04T13:29:23.280000", "content": "Congratulations! Nice work
", "votes": 1, "replies": [] }, { "id": 1261861, "author_name": "Muhammad Ahmed", "author_url": "", "post_date": "2021-04-03T14:21:47.487000", "content": "Congratulations!
", "votes": 1, "replies": [] }, { "id": 1260994, "author_name": "Christian Bruckner", "author_url": "", "post_date": "2021-04-02T15:38:59.537000", "content": "Congratulations, great work!! What kind of hardware did you use for computation (GPU,.. )?
\nAnd, thanks to many notebook editors here - they all keep us learning from each other.
I used 1 RTX 6000. But tbh the dataset of this competition was quite small and a collab pro account would be also enough.
", "votes": 0, "replies": [] } ] }, { "id": 1259140, "author_name": "Wonho Song", "author_url": "", "post_date": "2021-04-01T07:07:40.537000", "content": "Congrats for 1st place holder and thanks for ur kind explanation! What a nice solution …
\nI have a question here, Are u using wbf preprocessed images or raw images?
\nThanks.
Thanks!
\nI used wbf only on predicted boxes. Didn't apply it before training.
", "votes": 2, "replies": [] } ] }, { "id": 1258902, "author_name": "Liam Nguyen", "author_url": "", "post_date": "2021-04-01T02:11:04.013000", "content": "Hi Faith @fatihozturk, I would like to congratulate you and your teammates on your work. Besides, I have some questions and hope that you will answer them
\nHi congratulations,
\nI have some questions:
\n1) Did you apply 2nd class filter?
\n2) How can you assemble your model (I guess you have 2nd class filter already) with another public notebook with 2nd class filter also?
\nEx: Assume 5 images in your model [1-boxes, 2-boxes, 3-nofinding, 4-boxes, 5-boxes] and other notebook [1-nofinding, 2-boxes, 3-boxes, 4-boxes, 5-nofinding], how can you merge them? Or you merge them all before then apply your 2nd classifier?
\n3) Did you merge all your model once at the final? or step by step like above? what is the IOU WBF did you select?
Hi thanks!
\n1) Applied it only on top of yolo 0.154 and it became 0.214. Then their ensemble with detection 236 submission became 279.
\n2) This was not a problem because until reaching 0.301 submission, all individual submissions already had class 14 preds in them. After switching to my validated models, since I defined my own function, I could be able to ensemble based on sub class groups. No need to ensemble all classes from the individual submissions.
\n3) It was a step by step process. Each ensemble had their own iou and weights but most of the time was giving 5 weight to better submission and 1 to the new one. IOU was also around 0.40-0.60 most of the time.
So detailed explanation, thankyou
", "votes": 1, "replies": [] } ] }, { "id": 1258223, "author_name": "tik_boa", "author_url": "", "post_date": "2021-03-31T13:07:53.517000", "content": "Congratulations on your outstanding results. In the treatment of multiple doctors' annotations, do you use wbf fusion or another way to deal with it?
", "votes": 1, "replies": [ { "id": 1258225, "author_name": "Fatih Öztürk", "author_url": "", "post_date": "2021-03-31T13:09:38.237000", "content": "Thanks! I didn't do anything for it and kept all the boxes. During my WBF ensembling, they have been handled I guess. It was better to have lots of boxes to blend further.
", "votes": 0, "replies": [] } ] }, { "id": 1260637, "author_name": "Heroseo", "author_url": "", "post_date": "2021-04-02T09:32:37.927000", "content": "Congrats on 1st place! @fatihozturk @socom20 @morizin @avsanjay
\nAnd thank you for your kind explanation. Well done!
Yesterday I learned what WBF is, and today I have learned better useful an approach : Class Based Fusing. All very useful informations in total. Thank you for sharing your experience.
", "votes": 2, "replies": [ { "id": 1259352, "author_name": "Fatih Öztürk", "author_url": "", "post_date": "2021-04-01T10:25:12.003000", "content": "Thanks, Selman! Happy to hear that you find class-based fusing useful too :)
", "votes": 2, "replies": [] } ] }, { "id": 1259252, "author_name": "MaChaogong", "author_url": "", "post_date": "2021-04-01T09:03:02.463000", "content": "Congrats for 1st!
", "votes": 2, "replies": [] }, { "id": 1258878, "author_name": "t.nakajima", "author_url": "", "post_date": "2021-04-01T01:07:43.200000", "content": "Congratz !
", "votes": 2, "replies": [] }, { "id": 1258803, "author_name": "Chris Deotte", "author_url": "", "post_date": "2021-03-31T22:40:22.600000", "content": "Congratulations. Great story. It must have been exicting watching your public LB climb so much. It looks like Yolo ensembles well with WBF.
\nOur situation was similar but more scary. As we ensembled more models with WBF (Yolo, EffDet, VFNet) our CV kept increasing but our LB did not increase and many times it decreased. We wanted to break LB 0.300 on public but couldn't. Thankfully, we selected our best CV for our final sub. It turns out that as we ensembled and increased CV that our private LB was climbing each time but we don't know for sure during the competition.
", "votes": 2, "replies": [ { "id": 1259314, "author_name": "Fatih Öztürk", "author_url": "", "post_date": "2021-04-01T10:00:47.923000", "content": "Thanks, Chris, and congrats on your team too! It was indeed exciting :)
\nFor my side, I can say that I was a bit lucky at the beginning because my first ensembles with WBF were intuitive and they did work well and reached lb 0.301 so quickly. During WBF I used iou=0.4-0.5 and always gave higher weight to ensembles subs compared to single model subs.
\nAfter switching to validated subs, I also experienced inconsistency between my CV and LB from time to time and only kept changes if both increased at the same time. However, I was also aware that the validation set up itself is not a good representative of the test set itself and relying on LB still can be useful because it's the only sample we can see the results from the test set. So my final subs were relying on both validated sub and LB based sub :)
", "votes": 1, "replies": [] } ] }, { "id": 1258722, "author_name": "Javier Reinoso Velasco", "author_url": "", "post_date": "2021-03-31T20:35:54.737000", "content": "Congratulations and thank you very much for sharing your solution in such detail. It will help us to keep learning. Great job !
", "votes": 2, "replies": [] }, { "id": 1258240, "author_name": "Sunghyun Jun", "author_url": "", "post_date": "2021-03-31T13:26:54.680000", "content": "Congrats and thanks for your details. It is absolutely amazing!
", "votes": 2, "replies": [] }, { "id": 1258232, "author_name": "Wang Xing", "author_url": "", "post_date": "2021-03-31T13:17:08.080000", "content": "Congrats on the First Place!!!
\nThanks for your detailed explanation, Congrats Excellent teammates @fatihozturk @avsanjay @morizin @socom20
Thank you,
\nCongratz on your solo silver
Huge congratulations and thanks for sharing your path, very helpful👍
", "votes": 2, "replies": [] }, { "id": 1258159, "author_name": "Awsaf", "author_url": "", "post_date": "2021-03-31T11:54:14.767000", "content": "congrats 😄
", "votes": 2, "replies": [] }, { "id": 2591601, "author_name": "SANJEEV BHANDARI", "author_url": "", "post_date": "2024-01-08T05:21:53.897000", "content": "Thanks for your writing. It was clear and concise, helped me learned many new things. Congrats, on winning first price
", "votes": 0, "replies": [] }, { "id": 1597681, "author_name": "Oluwaseun Sokunbi", "author_url": "", "post_date": "2021-11-27T19:16:21.997000", "content": "I'm new here but i know this is a huge success.
\ncongratulations @faithozturk