Despite some evidence of randomness I'd like to share the ideas we used:\n* 10 model ensemble based on local CV and decent LB.\n* Different predictions between models (regression, bins and ordinal regression)\n* Some of it used bags of tiles and other stack the tiles in squares\n* Efficient nets (I trained only b0 but partner had a few b4\n* My models were trained in two steps. First a model with an attention layer is made (was shared by me in some thread). Then this attention layer and model is reused to predict weights for tiles. Then a model is retrained with a lower number of tiles (9 or 16). I have some 9 tiles models that were both fast and were going at 0.90CV+. On top of it it allowed us to inspect a larger amount of tiles during inference (128 tiles) and just select the best 9 or 16.\n* My partner used in his model a NetVlad layer which maybe hell talk about in this thread.\n* Ensembling with mean + round was better than majority voting for LB (and is what we used) but actually our best solution uses majority voting (which we didnt select).\n* We also built a CV without duplicates and without \"suspicious slides\".
\n\nIn the last weeks after making the team and how obvious it seemed the shake up would be big I started to mistrust LB and try to bring diversity to the ensemble. As long as a model was at LB > 0.88 that was good enough if the CV was among the best ones.
\n\nLearned a lot during this competition. Thanks to organizer.
\n\nNote: We also have a solution at 0.936 that we didn't select :(
", "messages": [ { "id": 940466, "postDate": "2020-07-23T01:03:53.383Z", "content": "Despite some evidence of randomness I'd like to share the ideas we used:\n* 10 model ensemble based on local CV and decent LB.\n* Different predictions between models (regression, bins and ordinal regression)\n* Some of it used bags of tiles and other stack the tiles in squares\n* Efficient nets (I trained only b0 but partner had a few b4\n* My models were trained in two steps. First a model with an attention layer is made (was shared by me in some thread). Then this attention layer and model is reused to predict weights for tiles. Then a model is retrained with a lower number of tiles (9 or 16). I have some 9 tiles models that were both fast and were going at 0.90CV+. On top of it it allowed us to inspect a larger amount of tiles during inference (128 tiles) and just select the best 9 or 16.\n* My partner used in his model a NetVlad layer which maybe hell talk about in this thread.\n* Ensembling with mean + round was better than majority voting for LB (and is what we used) but actually our best solution uses majority voting (which we didnt select).\n* We also built a CV without duplicates and without \"suspicious slides\".
\n\nIn the last weeks after making the team and how obvious it seemed the shake up would be big I started to mistrust LB and try to bring diversity to the ensemble. As long as a model was at LB > 0.88 that was good enough if the CV was among the best ones.
\n\nLearned a lot during this competition. Thanks to organizer.
\n\nNote: We also have a solution at 0.936 that we didn't select :(
", "rawMarkdown": "Despite some evidence of randomness I'd like to share the ideas we used:\n* 10 model ensemble based on local CV and decent LB.\n* Different predictions between models (regression, bins and ordinal regression)\n* Some of it used bags of tiles and other stack the tiles in squares\n* Efficient nets (I trained only b0 but partner had a few b4\n* My models were trained in two steps. First a model with an attention layer is made (was shared by me in some thread). Then this attention layer and model is reused to predict weights for tiles. Then a model is retrained with a lower number of tiles (9 or 16). I have some 9 tiles models that were both fast and were going at 0.90CV+. On top of it it allowed us to inspect a larger amount of tiles during inference (128 tiles) and just select the best 9 or 16.\n* My partner used in his model a NetVlad layer which maybe hell talk about in this thread.\n* Ensembling with mean + round was better than majority voting for LB (and is what we used) but actually our best solution uses majority voting (which we didnt select).\n* We also built a CV without duplicates and without \"suspicious slides\".\n\nIn the last weeks after making the team and how obvious it seemed the shake up would be big I started to mistrust LB and try to bring diversity to the ensemble. As long as a model was at LB > 0.88 that was good enough if the CV was among the best ones.\n\nLearned a lot during this competition. Thanks to organizer.\n\nNote: We also have a solution at 0.936 that we didn't select :(", "votes": 37 }, { "id": 941332, "postDate": "2020-07-23T07:08:19.053Z", "content": "Thanks every one who shared their ideas in this competition !\nHere is my code: solution code for PANDA \nFeel free to ask me about any questions.\nWe are lucky to get into gold zone in the private LB, and this is my second time to survive in big shake up. Last time is the Severstal: Steel Defect Detection.
\n\nprivate lb:0.920 ; public lb: 0.881 ; local kappa: 0.8952 ; karolinska kappa: 0.8976 ; radboud: 0.8704
\n\nDetail:thank my teammate Arnaud , I did the same thing as he did, train a Efficientnet-b0 with Attention layer to generate weight for each tiles . The number of tiles used to train is 64, and then I select the top 16 tiles as the training tiles for the 2nd stage model.
I use Blue Ratio Selection for selecting the top 36 tiles and to accelerate the selection function, firstly I select 128 tiles based on sum of tile pixels and then I select 16 tiles based on them with blue ratio value.
\n\nIn inference, all model use 8 TTA same as public kernel of lafoss.
\n\nAs Arnaud described above , our final submission ensemble 10 model.
", "rawMarkdown": "Thanks every one who shared their ideas in this competition !\nHere is my code: [solution code for PANDA ](https://github.com/ChienYiChi/kaggle-panda-challenge)\nFeel free to ask me about any questions.\nWe are lucky to get into gold zone in the private LB, and this is my second time to survive in big shake up. Last time is the [Severstal: Steel Defect Detection](https://www.kaggle.com/c/severstal-steel-defect-detection). \n\n### Best model of Private LeaderBoard\n#### **1st stage model**: efficientnet-b0 with attention layer, regression type, 64 tiles for train \n#### **2nd stage model**: efficientnet-b0 with NetVLAD layer, regression type, top 16 tiles for train\n#### **Score**: \n- private lb:0.920 ; public lb: 0.881 ; local kappa: 0.8952 ; karolinska kappa: 0.8976 ; radboud: 0.8704\n#### **Detail**:\n- thank my teammate Arnaud , I did the same thing as he did, train a Efficientnet-b0 with Attention layer to generate weight for each tiles . The number of tiles used to train is 64, and then I select the top 16 tiles as the training tiles for the 2nd stage model.\n- train the 2nd stage to predict the final score. I use efficientnet-b0 with a NetVLAD layer to generate the final feature for final prediction. The NetVLAD outperform adaptive max pooling and average pooling layer. \n\n### Best model of Public LeaderBoard \n#### **Model**: efficientnet-b4 with NetVLAD layer, regression type, 36 tiles selected by Blue Ratio Selection.\n#### **Score**:\n- private lb: 0.913 ; public lb: 0.902 ; local kappa: 0.8826 ; karolinska kappa: 0.9047 ; radboud: 0.8335 \n#### **Detail**:\nI use Blue Ratio Selection for selecting the top 36 tiles and to accelerate the selection function, firstly I select 128 tiles based on sum of tile pixels and then I select 16 tiles based on them with blue ratio value. \n\nIn inference, all model use 8 TTA same as public kernel of lafoss. \n\nAs Arnaud described above , our final submission ensemble 10 model.\n", "votes": 5 }, { "id": 940475, "postDate": "2020-07-23T01:13:53.627Z", "content": "Additionally my code is available on github (in a messy state :)) push/pulling was the easiest way for me when using GCP to train bigger models.\nhere: https://github.com/arroqc/pandacancer_kaggle
", "rawMarkdown": "Additionally my code is available on github (in a messy state :)) push/pulling was the easiest way for me when using GCP to train bigger models.\nhere: https://github.com/arroqc/pandacancer_kaggle", "votes": 4, "replies": [ { "id": 940481, "postDate": "2020-07-23T01:22:01.143Z", "content": "Congratulations @arroqc . Your code is still private :)
", "rawMarkdown": "Congratulations @arroqc . Your code is still private :) ", "votes": 1 }, { "id": 940482, "postDate": "2020-07-23T01:23:37.907Z", "content": "Sorry about that, fixed.
", "rawMarkdown": "Sorry about that, fixed.", "votes": 1 }, { "id": 940563, "postDate": "2020-07-23T02:57:46.637Z", "content": "congrats
", "rawMarkdown": "congrats " } ] }, { "id": 941756, "postDate": "2020-07-23T11:52:21.803Z", "content": "Thanks guys for sharing your githubs, I'll work on those right away. Congratulations on your medal ;)
", "rawMarkdown": "Thanks guys for sharing your githubs, I'll work on those right away. Congratulations on your medal ;)", "votes": 1 }, { "id": 940506, "postDate": "2020-07-23T01:54:00.997Z", "content": "@arroqc Congratulations for you gold, looking forward to work with you in future.
", "rawMarkdown": "@arroqc Congratulations for you gold, looking forward to work with you in future.", "votes": 1 }, { "id": 948158, "postDate": "2020-07-27T17:43:24.413Z", "content": "@arroqc \nCongrats & thanks for sharing your smart solution!
\n\n> We also built a CV without duplicates and without \"suspicious slides\".
\n\nI'm curious about this part. How did you achieve this?
", "rawMarkdown": "@arroqc \nCongrats & thanks for sharing your smart solution!\n\n> We also built a CV without duplicates and without \"suspicious slides\".\n\nI'm curious about this part. How did you achieve this?", "replies": [ { "id": 948412, "postDate": "2020-07-27T23:04:48.223Z", "content": "@yukkyo We just used some kernel someone shared that uses imagehash. For duplicates we used a lower threshold than suggested and got quite a few false positives but we don't think it matters much since all it did was to force those slides to be in the same splits. For suspicious someone released a list at some point of weird cases like blank slides, no mask, pen marks etc. I assumed these were likely to be not representative of the test set so I removed them from the validation splits.
\n\nThat being said it is not clear this strategy really paid off. It just seemed like a good idea at the time amd we didnt really have a way of testing because we believed the LB to be a poor predictor of the private.
", "rawMarkdown": "@yukkyo We just used some kernel someone shared that uses imagehash. For duplicates we used a lower threshold than suggested and got quite a few false positives but we don't think it matters much since all it did was to force those slides to be in the same splits. For suspicious someone released a list at some point of weird cases like blank slides, no mask, pen marks etc. I assumed these were likely to be not representative of the test set so I removed them from the validation splits.\n\nThat being said it is not clear this strategy really paid off. It just seemed like a good idea at the time amd we didnt really have a way of testing because we believed the LB to be a poor predictor of the private.", "votes": 1 }, { "id": 948468, "postDate": "2020-07-28T01:33:43.327Z", "content": "@arroqc Thanks for sharing the details! It's very helpful.
", "rawMarkdown": "@arroqc Thanks for sharing the details! It's very helpful." } ] }, { "id": 945866, "postDate": "2020-07-26T07:30:31.527Z", "content": "Congratulation with 8th place and thanks for sharing your ideas!
", "rawMarkdown": "Congratulation with 8th place and thanks for sharing your ideas!" }, { "id": 945830, "postDate": "2020-07-26T06:54:01.607Z", "content": "Hi @arroqc,
\n\nYour attention approach is interesting. Do I use batch size of 1 (1 bag of n_tiles from same slide) to train the attention model?
", "rawMarkdown": "Hi @arroqc,\n\nYour attention approach is interesting. Do I use batch size of 1 (1 bag of n_tiles from same slide) to train the attention model?", "replies": [ { "id": 945936, "postDate": "2020-07-26T08:22:49.243Z", "content": "Batch size should be greater than 4
", "rawMarkdown": "Batch size should be greater than 4 ", "votes": 1 }, { "id": 946870, "postDate": "2020-07-26T23:04:56.970Z", "content": "No, use as large a batch as you can while training at least 32 tiles per case. I also add a lot of randomness (random tile selection weighted by \"darkness\". augmentated datasets etc.) to help the attention train to find the best tiles.
", "rawMarkdown": "No, use as large a batch as you can while training at least 32 tiles per case. I also add a lot of randomness (random tile selection weighted by \"darkness\". augmentated datasets etc.) to help the attention train to find the best tiles.", "votes": 1 } ] }, { "id": 944105, "postDate": "2020-07-24T20:45:47.657Z", "content": "Congratulations and thank you for sharing your code.
", "rawMarkdown": "Congratulations and thank you for sharing your code." }, { "id": 943171, "postDate": "2020-07-24T07:22:32.630Z", "content": "Congrats @arroqc and your team, you contributed a lot by sharing your ideas during the 3 past months
", "rawMarkdown": "Congrats @arroqc and your team, you contributed a lot by sharing your ideas during the 3 past months" }, { "id": 940623, "postDate": "2020-07-23T03:57:22.947Z", "content": "Congrats @arroqc
", "rawMarkdown": "Congrats @arroqc " }, { "id": 940565, "postDate": "2020-07-23T03:01:04.717Z", "content": "Congrats on 8th place and gold medal @arroqc and thanks for sharing details solution.
", "rawMarkdown": "Congrats on 8th place and gold medal @arroqc and thanks for sharing details solution." }, { "id": 947913, "postDate": "2020-07-27T14:53:34.680Z", "content": "Cool...Thank you for sharing 👍
", "rawMarkdown": "Cool...Thank you for sharing 👍 " } ], "comments": [ { "id": 941332, "author_name": "ChienYiChi", "author_url": "", "post_date": "2020-07-23T07:08:19.053000", "content": "Thanks every one who shared their ideas in this competition !\nHere is my code: solution code for PANDA \nFeel free to ask me about any questions.\nWe are lucky to get into gold zone in the private LB, and this is my second time to survive in big shake up. Last time is the Severstal: Steel Defect Detection.
\n\nprivate lb:0.920 ; public lb: 0.881 ; local kappa: 0.8952 ; karolinska kappa: 0.8976 ; radboud: 0.8704
\n\nDetail:thank my teammate Arnaud , I did the same thing as he did, train a Efficientnet-b0 with Attention layer to generate weight for each tiles . The number of tiles used to train is 64, and then I select the top 16 tiles as the training tiles for the 2nd stage model.
I use Blue Ratio Selection for selecting the top 36 tiles and to accelerate the selection function, firstly I select 128 tiles based on sum of tile pixels and then I select 16 tiles based on them with blue ratio value.
\n\nIn inference, all model use 8 TTA same as public kernel of lafoss.
\n\nAs Arnaud described above , our final submission ensemble 10 model.
", "votes": 5, "replies": [] }, { "id": 940475, "author_name": "Arnaud Roussel", "author_url": "", "post_date": "2020-07-23T01:13:53.627000", "content": "Additionally my code is available on github (in a messy state :)) push/pulling was the easiest way for me when using GCP to train bigger models.\nhere: https://github.com/arroqc/pandacancer_kaggle
", "votes": 4, "replies": [ { "id": 940481, "author_name": "Nirjhar Roy", "author_url": "", "post_date": "2020-07-23T01:22:01.143000", "content": "Congratulations @arroqc . Your code is still private :)
", "votes": 1, "replies": [] }, { "id": 940482, "author_name": "Arnaud Roussel", "author_url": "", "post_date": "2020-07-23T01:23:37.907000", "content": "Sorry about that, fixed.
", "votes": 1, "replies": [] }, { "id": 940563, "author_name": "yuvaramsingh", "author_url": "", "post_date": "2020-07-23T02:57:46.637000", "content": "congrats
", "votes": 0, "replies": [] } ] }, { "id": 941756, "author_name": "Benjamin Dubreu", "author_url": "", "post_date": "2020-07-23T11:52:21.803000", "content": "Thanks guys for sharing your githubs, I'll work on those right away. Congratulations on your medal ;)
", "votes": 1, "replies": [] }, { "id": 940506, "author_name": "Dracarys", "author_url": "", "post_date": "2020-07-23T01:54:00.997000", "content": "@arroqc Congratulations for you gold, looking forward to work with you in future.
", "votes": 1, "replies": [] }, { "id": 948158, "author_name": "fam_taro", "author_url": "", "post_date": "2020-07-27T17:43:24.413000", "content": "@arroqc \nCongrats & thanks for sharing your smart solution!
\n\n> We also built a CV without duplicates and without \"suspicious slides\".
\n\nI'm curious about this part. How did you achieve this?
", "votes": 0, "replies": [ { "id": 948412, "author_name": "Arnaud Roussel", "author_url": "", "post_date": "2020-07-27T23:04:48.223000", "content": "@yukkyo We just used some kernel someone shared that uses imagehash. For duplicates we used a lower threshold than suggested and got quite a few false positives but we don't think it matters much since all it did was to force those slides to be in the same splits. For suspicious someone released a list at some point of weird cases like blank slides, no mask, pen marks etc. I assumed these were likely to be not representative of the test set so I removed them from the validation splits.
\n\nThat being said it is not clear this strategy really paid off. It just seemed like a good idea at the time amd we didnt really have a way of testing because we believed the LB to be a poor predictor of the private.
", "votes": 1, "replies": [] }, { "id": 948468, "author_name": "fam_taro", "author_url": "", "post_date": "2020-07-28T01:33:43.327000", "content": "@arroqc Thanks for sharing the details! It's very helpful.
", "votes": 0, "replies": [] } ] }, { "id": 945866, "author_name": "Artem Shibaev", "author_url": "", "post_date": "2020-07-26T07:30:31.527000", "content": "Congratulation with 8th place and thanks for sharing your ideas!
", "votes": 0, "replies": [] }, { "id": 945830, "author_name": "Yousef Rabi", "author_url": "", "post_date": "2020-07-26T06:54:01.607000", "content": "Hi @arroqc,
\n\nYour attention approach is interesting. Do I use batch size of 1 (1 bag of n_tiles from same slide) to train the attention model?
", "votes": 0, "replies": [ { "id": 945936, "author_name": "ChienYiChi", "author_url": "", "post_date": "2020-07-26T08:22:49.243000", "content": "Batch size should be greater than 4
", "votes": 1, "replies": [] }, { "id": 946870, "author_name": "Arnaud Roussel", "author_url": "", "post_date": "2020-07-26T23:04:56.970000", "content": "No, use as large a batch as you can while training at least 32 tiles per case. I also add a lot of randomness (random tile selection weighted by \"darkness\". augmentated datasets etc.) to help the attention train to find the best tiles.
", "votes": 1, "replies": [] } ] }, { "id": 944105, "author_name": "Yousef Rabi", "author_url": "", "post_date": "2020-07-24T20:45:47.657000", "content": "Congratulations and thank you for sharing your code.
", "votes": 0, "replies": [] }, { "id": 943171, "author_name": "Alex", "author_url": "", "post_date": "2020-07-24T07:22:32.630000", "content": "Congrats @arroqc and your team, you contributed a lot by sharing your ideas during the 3 past months
", "votes": 0, "replies": [] }, { "id": 940623, "author_name": "Kurian Benoy", "author_url": "", "post_date": "2020-07-23T03:57:22.947000", "content": "Congrats @arroqc
", "votes": 0, "replies": [] }, { "id": 940565, "author_name": "KhanhVD", "author_url": "", "post_date": "2020-07-23T03:01:04.717000", "content": "Congrats on 8th place and gold medal @arroqc and thanks for sharing details solution.
", "votes": 0, "replies": [] }, { "id": 947913, "author_name": "Jaseem C K", "author_url": "", "post_date": "2020-07-27T14:53:34.680000", "content": "Cool...Thank you for sharing 👍
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "940466": "Despite some evidence of randomness I'd like to share the ideas we used:\n* 10 model ensemble based on local CV and decent LB.\n* Different predictions between models (regression, bins and ordinal regression)\n* Some of it used bags of tiles and other stack the tiles in squares\n* Efficient nets (I trained only b0 but partner had a few b4\n* My models were trained in two steps. First a model with an attention layer is made (was shared by me in some thread). Then this attention layer and model is reused to predict weights for tiles. Then a model is retrained with a lower number of tiles (9 or 16). I have some 9 tiles models that were both fast and were going at 0.90CV+. On top of it it allowed us to inspect a larger amount of tiles during inference (128 tiles) and just select the best 9 or 16.\n* My partner used in his model a NetVlad layer which maybe hell talk about in this thread.\n* Ensembling with mean + round was better than majority voting for LB (and is what we used) but actually our best solution uses majority voting (which we didnt select).\n* We also built a CV without duplicates and without \"suspicious slides\".\n\nIn the last weeks after making the team and how obvious it seemed the shake up would be big I started to mistrust LB and try to bring diversity to the ensemble. As long as a model was at LB > 0.88 that was good enough if the CV was among the best ones.\n\nLearned a lot during this competition. Thanks to organizer.\n\nNote: We also have a solution at 0.936 that we didn't select :(", "941332": "Thanks every one who shared their ideas in this competition !\nHere is my code: [solution code for PANDA ](https://github.com/ChienYiChi/kaggle-panda-challenge)\nFeel free to ask me about any questions.\nWe are lucky to get into gold zone in the private LB, and this is my second time to survive in big shake up. Last time is the [Severstal: Steel Defect Detection](https://www.kaggle.com/c/severstal-steel-defect-detection). \n\n### Best model of Private LeaderBoard\n#### **1st stage model**: efficientnet-b0 with attention layer, regression type, 64 tiles for train \n#### **2nd stage model**: efficientnet-b0 with NetVLAD layer, regression type, top 16 tiles for train\n#### **Score**: \n- private lb:0.920 ; public lb: 0.881 ; local kappa: 0.8952 ; karolinska kappa: 0.8976 ; radboud: 0.8704\n#### **Detail**:\n- thank my teammate Arnaud , I did the same thing as he did, train a Efficientnet-b0 with Attention layer to generate weight for each tiles . The number of tiles used to train is 64, and then I select the top 16 tiles as the training tiles for the 2nd stage model.\n- train the 2nd stage to predict the final score. I use efficientnet-b0 with a NetVLAD layer to generate the final feature for final prediction. The NetVLAD outperform adaptive max pooling and average pooling layer. \n\n### Best model of Public LeaderBoard \n#### **Model**: efficientnet-b4 with NetVLAD layer, regression type, 36 tiles selected by Blue Ratio Selection.\n#### **Score**:\n- private lb: 0.913 ; public lb: 0.902 ; local kappa: 0.8826 ; karolinska kappa: 0.9047 ; radboud: 0.8335 \n#### **Detail**:\nI use Blue Ratio Selection for selecting the top 36 tiles and to accelerate the selection function, firstly I select 128 tiles based on sum of tile pixels and then I select 16 tiles based on them with blue ratio value. \n\nIn inference, all model use 8 TTA same as public kernel of lafoss. \n\nAs Arnaud described above , our final submission ensemble 10 model.\n", "940475": "Additionally my code is available on github (in a messy state :)) push/pulling was the easiest way for me when using GCP to train bigger models.\nhere: https://github.com/arroqc/pandacancer_kaggle", "941756": "Thanks guys for sharing your githubs, I'll work on those right away. Congratulations on your medal ;)", "940506": "@arroqc Congratulations for you gold, looking forward to work with you in future.", "948158": "@arroqc \nCongrats & thanks for sharing your smart solution!\n\n> We also built a CV without duplicates and without \"suspicious slides\".\n\nI'm curious about this part. How did you achieve this?", "945866": "Congratulation with 8th place and thanks for sharing your ideas!", "945830": "Hi @arroqc,\n\nYour attention approach is interesting. Do I use batch size of 1 (1 bag of n_tiles from same slide) to train the attention model?", "944105": "Congratulations and thank you for sharing your code.", "943171": "Congrats @arroqc and your team, you contributed a lot by sharing your ideas during the 3 past months", "940623": "Congrats @arroqc ", "940565": "Congrats on 8th place and gold medal @arroqc and thanks for sharing details solution.", "947913": "Cool...Thank you for sharing 👍 " } }