![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1342122%2F741dd449e72907360f2a0d83c8870f70%2FScreen%20Shot%202020-05-14%20at%2010.38.55%20PM.png?generation=1589510360183209&alt=media\")
A while back I made a thread that tracked odd/blank/suspicious slides. Since then I have gone through many iterations of how to leverage it in my pipeline as well as a clean way to collect it all.
\n\nI finally took the time to make a notebook putting all of the findings together. I also included my data loader class that I use in my work.
\n\n
A while back I made a thread that tracked odd/blank/suspicious slides. Since then I have gone through many iterations of how to leverage it in my pipeline as well as a clean way to collect it all.
\n\nI finally took the time to make a notebook putting all of the findings together. I also included my data loader class that I use in my work.
\n\n
hi @dannellyz good work on putting all at a single place . i went through the list and i saw the reason 'Background only' is a subset of 'blank'. any difference between them?
", "rawMarkdown": "hi @dannellyz good work on putting all at a single place . i went through the list and i saw the reason 'Background only' is a subset of 'blank'. any difference between them?", "votes": 1 }, { "id": 850014, "postDate": "2020-05-16T08:49:27.310Z", "content": "Did excluding these slides improve your score?
", "rawMarkdown": "Did excluding these slides improve your score?", "votes": 1, "replies": [ { "id": 850570, "postDate": "2020-05-16T19:16:38.660Z", "content": "I have only ever used the cleaned list, but I would be interested in what you find!
", "rawMarkdown": "I have only ever used the cleaned list, but I would be interested in what you find!" }, { "id": 850656, "postDate": "2020-05-16T21:48:00.233Z", "content": "Alright, I will let you know
", "rawMarkdown": "Alright, I will let you know", "votes": 1 }, { "id": 851562, "postDate": "2020-05-17T18:01:38.977Z", "content": "In my experience it did help my cv score but did not improve as much my lb score. I got a 0.05 cv boost and did not get that much of a lb score boost(but still got one of my top single model result) . Maybe the images in the test set does have some weird images after all? haha
", "rawMarkdown": "In my experience it did help my cv score but did not improve as much my lb score. I got a 0.05 cv boost and did not get that much of a lb score boost(but still got one of my top single model result) . Maybe the images in the test set does have some weird images after all? haha", "votes": 2 }, { "id": 851861, "postDate": "2020-05-18T01:21:12.647Z", "content": "It's the same for me. CV increased from 0.865 to 0.882, but lb is still 0.88 using clean slides
", "rawMarkdown": "It's the same for me. CV increased from 0.865 to 0.882, but lb is still 0.88 using clean slides", "votes": 3 }, { "id": 853525, "postDate": "2020-05-19T08:49:08.183Z", "content": "@shujun717 @yannmajewski Could it be because the pen marked images (i.e. label 3, 4 and 5) are the ones that are hard to predict, and when you remove those, the CV naturally gets better. My model's accuracy of label 0 and 1 is high while e.g. 3,4 and 5 gets lower accuracy.
", "rawMarkdown": "@shujun717 @yannmajewski Could it be because the pen marked images (i.e. label 3, 4 and 5) are the ones that are hard to predict, and when you remove those, the CV naturally gets better. My model's accuracy of label 0 and 1 is high while e.g. 3,4 and 5 gets lower accuracy.", "votes": 1 }, { "id": 853634, "postDate": "2020-05-19T10:55:02.777Z", "content": "You're probably correct. You can say these are hard examples, but i can't imagine hard examples having <3% tissue being actually good examples, so I think it still makes sense to not include them
", "rawMarkdown": "You're probably correct. You can say these are hard examples, but i can't imagine hard examples having <3% tissue being actually good examples, so I think it still makes sense to not include them" }, { "id": 855392, "postDate": "2020-05-20T21:59:42.127Z", "content": "Actually that I saw in my experiment with this data is the following: overall val has improved from 0.889 to 0.902, but this improvement is resulted only by Radboud 0.842->0.869 (mostly because excluding \"no cancerous tissue but ISUP > 0\", which are supposed to be incorrect, I expect), while Karolinska remains nearly the same of ~0.913, and LB is also about the same of 0.90, since it correlates mostly with Karolinska.
", "rawMarkdown": "Actually that I saw in my experiment with this data is the following: overall val has improved from 0.889 to 0.902, but this improvement is resulted only by Radboud 0.842->0.869 (mostly because excluding \"no cancerous tissue but ISUP > 0\", which are supposed to be incorrect, I expect), while Karolinska remains nearly the same of ~0.913, and LB is also about the same of 0.90, since it correlates mostly with Karolinska. ", "votes": 4 } ] }, { "id": 848575, "postDate": "2020-05-15T04:44:14.920Z", "content": "great works!
", "rawMarkdown": "great works!" }, { "id": 854537, "postDate": "2020-05-20T05:09:39.403Z", "content": "@geertlitjens \nI was curious to know the purpose of Pen Markings on slide.\nHow these markings affect the models learning,should such slides be removed from training set?
", "rawMarkdown": " @geertlitjens \nI was curious to know the purpose of Pen Markings on slide.\nHow these markings affect the models learning,should such slides be removed from training set?" }, { "id": 850039, "postDate": "2020-05-16T09:22:09.747Z", "content": "Many thanks @dannellyz! I have a question: Sometimes the tissue has a blackish color around its edges (not like a single pen mark but a coloring around the entire tissue), would that be considered pen marks (marking cancerous tissue)? Probably a question for the hosts of this competitions :-)
", "rawMarkdown": "Many thanks @dannellyz! I have a question: Sometimes the tissue has a blackish color around its edges (not like a single pen mark but a coloring around the entire tissue), would that be considered pen marks (marking cancerous tissue)? Probably a question for the hosts of this competitions :-)", "replies": [ { "id": 850255, "postDate": "2020-05-16T13:11:03.823Z", "content": "I am not sure on that one, but @rohitsingh9990 may have a better lead on that as it was his work that discovered them.
", "rawMarkdown": "I am not sure on that one, but @rohitsingh9990 may have a better lead on that as it was his work that discovered them. " }, { "id": 850275, "postDate": "2020-05-16T13:27:44.617Z", "content": "i was also not sure about that so i had considered them as pen markers.
", "rawMarkdown": "i was also not sure about that so i had considered them as pen markers.", "votes": 1 }, { "id": 850607, "postDate": "2020-05-16T20:13:51.837Z", "content": "Hi, good question. When tissue comes into the lab from a biopsy procedure, it is often inked around the edges (variety of color is possible, black/green/blue/red, etc.) to track that specific biopsy. During a biopsy procedure, up to 12 biopsies from varying locations in the prostate can be taken. The ink around the tissue edges helps the pathologist figure out which biopsy is from which location. In addition, sometimes they use multiple cuts through the same biopsy to help with diagnosis, the color then helps to know which ones are from the same biopsy. How the colors are used differs a bit from lab to lab, but they do not have diagnostic implications (e.g. it is not related to cancer/no cancer or specific grades)
", "rawMarkdown": "Hi, good question. When tissue comes into the lab from a biopsy procedure, it is often inked around the edges (variety of color is possible, black/green/blue/red, etc.) to track that specific biopsy. During a biopsy procedure, up to 12 biopsies from varying locations in the prostate can be taken. The ink around the tissue edges helps the pathologist figure out which biopsy is from which location. In addition, sometimes they use multiple cuts through the same biopsy to help with diagnosis, the color then helps to know which ones are from the same biopsy. How the colors are used differs a bit from lab to lab, but they do not have diagnostic implications (e.g. it is not related to cancer/no cancer or specific grades)", "votes": 4 }, { "id": 850664, "postDate": "2020-05-16T21:56:13.883Z", "content": "Thanks for clarification.
", "rawMarkdown": "Thanks for clarification." } ] }, { "id": 850313, "postDate": "2020-05-16T14:03:39.100Z", "content": "Thank you for the great analysis!
", "rawMarkdown": "Thank you for the great analysis!", "votes": 1 } ], "comments": [ { "id": 851985, "author_name": "yuvaramsingh", "author_url": "", "post_date": "2020-05-18T05:05:01.307000", "content": "hi @dannellyz good work on putting all at a single place . i went through the list and i saw the reason 'Background only' is a subset of 'blank'. any difference between them?
", "votes": 1, "replies": [] }, { "id": 850014, "author_name": "Shujun", "author_url": "", "post_date": "2020-05-16T08:49:27.310000", "content": "Did excluding these slides improve your score?
", "votes": 1, "replies": [ { "id": 850570, "author_name": "Zac Dannelly", "author_url": "", "post_date": "2020-05-16T19:16:38.660000", "content": "I have only ever used the cleaned list, but I would be interested in what you find!
", "votes": 0, "replies": [] }, { "id": 850656, "author_name": "Shujun", "author_url": "", "post_date": "2020-05-16T21:48:00.233000", "content": "Alright, I will let you know
", "votes": 1, "replies": [] }, { "id": 851562, "author_name": "Yann Majewski", "author_url": "", "post_date": "2020-05-17T18:01:38.977000", "content": "In my experience it did help my cv score but did not improve as much my lb score. I got a 0.05 cv boost and did not get that much of a lb score boost(but still got one of my top single model result) . Maybe the images in the test set does have some weird images after all? haha
", "votes": 2, "replies": [] }, { "id": 851861, "author_name": "Shujun", "author_url": "", "post_date": "2020-05-18T01:21:12.647000", "content": "It's the same for me. CV increased from 0.865 to 0.882, but lb is still 0.88 using clean slides
", "votes": 3, "replies": [] }, { "id": 853525, "author_name": "Alex", "author_url": "", "post_date": "2020-05-19T08:49:08.183000", "content": "@shujun717 @yannmajewski Could it be because the pen marked images (i.e. label 3, 4 and 5) are the ones that are hard to predict, and when you remove those, the CV naturally gets better. My model's accuracy of label 0 and 1 is high while e.g. 3,4 and 5 gets lower accuracy.
", "votes": 1, "replies": [] }, { "id": 853634, "author_name": "Shujun", "author_url": "", "post_date": "2020-05-19T10:55:02.777000", "content": "You're probably correct. You can say these are hard examples, but i can't imagine hard examples having <3% tissue being actually good examples, so I think it still makes sense to not include them
", "votes": 0, "replies": [] }, { "id": 855392, "author_name": "Iafoss", "author_url": "", "post_date": "2020-05-20T21:59:42.127000", "content": "Actually that I saw in my experiment with this data is the following: overall val has improved from 0.889 to 0.902, but this improvement is resulted only by Radboud 0.842->0.869 (mostly because excluding \"no cancerous tissue but ISUP > 0\", which are supposed to be incorrect, I expect), while Karolinska remains nearly the same of ~0.913, and LB is also about the same of 0.90, since it correlates mostly with Karolinska.
", "votes": 4, "replies": [] } ] }, { "id": 848575, "author_name": "DatNT", "author_url": "", "post_date": "2020-05-15T04:44:14.920000", "content": "great works!
", "votes": 0, "replies": [] }, { "id": 854537, "author_name": "Jaideep", "author_url": "", "post_date": "2020-05-20T05:09:39.403000", "content": "@geertlitjens \nI was curious to know the purpose of Pen Markings on slide.\nHow these markings affect the models learning,should such slides be removed from training set?
", "votes": 0, "replies": [] }, { "id": 850039, "author_name": "Alex", "author_url": "", "post_date": "2020-05-16T09:22:09.747000", "content": "Many thanks @dannellyz! I have a question: Sometimes the tissue has a blackish color around its edges (not like a single pen mark but a coloring around the entire tissue), would that be considered pen marks (marking cancerous tissue)? Probably a question for the hosts of this competitions :-)
", "votes": 0, "replies": [ { "id": 850255, "author_name": "Zac Dannelly", "author_url": "", "post_date": "2020-05-16T13:11:03.823000", "content": "I am not sure on that one, but @rohitsingh9990 may have a better lead on that as it was his work that discovered them.
", "votes": 0, "replies": [] }, { "id": 850275, "author_name": "Dracarys", "author_url": "", "post_date": "2020-05-16T13:27:44.617000", "content": "i was also not sure about that so i had considered them as pen markers.
", "votes": 1, "replies": [] }, { "id": 850607, "author_name": "GeertLitjens", "author_url": "", "post_date": "2020-05-16T20:13:51.837000", "content": "Hi, good question. When tissue comes into the lab from a biopsy procedure, it is often inked around the edges (variety of color is possible, black/green/blue/red, etc.) to track that specific biopsy. During a biopsy procedure, up to 12 biopsies from varying locations in the prostate can be taken. The ink around the tissue edges helps the pathologist figure out which biopsy is from which location. In addition, sometimes they use multiple cuts through the same biopsy to help with diagnosis, the color then helps to know which ones are from the same biopsy. How the colors are used differs a bit from lab to lab, but they do not have diagnostic implications (e.g. it is not related to cancer/no cancer or specific grades)
", "votes": 4, "replies": [] }, { "id": 850664, "author_name": "Dracarys", "author_url": "", "post_date": "2020-05-16T21:56:13.883000", "content": "Thanks for clarification.
", "votes": 0, "replies": [] } ] }, { "id": 850313, "author_name": "Hironobukawaguchi", "author_url": "", "post_date": "2020-05-16T14:03:39.100000", "content": "Thank you for the great analysis!
", "votes": 1, "replies": [] } ], "raw_markdown_by_id": { "848491": "A while back I made a thread that [tracked odd/blank/suspicious slides](https://www.kaggle.com/c/prostate-cancer-grade-assessment/discussion/148060). Since then I have gone through many iterations of how to leverage it in my pipeline as well as a clean way to collect it all. \n\nI finally took the time to make a [notebook putting all of the findings together](https://www.kaggle.com/dannellyz/collection-of-600-suspicious-slides-data-loader/). I also included my data loader class that I use in my work.\n\n", "851985": "hi @dannellyz good work on putting all at a single place . i went through the list and i saw the reason 'Background only' is a subset of 'blank'. any difference between them?", "850014": "Did excluding these slides improve your score?", "848575": "great works!", "854537": " @geertlitjens \nI was curious to know the purpose of Pen Markings on slide.\nHow these markings affect the models learning,should such slides be removed from training set?", "850039": "Many thanks @dannellyz! I have a question: Sometimes the tissue has a blackish color around its edges (not like a single pen mark but a coloring around the entire tissue), would that be considered pen marks (marking cancerous tissue)? Probably a question for the hosts of this competitions :-)", "850313": "Thank you for the great analysis!" } }