Many of you already know that there are 100 images without a mask. \nDuring my EDA I found two more suspicious things:\n* There are 4 masks that only contain background values\n* There are 85 masks that do not contain pixels marked as cancerous although the image is labeled with an ISUP grade > 0
\n\nIt might be a good idea to avoid using these test cases for training you model but instead use them for validation. I am not medically trained but for know I am assuming the isup_grade is still correct for the concering image_ids.
Details and a .csv containing all suspicious image_ids can be found in my EDA.
Many of you already know that there are 100 images without a mask. \nDuring my EDA I found two more suspicious things:\n* There are 4 masks that only contain background values\n* There are 85 masks that do not contain pixels marked as cancerous although the image is labeled with an ISUP grade > 0
\n\nIt might be a good idea to avoid using these test cases for training you model but instead use them for validation. I am not medically trained but for know I am assuming the isup_grade is still correct for the concering image_ids.
Details and a .csv containing all suspicious image_ids can be found in my EDA.
From https://www.kaggle.com/wouterbulten/getting-started-with-the-panda-dataset:
\n\n\n\n\nThe label masks of Radboudumc were semi-automatically generated by several deep learning algorithms, contain noise, and can be considered as weakly-supervised labels. The label masks of Karolinska were semi-autotomatically generated based on annotations by a pathologist.
\n
I noticed the same thing you did. The 85 masks that do not contain pixels labeled as cancerous even though ISUP grade > 0 are from Radboudumc, so the DL model probably just missed them. These are probably tougher cases so might be a good set to test your model against harder cases.
\n\nThe Karolinska labels were generated based on pathologist annotations, so likely are more reliable. However, they are not as granular as the DL-generated labels.
\n\nIn any case, these seem to be weak labels, so we need to be careful about using them.
", "rawMarkdown": "From https://www.kaggle.com/wouterbulten/getting-started-with-the-panda-dataset:\n\n> The label masks of Radboudumc were semi-automatically generated by several deep learning algorithms, contain noise, and can be considered as weakly-supervised labels. The label masks of Karolinska were semi-autotomatically generated based on annotations by a pathologist.\n\nI noticed the same thing you did. The 85 masks that do not contain pixels labeled as cancerous even though ISUP grade > 0 are from Radboudumc, so the DL model probably just missed them. These are probably tougher cases so might be a good set to test your model against harder cases. \n\nThe Karolinska labels were generated based on pathologist annotations, so likely are more reliable. However, they are not as granular as the DL-generated labels.\n\nIn any case, these seem to be weak labels, so we need to be careful about using them.", "votes": 12, "replies": [ { "id": 818277, "postDate": "2020-04-23T18:40:43.293Z", "content": "Thank you for explanation. This helped a lot!
", "rawMarkdown": "Thank you for explanation. This helped a lot!" }, { "id": 820897, "postDate": "2020-04-25T19:40:09.933Z", "rawMarkdown": "" }, { "id": 820899, "postDate": "2020-04-25T19:42:30.477Z", "content": "\n\nwhen doing augmentation on the images we have to apply the same augmentation to the mask too?
", "rawMarkdown": "@iamleonie,\n\nwhen doing augmentation on the images we have to apply the same augmentation to the mask too?" }, { "id": 823772, "postDate": "2020-04-27T21:55:40.080Z", "content": "@oscarrangel you only need to apply geometric transformations to both (e.g. rotation, scale, shift, ...), other non geometric transformations (e.g. brightness, contrast, ..) only need to be applied to the image.
", "rawMarkdown": "@oscarrangel you only need to apply geometric transformations to both (e.g. rotation, scale, shift, ...), other non geometric transformations (e.g. brightness, contrast, ..) only need to be applied to the image.", "votes": 1 }, { "id": 823812, "postDate": "2020-04-27T23:07:59.657Z", "content": "@ngcferreira thanks!
", "rawMarkdown": "@ngcferreira thanks!" } ] }, { "id": 824691, "postDate": "2020-04-28T14:39:02.333Z", "content": "Good work!
", "rawMarkdown": "Good work!" }, { "id": 820851, "postDate": "2020-04-25T19:00:03.323Z", "content": "Can you Please tell me what are the masks exactly? And when I try to open a mask using openslide why it shows a black canvas . Also why after adding cmap the pattern appears and how people got to know that this will work?\nI am trying to learn here
", "rawMarkdown": "Can you Please tell me what are the masks exactly? And when I try to open a mask using openslide why it shows a black canvas . Also why after adding cmap the pattern appears and how people got to know that this will work?\nI am trying to learn here", "replies": [ { "id": 820886, "postDate": "2020-04-25T19:29:28.583Z", "content": "The details and a list of the images are found in their EDA.
\n\nAnd for your question about viewing the masks: the masks are not image data like the WSIs. Instead of containing a range of values from 0 to 255, they only go up to a maximum of 6, representing the different class labels (check the dataset description for details on mask labels). Therefor when you try to visualize the mask, it will appear very dark as every value is close to 0. Applying the color map fixes the problem by assigning each label between 0 and 6 a distinct color.
", "rawMarkdown": "The details and a list of the images are found in their [EDA](https://www.kaggle.com/iamleonie/panda-eda-visualizations-suspicious-data).\n\nAnd for your question about viewing the masks: the masks are not image data like the WSIs. Instead of containing a range of values from 0 to 255, they only go up to a maximum of 6, representing the different class labels (check the dataset description for details on mask labels). Therefor when you try to visualize the mask, it will appear very dark as every value is close to 0. Applying the color map fixes the problem by assigning each label between 0 and 6 a distinct color.", "votes": 4 }, { "id": 820941, "postDate": "2020-04-25T19:59:17.803Z", "content": "Thanks a Lot for the help, I now understand everything
", "rawMarkdown": "Thanks a Lot for the help, I now understand everything" }, { "id": 867379, "postDate": "2020-05-30T08:24:20.213Z", "content": "Thanks for this reply @matthewmasters
", "rawMarkdown": "Thanks for this reply @matthewmasters " } ] } ], "comments": [ { "id": 818246, "author_name": "Ian Pan", "author_url": "", "post_date": "2020-04-23T18:09:23.710000", "content": "From https://www.kaggle.com/wouterbulten/getting-started-with-the-panda-dataset:
\n\n\n\n\nThe label masks of Radboudumc were semi-automatically generated by several deep learning algorithms, contain noise, and can be considered as weakly-supervised labels. The label masks of Karolinska were semi-autotomatically generated based on annotations by a pathologist.
\n
I noticed the same thing you did. The 85 masks that do not contain pixels labeled as cancerous even though ISUP grade > 0 are from Radboudumc, so the DL model probably just missed them. These are probably tougher cases so might be a good set to test your model against harder cases.
\n\nThe Karolinska labels were generated based on pathologist annotations, so likely are more reliable. However, they are not as granular as the DL-generated labels.
\n\nIn any case, these seem to be weak labels, so we need to be careful about using them.
", "votes": 12, "replies": [ { "id": 818277, "author_name": "Leonie", "author_url": "", "post_date": "2020-04-23T18:40:43.293000", "content": "Thank you for explanation. This helped a lot!
", "votes": 0, "replies": [] }, { "id": 820897, "author_name": "TheStoneMX", "author_url": "", "post_date": "2020-04-25T19:40:09.933000", "content": "", "votes": 0, "replies": [] }, { "id": 820899, "author_name": "TheStoneMX", "author_url": "", "post_date": "2020-04-25T19:42:30.477000", "content": "\n\nwhen doing augmentation on the images we have to apply the same augmentation to the mask too?
", "votes": 0, "replies": [] }, { "id": 823772, "author_name": "Nuno Ferreira", "author_url": "", "post_date": "2020-04-27T21:55:40.080000", "content": "@oscarrangel you only need to apply geometric transformations to both (e.g. rotation, scale, shift, ...), other non geometric transformations (e.g. brightness, contrast, ..) only need to be applied to the image.
", "votes": 1, "replies": [] }, { "id": 823812, "author_name": "TheStoneMX", "author_url": "", "post_date": "2020-04-27T23:07:59.657000", "content": "@ngcferreira thanks!
", "votes": 0, "replies": [] } ] }, { "id": 824691, "author_name": "", "author_url": "", "post_date": "2020-04-28T14:39:02.333000", "content": "Good work!
", "votes": 0, "replies": [] }, { "id": 820851, "author_name": "Mr_KnowNothing", "author_url": "", "post_date": "2020-04-25T19:00:03.323000", "content": "Can you Please tell me what are the masks exactly? And when I try to open a mask using openslide why it shows a black canvas . Also why after adding cmap the pattern appears and how people got to know that this will work?\nI am trying to learn here
", "votes": 0, "replies": [ { "id": 820886, "author_name": "Matt", "author_url": "", "post_date": "2020-04-25T19:29:28.583000", "content": "The details and a list of the images are found in their EDA.
\n\nAnd for your question about viewing the masks: the masks are not image data like the WSIs. Instead of containing a range of values from 0 to 255, they only go up to a maximum of 6, representing the different class labels (check the dataset description for details on mask labels). Therefor when you try to visualize the mask, it will appear very dark as every value is close to 0. Applying the color map fixes the problem by assigning each label between 0 and 6 a distinct color.
", "votes": 4, "replies": [] }, { "id": 820941, "author_name": "Mr_KnowNothing", "author_url": "", "post_date": "2020-04-25T19:59:17.803000", "content": "Thanks a Lot for the help, I now understand everything
", "votes": 0, "replies": [] }, { "id": 867379, "author_name": "Kurian Benoy", "author_url": "", "post_date": "2020-05-30T08:24:20.213000", "content": "Thanks for this reply @matthewmasters
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "818226": "Many of you already know that there are 100 images without a mask. \nDuring my EDA I found two more suspicious things:\n* There are 4 masks that only contain background values\n* There are 85 masks that do not contain pixels marked as cancerous although the image is labeled with an ISUP grade > 0\n\nIt might be a good idea to avoid using these test cases for training you model but instead use them for validation. I am not medically trained but for know I am assuming the `isup_grade` is still correct for the concering `image_id`s.\n\nDetails and a .csv containing all suspicious `image_id`s can be found in [my EDA](https://www.kaggle.com/iamleonie/panda-eda-visualizations-suspicious-data).", "818246": "From https://www.kaggle.com/wouterbulten/getting-started-with-the-panda-dataset:\n\n> The label masks of Radboudumc were semi-automatically generated by several deep learning algorithms, contain noise, and can be considered as weakly-supervised labels. The label masks of Karolinska were semi-autotomatically generated based on annotations by a pathologist.\n\nI noticed the same thing you did. The 85 masks that do not contain pixels labeled as cancerous even though ISUP grade > 0 are from Radboudumc, so the DL model probably just missed them. These are probably tougher cases so might be a good set to test your model against harder cases. \n\nThe Karolinska labels were generated based on pathologist annotations, so likely are more reliable. However, they are not as granular as the DL-generated labels.\n\nIn any case, these seem to be weak labels, so we need to be careful about using them.", "824691": "Good work!", "820851": "Can you Please tell me what are the masks exactly? And when I try to open a mask using openslide why it shows a black canvas . Also why after adding cmap the pattern appears and how people got to know that this will work?\nI am trying to learn here" } }