i know with only 2 days left i cant improve my training process anymore. but, i am too curious to know how to handle noisy data. most of the top submissions are handling this noisy label problem in one form or another. it will be helpful if some of those ideas can be shared here.
\n\nlooking forward to learn this and implement on my next competition. :D
", "messages": [ { "id": 939160, "postDate": "2020-07-22T04:15:07.063Z", "content": "Although I wanted to try many methods, I was too weak in computing resource to deal with the noise label.
", "rawMarkdown": "Although I wanted to try many methods, I was too weak in computing resource to deal with the noise label.", "votes": 1, "replies": [ { "id": 939212, "postDate": "2020-07-22T05:20:55.897Z", "content": "can you share some of those methods so it will help others to use them in next competitions .
", "rawMarkdown": "can you share some of those methods so it will help others to use them in next competitions . " }, { "id": 939222, "postDate": "2020-07-22T05:25:23.607Z", "content": "https://www.kaggle.com/c/prostate-cancer-grade-assessment/discussion/168833
", "rawMarkdown": "https://www.kaggle.com/c/prostate-cancer-grade-assessment/discussion/168833", "votes": 1 }, { "id": 939843, "postDate": "2020-07-22T14:10:51.593Z", "content": "thanks
", "rawMarkdown": "thanks" } ] }, { "id": 936766, "postDate": "2020-07-20T14:05:22.913Z", "content": "i know with only 2 days left i cant improve my training process anymore. but, i am too curious to know how to handle noisy data. most of the top submissions are handling this noisy label problem in one form or another. it will be helpful if some of those ideas can be shared here.
\n\nlooking forward to learn this and implement on my next competition. :D
", "rawMarkdown": "i know with only 2 days left i cant improve my training process anymore. but, i am too curious to know how to handle noisy data. most of the top submissions are handling this noisy label problem in one form or another. it will be helpful if some of those ideas can be shared here. \n\nlooking forward to learn this and implement on my next competition. :D ", "votes": 2 }, { "id": 936794, "postDate": "2020-07-20T14:28:35.227Z", "content": "interesting question. What do you exactly mean with noisy? I have noticed that the images have differently colors,\nmostly some variation of red and purple. I have always asked myself during this competition if reducing to graysacle might help. I used samplewised normalizing to hopefully adress without using grayscale. My problem is, that I have a high hould out score, about 0.91, but but only 0.55 on the LB (in my opinion I didtn overfit to my holdout set.)
", "rawMarkdown": "interesting question. What do you exactly mean with noisy? I have noticed that the images have differently colors,\nmostly some variation of red and purple. I have always asked myself during this competition if reducing to graysacle might help. I used samplewised normalizing to hopefully adress without using grayscale. My problem is, that I have a high hould out score, about 0.91, but but only 0.55 on the LB (in my opinion I didtn overfit to my holdout set.)", "replies": [ { "id": 936968, "postDate": "2020-07-20T16:20:54.643Z", "content": "by noisy, i refer to the label. there are some training data which are not having the correct label which is affecting the quality of our model. if we device a better algorithm to filter these samples out of the training data, the final model trained on this filtered data will be much better .
", "rawMarkdown": "by noisy, i refer to the label. there are some training data which are not having the correct label which is affecting the quality of our model. if we device a better algorithm to filter these samples out of the training data, the final model trained on this filtered data will be much better ." }, { "id": 937098, "postDate": "2020-07-20T18:08:30.377Z", "content": "Ah yes I remember to have read it in the description but I had no time and no idea how to address it, thanks for the clarification. I'm excited if this secret gets revealed.
", "rawMarkdown": "Ah yes I remember to have read it in the description but I had no time and no idea how to address it, thanks for the clarification. I'm excited if this secret gets revealed. ", "votes": 1 } ] } ], "comments": [ { "id": 939160, "author_name": "seefun", "author_url": "", "post_date": "2020-07-22T04:15:07.063000", "content": "Although I wanted to try many methods, I was too weak in computing resource to deal with the noise label.
", "votes": 1, "replies": [ { "id": 939212, "author_name": "yuvaramsingh", "author_url": "", "post_date": "2020-07-22T05:20:55.897000", "content": "can you share some of those methods so it will help others to use them in next competitions .
", "votes": 0, "replies": [] }, { "id": 939222, "author_name": "seefun", "author_url": "", "post_date": "2020-07-22T05:25:23.607000", "content": "https://www.kaggle.com/c/prostate-cancer-grade-assessment/discussion/168833
", "votes": 1, "replies": [] }, { "id": 939843, "author_name": "yuvaramsingh", "author_url": "", "post_date": "2020-07-22T14:10:51.593000", "content": "thanks
", "votes": 0, "replies": [] } ] }, { "id": 936794, "author_name": "Andreas Horlbeck", "author_url": "", "post_date": "2020-07-20T14:28:35.227000", "content": "interesting question. What do you exactly mean with noisy? I have noticed that the images have differently colors,\nmostly some variation of red and purple. I have always asked myself during this competition if reducing to graysacle might help. I used samplewised normalizing to hopefully adress without using grayscale. My problem is, that I have a high hould out score, about 0.91, but but only 0.55 on the LB (in my opinion I didtn overfit to my holdout set.)
", "votes": 0, "replies": [ { "id": 936968, "author_name": "yuvaramsingh", "author_url": "", "post_date": "2020-07-20T16:20:54.643000", "content": "by noisy, i refer to the label. there are some training data which are not having the correct label which is affecting the quality of our model. if we device a better algorithm to filter these samples out of the training data, the final model trained on this filtered data will be much better .
", "votes": 0, "replies": [] }, { "id": 937098, "author_name": "Andreas Horlbeck", "author_url": "", "post_date": "2020-07-20T18:08:30.377000", "content": "Ah yes I remember to have read it in the description but I had no time and no idea how to address it, thanks for the clarification. I'm excited if this secret gets revealed.
", "votes": 1, "replies": [] } ] } ], "raw_markdown_by_id": { "939160": "Although I wanted to try many methods, I was too weak in computing resource to deal with the noise label.", "936766": "i know with only 2 days left i cant improve my training process anymore. but, i am too curious to know how to handle noisy data. most of the top submissions are handling this noisy label problem in one form or another. it will be helpful if some of those ideas can be shared here. \n\nlooking forward to learn this and implement on my next competition. :D ", "936794": "interesting question. What do you exactly mean with noisy? I have noticed that the images have differently colors,\nmostly some variation of red and purple. I have always asked myself during this competition if reducing to graysacle might help. I used samplewised normalizing to hopefully adress without using grayscale. My problem is, that I have a high hould out score, about 0.91, but but only 0.55 on the LB (in my opinion I didtn overfit to my holdout set.)" } }