Just wondering how much improvement do people observe from pre-training on external dataset & transfer learning ?
\n** My disappointing experiment **
\nI tried to do pretraining on 50% of the 30K images from rsna-pneumonia-detection-challenge. ( This dataset has about 10K bounding boxes, and 2 class - normal and abnormal ) My Yolo model was able to achieve good result on pre-training task ( mAP 0.5 = 0.70 )
\nThen, I applied the pretrained weight on the current comp dataset. I notice that in the early epochs ( epoch < 10) the transfer learning weight outperforms the default Imagenet pretrained weight but after that (from 10th epoch until the end ) the local CV actually looks worse.
\nOn LB, model with transfer learning only outpeforms imagenet baseline by 0.001 ( basically, external data has no effect under my current setup). I am very disappointed, considering that the external data is very similar to the current comp dataset except that it has less classes.
", "messages": [ { "id": 1236881, "postDate": "2021-03-13T14:35:06.470Z", "content": "Just wondering how much improvement do people observe from pre-training on external dataset & transfer learning ?
\n** My disappointing experiment **
\nI tried to do pretraining on 50% of the 30K images from rsna-pneumonia-detection-challenge. ( This dataset has about 10K bounding boxes, and 2 class - normal and abnormal ) My Yolo model was able to achieve good result on pre-training task ( mAP 0.5 = 0.70 )
\nThen, I applied the pretrained weight on the current comp dataset. I notice that in the early epochs ( epoch < 10) the transfer learning weight outperforms the default Imagenet pretrained weight but after that (from 10th epoch until the end ) the local CV actually looks worse.
\nOn LB, model with transfer learning only outpeforms imagenet baseline by 0.001 ( basically, external data has no effect under my current setup). I am very disappointed, considering that the external data is very similar to the current comp dataset except that it has less classes.
", "rawMarkdown": "Just wondering how much improvement do people observe from pre-training on external dataset & transfer learning ? \n\n\n** My disappointing experiment **\n\n\n\nI tried to do pretraining on 50% of the 30K images from [ rsna-pneumonia-detection-challenge](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge). ( This dataset has about 10K bounding boxes, and 2 class - normal and abnormal ) My Yolo model was able to achieve good result on pre-training task ( mAP 0.5 = 0.70 )\n\nThen, I applied the pretrained weight on the current comp dataset. I notice that in the early epochs ( epoch < 10) the transfer learning weight outperforms the default Imagenet pretrained weight but after that (from 10th epoch until the end ) the local CV actually looks worse. \n\nOn LB, model with transfer learning only outpeforms imagenet baseline by 0.001 ( basically, external data has no effect under my current setup). I am very disappointed, considering that the external data is very similar to the current comp dataset except that it has less classes.\n\n\n\n\n", "votes": 11 }, { "id": 1238463, "postDate": "2021-03-15T01:40:14.637Z", "content": "I felt the same, bro. We'll find another way to utilize it
", "rawMarkdown": "I felt the same, bro. We'll find another way to utilize it", "replies": [ { "id": 1241283, "postDate": "2021-03-17T01:27:46.267Z", "content": "\nAny further luck down this path ?
\nI saw your other post regarding weakly labelled detection. Instead of learning localization without bboxes label, have you thought about / tried training the detection network's CNN backbone weight on classification task (and then load it back for detection use ) ?
", "rawMarkdown": "@namgalielei \n\nAny further luck down this path ? \n\nI saw your other post regarding weakly labelled detection. Instead of learning localization without bboxes label, have you thought about / tried training the detection network's CNN backbone weight on classification task (and then load it back for detection use ) ?\n\n\n" }, { "id": 1241440, "postDate": "2021-03-17T03:31:20.443Z", "content": "Yeah I am currently getting a hand on it. The very first problem to solve is to unify the label set on classification task before any training is conducted. The second one is that training on external data takes a huge amount of time
", "rawMarkdown": "Yeah I am currently getting a hand on it. The very first problem to solve is to unify the label set on classification task before any training is conducted. The second one is that training on external data takes a huge amount of time " } ] } ], "comments": [ { "id": 1238463, "author_name": "Liam Nguyen", "author_url": "", "post_date": "2021-03-15T01:40:14.637000", "content": "I felt the same, bro. We'll find another way to utilize it
", "votes": 0, "replies": [ { "id": 1241283, "author_name": "NakedKoala", "author_url": "", "post_date": "2021-03-17T01:27:46.267000", "content": "\nAny further luck down this path ?
\nI saw your other post regarding weakly labelled detection. Instead of learning localization without bboxes label, have you thought about / tried training the detection network's CNN backbone weight on classification task (and then load it back for detection use ) ?
", "votes": 0, "replies": [] }, { "id": 1241440, "author_name": "Liam Nguyen", "author_url": "", "post_date": "2021-03-17T03:31:20.443000", "content": "Yeah I am currently getting a hand on it. The very first problem to solve is to unify the label set on classification task before any training is conducted. The second one is that training on external data takes a huge amount of time
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "1236881": "Just wondering how much improvement do people observe from pre-training on external dataset & transfer learning ? \n\n\n** My disappointing experiment **\n\n\n\nI tried to do pretraining on 50% of the 30K images from [ rsna-pneumonia-detection-challenge](https://www.kaggle.com/c/rsna-pneumonia-detection-challenge). ( This dataset has about 10K bounding boxes, and 2 class - normal and abnormal ) My Yolo model was able to achieve good result on pre-training task ( mAP 0.5 = 0.70 )\n\nThen, I applied the pretrained weight on the current comp dataset. I notice that in the early epochs ( epoch < 10) the transfer learning weight outperforms the default Imagenet pretrained weight but after that (from 10th epoch until the end ) the local CV actually looks worse. \n\nOn LB, model with transfer learning only outpeforms imagenet baseline by 0.001 ( basically, external data has no effect under my current setup). I am very disappointed, considering that the external data is very similar to the current comp dataset except that it has less classes.\n\n\n\n\n", "1238463": "I felt the same, bro. We'll find another way to utilize it" } }