To get beyond LB0.93 , I guess I need to increase image size. But large size image takes really long :( \nIs it necessary to use equal or larger than 224x224 size to get more accuracy?
", "messages": [ { "id": 427920, "postDate": "2018-11-26T11:54:09.057Z", "content": "To get beyond LB0.93 , I guess I need to increase image size. But large size image takes really long :( \nIs it necessary to use equal or larger than 224x224 size to get more accuracy?
", "rawMarkdown": "To get beyond LB0.93 , I guess I need to increase image size. But large size image takes really long :( \nIs it necessary to use equal or larger than 224x224 size to get more accuracy?", "votes": 3 }, { "id": 428247, "postDate": "2018-11-27T01:19:05.673Z", "content": "I used smaller image sizes at first 64px then continue fine-tuning at 128px. I don't think you need to increase the image size to get beyond LB 0.93, instead focus on bigger batch sizes and work on a good training lr schedule.
", "rawMarkdown": "I used smaller image sizes at first 64px then continue fine-tuning at 128px. I don't think you need to increase the image size to get beyond LB 0.93, instead focus on bigger batch sizes and work on a good training lr schedule.", "votes": 1, "replies": [ { "id": 428568, "postDate": "2018-11-27T13:52:03.863Z", "content": "Is fine tunning better than training 128size from beginning?\nAny loss of accuracy?
", "rawMarkdown": "Is fine tunning better than training 128size from beginning?\nAny loss of accuracy? " }, { "id": 428687, "postDate": "2018-11-27T18:06:39.577Z", "content": "No loss in accuracy. From my experiments its faster to train this way overall and usually the model will be less prone to overfitting after re-sizing to 128px.
", "rawMarkdown": "No loss in accuracy. From my experiments its faster to train this way overall and usually the model will be less prone to overfitting after re-sizing to 128px." }, { "id": 428689, "postDate": "2018-11-27T18:09:30.280Z", "content": "You mean you train first using 64px then fine-tune it to 128px. question, how many epochs are you using and its batch size for 64px?
", "rawMarkdown": "You mean you train first using 64px then fine-tune it to 128px. question, how many epochs are you using and its batch size for 64px?", "votes": 10 }, { "id": 429509, "postDate": "2018-11-29T00:33:31.337Z", "content": "Thanks for sharing your experience!
", "rawMarkdown": "Thanks for sharing your experience! " } ] }, { "id": 427979, "postDate": "2018-11-26T14:17:21.417Z", "content": "I was able to use 128x128 to get 94.3 and I presume others may have gotten higher on similar image size. What are size are you using at the moment?
", "rawMarkdown": "I was able to use 128x128 to get 94.3 and I presume others may have gotten higher on similar image size. What are size are you using at the moment?", "votes": 1, "replies": [ { "id": 427985, "postDate": "2018-11-26T14:27:25.467Z", "content": "what kind of NN are you using to achieve 0.943? is it a single model or ensemble?
", "rawMarkdown": "what kind of NN are you using to achieve 0.943? is it a single model or ensemble?", "votes": 1 }, { "id": 428209, "postDate": "2018-11-26T23:27:14.790Z", "content": "A single Xception model obtains that score - nothing really fancy actually (as already mentioned by Heng in his other post).
", "rawMarkdown": "A single Xception model obtains that score - nothing really fancy actually (as already mentioned by Heng in his other post)." }, { "id": 428216, "postDate": "2018-11-26T23:40:07.957Z", "content": "I used 64 or 96size. If 128x128 size is sufficent, I will try it first. Thanks!
", "rawMarkdown": "I used 64 or 96size. If 128x128 size is sufficent, I will try it first. Thanks!" }, { "id": 428609, "postDate": "2018-11-27T15:19:10.960Z", "content": "@wilanw may I ask how long does it take for your Xception to complete 1 epoch of 50M training data? Because I use Xception with 128x128 too, but my model is super slow (i.e. 1 week+ for 50M data)
", "rawMarkdown": "@wilanw may I ask how long does it take for your Xception to complete 1 epoch of 50M training data? Because I use Xception with 128x128 too, but my model is super slow (i.e. 1 week+ for 50M data)", "votes": 1 }, { "id": 428660, "postDate": "2018-11-27T17:10:54.680Z", "content": "is your Xception model based on Pytorch or keras?
", "rawMarkdown": "is your Xception model based on Pytorch or keras?", "votes": 1 }, { "id": 428858, "postDate": "2018-11-28T01:34:20.853Z", "content": "It's based on Keras - using the pretrained imagenet weights as well.
", "rawMarkdown": "It's based on Keras - using the pretrained imagenet weights as well. ", "votes": 1 }, { "id": 428974, "postDate": "2018-11-28T06:52:19.673Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 428220, "postDate": "2018-11-26T23:52:02.043Z", "content": "Try to train on 64, then keep finetuning on larger images. The image size things are a kind of marginal benefit.\n:D
", "rawMarkdown": "Try to train on 64, then keep finetuning on larger images. The image size things are a kind of marginal benefit.\n:D", "votes": 2, "replies": [ { "id": 428239, "postDate": "2018-11-27T00:59:59.360Z", "content": "How do you change the image size for fine tuning?
", "rawMarkdown": "How do you change the image size for fine tuning?" }, { "id": 428571, "postDate": "2018-11-27T13:57:13.503Z", "content": "I’m wonder whether fine tunning have any loss of accuracy or not
", "rawMarkdown": "I’m wonder whether fine tunning have any loss of accuracy or not" }, { "id": 428648, "postDate": "2018-11-27T16:54:55.970Z", "content": "64 --> 96 --> 128 --> larger...
", "rawMarkdown": "64 --> 96 --> 128 --> larger..." } ] }, { "id": 431870, "postDate": "2018-12-03T03:35:01.607Z", "content": "not necessary for 0.93 I think. For my previous experiment, 25k/class training samples + 96*96size, no tta, no augmentation can get 0.935
", "rawMarkdown": "not necessary for 0.93 I think. For my previous experiment, 25k/class training samples + 96*96size, no tta, no augmentation can get 0.935" }, { "id": 430719, "postDate": "2018-11-30T22:06:28.110Z", "content": "When I resized to bigger image size, I just never could recover and beat the previous score.
", "rawMarkdown": "When I resized to bigger image size, I just never could recover and beat the previous score. " }, { "id": 428815, "postDate": "2018-11-27T23:50:04.433Z", "content": "When I finetuned my already pretty good model trained on image size 128*128 with image size 224*224, I got this strange phenomenon. I tried various learning rates but the finetuned model couldn't recover the previous score or improve it.
", "rawMarkdown": "When I finetuned my already pretty good model trained on image size 128*128 with image size 224*224, I got this strange phenomenon. I tried various learning rates but the finetuned model couldn't recover the previous score or improve it. ", "replies": [ { "id": 428842, "postDate": "2018-11-28T00:47:02.227Z", "content": "@HuyenNguyen, I had the same problem when I went from 80x80 image size to 128x128 but trained the 2nd model from scratch, for longer, and more data. So I am not sure what the issue is. Like you I changed my learning rate, batch size etc. but did not help still got lower score than the 80x80.
", "rawMarkdown": "@HuyenNguyen, I had the same problem when I went from 80x80 image size to 128x128 but trained the 2nd model from scratch, for longer, and more data. So I am not sure what the issue is. Like you I changed my learning rate, batch size etc. but did not help still got lower score than the 80x80.", "votes": 1 }, { "id": 429511, "postDate": "2018-11-29T00:36:56.163Z", "content": "Oh , I confused.... did you check optimizer or lr schedule?
", "rawMarkdown": "Oh , I confused.... did you check optimizer or lr schedule? " }, { "id": 430640, "postDate": "2018-11-30T19:11:34.397Z", "content": "224*224 is too big for such huge dataset... I wonder how you guys manage to train with this size in reasonnable amount of time.
", "rawMarkdown": "224*224 is too big for such huge dataset... I wonder how you guys manage to train with this size in reasonnable amount of time. " }, { "id": 431859, "postDate": "2018-12-03T02:53:52.230Z", "content": "omg.. I also have same problem when I finetunned 128x128 image from 96x96 images.
", "rawMarkdown": "omg.. I also have same problem when I finetunned 128x128 image from 96x96 images. " } ] }, { "id": 428243, "postDate": "2018-11-27T01:05:51.450Z", "content": "I used a variety of image sizes, including 96, 128, 144 and 196. At this point my model performance increases with size, but my models haven't converged yet for size 196.
", "rawMarkdown": "I used a variety of image sizes, including 96, 128, 144 and 196. At this point my model performance increases with size, but my models haven't converged yet for size 196.", "replies": [ { "id": 428373, "postDate": "2018-11-27T06:58:10.473Z", "content": "When you use larger image size, do you reduce the learning rate?
", "rawMarkdown": "When you use larger image size, do you reduce the learning rate? " }, { "id": 428570, "postDate": "2018-11-27T13:53:58.483Z", "content": "How many times did you need to get the final result of 144 size?
", "rawMarkdown": "How many times did you need to get the final result of 144 size?" } ] }, { "id": 427926, "postDate": "2018-11-26T12:04:52.413Z", "content": "I used 128x128 so far. I am thinking about increasing the input size too...
", "rawMarkdown": "I used 128x128 so far. I am thinking about increasing the input size too...", "replies": [ { "id": 428215, "postDate": "2018-11-26T23:38:57.763Z", "content": "Until now I used 64 or 96 size. Thanks for you comment!
", "rawMarkdown": "Until now I used 64 or 96 size. Thanks for you comment! " } ] }, { "id": 430215, "postDate": "2018-11-30T03:12:00.027Z", "rawMarkdown": "", "votes": 2, "isDeleted": true } ], "comments": [ { "id": 428247, "author_name": "James Requa", "author_url": "", "post_date": "2018-11-27T01:19:05.673000", "content": "I used smaller image sizes at first 64px then continue fine-tuning at 128px. I don't think you need to increase the image size to get beyond LB 0.93, instead focus on bigger batch sizes and work on a good training lr schedule.
", "votes": 1, "replies": [ { "id": 428568, "author_name": "hahaha", "author_url": "", "post_date": "2018-11-27T13:52:03.863000", "content": "Is fine tunning better than training 128size from beginning?\nAny loss of accuracy?
", "votes": 0, "replies": [] }, { "id": 428687, "author_name": "James Requa", "author_url": "", "post_date": "2018-11-27T18:06:39.577000", "content": "No loss in accuracy. From my experiments its faster to train this way overall and usually the model will be less prone to overfitting after re-sizing to 128px.
", "votes": 0, "replies": [] }, { "id": 428689, "author_name": "", "author_url": "", "post_date": "2018-11-27T18:09:30.280000", "content": "You mean you train first using 64px then fine-tune it to 128px. question, how many epochs are you using and its batch size for 64px?
", "votes": 10, "replies": [] }, { "id": 429509, "author_name": "hahaha", "author_url": "", "post_date": "2018-11-29T00:33:31.337000", "content": "Thanks for sharing your experience!
", "votes": 0, "replies": [] } ] }, { "id": 427979, "author_name": "wilan", "author_url": "", "post_date": "2018-11-26T14:17:21.417000", "content": "I was able to use 128x128 to get 94.3 and I presume others may have gotten higher on similar image size. What are size are you using at the moment?
", "votes": 1, "replies": [ { "id": 427985, "author_name": "", "author_url": "", "post_date": "2018-11-26T14:27:25.467000", "content": "what kind of NN are you using to achieve 0.943? is it a single model or ensemble?
", "votes": 1, "replies": [] }, { "id": 428209, "author_name": "wilan", "author_url": "", "post_date": "2018-11-26T23:27:14.790000", "content": "A single Xception model obtains that score - nothing really fancy actually (as already mentioned by Heng in his other post).
", "votes": 0, "replies": [] }, { "id": 428216, "author_name": "hahaha", "author_url": "", "post_date": "2018-11-26T23:40:07.957000", "content": "I used 64 or 96size. If 128x128 size is sufficent, I will try it first. Thanks!
", "votes": 0, "replies": [] }, { "id": 428609, "author_name": "Neuron Engineer", "author_url": "", "post_date": "2018-11-27T15:19:10.960000", "content": "@wilanw may I ask how long does it take for your Xception to complete 1 epoch of 50M training data? Because I use Xception with 128x128 too, but my model is super slow (i.e. 1 week+ for 50M data)
", "votes": 1, "replies": [] }, { "id": 428660, "author_name": "", "author_url": "", "post_date": "2018-11-27T17:10:54.680000", "content": "is your Xception model based on Pytorch or keras?
", "votes": 1, "replies": [] }, { "id": 428858, "author_name": "wilan", "author_url": "", "post_date": "2018-11-28T01:34:20.853000", "content": "It's based on Keras - using the pretrained imagenet weights as well.
", "votes": 1, "replies": [] }, { "id": 428974, "author_name": "", "author_url": "", "post_date": "2018-11-28T06:52:19.673000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 428220, "author_name": "Kulbear", "author_url": "", "post_date": "2018-11-26T23:52:02.043000", "content": "Try to train on 64, then keep finetuning on larger images. The image size things are a kind of marginal benefit.\n:D
", "votes": 2, "replies": [ { "id": 428239, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2018-11-27T00:59:59.360000", "content": "How do you change the image size for fine tuning?
", "votes": 0, "replies": [] }, { "id": 428571, "author_name": "hahaha", "author_url": "", "post_date": "2018-11-27T13:57:13.503000", "content": "I’m wonder whether fine tunning have any loss of accuracy or not
", "votes": 0, "replies": [] }, { "id": 428648, "author_name": "Kulbear", "author_url": "", "post_date": "2018-11-27T16:54:55.970000", "content": "64 --> 96 --> 128 --> larger...
", "votes": 0, "replies": [] } ] }, { "id": 431870, "author_name": "Yiheng Wang", "author_url": "", "post_date": "2018-12-03T03:35:01.607000", "content": "not necessary for 0.93 I think. For my previous experiment, 25k/class training samples + 96*96size, no tta, no augmentation can get 0.935
", "votes": 0, "replies": [] }, { "id": 430719, "author_name": "HuyenNguyen", "author_url": "", "post_date": "2018-11-30T22:06:28.110000", "content": "When I resized to bigger image size, I just never could recover and beat the previous score.
", "votes": 0, "replies": [] }, { "id": 428815, "author_name": "HuyenNguyen", "author_url": "", "post_date": "2018-11-27T23:50:04.433000", "content": "When I finetuned my already pretty good model trained on image size 128*128 with image size 224*224, I got this strange phenomenon. I tried various learning rates but the finetuned model couldn't recover the previous score or improve it.
", "votes": 0, "replies": [ { "id": 428842, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2018-11-28T00:47:02.227000", "content": "@HuyenNguyen, I had the same problem when I went from 80x80 image size to 128x128 but trained the 2nd model from scratch, for longer, and more data. So I am not sure what the issue is. Like you I changed my learning rate, batch size etc. but did not help still got lower score than the 80x80.
", "votes": 1, "replies": [] }, { "id": 429511, "author_name": "hahaha", "author_url": "", "post_date": "2018-11-29T00:36:56.163000", "content": "Oh , I confused.... did you check optimizer or lr schedule?
", "votes": 0, "replies": [] }, { "id": 430640, "author_name": "Serigne ", "author_url": "", "post_date": "2018-11-30T19:11:34.397000", "content": "224*224 is too big for such huge dataset... I wonder how you guys manage to train with this size in reasonnable amount of time.
", "votes": 0, "replies": [] }, { "id": 431859, "author_name": "hahaha", "author_url": "", "post_date": "2018-12-03T02:53:52.230000", "content": "omg.. I also have same problem when I finetunned 128x128 image from 96x96 images.
", "votes": 0, "replies": [] } ] }, { "id": 428243, "author_name": "Peiyuan Liao", "author_url": "", "post_date": "2018-11-27T01:05:51.450000", "content": "I used a variety of image sizes, including 96, 128, 144 and 196. At this point my model performance increases with size, but my models haven't converged yet for size 196.
", "votes": 0, "replies": [ { "id": 428373, "author_name": "HuyenNguyen", "author_url": "", "post_date": "2018-11-27T06:58:10.473000", "content": "When you use larger image size, do you reduce the learning rate?
", "votes": 0, "replies": [] }, { "id": 428570, "author_name": "hahaha", "author_url": "", "post_date": "2018-11-27T13:53:58.483000", "content": "How many times did you need to get the final result of 144 size?
", "votes": 0, "replies": [] } ] }, { "id": 427926, "author_name": "beluga", "author_url": "", "post_date": "2018-11-26T12:04:52.413000", "content": "I used 128x128 so far. I am thinking about increasing the input size too...
", "votes": 0, "replies": [ { "id": 428215, "author_name": "hahaha", "author_url": "", "post_date": "2018-11-26T23:38:57.763000", "content": "Until now I used 64 or 96 size. Thanks for you comment!
", "votes": 0, "replies": [] } ] }, { "id": 430215, "author_name": "", "author_url": "", "post_date": "2018-11-30T03:12:00.027000", "content": "", "votes": 2, "replies": [] } ], "raw_markdown_by_id": { "427920": "To get beyond LB0.93 , I guess I need to increase image size. But large size image takes really long :( \nIs it necessary to use equal or larger than 224x224 size to get more accuracy?", "428247": "I used smaller image sizes at first 64px then continue fine-tuning at 128px. I don't think you need to increase the image size to get beyond LB 0.93, instead focus on bigger batch sizes and work on a good training lr schedule.", "427979": "I was able to use 128x128 to get 94.3 and I presume others may have gotten higher on similar image size. What are size are you using at the moment?", "428220": "Try to train on 64, then keep finetuning on larger images. The image size things are a kind of marginal benefit.\n:D", "431870": "not necessary for 0.93 I think. For my previous experiment, 25k/class training samples + 96*96size, no tta, no augmentation can get 0.935", "430719": "When I resized to bigger image size, I just never could recover and beat the previous score. ", "428815": "When I finetuned my already pretty good model trained on image size 128*128 with image size 224*224, I got this strange phenomenon. I tried various learning rates but the finetuned model couldn't recover the previous score or improve it. ", "428243": "I used a variety of image sizes, including 96, 128, 144 and 196. At this point my model performance increases with size, but my models haven't converged yet for size 196.", "427926": "I used 128x128 so far. I am thinking about increasing the input size too...", "430215": "" } }