I wanted to share that I have been training models with image size 512 and 1024, what I found interesting is the following:
\nFor example if we have backbone A and backbone B and train each backbone with the same hyperparammeters for image size 512, backbone A has a better CV compared to B.
\nWhat is interesting is that when I do the same for image size 1024, backbone B has a much better CV compared to A.
\nThe main idea was to train with smaller image size and then if I have good results I can use bigger image size, nevertheless this experiment suggest that results from using image size 512 will not always have the same behaviour for image size 1024.
\nFor experts in computer vision this could not be an amazing discovery but for the rest that are learning I found it was interesting to share this
", "messages": [ { "id": 2067491, "postDate": "2022-12-16T19:13:08.017Z", "content": "I wanted to share that I have been training models with image size 512 and 1024, what I found interesting is the following:
\nFor example if we have backbone A and backbone B and train each backbone with the same hyperparammeters for image size 512, backbone A has a better CV compared to B.
\nWhat is interesting is that when I do the same for image size 1024, backbone B has a much better CV compared to A.
\nThe main idea was to train with smaller image size and then if I have good results I can use bigger image size, nevertheless this experiment suggest that results from using image size 512 will not always have the same behaviour for image size 1024.
\nFor experts in computer vision this could not be an amazing discovery but for the rest that are learning I found it was interesting to share this
", "rawMarkdown": "I wanted to share that I have been training models with image size 512 and 1024, what I found interesting is the following:\n\nFor example if we have backbone A and backbone B and train each backbone with the same hyperparammeters for image size 512, backbone A has a better CV compared to B.\n\nWhat is interesting is that when I do the same for image size 1024, backbone B has a much better CV compared to A.\n\nThe main idea was to train with smaller image size and then if I have good results I can use bigger image size, nevertheless this experiment suggest that results from using image size 512 will not always have the same behaviour for image size 1024.\n\nFor experts in computer vision this could not be an amazing discovery but for the rest that are learning I found it was interesting to share this", "votes": 17 }, { "id": 2067508, "postDate": "2022-12-16T19:30:34.720Z", "content": "this is a common observation in this competition data. Even for one single backbone, different oversampling strategy gives non-similar results for different size.
\nI think the reasons could be:
\noversampling, pos weight can easily leads to over fitting to a particular one pos sample.
\nif you want to visualise learned feature space you can write code to do playground like this https://playground.tensorflow.org.
\nThen change the loss function to focal loss, (heavily) weighted BCE, etc for different data of pos to background ratio. you can see That the learned feature space can changed drastically for different samples
on a side note, i think some kagglers may observed that some models are stuck are learning to predict only background class. This is when you have only cv in the range of 0.05
", "rawMarkdown": "this is a common observation in this competition data. Even for one single backbone, different oversampling strategy gives non-similar results for different size. \n\nI think the reasons could be:\n1. too little pos data to make the cv metric (and lb) reliable in the first place \n2. you may think we are having a 2 class binary problem, but it is not true. it is one class vs background (not a cat versus dog problem but cat versus random images problem)\nwe only have 54705*0.02=1094 pos images.\n\noversampling, pos weight can easily leads to over fitting to a particular one pos sample.\n\n---\n\nif you want to visualise learned feature space you can write code to do playground like this https://playground.tensorflow.org.\nThen change the loss function to focal loss, (heavily) weighted BCE, etc for different data of pos to background ratio. you can see That the learned feature space can changed drastically for different samples\n\n--- \non a side note, i think some kagglers may observed that some models are stuck are learning to predict only background class. This is when you have only cv in the range of 0.05", "votes": 7, "replies": [ { "id": 2068223, "postDate": "2022-12-17T16:19:17.247Z", "content": "\n\ni think some kagglers may observed that some models are stuck are learning to predict only background class. This is when you have only cv in the range of 0.05
\n
š This is exactly me. I have accounted for the imbalance in my loss function, but am still stuck in the 0.05 range. Do you have any suggestions?
\nThanks!
", "rawMarkdown": ">i think some kagglers may observed that some models are stuck are learning to predict only background class. This is when you have only cv in the range of 0.05\n\nš This is exactly me. I have accounted for the imbalance in my loss function, but am still stuck in the 0.05 range. Do you have any suggestions?\n\nThanks!", "replies": [ { "id": 2068377, "postDate": "2022-12-17T20:06:12.637Z", "content": "efficentnetb2/3/4 is one that not sensitive to the kaggle dataset here.
\nuse batch with pos/to neg in the range of 1/32 to 1/16. and rate from 1e-3, deccrease to 1-5
for larger model (e.g. >0.85 top-1 in imagenet), you still can get it to train by lowering the learning rate (initial rate = 1e-4) or use pos/to neg in range of 1/2 to 1/8. But even if these can achieve good CV (say in the range of 0.40), they may perform poorly in LB(only in range of 0.35). A better solution for large model is the use of segmentation as aux loss (from external data or pesudo mask from small model)
", "rawMarkdown": "efficentnetb2/3/4 is one that not sensitive to the kaggle dataset here.\nuse batch with pos/to neg in the range of 1/32 to 1/16. and rate from 1e-3, deccrease to 1-5\n\nfor larger model (e.g. >0.85 top-1 in imagenet), you still can get it to train by lowering the learning rate (initial rate = 1e-4) or use pos/to neg in range of 1/2 to 1/8. But even if these can achieve good CV (say in the range of 0.40), they may perform poorly in LB(only in range of 0.35). A better solution for large model is the use of segmentation as aux loss (from external data or pesudo mask from small model)\n", "votes": 7 }, { "id": 2068378, "postDate": "2022-12-17T20:09:53.953Z", "content": "That's very helpful, thank you for sharing your thoughts! š
", "rawMarkdown": "That's very helpful, thank you for sharing your thoughts! š" } ] } ] } ], "comments": [ { "id": 2067508, "author_name": "hengck23", "author_url": "", "post_date": "2022-12-16T19:30:34.720000", "content": "this is a common observation in this competition data. Even for one single backbone, different oversampling strategy gives non-similar results for different size.
\nI think the reasons could be:
\noversampling, pos weight can easily leads to over fitting to a particular one pos sample.
\nif you want to visualise learned feature space you can write code to do playground like this https://playground.tensorflow.org.
\nThen change the loss function to focal loss, (heavily) weighted BCE, etc for different data of pos to background ratio. you can see That the learned feature space can changed drastically for different samples
on a side note, i think some kagglers may observed that some models are stuck are learning to predict only background class. This is when you have only cv in the range of 0.05
", "votes": 7, "replies": [ { "id": 2068223, "author_name": "Julia Nickerson", "author_url": "", "post_date": "2022-12-17T16:19:17.247000", "content": "\n\ni think some kagglers may observed that some models are stuck are learning to predict only background class. This is when you have only cv in the range of 0.05
\n
š This is exactly me. I have accounted for the imbalance in my loss function, but am still stuck in the 0.05 range. Do you have any suggestions?
\nThanks!
", "votes": 0, "replies": [ { "id": 2068377, "author_name": "hengck23", "author_url": "", "post_date": "2022-12-17T20:06:12.637000", "content": "efficentnetb2/3/4 is one that not sensitive to the kaggle dataset here.
\nuse batch with pos/to neg in the range of 1/32 to 1/16. and rate from 1e-3, deccrease to 1-5
for larger model (e.g. >0.85 top-1 in imagenet), you still can get it to train by lowering the learning rate (initial rate = 1e-4) or use pos/to neg in range of 1/2 to 1/8. But even if these can achieve good CV (say in the range of 0.40), they may perform poorly in LB(only in range of 0.35). A better solution for large model is the use of segmentation as aux loss (from external data or pesudo mask from small model)
", "votes": 7, "replies": [] }, { "id": 2068378, "author_name": "Julia Nickerson", "author_url": "", "post_date": "2022-12-17T20:09:53.953000", "content": "That's very helpful, thank you for sharing your thoughts! š
", "votes": 0, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2067491": "I wanted to share that I have been training models with image size 512 and 1024, what I found interesting is the following:\n\nFor example if we have backbone A and backbone B and train each backbone with the same hyperparammeters for image size 512, backbone A has a better CV compared to B.\n\nWhat is interesting is that when I do the same for image size 1024, backbone B has a much better CV compared to A.\n\nThe main idea was to train with smaller image size and then if I have good results I can use bigger image size, nevertheless this experiment suggest that results from using image size 512 will not always have the same behaviour for image size 1024.\n\nFor experts in computer vision this could not be an amazing discovery but for the rest that are learning I found it was interesting to share this", "2067508": "this is a common observation in this competition data. Even for one single backbone, different oversampling strategy gives non-similar results for different size. \n\nI think the reasons could be:\n1. too little pos data to make the cv metric (and lb) reliable in the first place \n2. you may think we are having a 2 class binary problem, but it is not true. it is one class vs background (not a cat versus dog problem but cat versus random images problem)\nwe only have 54705*0.02=1094 pos images.\n\noversampling, pos weight can easily leads to over fitting to a particular one pos sample.\n\n---\n\nif you want to visualise learned feature space you can write code to do playground like this https://playground.tensorflow.org.\nThen change the loss function to focal loss, (heavily) weighted BCE, etc for different data of pos to background ratio. you can see That the learned feature space can changed drastically for different samples\n\n--- \non a side note, i think some kagglers may observed that some models are stuck are learning to predict only background class. This is when you have only cv in the range of 0.05" } }