We are in the final days of the competition, and inference is becoming one of the most important factors. I've been \"brute forcing\" augmentations to see which perform better, but it's painstaking, and I have no clue which ones would be expected to be better or worse since I have no domain knowledge.
\nWhich test augmentations are the best for TTA? Would crop and cutout be reasonable for inference, or only hue/brightness/flips? Looking forwards to hearing from everyone. Thanks!
", "messages": [ { "id": 1056708, "postDate": "2020-10-22T02:13:39.523Z", "content": "We are in the final days of the competition, and inference is becoming one of the most important factors. I've been \"brute forcing\" augmentations to see which perform better, but it's painstaking, and I have no clue which ones would be expected to be better or worse since I have no domain knowledge.
\nWhich test augmentations are the best for TTA? Would crop and cutout be reasonable for inference, or only hue/brightness/flips? Looking forwards to hearing from everyone. Thanks!
", "rawMarkdown": "We are in the final days of the competition, and inference is becoming one of the most important factors. I've been \"brute forcing\" augmentations to see which perform better, but it's painstaking, and I have no clue which ones would be expected to be better or worse since I have no domain knowledge.\n\nWhich test augmentations are the best for TTA? Would crop and cutout be reasonable for inference, or only hue/brightness/flips? Looking forwards to hearing from everyone. Thanks!", "votes": 9 }, { "id": 1057434, "postDate": "2020-10-22T16:31:52.570Z", "content": "the scans we're dealing with are not the typical real-life images we are familiar with. Increase the brightness and contrast of an image of a dog and you still have a dog. Increase the brightness of lung tissue and it becomes bone tissue. Flip the image and the RV becomes the LV!!
\nAugmentation is all about changing the data point slightly so that the label doesn't change. Therefore, I would think that the best kind of augmentation is the one that mimics the artefacts produced when performing the scan. Maybe a slight rotation or some random noise. Other than that I'm not really sure you can do augmentation at all.
You're right, it's not so simple as in, for example, flower classification task.
", "rawMarkdown": "You're right, it's not so simple as in, for example, flower classification task." } ] }, { "id": 1056771, "postDate": "2020-10-22T04:06:34.200Z", "content": "@stanleyjzheng For me, Rotate 90 didn't help at all, I’m not sure if rotate will help at all. As for the other flip and the likes We keep getting same score as the original one. Havnt tried others yet
", "rawMarkdown": "@stanleyjzheng For me, Rotate 90 didn't help at all, I’m not sure if rotate will help at all. As for the other flip and the likes We keep getting same score as the original one. Havnt tried others yet ", "votes": 4, "replies": [ { "id": 1056776, "postDate": "2020-10-22T04:14:14.337Z", "content": "Interesting - I'm currently running a few flips submissions, can't seem to get hue/contrast/brightness working for some reason (weird albumentations bug perhaps). Wondering if crop, cutout, etc. would make it worse too. I'm so lost at what augmentations would help
", "rawMarkdown": "Interesting - I'm currently running a few flips submissions, can't seem to get hue/contrast/brightness working for some reason (weird albumentations bug perhaps). Wondering if crop, cutout, etc. would make it worse too. I'm so lost at what augmentations would help", "votes": 3 } ] }, { "id": 1061308, "postDate": "2020-10-26T23:42:00.633Z", "content": "I used augmentation but with just small changes than normal, to reflect the different position and sizes, I added random stretch for X and Y axis and random rotate just within 5,10 & 20 degrees.
", "rawMarkdown": "I used augmentation but with just small changes than normal, to reflect the different position and sizes, I added random stretch for X and Y axis and random rotate just within 5,10 & 20 degrees. ", "votes": 1, "replies": [ { "id": 1061311, "postDate": "2020-10-26T23:44:39.747Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1061325, "postDate": "2020-10-27T00:10:11.857Z", "content": "Congratz to a great score and the bronze. Due to the the time limit I didn't use TTA to the Inference, used it instead to stacking many models and folds.The plan was some more models and longer training/epochs to the stacking, of course, with different setups, I came a bit on the road but not all the way to the goal.
", "rawMarkdown": "Congratz to a great score and the bronze. Due to the the time limit I didn't use TTA to the Inference, used it instead to stacking many models and folds.The plan was some more models and longer training/epochs to the stacking, of course, with different setups, I came a bit on the road but not all the way to the goal.", "votes": 1 } ] }, { "id": 1058495, "postDate": "2020-10-23T19:23:36.930Z", "content": "from last year's competition:
\nhttps://www.kaggle.com/c/rsna-intracranial-hemorrhage-detection/discussion/117293
Augmentation for CNN:
\nHeavy augmentation including crop and resize back, affine (360 degree rotation), contrast and brightness, gamma correction, blurring and sharpening, mirroring, optical distortion, grid distortion, elastic transform …
I don't think there is a time to do TTA in 9 hours.
", "rawMarkdown": "I don't think there is a time to do TTA in 9 hours.", "votes": 2, "replies": [ { "id": 1057492, "postDate": "2020-10-22T17:43:36.920Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1057521, "postDate": "2020-10-22T18:08:43.417Z", "content": "and I'm here just starting my stage one image-level model. Is recall of about .5-.3 normal early on in training? and for how many epochs should I train my model? and what does TTA mean?
", "rawMarkdown": "and I'm here just starting my stage one image-level model. Is recall of about .5-.3 normal early on in training? and for how many epochs should I train my model? and what does TTA mean?" }, { "id": 1057530, "postDate": "2020-10-22T18:19:29.270Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1057536, "postDate": "2020-10-22T18:24:17.460Z", "content": "so is it like you augment the test data make the coresponding prediction and then average the results? That doesn't sound like it improves your score much, lol.
", "rawMarkdown": "so is it like you augment the test data make the coresponding prediction and then average the results? That doesn't sound like it improves your score much, lol.", "votes": -2 } ] }, { "id": 1057051, "postDate": "2020-10-22T10:09:45.237Z", "content": "@stanleyjzheng I use Vertical and Horizontal Flip and it's work not really bad but in my eyes Resize and scaling would be useful (depends on your hardware resource), also check the contrast/brightness and results was good. Looking forwards to you if any ppl can advice new idea.
\nThank you
Odd that we get slightly different results with TTA - just tried a TTAx2 submission with vertical, horizontal, and 90deg flips, and got a worse score. Now I'm super confused….
", "rawMarkdown": "Odd that we get slightly different results with TTA - just tried a TTAx2 submission with vertical, horizontal, and 90deg flips, and got a worse score. Now I'm super confused....", "votes": 1 }, { "id": 1057371, "postDate": "2020-10-22T15:36:36.923Z", "content": "for me as well, flips did not help
\naffine helped me but improvements was marginal
the scans we're dealing with are not the typical real-life images we are familiar with. Increase the brightness and contrast of an image of a dog and you still have a dog. Increase the brightness of lung tissue and it becomes bone tissue. Flip the image and the RV becomes the LV!!
\nAugmentation is all about changing the data point slightly so that the label doesn't change. Therefore, I would think that the best kind of augmentation is the one that mimics the artefacts produced when performing the scan. Maybe a slight rotation or some random noise. Other than that I'm not really sure you can do augmentation at all.
You're right, it's not so simple as in, for example, flower classification task.
", "votes": 0, "replies": [] } ] }, { "id": 1056771, "author_name": "gao-hongnan", "author_url": "", "post_date": "2020-10-22T04:06:34.200000", "content": "@stanleyjzheng For me, Rotate 90 didn't help at all, I’m not sure if rotate will help at all. As for the other flip and the likes We keep getting same score as the original one. Havnt tried others yet
", "votes": 4, "replies": [ { "id": 1056776, "author_name": "Stanley Zheng", "author_url": "", "post_date": "2020-10-22T04:14:14.337000", "content": "Interesting - I'm currently running a few flips submissions, can't seem to get hue/contrast/brightness working for some reason (weird albumentations bug perhaps). Wondering if crop, cutout, etc. would make it worse too. I'm so lost at what augmentations would help
", "votes": 3, "replies": [] } ] }, { "id": 1061308, "author_name": "Kirderf", "author_url": "", "post_date": "2020-10-26T23:42:00.633000", "content": "I used augmentation but with just small changes than normal, to reflect the different position and sizes, I added random stretch for X and Y axis and random rotate just within 5,10 & 20 degrees.
", "votes": 1, "replies": [ { "id": 1061311, "author_name": "", "author_url": "", "post_date": "2020-10-26T23:44:39.747000", "content": "", "votes": 0, "replies": [] }, { "id": 1061325, "author_name": "Kirderf", "author_url": "", "post_date": "2020-10-27T00:10:11.857000", "content": "Congratz to a great score and the bronze. Due to the the time limit I didn't use TTA to the Inference, used it instead to stacking many models and folds.The plan was some more models and longer training/epochs to the stacking, of course, with different setups, I came a bit on the road but not all the way to the goal.
", "votes": 1, "replies": [] } ] }, { "id": 1058495, "author_name": "Kamal Das", "author_url": "", "post_date": "2020-10-23T19:23:36.930000", "content": "from last year's competition:
\nhttps://www.kaggle.com/c/rsna-intracranial-hemorrhage-detection/discussion/117293
Augmentation for CNN:
\nHeavy augmentation including crop and resize back, affine (360 degree rotation), contrast and brightness, gamma correction, blurring and sharpening, mirroring, optical distortion, grid distortion, elastic transform …
I don't think there is a time to do TTA in 9 hours.
", "votes": 2, "replies": [ { "id": 1057492, "author_name": "", "author_url": "", "post_date": "2020-10-22T17:43:36.920000", "content": "", "votes": 0, "replies": [] }, { "id": 1057521, "author_name": "DarkCube", "author_url": "", "post_date": "2020-10-22T18:08:43.417000", "content": "and I'm here just starting my stage one image-level model. Is recall of about .5-.3 normal early on in training? and for how many epochs should I train my model? and what does TTA mean?
", "votes": 0, "replies": [] }, { "id": 1057530, "author_name": "", "author_url": "", "post_date": "2020-10-22T18:19:29.270000", "content": "", "votes": 0, "replies": [] }, { "id": 1057536, "author_name": "DarkCube", "author_url": "", "post_date": "2020-10-22T18:24:17.460000", "content": "so is it like you augment the test data make the coresponding prediction and then average the results? That doesn't sound like it improves your score much, lol.
", "votes": -2, "replies": [] } ] }, { "id": 1057051, "author_name": "", "author_url": "", "post_date": "2020-10-22T10:09:45.237000", "content": "@stanleyjzheng I use Vertical and Horizontal Flip and it's work not really bad but in my eyes Resize and scaling would be useful (depends on your hardware resource), also check the contrast/brightness and results was good. Looking forwards to you if any ppl can advice new idea.
\nThank you
Odd that we get slightly different results with TTA - just tried a TTAx2 submission with vertical, horizontal, and 90deg flips, and got a worse score. Now I'm super confused….
", "votes": 1, "replies": [] }, { "id": 1057371, "author_name": "Kamal Das", "author_url": "", "post_date": "2020-10-22T15:36:36.923000", "content": "for me as well, flips did not help
\naffine helped me but improvements was marginal