I try to use mixup (data and loss) on various experiments but got poor performance compare to normal.
", "messages": [ { "id": 2094879, "postDate": "2023-01-11T04:15:39.697Z", "content": "I try to use mixup (data and loss) on various experiments but got poor performance compare to normal.
", "rawMarkdown": "I try to use mixup (data and loss) on various experiments but got poor performance compare to normal.", "votes": 4 }, { "id": 2099469, "postDate": "2023-01-14T13:16:00.077Z", "content": "What is your mixing hyperparameter value? Maybe it is just too large..
\nThe Devastator.
", "rawMarkdown": "What is your mixing hyperparameter value? Maybe it is just too large..\n\nThe Devastator.\n", "replies": [ { "id": 2109667, "postDate": "2023-01-21T15:12:06.057Z", "content": "I set alpha = 0.2 and prob = 0.5
", "rawMarkdown": "I set alpha = 0.2 and prob = 0.5 ", "replies": [ { "id": 2109712, "postDate": "2023-01-21T15:44:47.927Z", "content": "I have the same issue. I found at first it could work when training, however, it crushed after epoch 4. And the best threshold goes down quickly. Finally, It died…
\n100%|██████████| 2678/2678 [25:51<00:00, 1.73it/s]
\n100%|██████████| 685/685 [00:55<00:00, 12.29it/s]
\nloss:0.5541952940801617
\n1gtf_efficientnetv2_s_epoch0.pth
\nwithout optimization: 0.088
\nwith optimization: 0.160
\nbest_thresh: 0.88336
\nlr = 0.0003
\n100%|██████████| 2678/2678 [25:38<00:00, 1.74it/s]
\n100%|██████████| 685/685 [00:56<00:00, 12.19it/s]
\nloss:0.4798278690825611
\n2gtf_efficientnetv2_s_epoch1.pth
\nwithout optimization: 0.157
\nwith optimization: 0.228
\nbest_thresh: 0.85348
\nlr = 0.00024199999999999997
\n100%|██████████| 2678/2678 [25:45<00:00, 1.73it/s]
\n100%|██████████| 685/685 [00:56<00:00, 12.22it/s]
\nloss:0.42614720978041265
\n3gtf_efficientnetv2_s_epoch2.pth
\nwithout optimization: 0.146
\nwith optimization: 0.215
\nbest_thresh: 0.85648
\nlr = 0.00018399999999999997
\n100%|██████████| 2678/2678 [25:56<00:00, 1.72it/s]
\n100%|██████████| 685/685 [00:56<00:00, 12.17it/s]
\nloss:0.38675757105197256
\n4gtf_efficientnetv2_s_epoch3.pth
\nwithout optimization: 0.185
\nwith optimization: 0.226
\nbest_thresh: 0.06632
\nlr = 0.000126
\n100%|██████████| 2678/2678 [25:45<00:00, 1.73it/s]
\n100%|██████████| 685/685 [00:56<00:00, 12.10it/s]
\nloss:0.3563341726040533
\n5gtf_efficientnetv2_s_epoch4.pth
\nwithout optimization: 0.123
\nwith optimization: 0.223
\nbest_thresh: 0.015870000000000002
\nlr = 6.8e-05
\n100%|██████████| 2678/2678 [25:47<00:00, 1.73it/s]
\n100%|██████████| 685/685 [00:56<00:00, 12.12it/s]
\nloss:0.33278957070916787
\n6gtf_efficientnetv2_s_epoch5.pth
\nwithout optimization: 0.108
\nwith optimization: 0.209
\nbest_thresh: 0.0072
\nlr = 1e-05
\n100%|██████████| 2678/2678 [25:51<00:00, 1.73it/s]
\n100%|██████████| 685/685 [00:56<00:00, 12.15it/s]
\nloss:0.31562600300553817
\n7gtf_efficientnetv2_s_epoch6.pth
\nwithout optimization: 0.102
\nwith optimization: 0.206
\nbest_thresh: 0.00334
\nlr = 1e-05
\n100%|██████████| 2678/2678 [25:53<00:00, 1.72it/s]
\n100%|██████████| 685/685 [00:55<00:00, 12.23it/s]
\nloss:0.3020148665037026
\n8gtf_efficientnetv2_s_epoch7.pth
\nwithout optimization: 0.093
\nwith optimization: 0.193
\nbest_thresh: 0.00232
\nlr = 1e-05
\n100%|██████████| 2678/2678 [25:43<00:00, 1.74it/s]
\n100%|██████████| 685/685 [00:56<00:00, 12.20it/s]
\nloss:0.2908990504819062
\n9gtf_efficientnetv2_s_epoch8.pth
\nwithout optimization: 0.086
\nwith optimization: 0.209
\nbest_thresh: 0.0014900000000000002
\nlr = 1e-05
\n100%|██████████| 2678/2678 [25:45<00:00, 1.73it/s]
\n100%|██████████| 685/685 [00:55<00:00, 12.40it/s]
\nloss:0.2819449872065049
\n10gtf_efficientnetv2_s_epoch9.pth
\nwithout optimization: 0.099
\nwith optimization: 0.199
\nbest_thresh: 0.0031600000000000005
\nlr = 1e-05
Okay, my fault. It actually could work when I change my upsample rate. My code is like LB 0.4 -> 0.43 when using mixup.
", "rawMarkdown": "Okay, my fault. It actually could work when I change my upsample rate. My code is like LB 0.4 -> 0.43 when using mixup." }, { "id": 2111672, "postDate": "2023-01-23T04:46:37.393Z", "content": "Thank you for this point :D .
", "rawMarkdown": "Thank you for this point :D ." } ] } ] } ], "comments": [ { "id": 2099469, "author_name": "The Devastator", "author_url": "", "post_date": "2023-01-14T13:16:00.077000", "content": "What is your mixing hyperparameter value? Maybe it is just too large..
\nThe Devastator.
", "votes": 0, "replies": [ { "id": 2109667, "author_name": "SenTran", "author_url": "", "post_date": "2023-01-21T15:12:06.057000", "content": "I set alpha = 0.2 and prob = 0.5
", "votes": 0, "replies": [ { "id": 2109712, "author_name": "ChenxiangSun@NJU", "author_url": "", "post_date": "2023-01-21T15:44:47.927000", "content": "I have the same issue. I found at first it could work when training, however, it crushed after epoch 4. And the best threshold goes down quickly. Finally, It died…
\n100%|██████████| 2678/2678 [25:51<00:00, 1.73it/s]
\n100%|██████████| 685/685 [00:55<00:00, 12.29it/s]
\nloss:0.5541952940801617
\n1gtf_efficientnetv2_s_epoch0.pth
\nwithout optimization: 0.088
\nwith optimization: 0.160
\nbest_thresh: 0.88336
\nlr = 0.0003
\n100%|██████████| 2678/2678 [25:38<00:00, 1.74it/s]
\n100%|██████████| 685/685 [00:56<00:00, 12.19it/s]
\nloss:0.4798278690825611
\n2gtf_efficientnetv2_s_epoch1.pth
\nwithout optimization: 0.157
\nwith optimization: 0.228
\nbest_thresh: 0.85348
\nlr = 0.00024199999999999997
\n100%|██████████| 2678/2678 [25:45<00:00, 1.73it/s]
\n100%|██████████| 685/685 [00:56<00:00, 12.22it/s]
\nloss:0.42614720978041265
\n3gtf_efficientnetv2_s_epoch2.pth
\nwithout optimization: 0.146
\nwith optimization: 0.215
\nbest_thresh: 0.85648
\nlr = 0.00018399999999999997
\n100%|██████████| 2678/2678 [25:56<00:00, 1.72it/s]
\n100%|██████████| 685/685 [00:56<00:00, 12.17it/s]
\nloss:0.38675757105197256
\n4gtf_efficientnetv2_s_epoch3.pth
\nwithout optimization: 0.185
\nwith optimization: 0.226
\nbest_thresh: 0.06632
\nlr = 0.000126
\n100%|██████████| 2678/2678 [25:45<00:00, 1.73it/s]
\n100%|██████████| 685/685 [00:56<00:00, 12.10it/s]
\nloss:0.3563341726040533
\n5gtf_efficientnetv2_s_epoch4.pth
\nwithout optimization: 0.123
\nwith optimization: 0.223
\nbest_thresh: 0.015870000000000002
\nlr = 6.8e-05
\n100%|██████████| 2678/2678 [25:47<00:00, 1.73it/s]
\n100%|██████████| 685/685 [00:56<00:00, 12.12it/s]
\nloss:0.33278957070916787
\n6gtf_efficientnetv2_s_epoch5.pth
\nwithout optimization: 0.108
\nwith optimization: 0.209
\nbest_thresh: 0.0072
\nlr = 1e-05
\n100%|██████████| 2678/2678 [25:51<00:00, 1.73it/s]
\n100%|██████████| 685/685 [00:56<00:00, 12.15it/s]
\nloss:0.31562600300553817
\n7gtf_efficientnetv2_s_epoch6.pth
\nwithout optimization: 0.102
\nwith optimization: 0.206
\nbest_thresh: 0.00334
\nlr = 1e-05
\n100%|██████████| 2678/2678 [25:53<00:00, 1.72it/s]
\n100%|██████████| 685/685 [00:55<00:00, 12.23it/s]
\nloss:0.3020148665037026
\n8gtf_efficientnetv2_s_epoch7.pth
\nwithout optimization: 0.093
\nwith optimization: 0.193
\nbest_thresh: 0.00232
\nlr = 1e-05
\n100%|██████████| 2678/2678 [25:43<00:00, 1.74it/s]
\n100%|██████████| 685/685 [00:56<00:00, 12.20it/s]
\nloss:0.2908990504819062
\n9gtf_efficientnetv2_s_epoch8.pth
\nwithout optimization: 0.086
\nwith optimization: 0.209
\nbest_thresh: 0.0014900000000000002
\nlr = 1e-05
\n100%|██████████| 2678/2678 [25:45<00:00, 1.73it/s]
\n100%|██████████| 685/685 [00:55<00:00, 12.40it/s]
\nloss:0.2819449872065049
\n10gtf_efficientnetv2_s_epoch9.pth
\nwithout optimization: 0.099
\nwith optimization: 0.199
\nbest_thresh: 0.0031600000000000005
\nlr = 1e-05
Okay, my fault. It actually could work when I change my upsample rate. My code is like LB 0.4 -> 0.43 when using mixup.
", "votes": 0, "replies": [] }, { "id": 2111672, "author_name": "SenTran", "author_url": "", "post_date": "2023-01-23T04:46:37.393000", "content": "Thank you for this point :D .
", "votes": 0, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2094879": "I try to use mixup (data and loss) on various experiments but got poor performance compare to normal.", "2099469": "What is your mixing hyperparameter value? Maybe it is just too large..\n\nThe Devastator.\n" } }