{
  "id": 222707,
  "title": "Variance in LB score even with same hyperparameters and dataset (Yolov5x)",
  "url": "/competitions/vinbigdata-chest-xray-abnormalities-detection/discussion/222707",
  "author_name": "adriel cabral",
  "post_date": "2021-02-28T19:02:54.107000",
  "votes": 27,
  "comment_count": 111,
  "views": 0,
  "content": "<p>I use yolov5x with 2 class filters for my submissions and get a variance of LB score around of (0.2 - 0.247). I was waiting a low variance, for example, 0.23-0.247 or 0.24-0.247, but 0.2-0.247 i think is too high. I don't know why it happens.</p>\n<ul>\n<li>All training have CV &gt;= 0.4</li>\n<li>My max CV was 0.435, unfortunately, have a low LB score (0.208)</li>\n<li>For inference, i use conf_thres: 0.001, my \"better model\" get lb 0.240 with conf_thres: 0.01 and 0.247 with conf_thres: 0.001</li>\n</ul>\n<p>Anyone that use Yolov5 have the same \"problem\" ?<br>\nTo be honest, i use yolov5 because it is much easier to train and make inference (this is my first contact with object detection and DL). I will try to use other models</p>",
  "messages": [
    {
      "id": 1221216,
      "postDate": "2021-02-28T19:02:54.107Z",
      "content": "<p>I use yolov5x with 2 class filters for my submissions and get a variance of LB score around of (0.2 - 0.247). I was waiting a low variance, for example, 0.23-0.247 or 0.24-0.247, but 0.2-0.247 i think is too high. I don't know why it happens.</p>\n<ul>\n<li>All training have CV &gt;= 0.4</li>\n<li>My max CV was 0.435, unfortunately, have a low LB score (0.208)</li>\n<li>For inference, i use conf_thres: 0.001, my \"better model\" get lb 0.240 with conf_thres: 0.01 and 0.247 with conf_thres: 0.001</li>\n</ul>\n<p>Anyone that use Yolov5 have the same \"problem\" ?<br>\nTo be honest, i use yolov5 because it is much easier to train and make inference (this is my first contact with object detection and DL). I will try to use other models</p>",
      "rawMarkdown": "I use yolov5x with 2 class filters for my submissions and get a variance of LB score around of (0.2 - 0.247). I was waiting a low variance, for example, 0.23-0.247 or 0.24-0.247, but 0.2-0.247 i think is too high. I don't know why it happens.\n\n- All training have CV >= 0.4\n- My max CV was 0.435, unfortunately, have a low LB score (0.208)\n- For inference, i use conf_thres: 0.001, my \"better model\" get lb 0.240 with conf_thres: 0.01 and 0.247 with conf_thres: 0.001\n\nAnyone that use Yolov5 have the same \"problem\" ?\nTo be honest, i use yolov5 because it is much easier to train and make inference (this is my first contact with object detection and DL). I will try to use other models\n\n\n",
      "votes": 27
    },
    {
      "id": 1221394,
      "postDate": "2021-03-01T00:12:09.110Z",
      "content": "<p>Yes, I experienced the same. IMO this is not specific to yolov5 but caused by very few cases of rare classes in the public test set. Your model may or may not predict well those cases. As shown <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/220761\" target=\"_blank\">here</a>, the LB score is much more stable for frequent classes. So maybe we can rely on LB score for frequent classes or CV score for model selection.</p>",
      "rawMarkdown": "Yes, I experienced the same. IMO this is not specific to yolov5 but caused by very few cases of rare classes in the public test set. Your model may or may not predict well those cases. As shown [here](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/220761), the LB score is much more stable for frequent classes. So maybe we can rely on LB score for frequent classes or CV score for model selection.",
      "votes": 6
    },
    {
      "id": 1241717,
      "postDate": "2021-03-17T07:14:39.713Z",
      "content": "<p>My scores are very unstable using Yolov5 too</p>\n<p>All models are yolov5x, and scoring after run <code>2 class filter</code></p>\n<table>\n<thead>\n<tr>\n<th>Model</th>\n<th>Val</th>\n<th>LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Fold 0</td>\n<td>0.417</td>\n<td>0.195</td>\n</tr>\n<tr>\n<td>Fold 2</td>\n<td>0.419</td>\n<td>0.231</td>\n</tr>\n<tr>\n<td>Fold 4</td>\n<td>0.432</td>\n<td>0.217</td>\n</tr>\n<tr>\n<td>Ensemble (NMS .5)</td>\n<td>-</td>\n<td>0.225</td>\n</tr>\n</tbody>\n</table>\n<p>My data are split using <code>stratified k fold</code></p>",
      "rawMarkdown": "My scores are very unstable using Yolov5 too\n\nAll models are yolov5x, and scoring after run `2 class filter`\n\n| Model |  Val | LB |\n|--|--|--|\n| Fold 0 | 0.417 | 0.195 |\n| Fold 2 | 0.419 | 0.231 |\n| Fold 4 | 0.432 | 0.217 | \n| Ensemble (NMS .5) | - | 0.225 |\n\nMy data are split using `stratified k fold`",
      "votes": 1,
      "replies": [
        {
          "id": 1252879,
          "postDate": "2021-03-26T06:20:47.137Z",
          "content": "<p>From ur experiments, what ensemble strategy works best? I tried several methods but my LB score couldn't improve.</p>",
          "rawMarkdown": "From ur experiments, what ensemble strategy works best? I tried several methods but my LB score couldn't improve."
        },
        {
          "id": 1253081,
          "postDate": "2021-03-26T10:50:55.663Z",
          "content": "<p>I could not reproduce my best result, If I remember correctly, I have used TTA and ensemble of 5 folds with iou 0.5 and score threshold 0.001</p>",
          "rawMarkdown": "I could not reproduce my best result, If I remember correctly, I have used TTA and ensemble of 5 folds with iou 0.5 and score threshold 0.001\n"
        },
        {
          "id": 1253718,
          "postDate": "2021-03-27T01:37:21.607Z",
          "content": "<p>Thank for ur advisement.</p>",
          "rawMarkdown": "Thank for ur advisement."
        }
      ]
    },
    {
      "id": 1228729,
      "postDate": "2021-03-06T17:39:21.723Z",
      "content": "<p>During my model selection I experienced some model achieving high score in some certain classes (CV) therefore have high LB score while \"perfect\" model which achieved more well round number for all labels has lower LB score. For this reason, I think the variance is reasonable. </p>",
      "rawMarkdown": "During my model selection I experienced some model achieving high score in some certain classes (CV) therefore have high LB score while \"perfect\" model which achieved more well round number for all labels has lower LB score. For this reason, I think the variance is reasonable. ",
      "votes": 1,
      "replies": [
        {
          "id": 1228730,
          "postDate": "2021-03-06T17:41:28.893Z",
          "content": "<p>Yeah, i think the same</p>",
          "rawMarkdown": "Yeah, i think the same"
        },
        {
          "id": 1228733,
          "postDate": "2021-03-06T17:46:32.703Z",
          "content": "<p>In addition, I think ensemble might be the key boost in this competition. Currently, my best model achieved 0.193 but I haven't found any more architectures to experiment with. I think I will go to cleaning data process.</p>",
          "rawMarkdown": "In addition, I think ensemble might be the key boost in this competition. Currently, my best model achieved 0.193 but I haven't found any more architectures to experiment with. I think I will go to cleaning data process."
        }
      ]
    },
    {
      "id": 1228112,
      "postDate": "2021-03-06T06:11:29.137Z",
      "content": "<p><a href=\"https://www.kaggle.com/adrielcabral\" target=\"_blank\">@adrielcabral</a>   what aug did you use?</p>",
      "rawMarkdown": " @adrielcabral   what aug did you use?",
      "votes": 1,
      "replies": [
        {
          "id": 1228476,
          "postDate": "2021-03-06T12:50:05.127Z",
          "content": "<p>hsv_h: 0.015<br>\nhsv_s: 0.7<br>\nhsv_v: 0.4<br>\ndegrees: 0.0<br>\ntranslate: 0.2<br>\nscale: 0.6<br>\nshear: 0.0<br>\nperspective: 0.0<br>\nflipud: 0.2<br>\nfliplr: 0.5<br>\nmosaic: 1.0<br>\nmixup: 0.0</p>",
          "rawMarkdown": "hsv_h: 0.015\nhsv_s: 0.7\nhsv_v: 0.4\ndegrees: 0.0\ntranslate: 0.2\nscale: 0.6\nshear: 0.0\nperspective: 0.0\nflipud: 0.2\nfliplr: 0.5\nmosaic: 1.0\nmixup: 0.0"
        },
        {
          "id": 1228555,
          "postDate": "2021-03-06T14:45:01.020Z",
          "content": "<p><a href=\"https://www.kaggle.com/adrielcabral\" target=\"_blank\">@adrielcabral</a> Did you use nms or wbf on dataset before train ?</p>",
          "rawMarkdown": "@adrielcabral Did you use nms or wbf on dataset before train ?"
        },
        {
          "id": 1228700,
          "postDate": "2021-03-06T16:52:08.247Z",
          "content": "<p>I use a pre-processing (fusion bboxes with IOU &gt; 0.4)</p>",
          "rawMarkdown": "I use a pre-processing (fusion bboxes with IOU > 0.4)",
          "votes": 1
        }
      ]
    },
    {
      "id": 1227987,
      "postDate": "2021-03-06T02:11:18.200Z",
      "content": "<p><a href=\"https://www.kaggle.com/adrielcabral\" target=\"_blank\">@adrielcabral</a> </p>\n<p>I am using YOLOV5 too but my CV stuck around 0.33</p>\n<p>Can I ask a few questions about your setup ? </p>\n<ol>\n<li>which release of yolov5 are you using ? I  am using 4.0</li>\n<li>Did you train the model with only abnormal data ( filter out images with no finding )</li>\n<li>What NMS IOU did you use for inference ?  Did you leave it at 0.45 ( default ) ? </li>\n<li>Any modification to the model ?  Or did you just use it out of the box ?   </li>\n</ol>\n<p>Thanks  for the info</p>",
      "rawMarkdown": "@adrielcabral \n\nI am using YOLOV5 too but my CV stuck around 0.33\n\nCan I ask a few questions about your setup ? \n\n1. which release of yolov5 are you using ? I  am using 4.0\n2. Did you train the model with only abnormal data ( filter out images with no finding )\n3. What NMS IOU did you use for inference ?  Did you leave it at 0.45 ( default ) ? \n4. Any modification to the model ?  Or did you just use it out of the box ?   \n\nThanks  for the info",
      "votes": 1,
      "replies": [
        {
          "id": 1228002,
          "postDate": "2021-03-06T02:56:22.877Z",
          "content": "<p>1- I use 4.0 too<br>\n2- i tried add negative samples (empty txts files) in training, but get less lb score. I use only abnormal data<br>\n3- NMS IOU: 0.5<br>\n4- no modification</p>\n<p>Did you try to use my settings?</p>",
          "rawMarkdown": "1- I use 4.0 too\n2- i tried add negative samples (empty txts files) in training, but get less lb score. I use only abnormal data\n3- NMS IOU: 0.5\n4- no modification\n\nDid you try to use my settings?"
        },
        {
          "id": 1228035,
          "postDate": "2021-03-06T04:07:41.197Z",
          "content": "<p>Yes, I copied your hyp.scratch.yaml trying to reproduce the 0.40+ CV….  but failed.</p>\n<p>2 further questions</p>\n<p>1) How did you split your data ?   I did 5 fold splits with sklearn's group KFold<br>\n2) What dataset do you use ? Do you use the original competition dataset ( with original aspect ratio ) ? Or did you use the 512 x 512 / 1024 x 1024 resized ones people having been sharing ?</p>",
          "rawMarkdown": "Yes, I copied your hyp.scratch.yaml trying to reproduce the 0.40+ CV....  but failed.\n\n2 further questions\n\n1) How did you split your data ?   I did 5 fold splits with sklearn's group KFold\n2) What dataset do you use ? Do you use the original competition dataset ( with original aspect ratio ) ? Or did you use the 512 x 512 / 1024 x 1024 resized ones people having been sharing ?",
          "votes": 1
        },
        {
          "id": 1228473,
          "postDate": "2021-03-06T12:48:56.077Z",
          "content": "<p>1 - I use group KFold too<br>\n2 - i use 1024x1024 resized data, but in training i choice 640x640</p>\n<p>I use a pre-processing (fusion bboxes with IOU &gt; 0.4)</p>",
          "rawMarkdown": "1 - I use group KFold too\n2 - i use 1024x1024 resized data, but in training i choice 640x640\n\nI use a pre-processing (fusion bboxes with IOU > 0.4)"
        },
        {
          "id": 1228574,
          "postDate": "2021-03-06T15:03:35.093Z",
          "rawMarkdown": "",
          "isDeleted": true
        },
        {
          "id": 1228924,
          "postDate": "2021-03-06T22:27:38.333Z",
          "content": "<p>Thanks a lot for all the info provided so far. </p>\n<p>I added the box fusion pre-processing.. Now my local CV is a bit above 0.40.  However, the LB result (just the detector, without 2nd stage classifier) gets worse than my previous version ( with CV score less than 0.33).  </p>\n<p>Is there anything special that you are doing during inference ?  Any additional processing ?  ( I did conf-thres=0.01 + IOU=0.5)</p>",
          "rawMarkdown": "Thanks a lot for all the info provided so far. \n\nI added the box fusion pre-processing.. Now my local CV is a bit above 0.40.  However, the LB result (just the detector, without 2nd stage classifier) gets worse than my previous version ( with CV score less than 0.33).  \n\nIs there anything special that you are doing during inference ?  Any additional processing ?  ( I did conf-thres=0.01 + IOU=0.5)"
        },
        {
          "id": 1228999,
          "postDate": "2021-03-07T01:03:15.100Z",
          "content": "<p>Using 2 class filter which model is better ? with cv &gt; 0.4 or cv 0.33 ? i tell all that i do, sorry i don't make more processing</p>",
          "rawMarkdown": "Using 2 class filter which model is better ? with cv > 0.4 or cv 0.33 ? i tell all that i do, sorry i don't make more processing",
          "votes": 1
        },
        {
          "id": 1245460,
          "postDate": "2021-03-19T20:05:46.027Z",
          "content": "<p>The same issue observed in my training with yolov5. Have you managed to figure out what’s going on for the pre-fused case? Thanks!</p>",
          "rawMarkdown": "The same issue observed in my training with yolov5. Have you managed to figure out what’s going on for the pre-fused case? Thanks!"
        }
      ]
    },
    {
      "id": 1245640,
      "postDate": "2021-03-20T01:34:32.927Z",
      "content": "<p>Hello,which parameters do you choose?I only get 0.33+ on cv.</p>",
      "rawMarkdown": "Hello,which parameters do you choose?I only get 0.33+ on cv."
    },
    {
      "id": 1233921,
      "postDate": "2021-03-10T19:17:21.190Z",
      "content": "<p>Hi, Adriel, I use yolov3 (with 2 class filter) and have a LB score 2.3. I am trying with yolo5X now, but only got LB around 2.0.  <br>\nYolo5x should achieve better performance than yolov3. Do you know any possible reasons? Thanks!<br>\nFYI, I already enable the \"multi-label\" for nms in yolov5.</p>",
      "rawMarkdown": "Hi, Adriel, I use yolov3 (with 2 class filter) and have a LB score 2.3. I am trying with yolo5X now, but only got LB around 2.0.  \nYolo5x should achieve better performance than yolov3. Do you know any possible reasons? Thanks!\nFYI, I already enable the \"multi-label\" for nms in yolov5.",
      "replies": [
        {
          "id": 1233987,
          "postDate": "2021-03-10T20:45:24.867Z",
          "content": "<ul>\n<li>what parameters do you use?</li>\n<li>img size</li>\n<li>make any pre, pos processing ?</li>\n</ul>",
          "rawMarkdown": "- what parameters do you use?\n- img size\n- make any pre, pos processing ?"
        },
        {
          "id": 1233999,
          "postDate": "2021-03-10T21:00:45.413Z",
          "content": "<p>Thanks for your reply!</p>\n<ul>\n<li>I use the default parameters.</li>\n<li>Img size 1024x1024.</li>\n<li>Since the default parameters already dealt with augmentation, I didn't do any extra pre.<br>\nAnd for Pos, I use conf = 0.001, and IOU = 0.4.<br>\nBasically, I use the same criterion for yolov3 and 5. Thanks! </li>\n</ul>",
          "rawMarkdown": "Thanks for your reply!\n- I use the default parameters.\n- Img size 1024x1024.\n- Since the default parameters already dealt with augmentation, I didn't do any extra pre.\nAnd for Pos, I use conf = 0.001, and IOU = 0.4.\nBasically, I use the same criterion for yolov3 and 5. Thanks! "
        },
        {
          "id": 1234008,
          "postDate": "2021-03-10T21:19:43.030Z",
          "content": "<p>I make a pre-processing (fusion bboxes with IOU &gt; 0.4) so …<br>\nU should try use my  hyperparameters, probably will not work for CV if u don't make the same pre-processing and my imgsize is 640x640</p>\n<p>I hope I've helped :)</p>",
          "rawMarkdown": "I make a pre-processing (fusion bboxes with IOU > 0.4) so ...\nU should try use my  hyperparameters, probably will not work for CV if u don't make the same pre-processing and my imgsize is 640x640\n\n\nI hope I've helped :)"
        },
        {
          "id": 1234012,
          "postDate": "2021-03-10T21:24:50.630Z",
          "content": "<p>Thanks! Will try your hyperparameters, and the fusion preprocessing.<br>\nBesides, any specific reason for using 640 instead of 1024?</p>",
          "rawMarkdown": "Thanks! Will try your hyperparameters, and the fusion preprocessing.\nBesides, any specific reason for using 640 instead of 1024?"
        },
        {
          "id": 1234019,
          "postDate": "2021-03-10T21:33:45.307Z",
          "content": "<p>1 - I tried use imgsize 1024x1024 but I got a lower lb score (old hyperparameters), if u want u can try now with the new hyperparameters.</p>\n<p>2 - 1024x1024 take a long time ( i use colab don't have GPU)</p>",
          "rawMarkdown": "1 - I tried use imgsize 1024x1024 but I got a lower lb score (old hyperparameters), if u want u can try now with the new hyperparameters.\n\n2 - 1024x1024 take a long time ( i use colab don't have GPU)\n"
        },
        {
          "id": 1239288,
          "postDate": "2021-03-15T15:35:19.157Z",
          "content": "<p>Thanks for sharing, Adriel :)  I'd like to hear your opinions on the following points:</p>\n<p>-- For YOLOv5x, I guess you already enabled the \"multi-label\" by modifying this line in <em>general.py</em>: <br>\n     multi_label &amp;= nc &gt; 1, am I correct?  Besides, do you use TTA for prediction?  </p>\n<p>--  Have you achieved a better score by using vfnet than yolov5? </p>",
          "rawMarkdown": "Thanks for sharing, Adriel :)  I'd like to hear your opinions on the following points:\n\n-- For YOLOv5x, I guess you already enabled the \"multi-label\" by modifying this line in *general.py*: \n     multi_label &= nc > 1, am I correct?  Besides, do you use TTA for prediction?  \n\n--  Have you achieved a better score by using vfnet than yolov5? "
        },
        {
          "id": 1239341,
          "postDate": "2021-03-15T16:30:28.187Z",
          "content": "<ul>\n<li>yeah,  I modified general.py file and TTA decrease lb score (for me), but u should try to use because others have a increase like <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/225391\" target=\"_blank\">here</a></li>\n<li>i get better score for 1 single fold, yolov5x (0.248) VFnet (0.255), but no is better than ensemble weights (0.268), i 'm tring work with VFnet</li>\n</ul>",
          "rawMarkdown": "- yeah,  I modified general.py file and TTA decrease lb score (for me), but u should try to use because others have a increase like [here](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/225391)\n- i get better score for 1 single fold, yolov5x (0.248) VFnet (0.255), but no is better than ensemble weights (0.268), i 'm tring work with VFnet"
        },
        {
          "id": 1239366,
          "postDate": "2021-03-15T16:50:46.043Z",
          "content": "<p>Got it, thank you! I am working on YOLOv5, and will also try VFnet. <br>\nHope we could reach 0.3 sooner or later :) </p>",
          "rawMarkdown": "Got it, thank you! I am working on YOLOv5, and will also try VFnet. \nHope we could reach 0.3 sooner or later :) "
        },
        {
          "id": 1239374,
          "postDate": "2021-03-15T16:55:53.883Z",
          "content": "<p>I hope too ✌️ That is my dream 😂😂</p>",
          "rawMarkdown": "I hope too ✌️ That is my dream 😂😂"
        },
        {
          "id": 1244427,
          "postDate": "2021-03-19T01:32:45.770Z",
          "content": "<p>Dear Adriel, I obtained 0.17+ for best yolov5 on the original data, but always get much lower values (0.15-) on pre-fusioned data. I tried nms and wbf (0.4 threshold), but neither works. Do you have any idea? I am currently using your configuration. It looks great.👍</p>",
          "rawMarkdown": "Dear Adriel, I obtained 0.17+ for best yolov5 on the original data, but always get much lower values (0.15-) on pre-fusioned data. I tried nms and wbf (0.4 threshold), but neither works. Do you have any idea? I am currently using your configuration. It looks great.👍"
        },
        {
          "id": 1244519,
          "postDate": "2021-03-19T03:24:02.947Z",
          "content": "<p>Well, I saw many people get a higher score with original data (with many overlapping boxes), But I don't know why. It should more reasonable if preprocessed data (fusion box) work better.</p>",
          "rawMarkdown": "Well, I saw many people get a higher score with original data (with many overlapping boxes), But I don't know why. It should more reasonable if preprocessed data (fusion box) work better."
        },
        {
          "id": 1245032,
          "postDate": "2021-03-19T12:03:19.783Z",
          "content": "<p><a href=\"https://www.kaggle.com/kuanzhang\" target=\"_blank\">@kuanzhang</a> sorry but i no have idea, I tried to use the original data at the start of the competition (for a long time), but I only get 0.219 in the lb score.</p>\n<ul>\n<li>I don't know if i do \"NMS\" because i do my own \"fusion bboxes\" , but looking at the results of NMS and mine, they are very similar.</li>\n</ul>",
          "rawMarkdown": "@kuanzhang sorry but i no have idea, I tried to use the original data at the start of the competition (for a long time), but I only get 0.219 in the lb score.\n\n- I don't know if i do \"NMS\" because i do my own \"fusion bboxes\" , but looking at the results of NMS and mine, they are very similar."
        },
        {
          "id": 1245646,
          "postDate": "2021-03-20T01:49:11.970Z",
          "content": "<p>Thanks, Adriel! Yeah, it is a little weird on that comparison. I guess I can only have to keep trying 😛</p>",
          "rawMarkdown": "Thanks, Adriel! Yeah, it is a little weird on that comparison. I guess I can only have to keep trying 😛"
        },
        {
          "id": 1245647,
          "postDate": "2021-03-20T01:53:39.187Z",
          "content": "<p>Hello,did you train on kaggle?When I use 1024 size,the cpu will Out Of Memory….</p>",
          "rawMarkdown": "Hello,did you train on kaggle?When I use 1024 size,the cpu will Out Of Memory...."
        }
      ]
    },
    {
      "id": 1225114,
      "postDate": "2021-03-03T10:29:54.873Z",
      "content": "<p>I have used yolov5m(release 1.0) to get LB 0.186. But on my val set, the best map@iou0.5 is 0.26</p>\n<p>Inference detail: <br>\nTTA: no<br>\nInput_size: 640x640<br>\nscore thres: 0.1</p>\n<p>does this means I shuold try bigger model(yolov5x) like you?</p>",
      "rawMarkdown": "I have used yolov5m(release 1.0) to get LB 0.186. But on my val set, the best map@iou0.5 is 0.26\n\nInference detail: \nTTA: no\nInput_size: 640x640\nscore thres: 0.1\n\ndoes this means I shuold try bigger model(yolov5x) like you?",
      "replies": [
        {
          "id": 1225132,
          "postDate": "2021-03-03T10:45:55.110Z",
          "content": "<p>using my configuration you get only 0.26 in val set ? well, i tryed yolov5x and yolov5l and get <br>\nsimilar results (yolov5x is a little better), you should try use too</p>",
          "rawMarkdown": "using my configuration you get only 0.26 in val set ? well, i tryed yolov5x and yolov5l and get \nsimilar results (yolov5x is a little better), you should try use too"
        },
        {
          "id": 1228061,
          "postDate": "2021-03-06T04:50:35.520Z",
          "content": "<p><a href=\"https://www.kaggle.com/tenggyut\" target=\"_blank\">@tenggyut</a> did you use 2-class fillter</p>",
          "rawMarkdown": "@tenggyut did you use 2-class fillter"
        },
        {
          "id": 1231918,
          "postDate": "2021-03-09T11:29:22.423Z",
          "content": "<p>no, I haven't</p>",
          "rawMarkdown": "no, I haven't",
          "votes": 1
        }
      ]
    },
    {
      "id": 1225028,
      "postDate": "2021-03-03T08:55:35.730Z",
      "content": "<p>Thank for sharing. How much did 2 class filter improve ur LB score than only using single yolov5?</p>",
      "rawMarkdown": "Thank for sharing. How much did 2 class filter improve ur LB score than only using single yolov5?",
      "replies": [
        {
          "id": 1225127,
          "postDate": "2021-03-03T10:42:24.927Z",
          "content": "<p>around of (0.04-0.06)</p>",
          "rawMarkdown": "around of (0.04-0.06)"
        },
        {
          "id": 1228725,
          "postDate": "2021-03-06T17:24:16.733Z",
          "content": "<p><a href=\"https://www.kaggle.com/adrielcabral\" target=\"_blank\">@adrielcabral</a> what thresholds did you use?</p>",
          "rawMarkdown": " @adrielcabral what thresholds did you use?"
        },
        {
          "id": 1228731,
          "postDate": "2021-03-06T17:44:06.120Z",
          "content": "<p>i use class filter of this <a href=\"https://www.kaggle.com/awsaf49/vinbigdata-2-class-filter\" target=\"_blank\">notebook</a> </p>",
          "rawMarkdown": "i use class filter of this [notebook](https://www.kaggle.com/awsaf49/vinbigdata-2-class-filter) ",
          "votes": 1
        }
      ]
    },
    {
      "id": 1224788,
      "postDate": "2021-03-03T04:21:07.707Z",
      "content": "<p>Hi, in Yolov5 by default are evaluated by mAP@.5. Are your CV measured at 0.5 IOU or 0.4 (competition metric)</p>",
      "rawMarkdown": "Hi, in Yolov5 by default are evaluated by mAP@.5. Are your CV measured at 0.5 IOU or 0.4 (competition metric)",
      "replies": [
        {
          "id": 1225125,
          "postDate": "2021-03-03T10:40:52.790Z",
          "content": "<p>at 0.5 IOU</p>",
          "rawMarkdown": "at 0.5 IOU"
        }
      ]
    },
    {
      "id": 1222635,
      "postDate": "2021-03-02T01:44:42.677Z",
      "content": "<p>I seeing pretty bad CV to LB correlation for yolov5. I have local 0.32+ and LB 0.15. What image sizes are you using? I tried 640 with yolo5x</p>",
      "rawMarkdown": "I seeing pretty bad CV to LB correlation for yolov5. I have local 0.32+ and LB 0.15. What image sizes are you using? I tried 640 with yolo5x",
      "replies": [
        {
          "id": 1223011,
          "postDate": "2021-03-02T11:21:21.850Z",
          "content": "<p>640x640 too</p>",
          "rawMarkdown": "640x640 too"
        }
      ]
    },
    {
      "id": 1221592,
      "postDate": "2021-03-01T05:40:46.003Z",
      "content": "<p>May I ask what is your configuration? </p>",
      "rawMarkdown": "May I ask what is your configuration? ",
      "replies": [
        {
          "id": 1221881,
          "postDate": "2021-03-01T11:24:35.907Z",
          "content": "<p>Yeah<br>\nMy configuration:</p>\n<p>lr0: 0.01<br>\nlrf: 0.032<br>\nmomentum: 0.937<br>\nweight_decay: 0.0005<br>\nwarmup_epochs: 3.0<br>\nwarmup_momentum: 0.8<br>\nwarmup_bias_lr: 0.1<br>\nbox: 0.1<br>\ncls: 1.0<br>\ncls_pw: 0.5<br>\nobj: 2.0<br>\nobj_pw: 0.5<br>\niou_t: 0.2<br>\nanchor_t: 4.0<br>\nanchors: 0<br>\nfl_gamma: 0.0<br>\nhsv_h: 0.015<br>\nhsv_s: 0.7<br>\nhsv_v: 0.4<br>\ndegrees: 0.0<br>\ntranslate: 0.2<br>\nscale: 0.6<br>\nshear: 0.0<br>\nperspective: 0.0<br>\nflipud: 0.2<br>\nfliplr: 0.5<br>\nmosaic: 1.0<br>\nmixup: 0.0</p>\n<p>training for 50 epochs </p>",
          "rawMarkdown": "Yeah\nMy configuration:\n\nlr0: 0.01\nlrf: 0.032\nmomentum: 0.937\nweight_decay: 0.0005\nwarmup_epochs: 3.0\nwarmup_momentum: 0.8\nwarmup_bias_lr: 0.1\nbox: 0.1\ncls: 1.0\ncls_pw: 0.5\nobj: 2.0\nobj_pw: 0.5\niou_t: 0.2\nanchor_t: 4.0\nanchors: 0\nfl_gamma: 0.0\nhsv_h: 0.015\nhsv_s: 0.7\nhsv_v: 0.4\ndegrees: 0.0\ntranslate: 0.2\nscale: 0.6\nshear: 0.0\nperspective: 0.0\nflipud: 0.2\nfliplr: 0.5\nmosaic: 1.0\nmixup: 0.0\n\ntraining for 50 epochs ",
          "votes": 4
        },
        {
          "id": 1222425,
          "postDate": "2021-03-01T19:28:03.213Z",
          "content": "<p>Oh thanks for sharing! How did you choose the parameters?</p>\n<p>With respect to other models, I can recommend mmdetection, it's a great library for detection and is relatively easy to use, just need to convert the data to coco format and use a config file. </p>",
          "rawMarkdown": "Oh thanks for sharing! How did you choose the parameters?\n\nWith respect to other models, I can recommend mmdetection, it's a great library for detection and is relatively easy to use, just need to convert the data to coco format and use a config file. ",
          "replies": [
            {
              "id": 1225629,
              "postDate": "2021-03-03T19:00:45.293Z",
              "content": "<blockquote>\n  <p>With respect to other models, I can recommend mmdetection, it's a great library for detection and is relatively easy to use, just need to convert the data to coco format and use a config file.</p>\n</blockquote>\n<p>convert my data to coco format, but when a go try train i get this erro:</p>\n<p>Expected condition, x and y to be on the same device, but condition is on cuda:0 and x and y are on cuda:0 and cpu respectively</p>\n<p>u know how fix it ?</p>",
              "rawMarkdown": "> With respect to other models, I can recommend mmdetection, it's a great library for detection and is relatively easy to use, just need to convert the data to coco format and use a config file.\n\n convert my data to coco format, but when a go try train i get this erro:\n\nExpected condition, x and y to be on the same device, but condition is on cuda:0 and x and y are on cuda:0 and cpu respectively\n\nu know how fix it ?\n\n\n"
            }
          ]
        },
        {
          "id": 1222619,
          "postDate": "2021-03-02T01:18:42.583Z",
          "content": "<p>I did several tests and arrived at these hyperparameters (if you want, u can increase [obj_pw], but will increase a number of FP). I will try use mmdetection i see that have <br>\nvarious models, very nice</p>",
          "rawMarkdown": "I did several tests and arrived at these hyperparameters (if you want, u can increase [obj_pw], but will increase a number of FP). I will try use mmdetection i see that have \nvarious models, very nice",
          "votes": 1
        },
        {
          "id": 1226085,
          "postDate": "2021-03-04T08:11:19.503Z",
          "content": "<p><a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> are you getting better results with mmdet models vs yolo5*? </p>",
          "rawMarkdown": "@hannes82 are you getting better results with mmdet models vs yolo5*? "
        },
        {
          "id": 1226105,
          "postDate": "2021-03-04T08:24:34.860Z",
          "content": "<blockquote>\n  <p>Expected condition, x and y to be on the same device, but condition is on cuda:0 and x and y are on cuda:0 and cpu respectively</p>\n</blockquote>\n<p><a href=\"https://www.kaggle.com/adrielcabral\" target=\"_blank\">@adrielcabral</a>  are you trying some custom models? it looks like the pytorch device is CPU for some tensors and GPU for others. I dont see this while trying out of the box models in mmdetect. </p>",
          "rawMarkdown": "> Expected condition, x and y to be on the same device, but condition is on cuda:0 and x and y are on cuda:0 and cpu respectively\n\n@adrielcabral  are you trying some custom models? it looks like the pytorch device is CPU for some tensors and GPU for others. I dont see this while trying out of the box models in mmdetect. "
        },
        {
          "id": 1226242,
          "postDate": "2021-03-04T11:21:56.670Z",
          "content": "<p><a href=\"https://www.kaggle.com/trushk\" target=\"_blank\">@trushk</a> yes, LB 0.258 with VfNet vs. 0.248 Yolov5. </p>",
          "rawMarkdown": "@trushk yes, LB 0.258 with VfNet vs. 0.248 Yolov5. ",
          "votes": 1
        },
        {
          "id": 1226534,
          "postDate": "2021-03-04T16:03:24.157Z",
          "content": "<p>Thanks <a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> . That is promising. Are you doing bbox processing before train and after infer. My yolo score is still not that high with pre nms. </p>",
          "rawMarkdown": "Thanks @hannes82 . That is promising. Are you doing bbox processing before train and after infer. My yolo score is still not that high with pre nms. ",
          "votes": 1
        },
        {
          "id": 1226695,
          "postDate": "2021-03-04T18:42:58.123Z",
          "content": "<p>I now do wbf before VfNet, but I am not sure it helps in terms of LB score. In case of yolov5 it didnt help.<br>\nI do post nms - this definetely helps..</p>",
          "rawMarkdown": "I now do wbf before VfNet, but I am not sure it helps in terms of LB score. In case of yolov5 it didnt help.\nI do post nms - this definetely helps.."
        },
        {
          "id": 1226708,
          "postDate": "2021-03-04T18:58:58.887Z",
          "content": "<p>thanks! that gives me some motivation to go down the path I had planned to do some more post proc. BTW mmdetect is pretty cool so far! it would be great if they have dasboards similar to wandb for yolo</p>",
          "rawMarkdown": "thanks! that gives me some motivation to go down the path I had planned to do some more post proc. BTW mmdetect is pretty cool so far! it would be great if they have dasboards similar to wandb for yolo",
          "votes": 1
        },
        {
          "id": 1226745,
          "postDate": "2021-03-04T19:56:38.783Z",
          "content": "<p>I somehow never tried wandb, maybe I should.. NMS is already included in test_cfg in mmdetection and if I am not mistaken you can also change it to soft NMS.</p>",
          "rawMarkdown": "I somehow never tried wandb, maybe I should.. NMS is already included in test_cfg in mmdetection and if I am not mistaken you can also change it to soft NMS."
        },
        {
          "id": 1226753,
          "postDate": "2021-03-04T20:12:57.533Z",
          "content": "<p>Ah good to know. Looking at the model config it looks like it uses NMS by default</p>\n<blockquote>\n  <p>test_cfg = dict(<br>\n      nms_pre=1000,<br>\n      min_bbox_size=0,<br>\n      score_thr=0.05,<br>\n      nms=dict(type='nms', iou_threshold=0.6),<br>\n      max_per_img=100)</p>\n</blockquote>",
          "rawMarkdown": "Ah good to know. Looking at the model config it looks like it uses NMS by default\n> test_cfg = dict(\n    nms_pre=1000,\n    min_bbox_size=0,\n    score_thr=0.05,\n    nms=dict(type='nms', iou_threshold=0.6),\n    max_per_img=100)\n"
        },
        {
          "id": 1226865,
          "postDate": "2021-03-05T00:41:49.713Z",
          "content": "<p><a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> What image sizes are you using? I used 1024x1024 for VFNET and heavy augs and LB score is horrible at 0.11  at 10 epochs. This is with 2 stage classifier. I will try some other changes as well (thresholds, NMS, lighter augs) </p>",
          "rawMarkdown": "@hannes82 What image sizes are you using? I used 1024x1024 for VFNET and heavy augs and LB score is horrible at 0.11  at 10 epochs. This is with 2 stage classifier. I will try some other changes as well (thresholds, NMS, lighter augs) ",
          "replies": [
            {
              "id": 1227504,
              "postDate": "2021-03-05T15:46:38.333Z",
              "content": "<blockquote>\n  <p><a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> What image sizes are you using? I used 1024x1024 for VFNET and heavy augs and LB score is horrible at 0.11  at 10 epochs. This is with 2 stage classifier. I will try some other changes as well (thresholds, NMS, lighter augs)</p>\n</blockquote>\n<p>what is your CV  and conf_tresh used for submission?</p>",
              "rawMarkdown": "> @hannes82 What image sizes are you using? I used 1024x1024 for VFNET and heavy augs and LB score is horrible at 0.11  at 10 epochs. This is with 2 stage classifier. I will try some other changes as well (thresholds, NMS, lighter augs)\n\n\nwhat is your CV  and conf_tresh used for submission?"
            }
          ]
        },
        {
          "id": 1226868,
          "postDate": "2021-03-05T00:46:46.483Z",
          "content": "<p>I use 2x downsampled from original size. Didn't use too much augmentation. Using pretrained weights was important for me.</p>",
          "rawMarkdown": "I use 2x downsampled from original size. Didn't use too much augmentation. Using pretrained weights was important for me.",
          "votes": 1
        },
        {
          "id": 1226869,
          "postDate": "2021-03-05T00:51:30.827Z",
          "content": "<p><a href=\"https://www.kaggle.com/adrielcabral\" target=\"_blank\">@adrielcabral</a> how do you use the models? I just use !python train.py {config_file} </p>",
          "rawMarkdown": "@adrielcabral how do you use the models? I just use !python train.py {config_file} \n",
          "replies": [
            {
              "id": 1227256,
              "postDate": "2021-03-05T11:02:24.393Z",
              "content": "<blockquote>\n  <p><a href=\"https://www.kaggle.com/adrielcabral\" target=\"_blank\">@adrielcabral</a> how do you use the models? I just use !python train.py {config_file}</p>\n</blockquote>\n<p>I solved the problem, my bad 😅</p>",
              "rawMarkdown": "> @adrielcabral how do you use the models? I just use !python train.py {config_file}\n\nI solved the problem, my bad 😅"
            }
          ]
        },
        {
          "id": 1226870,
          "postDate": "2021-03-05T00:52:12.740Z",
          "content": "<p>Thanks. Yeah I am using pretrained weights. I think its the augs that is killing it. Will run some more experiments. </p>",
          "rawMarkdown": "Thanks. Yeah I am using pretrained weights. I think its the augs that is killing it. Will run some more experiments. "
        },
        {
          "id": 1227875,
          "postDate": "2021-03-05T22:06:40.523Z",
          "content": "<p><a href=\"https://www.kaggle.com/adrielcabral\" target=\"_blank\">@adrielcabral</a> <a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> If anyone of you is interested in teaming up let me know. I will have more time to work on this competition now. If not, all the best! </p>",
          "rawMarkdown": "@adrielcabral @hannes82 If anyone of you is interested in teaming up let me know. I will have more time to work on this competition now. If not, all the best! ",
          "votes": 1,
          "replies": [
            {
              "id": 1228008,
              "postDate": "2021-03-06T03:01:14.060Z",
              "content": "<blockquote>\n  <p><a href=\"https://www.kaggle.com/adrielcabral\" target=\"_blank\">@adrielcabral</a> <a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> If anyone of you is interested in teaming up let me know. I will have more time to work on this competition now. If not, all the best!</p>\n</blockquote>\n<p>I will try alone, but thanks for the invitation and good luck for all of us !</p>",
              "rawMarkdown": "> @adrielcabral @hannes82 If anyone of you is interested in teaming up let me know. I will have more time to work on this competition now. If not, all the best!\n\nI will try alone, but thanks for the invitation and good luck for all of us !",
              "votes": 1
            }
          ]
        },
        {
          "id": 1228031,
          "postDate": "2021-03-06T03:56:09.103Z",
          "content": "<p>no problem. good luck! </p>",
          "rawMarkdown": "no problem. good luck! ",
          "votes": 1
        },
        {
          "id": 1228438,
          "postDate": "2021-03-06T11:33:12.823Z",
          "content": "<p>I think this time I will also go solo. But thanks and good luck to you both!</p>",
          "rawMarkdown": "I think this time I will also go solo. But thanks and good luck to you both!",
          "votes": 2
        },
        {
          "id": 1229253,
          "postDate": "2021-03-07T07:49:19.080Z",
          "content": "<p>may I ask what is your only detection scores for both vfnet and yolov5? <a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> I can not evaluate my 2 class filter correctly ://</p>",
          "rawMarkdown": "may I ask what is your only detection scores for both vfnet and yolov5? @hannes82 I can not evaluate my 2 class filter correctly ://"
        },
        {
          "id": 1229500,
          "postDate": "2021-03-07T11:45:50.007Z",
          "content": "<p>I think for yolov5 it was just 0.152 (single fold, no TTA at that time).. for vfnet I never tried to be honest.</p>",
          "rawMarkdown": "I think for yolov5 it was just 0.152 (single fold, no TTA at that time).. for vfnet I never tried to be honest."
        },
        {
          "id": 1230278,
          "postDate": "2021-03-08T01:05:48.540Z",
          "content": "<p><a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> </p>\n<blockquote>\n  <p>I somehow never tried wandb, maybe I should.. NMS is already included in test_cfg in mmdetection and if I am not mistaken you can also change it to soft NMS.</p>\n</blockquote>\n<p>This is how you enable tensorboard and wandb for mmdet. Pretty useful:</p>\n<p><a href=\"https://mmdetection.readthedocs.io/en/latest/tutorials/customize_runtime.html#log-config\" target=\"_blank\">https://mmdetection.readthedocs.io/en/latest/tutorials/customize_runtime.html#log-config</a></p>",
          "rawMarkdown": "@hannes82 \n\n>I somehow never tried wandb, maybe I should.. NMS is already included in test_cfg in mmdetection and if I am not mistaken you can also change it to soft NMS.\n\nThis is how you enable tensorboard and wandb for mmdet. Pretty useful:\n\nhttps://mmdetection.readthedocs.io/en/latest/tutorials/customize_runtime.html#log-config",
          "votes": 1
        },
        {
          "id": 1230288,
          "postDate": "2021-03-08T01:16:38.543Z",
          "content": "<p>Nice, I see. Thanks!</p>",
          "rawMarkdown": "Nice, I see. Thanks!"
        },
        {
          "id": 1231287,
          "postDate": "2021-03-08T20:36:58.657Z",
          "content": "<p><a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> Which VFNET pretrained model were you using? </p>",
          "rawMarkdown": "@hannes82 Which VFNET pretrained model were you using? "
        },
        {
          "id": 1231319,
          "postDate": "2021-03-08T21:50:55.150Z",
          "content": "<p>This one: vfnet_r50_fpn_mdconv_c3-c5_mstrain_2x_coco_20201027pth-6879c318.pth</p>",
          "rawMarkdown": "This one: vfnet_r50_fpn_mdconv_c3-c5_mstrain_2x_coco_20201027pth-6879c318.pth",
          "replies": [
            {
              "id": 1234729,
              "postDate": "2021-03-11T14:30:29.293Z",
              "content": "<blockquote>\n  <p>This one: vfnet_r50_fpn_mdconv_c3-c5_mstrain_2x_coco_20201027pth-6879c318.pth</p>\n</blockquote>\n<p>I'm trying train Vfnet (the same model), but i get a lb score lower then yolov5x u can sharing what imgsize u use and if use muil scale ?</p>",
              "rawMarkdown": "> This one: vfnet_r50_fpn_mdconv_c3-c5_mstrain_2x_coco_20201027pth-6879c318.pth\n\nI'm trying train Vfnet (the same model), but i get a lb score lower then yolov5x u can sharing what imgsize u use and if use muil scale ?"
            }
          ]
        },
        {
          "id": 1231333,
          "postDate": "2021-03-08T22:04:47.947Z",
          "content": "<p>yeah resnet50 is what I tried as well. </p>",
          "rawMarkdown": "yeah resnet50 is what I tried as well. "
        },
        {
          "id": 1231958,
          "postDate": "2021-03-09T11:57:17.593Z",
          "content": "<p><a href=\"https://www.kaggle.com/adrielcabral\" target=\"_blank\">@adrielcabral</a> Thank you so much for sharing so much information on what model you used , the exact configuration and processing techniques like 2 class filter . This is huge . Thanks so much . Thats a very good position on LB with it being your first comp</p>",
          "rawMarkdown": "@adrielcabral Thank you so much for sharing so much information on what model you used , the exact configuration and processing techniques like 2 class filter . This is huge . Thanks so much . Thats a very good position on LB with it being your first comp",
          "votes": 1,
          "replies": [
            {
              "id": 1232026,
              "postDate": "2021-03-09T12:43:08.273Z",
              "content": "<blockquote>\n  <p><a href=\"https://www.kaggle.com/adrielcabral\" target=\"_blank\">@adrielcabral</a> Thank you so much for sharing so much information on what model you used , the exact configuration and processing techniques like 2 class filter . This is huge . Thanks so much . Thats a very good position on LB with it being your first comp</p>\n</blockquote>\n<p>Thanks!, I am very happy with your comment</p>",
              "rawMarkdown": "> @adrielcabral Thank you so much for sharing so much information on what model you used , the exact configuration and processing techniques like 2 class filter . This is huge . Thanks so much . Thats a very good position on LB with it being your first comp\n\nThanks!, I am very happy with your comment"
            }
          ]
        },
        {
          "id": 1232021,
          "postDate": "2021-03-09T12:37:26.750Z",
          "rawMarkdown": "",
          "isDeleted": true
        },
        {
          "id": 1234866,
          "postDate": "2021-03-11T16:26:21.180Z",
          "content": "<p><a href=\"https://www.kaggle.com/adrielcabral\" target=\"_blank\">@adrielcabral</a> I use the original images 2 times downsampled from <a href=\"https://www.kaggle.com/raddar/vinbigdata-competition-jpg-data-2x-downsampled\" target=\"_blank\">this</a> dataset. <br>\nAnd yes, I use multiscaling:     dict(<br>\n        type='Resize',<br>\n        img_scale=[(1333,480),(1333,960)],<br>\n        multiscale_mode='range',<br>\n        keep_ratio=True)<br>\nbut not at test time.<br>\nYou may need to adjust test_cfg for a high score (setting score thres to a low value and maybe iou_threshold to a lower value). I am sure you also adjusted the learning rate to your batch size.</p>",
          "rawMarkdown": "@adrielcabral I use the original images 2 times downsampled from [this](https://www.kaggle.com/raddar/vinbigdata-competition-jpg-data-2x-downsampled) dataset. \nAnd yes, I use multiscaling:     dict(\n        type='Resize',\n        img_scale=[(1333,480),(1333,960)],\n        multiscale_mode='range',\n        keep_ratio=True)\nbut not at test time.\nYou may need to adjust test_cfg for a high score (setting score thres to a low value and maybe iou_threshold to a lower value). I am sure you also adjusted the learning rate to your batch size.",
          "votes": 2
        },
        {
          "id": 1234891,
          "postDate": "2021-03-11T16:54:07.910Z",
          "content": "<p>Thanks for sharing :) !!!</p>",
          "rawMarkdown": "Thanks for sharing :) !!!",
          "votes": 1
        },
        {
          "id": 1234951,
          "postDate": "2021-03-11T17:44:09.777Z",
          "content": "<p><a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> Thanks for sharing so much!</p>",
          "rawMarkdown": "@hannes82 Thanks for sharing so much!",
          "votes": 1
        },
        {
          "id": 1236524,
          "postDate": "2021-03-13T08:11:01.217Z",
          "content": "<p><a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> Thank you for sharing! I have recently tried out mmdetection (starting with retinanet as baseline). However, there was a huge discrepancy between my CV and LB. On CV, the maP was around ~0.3, but LB score was 0.03. </p>",
          "rawMarkdown": "@hannes82 Thank you for sharing! I have recently tried out mmdetection (starting with retinanet as baseline). However, there was a huge discrepancy between my CV and LB. On CV, the maP was around ~0.3, but LB score was 0.03. ",
          "votes": 1
        },
        {
          "id": 1236721,
          "postDate": "2021-03-13T11:29:04.560Z",
          "content": "<p><a href=\"https://www.kaggle.com/angqx95\" target=\"_blank\">@angqx95</a> I think the LB score is too low to have something to do with the common variation we observe between CV and LB. Maybe it has something to do with how you split the data in train and validation data? Do you use MultilabelStratifiedKFold or GroupKFold?</p>",
          "rawMarkdown": "@angqx95 I think the LB score is too low to have something to do with the common variation we observe between CV and LB. Maybe it has something to do with how you split the data in train and validation data? Do you use MultilabelStratifiedKFold or GroupKFold?"
        },
        {
          "id": 1237065,
          "postDate": "2021-03-13T17:40:36.133Z",
          "content": "<p><a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> i used GroupKFold. Below is the code i use for inference</p>\n<pre><code>for idx, row in tqdm(sub.iterrows(), total=len(sub), position=0, leave=True):\n  img_id = row['image_id']\n  meta = test_meta[test_meta['image_id']==img_id]\n  orig_h, orig_w = meta['dim0'].item(), meta['dim1'].item()\n  h_ratio, w_ratio = orig_h/512, orig_w/512\n\n  #inference\n  img = mmcv.imread('/content/test/'+img_id+'.png')\n  result = inference_detector(model, img)\n  string = \"\"\n  for class_, class_array in enumerate(result):\n    if class_array.shape[0]:\n      class_array = class_array.tolist()\n      for array in class_array:\n        array[0] = array[0] * w_ratio\n        array[1] = array[1] * h_ratio\n        array[2] = array[2] * w_ratio\n        array[3] = array[3] * h_ratio\n        string += '{} {:.2f} {} {} {} {} '.format(int(class_), array[4], int(array[0]), int(array[1]), int(array[2]), int(array[3]))\n  if len(string) == 0:\n    string += \"14 1 0 0 1 1\"\n\n  sub.loc[idx, 'PredictionString'] = string\n</code></pre>\n<p>I can't seem to identify what is wrong with the inference process</p>",
          "rawMarkdown": "@hannes82 i used GroupKFold. Below is the code i use for inference\n```\nfor idx, row in tqdm(sub.iterrows(), total=len(sub), position=0, leave=True):\n  img_id = row['image_id']\n  meta = test_meta[test_meta['image_id']==img_id]\n  orig_h, orig_w = meta['dim0'].item(), meta['dim1'].item()\n  h_ratio, w_ratio = orig_h/512, orig_w/512\n\n  #inference\n  img = mmcv.imread('/content/test/'+img_id+'.png')\n  result = inference_detector(model, img)\n  string = \"\"\n  for class_, class_array in enumerate(result):\n    if class_array.shape[0]:\n      class_array = class_array.tolist()\n      for array in class_array:\n        array[0] = array[0] * w_ratio\n        array[1] = array[1] * h_ratio\n        array[2] = array[2] * w_ratio\n        array[3] = array[3] * h_ratio\n        string += '{} {:.2f} {} {} {} {} '.format(int(class_), array[4], int(array[0]), int(array[1]), int(array[2]), int(array[3]))\n  if len(string) == 0:\n    string += \"14 1 0 0 1 1\"\n\n  sub.loc[idx, 'PredictionString'] = string\n```\n\nI can't seem to identify what is wrong with the inference process\n"
        },
        {
          "id": 1237118,
          "postDate": "2021-03-13T19:15:03.713Z",
          "content": "<p>Looks good to me although I don't use inference_detector. Maybe 512 is a bit small for good detections!?</p>",
          "rawMarkdown": "Looks good to me although I don't use inference_detector. Maybe 512 is a bit small for good detections!?"
        },
        {
          "id": 1237334,
          "postDate": "2021-03-14T04:04:25.273Z",
          "content": "<p>Thanks! really appreciate your help. I will explore and see where it went wrong. Btw, may i ask what is the AP of your vfnet without 2-class classifier?</p>",
          "rawMarkdown": "Thanks! really appreciate your help. I will explore and see where it went wrong. Btw, may i ask what is the AP of your vfnet without 2-class classifier?"
        },
        {
          "id": 1237737,
          "postDate": "2021-03-14T12:09:50.733Z",
          "content": "<p><a href=\"https://www.kaggle.com/trushk\" target=\"_blank\">@trushk</a> Did you use vfnet?</p>",
          "rawMarkdown": "@trushk Did you use vfnet?"
        },
        {
          "id": 1237876,
          "postDate": "2021-03-14T13:39:59.160Z",
          "rawMarkdown": "",
          "isDeleted": true
        },
        {
          "id": 1237882,
          "postDate": "2021-03-14T13:40:22.853Z",
          "content": "<p><a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> i get 0.432 in CV with Vfnet but only 0.228 in LB score … i think that the problem is a high number of detections per image because my single model (yolov5) 0.248 lb generate a cv file of 1.6 mb, but vfnet generate a cv file of 3.7mb,  high numbers of FP. i don't know what to do … </p>",
          "rawMarkdown": "@hannes82 i get 0.432 in CV with Vfnet but only 0.228 in LB score ... i think that the problem is a high number of detections per image because my single model (yolov5) 0.248 lb generate a cv file of 1.6 mb, but vfnet generate a cv file of 3.7mb,  high numbers of FP. i don't know what to do ... "
        },
        {
          "id": 1237896,
          "postDate": "2021-03-14T13:47:30.173Z",
          "content": "<p>I am not sure if it is the high number of FP.. I basically also predict 300 bboxes per image :-)</p>\n<p>I think I also tried it ones with 0.4 fusion threshold and got CV 0.44/LB 0.24 opposed to CV 0.40/LB 0.27 with 0.6 fusion threshold (although I also used another seed/fold which could also be the reason for the variation..)</p>",
          "rawMarkdown": "I am not sure if it is the high number of FP.. I basically also predict 300 bboxes per image :-)\n\nI think I also tried it ones with 0.4 fusion threshold and got CV 0.44/LB 0.24 opposed to CV 0.40/LB 0.27 with 0.6 fusion threshold (although I also used another seed/fold which could also be the reason for the variation..)",
          "votes": 1,
          "replies": [
            {
              "id": 1238363,
              "postDate": "2021-03-14T23:35:09.593Z",
              "content": "<p>Is it CV score on pos only images ?</p>",
              "rawMarkdown": "Is it CV score on pos only images ?"
            }
          ]
        },
        {
          "id": 1237919,
          "postDate": "2021-03-14T13:56:48.450Z",
          "content": "<p><a href=\"https://www.kaggle.com/angqx95\" target=\"_blank\">@angqx95</a> My best Vfnet got 0.199 (0.275 with 2-class prediction model)</p>",
          "rawMarkdown": "@angqx95 My best Vfnet got 0.199 (0.275 with 2-class prediction model)",
          "votes": 1
        },
        {
          "id": 1237928,
          "postDate": "2021-03-14T14:00:21.657Z",
          "content": "<p><a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> U make any pre-processing, sorry if i making a lot of questions</p>",
          "rawMarkdown": " @hannes82 U make any pre-processing, sorry if i making a lot of questions"
        },
        {
          "id": 1237941,
          "postDate": "2021-03-14T14:10:13.547Z",
          "content": "<p>That's fine. Just WBF..</p>",
          "rawMarkdown": "That's fine. Just WBF..",
          "votes": 2
        },
        {
          "id": 1238217,
          "postDate": "2021-03-14T18:43:17.190Z",
          "content": "<p><a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> Thank you very much for all the replies :)</p>",
          "rawMarkdown": "@hannes82 Thank you very much for all the replies :)",
          "votes": 1
        },
        {
          "id": 1238316,
          "postDate": "2021-03-14T21:45:14.823Z",
          "content": "<p>You are welcome :-)</p>",
          "rawMarkdown": "You are welcome :-)"
        },
        {
          "id": 1239171,
          "postDate": "2021-03-15T14:02:59.523Z",
          "content": "<p><a href=\"https://www.kaggle.com/alexandrecc\" target=\"_blank\">@alexandrecc</a> yes positive images only</p>",
          "rawMarkdown": "@alexandrecc yes positive images only"
        },
        {
          "id": 1239334,
          "postDate": "2021-03-15T16:24:54.060Z",
          "content": "<p>I was wondering if you have tried using the VFNet variant: <strong>vfnet_x101_64x4d_fpn_mdconv_c3-c5_mstrain_2x_coco</strong>. I tried using this variant, which was supposedly to be the best, but the LB was not even close to 0.1</p>",
          "rawMarkdown": "I was wondering if you have tried using the VFNet variant: **vfnet_x101_64x4d_fpn_mdconv_c3-c5_mstrain_2x_coco**. I tried using this variant, which was supposedly to be the best, but the LB was not even close to 0.1"
        },
        {
          "id": 1240945,
          "postDate": "2021-03-16T18:47:03.453Z",
          "content": "<p><a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> when you said <code>I do post nms</code> was this for yolo?  I thought yolov5 already does nms by default: <a href=\"https://github.com/ultralytics/yolov5/blob/ed2c74218d6d46605cc5fa68ce9bd6ece213abe4/detect.py#L75\" target=\"_blank\">https://github.com/ultralytics/yolov5/blob/ed2c74218d6d46605cc5fa68ce9bd6ece213abe4/detect.py#L75</a></p>",
          "rawMarkdown": "@hannes82 when you said `I do post nms` was this for yolo?  I thought yolov5 already does nms by default: https://github.com/ultralytics/yolov5/blob/ed2c74218d6d46605cc5fa68ce9bd6ece213abe4/detect.py#L75"
        },
        {
          "id": 1240959,
          "postDate": "2021-03-16T18:59:58.037Z",
          "content": "<p><a href=\"https://www.kaggle.com/angqx95\" target=\"_blank\">@angqx95</a> I think I tried it but had to reduce batch size to stay in the memory limit of kaggle.. so I thought this could be the reason for my worse results..</p>",
          "rawMarkdown": "@angqx95 I think I tried it but had to reduce batch size to stay in the memory limit of kaggle.. so I thought this could be the reason for my worse results..",
          "votes": 1
        },
        {
          "id": 1240963,
          "postDate": "2021-03-16T19:02:21.173Z",
          "content": "<p><a href=\"https://www.kaggle.com/danshan\" target=\"_blank\">@danshan</a> yes yolov5 and mmdetection vfnet include nms by default.. I just changed the iou threshold.</p>",
          "rawMarkdown": "@danshan yes yolov5 and mmdetection vfnet include nms by default.. I just changed the iou threshold."
        },
        {
          "id": 1245641,
          "postDate": "2021-03-20T01:38:18.537Z",
          "content": "<p>Is this the default hyparameters?</p>",
          "rawMarkdown": "Is this the default hyparameters?"
        }
      ]
    },
    {
      "id": 1221436,
      "postDate": "2021-03-01T01:26:38.563Z",
      "content": "<p>What is your LB score for detection model only? since the detection might be yield instability while training especially in the rare cases</p>",
      "rawMarkdown": "What is your LB score for detection model only? since the detection might be yield instability while training especially in the rare cases\n",
      "replies": [
        {
          "id": 1221883,
          "postDate": "2021-03-01T11:25:24.530Z",
          "rawMarkdown": "",
          "isDeleted": true
        },
        {
          "id": 1222129,
          "postDate": "2021-03-01T15:15:07.027Z",
          "content": "<p>get 0.188 with the \"best model\" (conf_thres: 0.01)</p>",
          "rawMarkdown": "get 0.188 with the \"best model\" (conf_thres: 0.01)"
        },
        {
          "id": 1222157,
          "postDate": "2021-03-01T15:40:59.520Z",
          "content": "<p>That's quite high for yolo5x. You had to ensemble 5 folds to achieve that result right?</p>",
          "rawMarkdown": "That's quite high for yolo5x. You had to ensemble 5 folds to achieve that result right?"
        },
        {
          "id": 1222285,
          "postDate": "2021-03-01T17:19:16.723Z",
          "content": "<p>I use Colab for train ( don't have GPU), so … i choice only use 1 fold for test results </p>",
          "rawMarkdown": "I use Colab for train ( don't have GPU), so ... i choice only use 1 fold for test results "
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 1221394,
      "author_name": "Hannes Öhler",
      "author_url": "",
      "post_date": "2021-03-01T00:12:09.110000",
      "content": "<p>Yes, I experienced the same. IMO this is not specific to yolov5 but caused by very few cases of rare classes in the public test set. Your model may or may not predict well those cases. As shown <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/220761\" target=\"_blank\">here</a>, the LB score is much more stable for frequent classes. So maybe we can rely on LB score for frequent classes or CV score for model selection.</p>",
      "votes": 6,
      "replies": []
    },
    {
      "id": 1241717,
      "author_name": "Phat Tran",
      "author_url": "",
      "post_date": "2021-03-17T07:14:39.713000",
      "content": "<p>My scores are very unstable using Yolov5 too</p>\n<p>All models are yolov5x, and scoring after run <code>2 class filter</code></p>\n<table>\n<thead>\n<tr>\n<th>Model</th>\n<th>Val</th>\n<th>LB</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>Fold 0</td>\n<td>0.417</td>\n<td>0.195</td>\n</tr>\n<tr>\n<td>Fold 2</td>\n<td>0.419</td>\n<td>0.231</td>\n</tr>\n<tr>\n<td>Fold 4</td>\n<td>0.432</td>\n<td>0.217</td>\n</tr>\n<tr>\n<td>Ensemble (NMS .5)</td>\n<td>-</td>\n<td>0.225</td>\n</tr>\n</tbody>\n</table>\n<p>My data are split using <code>stratified k fold</code></p>",
      "votes": 1,
      "replies": [
        {
          "id": 1252879,
          "author_name": "nOpe",
          "author_url": "",
          "post_date": "2021-03-26T06:20:47.137000",
          "content": "<p>From ur experiments, what ensemble strategy works best? I tried several methods but my LB score couldn't improve.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1253081,
          "author_name": "Phat Tran",
          "author_url": "",
          "post_date": "2021-03-26T10:50:55.663000",
          "content": "<p>I could not reproduce my best result, If I remember correctly, I have used TTA and ensemble of 5 folds with iou 0.5 and score threshold 0.001</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1253718,
          "author_name": "nOpe",
          "author_url": "",
          "post_date": "2021-03-27T01:37:21.607000",
          "content": "<p>Thank for ur advisement.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1228729,
      "author_name": "Zepyhr99",
      "author_url": "",
      "post_date": "2021-03-06T17:39:21.723000",
      "content": "<p>During my model selection I experienced some model achieving high score in some certain classes (CV) therefore have high LB score while \"perfect\" model which achieved more well round number for all labels has lower LB score. For this reason, I think the variance is reasonable. </p>",
      "votes": 1,
      "replies": [
        {
          "id": 1228730,
          "author_name": "adriel cabral",
          "author_url": "",
          "post_date": "2021-03-06T17:41:28.893000",
          "content": "<p>Yeah, i think the same</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1228733,
          "author_name": "Zepyhr99",
          "author_url": "",
          "post_date": "2021-03-06T17:46:32.703000",
          "content": "<p>In addition, I think ensemble might be the key boost in this competition. Currently, my best model achieved 0.193 but I haven't found any more architectures to experiment with. I think I will go to cleaning data process.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1228112,
      "author_name": "Empty258",
      "author_url": "",
      "post_date": "2021-03-06T06:11:29.137000",
      "content": "<p><a href=\"https://www.kaggle.com/adrielcabral\" target=\"_blank\">@adrielcabral</a>   what aug did you use?</p>",
      "votes": 1,
      "replies": [
        {
          "id": 1228476,
          "author_name": "adriel cabral",
          "author_url": "",
          "post_date": "2021-03-06T12:50:05.127000",
          "content": "<p>hsv_h: 0.015<br>\nhsv_s: 0.7<br>\nhsv_v: 0.4<br>\ndegrees: 0.0<br>\ntranslate: 0.2<br>\nscale: 0.6<br>\nshear: 0.0<br>\nperspective: 0.0<br>\nflipud: 0.2<br>\nfliplr: 0.5<br>\nmosaic: 1.0<br>\nmixup: 0.0</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1228555,
          "author_name": "Empty258",
          "author_url": "",
          "post_date": "2021-03-06T14:45:01.020000",
          "content": "<p><a href=\"https://www.kaggle.com/adrielcabral\" target=\"_blank\">@adrielcabral</a> Did you use nms or wbf on dataset before train ?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1228700,
          "author_name": "adriel cabral",
          "author_url": "",
          "post_date": "2021-03-06T16:52:08.247000",
          "content": "<p>I use a pre-processing (fusion bboxes with IOU &gt; 0.4)</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 1227987,
      "author_name": "NakedKoala",
      "author_url": "",
      "post_date": "2021-03-06T02:11:18.200000",
      "content": "<p><a href=\"https://www.kaggle.com/adrielcabral\" target=\"_blank\">@adrielcabral</a> </p>\n<p>I am using YOLOV5 too but my CV stuck around 0.33</p>\n<p>Can I ask a few questions about your setup ? </p>\n<ol>\n<li>which release of yolov5 are you using ? I  am using 4.0</li>\n<li>Did you train the model with only abnormal data ( filter out images with no finding )</li>\n<li>What NMS IOU did you use for inference ?  Did you leave it at 0.45 ( default ) ? </li>\n<li>Any modification to the model ?  Or did you just use it out of the box ?   </li>\n</ol>\n<p>Thanks  for the info</p>",
      "votes": 1,
      "replies": [
        {
          "id": 1228002,
          "author_name": "adriel cabral",
          "author_url": "",
          "post_date": "2021-03-06T02:56:22.877000",
          "content": "<p>1- I use 4.0 too<br>\n2- i tried add negative samples (empty txts files) in training, but get less lb score. I use only abnormal data<br>\n3- NMS IOU: 0.5<br>\n4- no modification</p>\n<p>Did you try to use my settings?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1228035,
          "author_name": "NakedKoala",
          "author_url": "",
          "post_date": "2021-03-06T04:07:41.197000",
          "content": "<p>Yes, I copied your hyp.scratch.yaml trying to reproduce the 0.40+ CV….  but failed.</p>\n<p>2 further questions</p>\n<p>1) How did you split your data ?   I did 5 fold splits with sklearn's group KFold<br>\n2) What dataset do you use ? Do you use the original competition dataset ( with original aspect ratio ) ? Or did you use the 512 x 512 / 1024 x 1024 resized ones people having been sharing ?</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1228473,
          "author_name": "adriel cabral",
          "author_url": "",
          "post_date": "2021-03-06T12:48:56.077000",
          "content": "<p>1 - I use group KFold too<br>\n2 - i use 1024x1024 resized data, but in training i choice 640x640</p>\n<p>I use a pre-processing (fusion bboxes with IOU &gt; 0.4)</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1228574,
          "author_name": "",
          "author_url": "",
          "post_date": "2021-03-06T15:03:35.093000",
          "content": "",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1228924,
          "author_name": "NakedKoala",
          "author_url": "",
          "post_date": "2021-03-06T22:27:38.333000",
          "content": "<p>Thanks a lot for all the info provided so far. </p>\n<p>I added the box fusion pre-processing.. Now my local CV is a bit above 0.40.  However, the LB result (just the detector, without 2nd stage classifier) gets worse than my previous version ( with CV score less than 0.33).  </p>\n<p>Is there anything special that you are doing during inference ?  Any additional processing ?  ( I did conf-thres=0.01 + IOU=0.5)</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1228999,
          "author_name": "adriel cabral",
          "author_url": "",
          "post_date": "2021-03-07T01:03:15.100000",
          "content": "<p>Using 2 class filter which model is better ? with cv &gt; 0.4 or cv 0.33 ? i tell all that i do, sorry i don't make more processing</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1245460,
          "author_name": "Kuan Zhang",
          "author_url": "",
          "post_date": "2021-03-19T20:05:46.027000",
          "content": "<p>The same issue observed in my training with yolov5. Have you managed to figure out what’s going on for the pre-fused case? Thanks!</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1245640,
      "author_name": "Zekun",
      "author_url": "",
      "post_date": "2021-03-20T01:34:32.927000",
      "content": "<p>Hello,which parameters do you choose?I only get 0.33+ on cv.</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1233921,
      "author_name": "Kuan Zhang",
      "author_url": "",
      "post_date": "2021-03-10T19:17:21.190000",
      "content": "<p>Hi, Adriel, I use yolov3 (with 2 class filter) and have a LB score 2.3. I am trying with yolo5X now, but only got LB around 2.0.  <br>\nYolo5x should achieve better performance than yolov3. Do you know any possible reasons? Thanks!<br>\nFYI, I already enable the \"multi-label\" for nms in yolov5.</p>",
      "votes": 0,
      "replies": [
        {
          "id": 1233987,
          "author_name": "adriel cabral",
          "author_url": "",
          "post_date": "2021-03-10T20:45:24.867000",
          "content": "<ul>\n<li>what parameters do you use?</li>\n<li>img size</li>\n<li>make any pre, pos processing ?</li>\n</ul>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1233999,
          "author_name": "Kuan Zhang",
          "author_url": "",
          "post_date": "2021-03-10T21:00:45.413000",
          "content": "<p>Thanks for your reply!</p>\n<ul>\n<li>I use the default parameters.</li>\n<li>Img size 1024x1024.</li>\n<li>Since the default parameters already dealt with augmentation, I didn't do any extra pre.<br>\nAnd for Pos, I use conf = 0.001, and IOU = 0.4.<br>\nBasically, I use the same criterion for yolov3 and 5. Thanks! </li>\n</ul>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1234008,
          "author_name": "adriel cabral",
          "author_url": "",
          "post_date": "2021-03-10T21:19:43.030000",
          "content": "<p>I make a pre-processing (fusion bboxes with IOU &gt; 0.4) so …<br>\nU should try use my  hyperparameters, probably will not work for CV if u don't make the same pre-processing and my imgsize is 640x640</p>\n<p>I hope I've helped :)</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1234012,
          "author_name": "Kuan Zhang",
          "author_url": "",
          "post_date": "2021-03-10T21:24:50.630000",
          "content": "<p>Thanks! Will try your hyperparameters, and the fusion preprocessing.<br>\nBesides, any specific reason for using 640 instead of 1024?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1234019,
          "author_name": "adriel cabral",
          "author_url": "",
          "post_date": "2021-03-10T21:33:45.307000",
          "content": "<p>1 - I tried use imgsize 1024x1024 but I got a lower lb score (old hyperparameters), if u want u can try now with the new hyperparameters.</p>\n<p>2 - 1024x1024 take a long time ( i use colab don't have GPU)</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1239288,
          "author_name": "Kuan Zhang",
          "author_url": "",
          "post_date": "2021-03-15T15:35:19.157000",
          "content": "<p>Thanks for sharing, Adriel :)  I'd like to hear your opinions on the following points:</p>\n<p>-- For YOLOv5x, I guess you already enabled the \"multi-label\" by modifying this line in <em>general.py</em>: <br>\n     multi_label &amp;= nc &gt; 1, am I correct?  Besides, do you use TTA for prediction?  </p>\n<p>--  Have you achieved a better score by using vfnet than yolov5? </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1239341,
          "author_name": "adriel cabral",
          "author_url": "",
          "post_date": "2021-03-15T16:30:28.187000",
          "content": "<ul>\n<li>yeah,  I modified general.py file and TTA decrease lb score (for me), but u should try to use because others have a increase like <a href=\"https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/225391\" target=\"_blank\">here</a></li>\n<li>i get better score for 1 single fold, yolov5x (0.248) VFnet (0.255), but no is better than ensemble weights (0.268), i 'm tring work with VFnet</li>\n</ul>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1239366,
          "author_name": "Kuan Zhang",
          "author_url": "",
          "post_date": "2021-03-15T16:50:46.043000",
          "content": "<p>Got it, thank you! I am working on YOLOv5, and will also try VFnet. <br>\nHope we could reach 0.3 sooner or later :) </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1239374,
          "author_name": "adriel cabral",
          "author_url": "",
          "post_date": "2021-03-15T16:55:53.883000",
          "content": "<p>I hope too ✌️ That is my dream 😂😂</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1244427,
          "author_name": "Kuan Zhang",
          "author_url": "",
          "post_date": "2021-03-19T01:32:45.770000",
          "content": "<p>Dear Adriel, I obtained 0.17+ for best yolov5 on the original data, but always get much lower values (0.15-) on pre-fusioned data. I tried nms and wbf (0.4 threshold), but neither works. Do you have any idea? I am currently using your configuration. It looks great.👍</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1244519,
          "author_name": "Phat Tran",
          "author_url": "",
          "post_date": "2021-03-19T03:24:02.947000",
          "content": "<p>Well, I saw many people get a higher score with original data (with many overlapping boxes), But I don't know why. It should more reasonable if preprocessed data (fusion box) work better.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1245032,
          "author_name": "adriel cabral",
          "author_url": "",
          "post_date": "2021-03-19T12:03:19.783000",
          "content": "<p><a href=\"https://www.kaggle.com/kuanzhang\" target=\"_blank\">@kuanzhang</a> sorry but i no have idea, I tried to use the original data at the start of the competition (for a long time), but I only get 0.219 in the lb score.</p>\n<ul>\n<li>I don't know if i do \"NMS\" because i do my own \"fusion bboxes\" , but looking at the results of NMS and mine, they are very similar.</li>\n</ul>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1245646,
          "author_name": "Kuan Zhang",
          "author_url": "",
          "post_date": "2021-03-20T01:49:11.970000",
          "content": "<p>Thanks, Adriel! Yeah, it is a little weird on that comparison. I guess I can only have to keep trying 😛</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1245647,
          "author_name": "Zekun",
          "author_url": "",
          "post_date": "2021-03-20T01:53:39.187000",
          "content": "<p>Hello,did you train on kaggle?When I use 1024 size,the cpu will Out Of Memory….</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1225114,
      "author_name": "YutongTeng",
      "author_url": "",
      "post_date": "2021-03-03T10:29:54.873000",
      "content": "<p>I have used yolov5m(release 1.0) to get LB 0.186. But on my val set, the best map@iou0.5 is 0.26</p>\n<p>Inference detail: <br>\nTTA: no<br>\nInput_size: 640x640<br>\nscore thres: 0.1</p>\n<p>does this means I shuold try bigger model(yolov5x) like you?</p>",
      "votes": 0,
      "replies": [
        {
          "id": 1225132,
          "author_name": "adriel cabral",
          "author_url": "",
          "post_date": "2021-03-03T10:45:55.110000",
          "content": "<p>using my configuration you get only 0.26 in val set ? well, i tryed yolov5x and yolov5l and get <br>\nsimilar results (yolov5x is a little better), you should try use too</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1228061,
          "author_name": "Empty258",
          "author_url": "",
          "post_date": "2021-03-06T04:50:35.520000",
          "content": "<p><a href=\"https://www.kaggle.com/tenggyut\" target=\"_blank\">@tenggyut</a> did you use 2-class fillter</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1231918,
          "author_name": "YutongTeng",
          "author_url": "",
          "post_date": "2021-03-09T11:29:22.423000",
          "content": "<p>no, I haven't</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 1225028,
      "author_name": "nOpe",
      "author_url": "",
      "post_date": "2021-03-03T08:55:35.730000",
      "content": "<p>Thank for sharing. How much did 2 class filter improve ur LB score than only using single yolov5?</p>",
      "votes": 0,
      "replies": [
        {
          "id": 1225127,
          "author_name": "adriel cabral",
          "author_url": "",
          "post_date": "2021-03-03T10:42:24.927000",
          "content": "<p>around of (0.04-0.06)</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1228725,
          "author_name": "Linus Johansson",
          "author_url": "",
          "post_date": "2021-03-06T17:24:16.733000",
          "content": "<p><a href=\"https://www.kaggle.com/adrielcabral\" target=\"_blank\">@adrielcabral</a> what thresholds did you use?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1228731,
          "author_name": "adriel cabral",
          "author_url": "",
          "post_date": "2021-03-06T17:44:06.120000",
          "content": "<p>i use class filter of this <a href=\"https://www.kaggle.com/awsaf49/vinbigdata-2-class-filter\" target=\"_blank\">notebook</a> </p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 1224788,
      "author_name": "Phat Tran",
      "author_url": "",
      "post_date": "2021-03-03T04:21:07.707000",
      "content": "<p>Hi, in Yolov5 by default are evaluated by mAP@.5. Are your CV measured at 0.5 IOU or 0.4 (competition metric)</p>",
      "votes": 0,
      "replies": [
        {
          "id": 1225125,
          "author_name": "adriel cabral",
          "author_url": "",
          "post_date": "2021-03-03T10:40:52.790000",
          "content": "<p>at 0.5 IOU</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1222635,
      "author_name": "Trushant Kalyanpur",
      "author_url": "",
      "post_date": "2021-03-02T01:44:42.677000",
      "content": "<p>I seeing pretty bad CV to LB correlation for yolov5. I have local 0.32+ and LB 0.15. What image sizes are you using? I tried 640 with yolo5x</p>",
      "votes": 0,
      "replies": [
        {
          "id": 1223011,
          "author_name": "adriel cabral",
          "author_url": "",
          "post_date": "2021-03-02T11:21:21.850000",
          "content": "<p>640x640 too</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1221592,
      "author_name": "Nguyen Phuc Dat",
      "author_url": "",
      "post_date": "2021-03-01T05:40:46.003000",
      "content": "<p>May I ask what is your configuration? </p>",
      "votes": 0,
      "replies": [
        {
          "id": 1221881,
          "author_name": "adriel cabral",
          "author_url": "",
          "post_date": "2021-03-01T11:24:35.907000",
          "content": "<p>Yeah<br>\nMy configuration:</p>\n<p>lr0: 0.01<br>\nlrf: 0.032<br>\nmomentum: 0.937<br>\nweight_decay: 0.0005<br>\nwarmup_epochs: 3.0<br>\nwarmup_momentum: 0.8<br>\nwarmup_bias_lr: 0.1<br>\nbox: 0.1<br>\ncls: 1.0<br>\ncls_pw: 0.5<br>\nobj: 2.0<br>\nobj_pw: 0.5<br>\niou_t: 0.2<br>\nanchor_t: 4.0<br>\nanchors: 0<br>\nfl_gamma: 0.0<br>\nhsv_h: 0.015<br>\nhsv_s: 0.7<br>\nhsv_v: 0.4<br>\ndegrees: 0.0<br>\ntranslate: 0.2<br>\nscale: 0.6<br>\nshear: 0.0<br>\nperspective: 0.0<br>\nflipud: 0.2<br>\nfliplr: 0.5<br>\nmosaic: 1.0<br>\nmixup: 0.0</p>\n<p>training for 50 epochs </p>",
          "votes": 4,
          "replies": []
        },
        {
          "id": 1222425,
          "author_name": "Hannes Öhler",
          "author_url": "",
          "post_date": "2021-03-01T19:28:03.213000",
          "content": "<p>Oh thanks for sharing! How did you choose the parameters?</p>\n<p>With respect to other models, I can recommend mmdetection, it's a great library for detection and is relatively easy to use, just need to convert the data to coco format and use a config file. </p>",
          "votes": 0,
          "replies": [
            {
              "id": 1225629,
              "author_name": "adriel cabral",
              "author_url": "",
              "post_date": "2021-03-03T19:00:45.293000",
              "content": "<blockquote>\n  <p>With respect to other models, I can recommend mmdetection, it's a great library for detection and is relatively easy to use, just need to convert the data to coco format and use a config file.</p>\n</blockquote>\n<p>convert my data to coco format, but when a go try train i get this erro:</p>\n<p>Expected condition, x and y to be on the same device, but condition is on cuda:0 and x and y are on cuda:0 and cpu respectively</p>\n<p>u know how fix it ?</p>",
              "votes": 0,
              "replies": []
            }
          ]
        },
        {
          "id": 1222619,
          "author_name": "adriel cabral",
          "author_url": "",
          "post_date": "2021-03-02T01:18:42.583000",
          "content": "<p>I did several tests and arrived at these hyperparameters (if you want, u can increase [obj_pw], but will increase a number of FP). I will try use mmdetection i see that have <br>\nvarious models, very nice</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1226085,
          "author_name": "Trushant Kalyanpur",
          "author_url": "",
          "post_date": "2021-03-04T08:11:19.503000",
          "content": "<p><a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> are you getting better results with mmdet models vs yolo5*? </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1226105,
          "author_name": "Trushant Kalyanpur",
          "author_url": "",
          "post_date": "2021-03-04T08:24:34.860000",
          "content": "<blockquote>\n  <p>Expected condition, x and y to be on the same device, but condition is on cuda:0 and x and y are on cuda:0 and cpu respectively</p>\n</blockquote>\n<p><a href=\"https://www.kaggle.com/adrielcabral\" target=\"_blank\">@adrielcabral</a>  are you trying some custom models? it looks like the pytorch device is CPU for some tensors and GPU for others. I dont see this while trying out of the box models in mmdetect. </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1226242,
          "author_name": "Hannes Öhler",
          "author_url": "",
          "post_date": "2021-03-04T11:21:56.670000",
          "content": "<p><a href=\"https://www.kaggle.com/trushk\" target=\"_blank\">@trushk</a> yes, LB 0.258 with VfNet vs. 0.248 Yolov5. </p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1226534,
          "author_name": "Trushant Kalyanpur",
          "author_url": "",
          "post_date": "2021-03-04T16:03:24.157000",
          "content": "<p>Thanks <a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> . That is promising. Are you doing bbox processing before train and after infer. My yolo score is still not that high with pre nms. </p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1226695,
          "author_name": "Hannes Öhler",
          "author_url": "",
          "post_date": "2021-03-04T18:42:58.123000",
          "content": "<p>I now do wbf before VfNet, but I am not sure it helps in terms of LB score. In case of yolov5 it didnt help.<br>\nI do post nms - this definetely helps..</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1226708,
          "author_name": "Trushant Kalyanpur",
          "author_url": "",
          "post_date": "2021-03-04T18:58:58.887000",
          "content": "<p>thanks! that gives me some motivation to go down the path I had planned to do some more post proc. BTW mmdetect is pretty cool so far! it would be great if they have dasboards similar to wandb for yolo</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1226745,
          "author_name": "Hannes Öhler",
          "author_url": "",
          "post_date": "2021-03-04T19:56:38.783000",
          "content": "<p>I somehow never tried wandb, maybe I should.. NMS is already included in test_cfg in mmdetection and if I am not mistaken you can also change it to soft NMS.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1226753,
          "author_name": "Trushant Kalyanpur",
          "author_url": "",
          "post_date": "2021-03-04T20:12:57.533000",
          "content": "<p>Ah good to know. Looking at the model config it looks like it uses NMS by default</p>\n<blockquote>\n  <p>test_cfg = dict(<br>\n      nms_pre=1000,<br>\n      min_bbox_size=0,<br>\n      score_thr=0.05,<br>\n      nms=dict(type='nms', iou_threshold=0.6),<br>\n      max_per_img=100)</p>\n</blockquote>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1226865,
          "author_name": "Trushant Kalyanpur",
          "author_url": "",
          "post_date": "2021-03-05T00:41:49.713000",
          "content": "<p><a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> What image sizes are you using? I used 1024x1024 for VFNET and heavy augs and LB score is horrible at 0.11  at 10 epochs. This is with 2 stage classifier. I will try some other changes as well (thresholds, NMS, lighter augs) </p>",
          "votes": 0,
          "replies": [
            {
              "id": 1227504,
              "author_name": "adriel cabral",
              "author_url": "",
              "post_date": "2021-03-05T15:46:38.333000",
              "content": "<blockquote>\n  <p><a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> What image sizes are you using? I used 1024x1024 for VFNET and heavy augs and LB score is horrible at 0.11  at 10 epochs. This is with 2 stage classifier. I will try some other changes as well (thresholds, NMS, lighter augs)</p>\n</blockquote>\n<p>what is your CV  and conf_tresh used for submission?</p>",
              "votes": 0,
              "replies": []
            }
          ]
        },
        {
          "id": 1226868,
          "author_name": "Hannes Öhler",
          "author_url": "",
          "post_date": "2021-03-05T00:46:46.483000",
          "content": "<p>I use 2x downsampled from original size. Didn't use too much augmentation. Using pretrained weights was important for me.</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1226869,
          "author_name": "Hannes Öhler",
          "author_url": "",
          "post_date": "2021-03-05T00:51:30.827000",
          "content": "<p><a href=\"https://www.kaggle.com/adrielcabral\" target=\"_blank\">@adrielcabral</a> how do you use the models? I just use !python train.py {config_file} </p>",
          "votes": 0,
          "replies": [
            {
              "id": 1227256,
              "author_name": "adriel cabral",
              "author_url": "",
              "post_date": "2021-03-05T11:02:24.393000",
              "content": "<blockquote>\n  <p><a href=\"https://www.kaggle.com/adrielcabral\" target=\"_blank\">@adrielcabral</a> how do you use the models? I just use !python train.py {config_file}</p>\n</blockquote>\n<p>I solved the problem, my bad 😅</p>",
              "votes": 0,
              "replies": []
            }
          ]
        },
        {
          "id": 1226870,
          "author_name": "Trushant Kalyanpur",
          "author_url": "",
          "post_date": "2021-03-05T00:52:12.740000",
          "content": "<p>Thanks. Yeah I am using pretrained weights. I think its the augs that is killing it. Will run some more experiments. </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1227875,
          "author_name": "Trushant Kalyanpur",
          "author_url": "",
          "post_date": "2021-03-05T22:06:40.523000",
          "content": "<p><a href=\"https://www.kaggle.com/adrielcabral\" target=\"_blank\">@adrielcabral</a> <a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> If anyone of you is interested in teaming up let me know. I will have more time to work on this competition now. If not, all the best! </p>",
          "votes": 1,
          "replies": [
            {
              "id": 1228008,
              "author_name": "adriel cabral",
              "author_url": "",
              "post_date": "2021-03-06T03:01:14.060000",
              "content": "<blockquote>\n  <p><a href=\"https://www.kaggle.com/adrielcabral\" target=\"_blank\">@adrielcabral</a> <a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> If anyone of you is interested in teaming up let me know. I will have more time to work on this competition now. If not, all the best!</p>\n</blockquote>\n<p>I will try alone, but thanks for the invitation and good luck for all of us !</p>",
              "votes": 1,
              "replies": []
            }
          ]
        },
        {
          "id": 1228031,
          "author_name": "Trushant Kalyanpur",
          "author_url": "",
          "post_date": "2021-03-06T03:56:09.103000",
          "content": "<p>no problem. good luck! </p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1228438,
          "author_name": "Hannes Öhler",
          "author_url": "",
          "post_date": "2021-03-06T11:33:12.823000",
          "content": "<p>I think this time I will also go solo. But thanks and good luck to you both!</p>",
          "votes": 2,
          "replies": []
        },
        {
          "id": 1229253,
          "author_name": "ZLLA",
          "author_url": "",
          "post_date": "2021-03-07T07:49:19.080000",
          "content": "<p>may I ask what is your only detection scores for both vfnet and yolov5? <a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> I can not evaluate my 2 class filter correctly ://</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1229500,
          "author_name": "Hannes Öhler",
          "author_url": "",
          "post_date": "2021-03-07T11:45:50.007000",
          "content": "<p>I think for yolov5 it was just 0.152 (single fold, no TTA at that time).. for vfnet I never tried to be honest.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1230278,
          "author_name": "Trushant Kalyanpur",
          "author_url": "",
          "post_date": "2021-03-08T01:05:48.540000",
          "content": "<p><a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> </p>\n<blockquote>\n  <p>I somehow never tried wandb, maybe I should.. NMS is already included in test_cfg in mmdetection and if I am not mistaken you can also change it to soft NMS.</p>\n</blockquote>\n<p>This is how you enable tensorboard and wandb for mmdet. Pretty useful:</p>\n<p><a href=\"https://mmdetection.readthedocs.io/en/latest/tutorials/customize_runtime.html#log-config\" target=\"_blank\">https://mmdetection.readthedocs.io/en/latest/tutorials/customize_runtime.html#log-config</a></p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1230288,
          "author_name": "Hannes Öhler",
          "author_url": "",
          "post_date": "2021-03-08T01:16:38.543000",
          "content": "<p>Nice, I see. Thanks!</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1231287,
          "author_name": "Trushant Kalyanpur",
          "author_url": "",
          "post_date": "2021-03-08T20:36:58.657000",
          "content": "<p><a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> Which VFNET pretrained model were you using? </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1231319,
          "author_name": "Hannes Öhler",
          "author_url": "",
          "post_date": "2021-03-08T21:50:55.150000",
          "content": "<p>This one: vfnet_r50_fpn_mdconv_c3-c5_mstrain_2x_coco_20201027pth-6879c318.pth</p>",
          "votes": 0,
          "replies": [
            {
              "id": 1234729,
              "author_name": "adriel cabral",
              "author_url": "",
              "post_date": "2021-03-11T14:30:29.293000",
              "content": "<blockquote>\n  <p>This one: vfnet_r50_fpn_mdconv_c3-c5_mstrain_2x_coco_20201027pth-6879c318.pth</p>\n</blockquote>\n<p>I'm trying train Vfnet (the same model), but i get a lb score lower then yolov5x u can sharing what imgsize u use and if use muil scale ?</p>",
              "votes": 0,
              "replies": []
            }
          ]
        },
        {
          "id": 1231333,
          "author_name": "Trushant Kalyanpur",
          "author_url": "",
          "post_date": "2021-03-08T22:04:47.947000",
          "content": "<p>yeah resnet50 is what I tried as well. </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1231958,
          "author_name": "Tensor Girl",
          "author_url": "",
          "post_date": "2021-03-09T11:57:17.593000",
          "content": "<p><a href=\"https://www.kaggle.com/adrielcabral\" target=\"_blank\">@adrielcabral</a> Thank you so much for sharing so much information on what model you used , the exact configuration and processing techniques like 2 class filter . This is huge . Thanks so much . Thats a very good position on LB with it being your first comp</p>",
          "votes": 1,
          "replies": [
            {
              "id": 1232026,
              "author_name": "adriel cabral",
              "author_url": "",
              "post_date": "2021-03-09T12:43:08.273000",
              "content": "<blockquote>\n  <p><a href=\"https://www.kaggle.com/adrielcabral\" target=\"_blank\">@adrielcabral</a> Thank you so much for sharing so much information on what model you used , the exact configuration and processing techniques like 2 class filter . This is huge . Thanks so much . Thats a very good position on LB with it being your first comp</p>\n</blockquote>\n<p>Thanks!, I am very happy with your comment</p>",
              "votes": 0,
              "replies": []
            }
          ]
        },
        {
          "id": 1232021,
          "author_name": "",
          "author_url": "",
          "post_date": "2021-03-09T12:37:26.750000",
          "content": "",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1234866,
          "author_name": "Hannes Öhler",
          "author_url": "",
          "post_date": "2021-03-11T16:26:21.180000",
          "content": "<p><a href=\"https://www.kaggle.com/adrielcabral\" target=\"_blank\">@adrielcabral</a> I use the original images 2 times downsampled from <a href=\"https://www.kaggle.com/raddar/vinbigdata-competition-jpg-data-2x-downsampled\" target=\"_blank\">this</a> dataset. <br>\nAnd yes, I use multiscaling:     dict(<br>\n        type='Resize',<br>\n        img_scale=[(1333,480),(1333,960)],<br>\n        multiscale_mode='range',<br>\n        keep_ratio=True)<br>\nbut not at test time.<br>\nYou may need to adjust test_cfg for a high score (setting score thres to a low value and maybe iou_threshold to a lower value). I am sure you also adjusted the learning rate to your batch size.</p>",
          "votes": 2,
          "replies": []
        },
        {
          "id": 1234891,
          "author_name": "adriel cabral",
          "author_url": "",
          "post_date": "2021-03-11T16:54:07.910000",
          "content": "<p>Thanks for sharing :) !!!</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1234951,
          "author_name": "Matthieu Planté",
          "author_url": "",
          "post_date": "2021-03-11T17:44:09.777000",
          "content": "<p><a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> Thanks for sharing so much!</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1236524,
          "author_name": "aqx",
          "author_url": "",
          "post_date": "2021-03-13T08:11:01.217000",
          "content": "<p><a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> Thank you for sharing! I have recently tried out mmdetection (starting with retinanet as baseline). However, there was a huge discrepancy between my CV and LB. On CV, the maP was around ~0.3, but LB score was 0.03. </p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1236721,
          "author_name": "Hannes Öhler",
          "author_url": "",
          "post_date": "2021-03-13T11:29:04.560000",
          "content": "<p><a href=\"https://www.kaggle.com/angqx95\" target=\"_blank\">@angqx95</a> I think the LB score is too low to have something to do with the common variation we observe between CV and LB. Maybe it has something to do with how you split the data in train and validation data? Do you use MultilabelStratifiedKFold or GroupKFold?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1237065,
          "author_name": "aqx",
          "author_url": "",
          "post_date": "2021-03-13T17:40:36.133000",
          "content": "<p><a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> i used GroupKFold. Below is the code i use for inference</p>\n<pre><code>for idx, row in tqdm(sub.iterrows(), total=len(sub), position=0, leave=True):\n  img_id = row['image_id']\n  meta = test_meta[test_meta['image_id']==img_id]\n  orig_h, orig_w = meta['dim0'].item(), meta['dim1'].item()\n  h_ratio, w_ratio = orig_h/512, orig_w/512\n\n  #inference\n  img = mmcv.imread('/content/test/'+img_id+'.png')\n  result = inference_detector(model, img)\n  string = \"\"\n  for class_, class_array in enumerate(result):\n    if class_array.shape[0]:\n      class_array = class_array.tolist()\n      for array in class_array:\n        array[0] = array[0] * w_ratio\n        array[1] = array[1] * h_ratio\n        array[2] = array[2] * w_ratio\n        array[3] = array[3] * h_ratio\n        string += '{} {:.2f} {} {} {} {} '.format(int(class_), array[4], int(array[0]), int(array[1]), int(array[2]), int(array[3]))\n  if len(string) == 0:\n    string += \"14 1 0 0 1 1\"\n\n  sub.loc[idx, 'PredictionString'] = string\n</code></pre>\n<p>I can't seem to identify what is wrong with the inference process</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1237118,
          "author_name": "Hannes Öhler",
          "author_url": "",
          "post_date": "2021-03-13T19:15:03.713000",
          "content": "<p>Looks good to me although I don't use inference_detector. Maybe 512 is a bit small for good detections!?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1237334,
          "author_name": "aqx",
          "author_url": "",
          "post_date": "2021-03-14T04:04:25.273000",
          "content": "<p>Thanks! really appreciate your help. I will explore and see where it went wrong. Btw, may i ask what is the AP of your vfnet without 2-class classifier?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1237737,
          "author_name": "Nguyen Phuc Dat",
          "author_url": "",
          "post_date": "2021-03-14T12:09:50.733000",
          "content": "<p><a href=\"https://www.kaggle.com/trushk\" target=\"_blank\">@trushk</a> Did you use vfnet?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1237876,
          "author_name": "",
          "author_url": "",
          "post_date": "2021-03-14T13:39:59.160000",
          "content": "",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1237882,
          "author_name": "adriel cabral",
          "author_url": "",
          "post_date": "2021-03-14T13:40:22.853000",
          "content": "<p><a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> i get 0.432 in CV with Vfnet but only 0.228 in LB score … i think that the problem is a high number of detections per image because my single model (yolov5) 0.248 lb generate a cv file of 1.6 mb, but vfnet generate a cv file of 3.7mb,  high numbers of FP. i don't know what to do … </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1237896,
          "author_name": "Hannes Öhler",
          "author_url": "",
          "post_date": "2021-03-14T13:47:30.173000",
          "content": "<p>I am not sure if it is the high number of FP.. I basically also predict 300 bboxes per image :-)</p>\n<p>I think I also tried it ones with 0.4 fusion threshold and got CV 0.44/LB 0.24 opposed to CV 0.40/LB 0.27 with 0.6 fusion threshold (although I also used another seed/fold which could also be the reason for the variation..)</p>",
          "votes": 1,
          "replies": [
            {
              "id": 1238363,
              "author_name": "Alexandre Cadrin-Chênevert",
              "author_url": "",
              "post_date": "2021-03-14T23:35:09.593000",
              "content": "<p>Is it CV score on pos only images ?</p>",
              "votes": 0,
              "replies": []
            }
          ]
        },
        {
          "id": 1237919,
          "author_name": "Hannes Öhler",
          "author_url": "",
          "post_date": "2021-03-14T13:56:48.450000",
          "content": "<p><a href=\"https://www.kaggle.com/angqx95\" target=\"_blank\">@angqx95</a> My best Vfnet got 0.199 (0.275 with 2-class prediction model)</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1237928,
          "author_name": "adriel cabral",
          "author_url": "",
          "post_date": "2021-03-14T14:00:21.657000",
          "content": "<p><a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> U make any pre-processing, sorry if i making a lot of questions</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1237941,
          "author_name": "Hannes Öhler",
          "author_url": "",
          "post_date": "2021-03-14T14:10:13.547000",
          "content": "<p>That's fine. Just WBF..</p>",
          "votes": 2,
          "replies": []
        },
        {
          "id": 1238217,
          "author_name": "aqx",
          "author_url": "",
          "post_date": "2021-03-14T18:43:17.190000",
          "content": "<p><a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> Thank you very much for all the replies :)</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1238316,
          "author_name": "Hannes Öhler",
          "author_url": "",
          "post_date": "2021-03-14T21:45:14.823000",
          "content": "<p>You are welcome :-)</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1239171,
          "author_name": "Hannes Öhler",
          "author_url": "",
          "post_date": "2021-03-15T14:02:59.523000",
          "content": "<p><a href=\"https://www.kaggle.com/alexandrecc\" target=\"_blank\">@alexandrecc</a> yes positive images only</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1239334,
          "author_name": "aqx",
          "author_url": "",
          "post_date": "2021-03-15T16:24:54.060000",
          "content": "<p>I was wondering if you have tried using the VFNet variant: <strong>vfnet_x101_64x4d_fpn_mdconv_c3-c5_mstrain_2x_coco</strong>. I tried using this variant, which was supposedly to be the best, but the LB was not even close to 0.1</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1240945,
          "author_name": "Daniel Shan",
          "author_url": "",
          "post_date": "2021-03-16T18:47:03.453000",
          "content": "<p><a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> when you said <code>I do post nms</code> was this for yolo?  I thought yolov5 already does nms by default: <a href=\"https://github.com/ultralytics/yolov5/blob/ed2c74218d6d46605cc5fa68ce9bd6ece213abe4/detect.py#L75\" target=\"_blank\">https://github.com/ultralytics/yolov5/blob/ed2c74218d6d46605cc5fa68ce9bd6ece213abe4/detect.py#L75</a></p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1240959,
          "author_name": "Hannes Öhler",
          "author_url": "",
          "post_date": "2021-03-16T18:59:58.037000",
          "content": "<p><a href=\"https://www.kaggle.com/angqx95\" target=\"_blank\">@angqx95</a> I think I tried it but had to reduce batch size to stay in the memory limit of kaggle.. so I thought this could be the reason for my worse results..</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1240963,
          "author_name": "Hannes Öhler",
          "author_url": "",
          "post_date": "2021-03-16T19:02:21.173000",
          "content": "<p><a href=\"https://www.kaggle.com/danshan\" target=\"_blank\">@danshan</a> yes yolov5 and mmdetection vfnet include nms by default.. I just changed the iou threshold.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1245641,
          "author_name": "Zekun",
          "author_url": "",
          "post_date": "2021-03-20T01:38:18.537000",
          "content": "<p>Is this the default hyparameters?</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 1221436,
      "author_name": "ZLLA",
      "author_url": "",
      "post_date": "2021-03-01T01:26:38.563000",
      "content": "<p>What is your LB score for detection model only? since the detection might be yield instability while training especially in the rare cases</p>",
      "votes": 0,
      "replies": [
        {
          "id": 1221883,
          "author_name": "",
          "author_url": "",
          "post_date": "2021-03-01T11:25:24.530000",
          "content": "",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1222129,
          "author_name": "adriel cabral",
          "author_url": "",
          "post_date": "2021-03-01T15:15:07.027000",
          "content": "<p>get 0.188 with the \"best model\" (conf_thres: 0.01)</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1222157,
          "author_name": "ZLLA",
          "author_url": "",
          "post_date": "2021-03-01T15:40:59.520000",
          "content": "<p>That's quite high for yolo5x. You had to ensemble 5 folds to achieve that result right?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1222285,
          "author_name": "adriel cabral",
          "author_url": "",
          "post_date": "2021-03-01T17:19:16.723000",
          "content": "<p>I use Colab for train ( don't have GPU), so … i choice only use 1 fold for test results </p>",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1221216": "I use yolov5x with 2 class filters for my submissions and get a variance of LB score around of (0.2 - 0.247). I was waiting a low variance, for example, 0.23-0.247 or 0.24-0.247, but 0.2-0.247 i think is too high. I don't know why it happens.\n\n- All training have CV >= 0.4\n- My max CV was 0.435, unfortunately, have a low LB score (0.208)\n- For inference, i use conf_thres: 0.001, my \"better model\" get lb 0.240 with conf_thres: 0.01 and 0.247 with conf_thres: 0.001\n\nAnyone that use Yolov5 have the same \"problem\" ?\nTo be honest, i use yolov5 because it is much easier to train and make inference (this is my first contact with object detection and DL). I will try to use other models\n\n\n",
    "1221394": "Yes, I experienced the same. IMO this is not specific to yolov5 but caused by very few cases of rare classes in the public test set. Your model may or may not predict well those cases. As shown [here](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/220761), the LB score is much more stable for frequent classes. So maybe we can rely on LB score for frequent classes or CV score for model selection.",
    "1241717": "My scores are very unstable using Yolov5 too\n\nAll models are yolov5x, and scoring after run `2 class filter`\n\n| Model |  Val | LB |\n|--|--|--|\n| Fold 0 | 0.417 | 0.195 |\n| Fold 2 | 0.419 | 0.231 |\n| Fold 4 | 0.432 | 0.217 | \n| Ensemble (NMS .5) | - | 0.225 |\n\nMy data are split using `stratified k fold`",
    "1228729": "During my model selection I experienced some model achieving high score in some certain classes (CV) therefore have high LB score while \"perfect\" model which achieved more well round number for all labels has lower LB score. For this reason, I think the variance is reasonable. ",
    "1228112": " @adrielcabral   what aug did you use?",
    "1227987": "@adrielcabral \n\nI am using YOLOV5 too but my CV stuck around 0.33\n\nCan I ask a few questions about your setup ? \n\n1. which release of yolov5 are you using ? I  am using 4.0\n2. Did you train the model with only abnormal data ( filter out images with no finding )\n3. What NMS IOU did you use for inference ?  Did you leave it at 0.45 ( default ) ? \n4. Any modification to the model ?  Or did you just use it out of the box ?   \n\nThanks  for the info",
    "1245640": "Hello,which parameters do you choose?I only get 0.33+ on cv.",
    "1233921": "Hi, Adriel, I use yolov3 (with 2 class filter) and have a LB score 2.3. I am trying with yolo5X now, but only got LB around 2.0.  \nYolo5x should achieve better performance than yolov3. Do you know any possible reasons? Thanks!\nFYI, I already enable the \"multi-label\" for nms in yolov5.",
    "1225114": "I have used yolov5m(release 1.0) to get LB 0.186. But on my val set, the best map@iou0.5 is 0.26\n\nInference detail: \nTTA: no\nInput_size: 640x640\nscore thres: 0.1\n\ndoes this means I shuold try bigger model(yolov5x) like you?",
    "1225028": "Thank for sharing. How much did 2 class filter improve ur LB score than only using single yolov5?",
    "1224788": "Hi, in Yolov5 by default are evaluated by mAP@.5. Are your CV measured at 0.5 IOU or 0.4 (competition metric)",
    "1222635": "I seeing pretty bad CV to LB correlation for yolov5. I have local 0.32+ and LB 0.15. What image sizes are you using? I tried 640 with yolo5x",
    "1221592": "May I ask what is your configuration? ",
    "1221436": "What is your LB score for detection model only? since the detection might be yield instability while training especially in the rare cases\n"
  }
}