Common approach to improve the score is to use detection model + 2-class classifier, as I also introduced in 📸VinBigData 2-class classifier complete pipeline.
\nDetection model is used to locate abnormal bounding box class, where 2-class classifier is used to classify \"No finding\" images. So far this approach achieves LB 0.230.
Can we consider 1-step pipeline that does not need 2-class classifier?
\nActually \"No finding\" class is also evaluated as \"bonding box\" in the mAP competition metric, as discussed in this discussion.
Here, I tried a new approach: learn \"No finding\" as bounding boxes where bounding box covers all the image in the detection model.
\nNote that \"No finding\" class must be submitted with the bounding box with 0 0 1 1, so you need to change bounding box area in the submission, after the prediction (which highly likely covers all the image area).
\nI also added extra post-processing to only use 1 \"No finding\" bounding box per each image, with the maximum probability (score) in the prediction.
I got almost same score with the 2-class classifier approach (LB 0.230), which indicates that we can experiment with the simple pipeline to focus on detection model.
\nThe code for this training is available with this kernel:
\n\nYou can just set use_class14=True in the Flags class.
Common approach to improve the score is to use detection model + 2-class classifier, as I also introduced in 📸VinBigData 2-class classifier complete pipeline.
\nDetection model is used to locate abnormal bounding box class, where 2-class classifier is used to classify \"No finding\" images. So far this approach achieves LB 0.230.
Can we consider 1-step pipeline that does not need 2-class classifier?
\nActually \"No finding\" class is also evaluated as \"bonding box\" in the mAP competition metric, as discussed in this discussion.
Here, I tried a new approach: learn \"No finding\" as bounding boxes where bounding box covers all the image in the detection model.
\nNote that \"No finding\" class must be submitted with the bounding box with 0 0 1 1, so you need to change bounding box area in the submission, after the prediction (which highly likely covers all the image area).
\nI also added extra post-processing to only use 1 \"No finding\" bounding box per each image, with the maximum probability (score) in the prediction.
I got almost same score with the 2-class classifier approach (LB 0.230), which indicates that we can experiment with the simple pipeline to focus on detection model.
\nThe code for this training is available with this kernel:
\n\nYou can just set use_class14=True in the Flags class.
I've also been training with Normal images (on a partial train dataset to enable validation) and post processing to change bbox format to 14 0 0 1 1 in the submission csv.
\nI tried the 2-class classifier and found its performance was not too different from the detection only model.
\nBut the main advantage of the detection-only approach is more reliable validation - I think it is harder to validate the detection+classifier pipeline correctly.
@corochann hello, 2-class classifier you mentioned, is trained on kaggle kernel?
\nis that convenient if i train it on my own server
I trained in my local machine for the dataset: https://www.kaggle.com/corochann/vinbigdata2classpred
\nYou can also check training configuration in flags.yaml.
appreciate it,dude🙌
", "rawMarkdown": "appreciate it,dude🙌" } ] }, { "id": 1212734, "postDate": "2021-02-21T14:27:06.597Z", "content": "I would like to know more details about the 'test_meta.csv' file in VinBigData 2-class classifier complete pipeline. Looking at 3000 rows, it seems that you did additional work on the test data, so please explain how it was generated and the columns '#dim0', '#dim1'.
", "rawMarkdown": "I would like to know more details about the 'test_meta.csv' file in VinBigData 2-class classifier complete pipeline. Looking at 3000 rows, it seems that you did additional work on the test data, so please explain how it was generated and the columns '#dim0', '#dim1'.", "votes": 1, "replies": [ { "id": 1213481, "postDate": "2021-02-22T06:27:40.920Z", "content": "test_meat.csv is generated in the same way with train_meta.csv file. It just saves the original image size, so that we can rescale-back the bounding box position to original image's position.
\nPlease refer https://www.kaggle.com/corochann/vinbigdata-create-test-meta
Thank you for your kind answer!
", "rawMarkdown": "Thank you for your kind answer!", "votes": 1 }, { "id": 1229615, "postDate": "2021-03-07T13:34:50.507Z", "content": "can you help me, dude,
\nfor submision, we simply dump all detection result that model prediction on test_data,
\nor if it's better that we set a threshold(like 0.5) to filter out some low-score results
It's better not to filter out low-score results, for mAP metric.
\nSee https://www.kaggle.com/its7171/map-understanding-with-code-and-its-tips
roger that.
\ncould you please answer my another stupid question
\ni'm still using coco_metric, is it better to use pascal_voc metric as vin-bigdata do?
It's better if you can align metric calculation with the organizer. But I think coco_metric also work fine to see the correlation between LB.
", "rawMarkdown": "It's better if you can align metric calculation with the organizer. But I think coco_metric also work fine to see the correlation between LB." } ] }, { "id": 1217724, "postDate": "2021-02-25T09:10:03.277Z", "content": "Is 1-stage training highly affected by the imbalance between No Finding bboxes and the other classes' boxes ?
", "rawMarkdown": "Is 1-stage training highly affected by the imbalance between No Finding bboxes and the other classes' boxes ? ", "replies": [ { "id": 1217863, "postDate": "2021-02-25T11:24:57.287Z", "content": "IMO, including No Finding bboxes is \"True\" distribution, evaluated in test dataset.
", "rawMarkdown": "IMO, including No Finding bboxes is \"True\" distribution, evaluated in test dataset." } ] }, { "id": 1248049, "postDate": "2021-03-22T09:44:33.750Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 1204454, "author_name": "InDSweTrust", "author_url": "", "post_date": "2021-02-16T06:43:57.470000", "content": "I've also been training with Normal images (on a partial train dataset to enable validation) and post processing to change bbox format to 14 0 0 1 1 in the submission csv.
\nI tried the 2-class classifier and found its performance was not too different from the detection only model.
\nBut the main advantage of the detection-only approach is more reliable validation - I think it is harder to validate the detection+classifier pipeline correctly.
@corochann hello, 2-class classifier you mentioned, is trained on kaggle kernel?
\nis that convenient if i train it on my own server
I trained in my local machine for the dataset: https://www.kaggle.com/corochann/vinbigdata2classpred
\nYou can also check training configuration in flags.yaml.
appreciate it,dude🙌
", "votes": 0, "replies": [] } ] }, { "id": 1212734, "author_name": "stonebell", "author_url": "", "post_date": "2021-02-21T14:27:06.597000", "content": "I would like to know more details about the 'test_meta.csv' file in VinBigData 2-class classifier complete pipeline. Looking at 3000 rows, it seems that you did additional work on the test data, so please explain how it was generated and the columns '#dim0', '#dim1'.
", "votes": 1, "replies": [ { "id": 1213481, "author_name": "corochann", "author_url": "", "post_date": "2021-02-22T06:27:40.920000", "content": "test_meat.csv is generated in the same way with train_meta.csv file. It just saves the original image size, so that we can rescale-back the bounding box position to original image's position.
\nPlease refer https://www.kaggle.com/corochann/vinbigdata-create-test-meta
Thank you for your kind answer!
", "votes": 1, "replies": [] }, { "id": 1229615, "author_name": "xuzhaocheng", "author_url": "", "post_date": "2021-03-07T13:34:50.507000", "content": "can you help me, dude,
\nfor submision, we simply dump all detection result that model prediction on test_data,
\nor if it's better that we set a threshold(like 0.5) to filter out some low-score results
It's better not to filter out low-score results, for mAP metric.
\nSee https://www.kaggle.com/its7171/map-understanding-with-code-and-its-tips
roger that.
\ncould you please answer my another stupid question
\ni'm still using coco_metric, is it better to use pascal_voc metric as vin-bigdata do?
It's better if you can align metric calculation with the organizer. But I think coco_metric also work fine to see the correlation between LB.
", "votes": 0, "replies": [] } ] }, { "id": 1217724, "author_name": "Liam Nguyen", "author_url": "", "post_date": "2021-02-25T09:10:03.277000", "content": "Is 1-stage training highly affected by the imbalance between No Finding bboxes and the other classes' boxes ?
", "votes": 0, "replies": [ { "id": 1217863, "author_name": "corochann", "author_url": "", "post_date": "2021-02-25T11:24:57.287000", "content": "IMO, including No Finding bboxes is \"True\" distribution, evaluated in test dataset.
", "votes": 0, "replies": [] } ] }, { "id": 1248049, "author_name": "", "author_url": "", "post_date": "2021-03-22T09:44:33.750000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1204201": "Common approach to improve the score is to use **detection model + 2-class classifier**, as I also introduced in [📸VinBigData 2-class classifier complete pipeline](https://www.kaggle.com/corochann/vinbigdata-2-class-classifier-complete-pipeline).\nDetection model is used to locate abnormal bounding box class, where 2-class classifier is used to classify \"No finding\" images. So far this approach achieves LB 0.230.\n\nCan we consider 1-step pipeline that does not need 2-class classifier?\nActually \"No finding\" class is also evaluated as \"bonding box\" in the mAP competition metric, as discussed in this [discussion](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/211971).\n\nHere, I tried a new approach: **learn \"No finding\" as bounding boxes where bounding box covers all the image in the detection model**.\nNote that \"No finding\" class must be submitted with the bounding box with `0 0 1 1`, so you need to change bounding box area in the submission, after the prediction (which highly likely covers all the image area).\nI also added extra post-processing to only use 1 \"No finding\" bounding box per each image, **with the maximum probability (score)** in the prediction.\n\nI got almost same score with the 2-class classifier approach (LB 0.230), which indicates that we can experiment with the simple pipeline to focus on detection model. \n\nThe code for this training is available with this kernel:\n - [📸VinBigData detectron2 train](https://www.kaggle.com/corochann/vinbigdata-detectron2-train)\n\nYou can just set `use_class14=True` in the `Flags` class.\n", "1204454": "I've also been training with Normal images (on a partial train dataset to enable validation) and post processing to change bbox format to 14 0 0 1 1 in the submission csv.\nI tried the 2-class classifier and found its performance was not too different from the detection only model. \nBut the main advantage of the detection-only approach is more reliable validation - I think it is harder to validate the detection+classifier pipeline correctly.", "1229607": "@corochann hello, 2-class classifier you mentioned, is trained on kaggle kernel?\nis that convenient if i train it on my own server", "1212734": "I would like to know more details about the 'test_meta.csv' file in VinBigData 2-class classifier complete pipeline. Looking at 3000 rows, it seems that you did additional work on the test data, so please explain how it was generated and the columns '#dim0', '#dim1'.", "1217724": "Is 1-stage training highly affected by the imbalance between No Finding bboxes and the other classes' boxes ? ", "1248049": "" } }