{ "id": 208837, "title": "[LB0.155] baseline solution", "url": "/competitions/vinbigdata-chest-xray-abnormalities-detection/discussion/208837", "author_name": "phalanx", "post_date": "2021-01-05T08:45:41.234000", "votes": 91, "comment_count": 37, "views": 0, "content": "

share my solution overview, it is just baseline solution

\n
[detection]\ndataset\n  - no external data\n  - image resolution: 512x512\n  - use only abnormal images (remove class 14)\nmodel\n  - sparse rcnn\n  - use this repo: https://github.com/PeizeSun/SparseR-CNN\n  - encoder: resnet34(imagenet pretrained)\n  - others: random initialize\ntrain\n  - single fold\n  - augmentation: hflip, scale, shift, random brightness/contrast\n  - cls loss: focal loss, regression loss: l1 and giou loss\n  - epochs: 60\n  - optimizer: Adam\n  - scheduler: cosine annealing\n\n[classification]\nclassify abnormal image\ndataset\n - no external data\n - image resolution: 512x512\n - use all images\nmodel\n  - resnet50(imagenet pretrained)\n train\n  - 5fold\n  - augmentation: hflip, scale, shift, rotate, random brightness/contrast\n  - loss: bce\n  - epochs: 15\n  - optimizer: Adam\n  - scheduler: cosine annealing\n\nCV: 0.332, LB: 0.155\n
", "messages": [ { "id": 1139204, "postDate": "2021-01-05T08:45:41.233Z", "content": "

share my solution overview, it is just baseline solution

\n
[detection]\ndataset\n  - no external data\n  - image resolution: 512x512\n  - use only abnormal images (remove class 14)\nmodel\n  - sparse rcnn\n  - use this repo: https://github.com/PeizeSun/SparseR-CNN\n  - encoder: resnet34(imagenet pretrained)\n  - others: random initialize\ntrain\n  - single fold\n  - augmentation: hflip, scale, shift, random brightness/contrast\n  - cls loss: focal loss, regression loss: l1 and giou loss\n  - epochs: 60\n  - optimizer: Adam\n  - scheduler: cosine annealing\n\n[classification]\nclassify abnormal image\ndataset\n - no external data\n - image resolution: 512x512\n - use all images\nmodel\n  - resnet50(imagenet pretrained)\n train\n  - 5fold\n  - augmentation: hflip, scale, shift, rotate, random brightness/contrast\n  - loss: bce\n  - epochs: 15\n  - optimizer: Adam\n  - scheduler: cosine annealing\n\nCV: 0.332, LB: 0.155\n
", "rawMarkdown": "share my solution overview, it is just baseline solution\n```\n[detection]\ndataset\n - no external data\n - image resolution: 512x512\n - use only abnormal images (remove class 14)\nmodel\n - sparse rcnn\n - use this repo: https://github.com/PeizeSun/SparseR-CNN\n - encoder: resnet34(imagenet pretrained)\n - others: random initialize\ntrain\n - single fold\n - augmentation: hflip, scale, shift, random brightness/contrast\n - cls loss: focal loss, regression loss: l1 and giou loss\n - epochs: 60\n - optimizer: Adam\n - scheduler: cosine annealing\n\n[classification]\nclassify abnormal image\ndataset\n - no external data\n - image resolution: 512x512\n - use all images\nmodel\n - resnet50(imagenet pretrained)\n train\n - 5fold\n - augmentation: hflip, scale, shift, rotate, random brightness/contrast\n - loss: bce\n - epochs: 15\n - optimizer: Adam\n - scheduler: cosine annealing\n\nCV: 0.332, LB: 0.155\n```", "votes": 91 }, { "id": 1151537, "postDate": "2021-01-13T11:56:31.943Z", "content": "

@phalanx, thank you for your post! What do you think is the best way to handle multiple bboxes per object from different radiologists when setting target bbox? To apply nms, take average or just leave as is? Thx

", "rawMarkdown": "@phalanx, thank you for your post! What do you think is the best way to handle multiple bboxes per object from different radiologists when setting target bbox? To apply nms, take average or just leave as is? Thx", "votes": 1, "replies": [ { "id": 1151575, "postDate": "2021-01-13T12:22:47.320Z", "content": "

Sorry, it is core of my approach now. After label cleaning, lb improve from 0.205 to 0.234. It is very important.

", "rawMarkdown": "Sorry, it is core of my approach now. After label cleaning, lb improve from 0.205 to 0.234. It is very important.", "votes": 11 }, { "id": 1164640, "postDate": "2021-01-22T13:49:10.160Z", "content": "

I support his statement because i got an improvement from 0.200 to 0.239

", "rawMarkdown": "I support his statement because i got an improvement from 0.200 to 0.239", "votes": 1 } ] }, { "id": 1145712, "postDate": "2021-01-09T09:49:56.337Z", "content": "

Thanks for sharing! This is really helpful for beginners like me!

", "rawMarkdown": "Thanks for sharing! This is really helpful for beginners like me! ", "votes": 1 }, { "id": 1139823, "postDate": "2021-01-05T16:33:57.967Z", "content": "

Thanks for sharing!
\nThis is a good start to the competition.

\n

p.s. I just remember the GWD competition. @phalanx

", "rawMarkdown": "Thanks for sharing!\nThis is a good start to the competition.\n\np.s. I just remember the GWD competition. @phalanx ", "votes": 1 }, { "id": 1139739, "postDate": "2021-01-05T15:36:49.467Z", "content": "

@phalanx How did you make the final prediction? First, classify and then detect only the positives with some threshold?

", "rawMarkdown": "@phalanx How did you make the final prediction? First, classify and then detect only the positives with some threshold?", "votes": 1, "replies": [ { "id": 1139754, "postDate": "2021-01-05T15:45:34.247Z", "content": "

Yes, I predict only abnormal images.

\n
abnormal_indexes = np.where(classification_pred > 0.5)[0]\nX = X[abnormal_indexes]\npreds = predict(model, X, mode='test')\n\n# normal image -> 14 1 0 0 1 1\n# abnlrmal image -> preds\nsub['PredictionString'] = postprocess(preds, abnormal_indexes)\n
", "rawMarkdown": "Yes, I predict only abnormal images.\n```\nabnormal_indexes = np.where(classification_pred > 0.5)[0]\nX = X[abnormal_indexes]\npreds = predict(model, X, mode='test')\n\n# normal image -> 14 1 0 0 1 1\n# abnlrmal image -> preds\nsub['PredictionString'] = postprocess(preds, abnormal_indexes)\n```", "votes": 9 } ] }, { "id": 1139259, "postDate": "2021-01-05T09:26:53.423Z", "content": "

There are differences in the models, but they are almost identical to mine.

", "rawMarkdown": "There are differences in the models, but they are almost identical to mine.", "votes": 1 }, { "id": 1150721, "postDate": "2021-01-12T19:47:18Z", "content": "

If you were to do this with torchvision detection models would it be one model (which predicts labels and boxes) or 2 models (multi-task learning)?

", "rawMarkdown": "If you were to do this with torchvision detection models would it be one model (which predicts labels and boxes) or 2 models (multi-task learning)?", "votes": 2, "replies": [ { "id": 1151265, "postDate": "2021-01-13T08:01:12.947Z", "content": "

If you're using torchvision FasterRCNN (or Mask RCNN), the model predicts both bounding boxes and labels. So in that case, its one model. @phalanx seems to be using a 2-stage (detection-classification approach)

", "rawMarkdown": "If you're using torchvision FasterRCNN (or Mask RCNN), the model predicts both bounding boxes and labels. So in that case, its one model. @phalanx seems to be using a 2-stage (detection-classification approach)", "votes": 1 }, { "id": 1151295, "postDate": "2021-01-13T08:14:43.310Z", "content": "

I use classification model and detection model.
\nif you use torchvision detection framework,

\n", "rawMarkdown": "I use classification model and detection model.\nif you use torchvision detection framework,\n- detection : use FasterRCNN to localize and classify thoracic abnormalities.\n- classification: use resnet34 for binary classification (normal or abnormal image).", "votes": 5 }, { "id": 1151316, "postDate": "2021-01-13T08:33:20.320Z", "content": "

Thank you, that makes sense now!

", "rawMarkdown": "Thank you, that makes sense now!" } ] }, { "id": 1139588, "postDate": "2021-01-05T14:01:41.213Z", "content": "

I also found that the K-fold Strategy is really important. I randomly split data into 5 folds. My model got 0.142 LB with fold 0, while getting just 0.109 with another fold.

", "rawMarkdown": "I also found that the K-fold Strategy is really important. I randomly split data into 5 folds. My model got 0.142 LB with fold 0, while getting just 0.109 with another fold.", "votes": 2, "replies": [ { "id": 1139712, "postDate": "2021-01-05T15:20:29.820Z", "content": "

I use multilabel stratified kfold.
\nhttps://github.com/trent-b/iterative-stratification
\nit is stable for me.

\n
# bboxes: [N, M, 5=(x1, y1,x2, y2, class_id)]\none_hot_labels = [np.eye(15)[bbox[:, -1].astype(int)].sum(0) for bbox in bboxes]    \nmskf = MultilabelStratifiedKFold(n_splits=5, shuffle=True, random_state=2021)\nfor fold, (train_index, val_index) in enumerate(mskf.split(X, one_hot_labels)):\n    train_index = [idx for idx in train_index if bboxes[idx][0, -1] != 14]\n    val_index = [idx for idx in val_index if bboxes[idx][0, -1] != 14]\n    X_train = X[train_index]\n    X_val = X[val_index]\n    y_train = bboxes[train_index]\n    y_val = bboxes[val_index]\n
\n

class distribution
\n\"\"

\n

LB:
\n[0.155, 0.162, ---, ---, ---]

", "rawMarkdown": "I use multilabel stratified kfold.\nhttps://github.com/trent-b/iterative-stratification\nit is stable for me.\n```\n# bboxes: [N, M, 5=(x1, y1,x2, y2, class_id)]\none_hot_labels = [np.eye(15)[bbox[:, -1].astype(int)].sum(0) for bbox in bboxes] \nmskf = MultilabelStratifiedKFold(n_splits=5, shuffle=True, random_state=2021)\nfor fold, (train_index, val_index) in enumerate(mskf.split(X, one_hot_labels)):\n train_index = [idx for idx in train_index if bboxes[idx][0, -1] != 14]\n val_index = [idx for idx in val_index if bboxes[idx][0, -1] != 14]\n X_train = X[train_index]\n X_val = X[val_index]\n y_train = bboxes[train_index]\n y_val = bboxes[val_index]\n```\n\nclass distribution\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1620223%2F802470b89d4294e49f46aca3aaf7b239%2F2021-01-06%200.18.21.jpg?generation=1609859965987071&alt=media)\n\nLB:\n[0.155, 0.162, ---, ---, ---]", "votes": 18 }, { "id": 1139752, "postDate": "2021-01-05T15:43:58.443Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1139855, "postDate": "2021-01-05T16:58:49.783Z", "content": "

Thank you for sharing!

\n

Is data leakage happening when we are using multilabel stratified k-fold since the ground truth coming from multiple radiologists? @phalanx

", "rawMarkdown": "Thank you for sharing! \n\nIs data leakage happening when we are using multilabel stratified k-fold since the ground truth coming from multiple radiologists? @phalanx ", "votes": 1 }, { "id": 1139897, "postDate": "2021-01-05T17:27:37.943Z", "content": "

X is unique image_id, so data leakage is not heappening.
\nBut I don't consider rad_id, so distribution of it may be skewed.

", "rawMarkdown": "`X` is unique image_id, so data leakage is not heappening.\nBut I don't consider `rad_id`, so distribution of it may be skewed.", "votes": 3 }, { "id": 1145247, "postDate": "2021-01-09T02:11:08.453Z", "content": "

I tried multilabel stratified kfold but it seems to be unstable at all. I reckon your models are stable because of the 2-stage detection approach.

", "rawMarkdown": "I tried multilabel stratified kfold but it seems to be unstable at all. I reckon your models are stable because of the 2-stage detection approach. " }, { "id": 1146609, "postDate": "2021-01-09T21:59:42.680Z", "content": "

it worked for me. Thanks for that. My model is more stable with this

", "rawMarkdown": "it worked for me. Thanks for that. My model is more stable with this" }, { "id": 1165440, "postDate": "2021-01-23T01:09:10.933Z", "content": "

What does M denote?

", "rawMarkdown": "What does M denote?" }, { "id": 1173452, "postDate": "2021-01-27T22:26:55.860Z", "content": "

I know that make CV is very important, but it's no take a long time ? Because, if i understand, u splited your data in 5 folds (train and valid), so u train in 5 diferents datas, how long time it's take ? i'm begginer in Object Detection and no have expericence in make CV in this data type.

", "rawMarkdown": "I know that make CV is very important, but it's no take a long time ? Because, if i understand, u splited your data in 5 folds (train and valid), so u train in 5 diferents datas, how long time it's take ? i'm begginer in Object Detection and no have expericence in make CV in this data type. " }, { "id": 1182978, "postDate": "2021-02-02T17:06:23.843Z", "content": "

@phalanx I don't fully understand your idea, can you explained it for me ?

", "rawMarkdown": "@phalanx I don't fully understand your idea, can you explained it for me ? " } ] }, { "id": 1584285, "postDate": "2021-11-16T12:15:18.143Z", "content": "

Hello, sorry to disturb, can you share the source code since the competition has finished? I am a beginner in detection, and I am doing a subject about X-ray detection. Your code would be very helpful for me, it would be very kind of you if you can open source the solution.

", "rawMarkdown": "Hello, sorry to disturb, can you share the source code since the competition has finished? I am a beginner in detection, and I am doing a subject about X-ray detection. Your code would be very helpful for me, it would be very kind of you if you can open source the solution." }, { "id": 1232504, "postDate": "2021-03-09T20:42:14.107Z", "content": "

Thank you for sharing this clean and well documented notebook, great baseline!

", "rawMarkdown": "Thank you for sharing this clean and well documented notebook, great baseline!" }, { "id": 1182181, "postDate": "2021-02-02T10:44:19.257Z", "content": "

Hi, nice baseline !

\n

You're not using non max suppression ?

\n

Thanks

", "rawMarkdown": "Hi, nice baseline !\n\nYou're not using non max suppression ?\n\nThanks" }, { "id": 1162102, "postDate": "2021-01-21T01:59:44.240Z", "content": "

Also are you guys finding better results to do NMS to the training dataset or leaving the training set as is and doing NMS to the predictions.

\n

Thanks,

", "rawMarkdown": "Also are you guys finding better results to do NMS to the training dataset or leaving the training set as is and doing NMS to the predictions. \n\nThanks," }, { "id": 1162096, "postDate": "2021-01-21T01:47:00.347Z", "content": "

Are you guys getting better scores after using NMS for the final prediction as a filter? I initially tested using NMS for the final results to remove false positive duplicate boxes and for IOU thresholds for NMS from 0.4-0.8, the LB score is not better.

\n

Thanks!

", "rawMarkdown": "Are you guys getting better scores after using NMS for the final prediction as a filter? I initially tested using NMS for the final results to remove false positive duplicate boxes and for IOU thresholds for NMS from 0.4-0.8, the LB score is not better. \n\nThanks!", "replies": [ { "id": 1162337, "postDate": "2021-01-21T05:28:46.703Z", "content": "

I improved my score with NMS

", "rawMarkdown": "I improved my score with NMS", "votes": 2 }, { "id": 1163620, "postDate": "2021-01-21T19:59:59.237Z", "content": "

FenomeN,

\n

Did you do NMS on the training data and then have predictions, or did you not filter the training data and then do NMS on the predicted data?

\n

Thanks,

", "rawMarkdown": "FenomeN,\n\nDid you do NMS on the training data and then have predictions, or did you not filter the training data and then do NMS on the predicted data?\n\nThanks," }, { "id": 1163914, "postDate": "2021-01-22T03:53:02.497Z", "content": "

Doing NMS in training data which lower the score for me both validation and inference.
\nBut doing NMS during inference which predicted multiple boxes on same location will improve your score

", "rawMarkdown": "Doing NMS in training data which lower the score for me both validation and inference.\nBut doing NMS during inference which predicted multiple boxes on same location will improve your score", "votes": 2 } ] }, { "id": 1161058, "postDate": "2021-01-20T09:50:25.657Z", "content": "

Hi @phalanx, may I ask you what is the CV/LB score of the detection model only (i.e. without post-processing predictions with the binary classifier)?

", "rawMarkdown": "Hi @phalanx, may I ask you what is the CV/LB score of the detection model only (i.e. without post-processing predictions with the binary classifier)?", "replies": [ { "id": 1161093, "postDate": "2021-01-20T10:24:48.027Z", "content": "

detection only: lb 0.178
\nadd classification: +0.027(lb 0.205), num of class 14: 1996
\nmodify detection model, fix classification model: +0.061(lb 0.266), num of class 14: 1996

\n

detection + classification: lb 0.266
\ndetection only: lb 0.240

", "rawMarkdown": "detection only: lb 0.178\nadd classification: +0.027(lb 0.205), num of class 14: 1996\nmodify detection model, fix classification model: +0.061(lb 0.266), num of class 14: 1996\n\ndetection + classification: lb 0.266\ndetection only: lb 0.240", "votes": 10 }, { "id": 1164600, "postDate": "2021-01-22T13:21:28.020Z", "content": "

Likewise, this is your result on LB. Can I ask some questions:

\n
    \n
  1. is the model which scores 0.266 on LB going through this process?
    2. \n
  2. Is your MultiStratifiedFold contributes any amount in your score? for ensembling or other cases
    3. \n
  3. Classification is an important task but have a question still: What AUC or Accuracy you achieve in it
    4. \n
  4. Have you tried with Strong or more Augmentations?
    5. \n
  5. You said 5 folds, is the score you posted in discussion topic ie CV: 0.332, LB: 0.155 is ensembled or individual score
    6. \n
  6. fix classification model you mean you had some problem in the network
    7. \n
\n

Sorry for So many questions
\nSorry and Thank you in Advance and Thank you for posting your solution which helps us a lot

", "rawMarkdown": "Likewise, this is your result on LB. Can I ask some questions:\n1. is the model which scores 0.266 on LB going through this process?\n2. Is your `MultiStratifiedFold` contributes any amount in your score? for ensembling or other cases\n3. Classification is an important task but have a question still: What AUC or Accuracy you achieve in it\n4. Have you tried with Strong or more Augmentations?\n5. You said 5 folds, is the score you posted in discussion topic ie `CV: 0.332, LB: 0.155` is ensembled or individual score\n6. `fix classification model` you mean you had some problem in the network\n\nSorry for So many questions\nSorry and Thank you in Advance and Thank you for posting your solution which helps us a lot" }, { "id": 1164649, "postDate": "2021-01-22T13:55:33.447Z", "content": "
    \n
  1. yes, classification + detection
    2. \n
  2. I don't know. But splitting the data using this technique is a natural choice for me.
    3. \n
  3. AUC: 0.93
    4. \n
  4. I just tried simple augmentation.
    5. \n
  5. yes, ensemble prediction
    6. \n
", "rawMarkdown": "1. yes, classification + detection\n2. I don't know. But splitting the data using this technique is a natural choice for me.\n3. AUC: 0.93\n4. I just tried simple augmentation.\n5. yes, ensemble prediction", "votes": 4 }, { "id": 1164678, "postDate": "2021-01-22T14:14:31.660Z", "content": "

Thank you for Fast Reply.

", "rawMarkdown": "Thank you for Fast Reply." } ] }, { "id": 1247228, "postDate": "2021-03-21T14:52:00.897Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1141195, "postDate": "2021-01-06T14:51:49.780Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 1151537, "author_name": "Gleb Anferov", "author_url": "", "post_date": "2021-01-13T11:56:31.943000", "content": "

@phalanx, thank you for your post! What do you think is the best way to handle multiple bboxes per object from different radiologists when setting target bbox? To apply nms, take average or just leave as is? Thx

", "votes": 1, "replies": [ { "id": 1151575, "author_name": "phalanx", "author_url": "", "post_date": "2021-01-13T12:22:47.320000", "content": "

Sorry, it is core of my approach now. After label cleaning, lb improve from 0.205 to 0.234. It is very important.

", "votes": 11, "replies": [] }, { "id": 1164640, "author_name": "Mohammed Rizin V K", "author_url": "", "post_date": "2021-01-22T13:49:10.160000", "content": "

I support his statement because i got an improvement from 0.200 to 0.239

", "votes": 1, "replies": [] } ] }, { "id": 1145712, "author_name": "Dongkyu Kim", "author_url": "", "post_date": "2021-01-09T09:49:56.337000", "content": "

Thanks for sharing! This is really helpful for beginners like me!

", "votes": 1, "replies": [] }, { "id": 1139823, "author_name": "Heroseo", "author_url": "", "post_date": "2021-01-05T16:33:57.967000", "content": "

Thanks for sharing!
\nThis is a good start to the competition.

\n

p.s. I just remember the GWD competition. @phalanx

", "votes": 1, "replies": [] }, { "id": 1139739, "author_name": "Peter", "author_url": "", "post_date": "2021-01-05T15:36:49.467000", "content": "

@phalanx How did you make the final prediction? First, classify and then detect only the positives with some threshold?

", "votes": 1, "replies": [ { "id": 1139754, "author_name": "phalanx", "author_url": "", "post_date": "2021-01-05T15:45:34.247000", "content": "

Yes, I predict only abnormal images.

\n
abnormal_indexes = np.where(classification_pred > 0.5)[0]\nX = X[abnormal_indexes]\npreds = predict(model, X, mode='test')\n\n# normal image -> 14 1 0 0 1 1\n# abnlrmal image -> preds\nsub['PredictionString'] = postprocess(preds, abnormal_indexes)\n
", "votes": 9, "replies": [] } ] }, { "id": 1139259, "author_name": "YujiAriyasu", "author_url": "", "post_date": "2021-01-05T09:26:53.423000", "content": "

There are differences in the models, but they are almost identical to mine.

", "votes": 1, "replies": [] }, { "id": 1150721, "author_name": "Mostafa Ibrahim", "author_url": "", "post_date": "2021-01-12T19:47:18", "content": "

If you were to do this with torchvision detection models would it be one model (which predicts labels and boxes) or 2 models (multi-task learning)?

", "votes": 2, "replies": [ { "id": 1151265, "author_name": "InDSweTrust", "author_url": "", "post_date": "2021-01-13T08:01:12.947000", "content": "

If you're using torchvision FasterRCNN (or Mask RCNN), the model predicts both bounding boxes and labels. So in that case, its one model. @phalanx seems to be using a 2-stage (detection-classification approach)

", "votes": 1, "replies": [] }, { "id": 1151295, "author_name": "phalanx", "author_url": "", "post_date": "2021-01-13T08:14:43.310000", "content": "

I use classification model and detection model.
\nif you use torchvision detection framework,

\n", "votes": 5, "replies": [] }, { "id": 1151316, "author_name": "InDSweTrust", "author_url": "", "post_date": "2021-01-13T08:33:20.320000", "content": "

Thank you, that makes sense now!

", "votes": 0, "replies": [] } ] }, { "id": 1139588, "author_name": "The fearless", "author_url": "", "post_date": "2021-01-05T14:01:41.213000", "content": "

I also found that the K-fold Strategy is really important. I randomly split data into 5 folds. My model got 0.142 LB with fold 0, while getting just 0.109 with another fold.

", "votes": 2, "replies": [ { "id": 1139712, "author_name": "phalanx", "author_url": "", "post_date": "2021-01-05T15:20:29.820000", "content": "

I use multilabel stratified kfold.
\nhttps://github.com/trent-b/iterative-stratification
\nit is stable for me.

\n
# bboxes: [N, M, 5=(x1, y1,x2, y2, class_id)]\none_hot_labels = [np.eye(15)[bbox[:, -1].astype(int)].sum(0) for bbox in bboxes]    \nmskf = MultilabelStratifiedKFold(n_splits=5, shuffle=True, random_state=2021)\nfor fold, (train_index, val_index) in enumerate(mskf.split(X, one_hot_labels)):\n    train_index = [idx for idx in train_index if bboxes[idx][0, -1] != 14]\n    val_index = [idx for idx in val_index if bboxes[idx][0, -1] != 14]\n    X_train = X[train_index]\n    X_val = X[val_index]\n    y_train = bboxes[train_index]\n    y_val = bboxes[val_index]\n
\n

class distribution
\n\"\"

\n

LB:
\n[0.155, 0.162, ---, ---, ---]

", "votes": 18, "replies": [] }, { "id": 1139752, "author_name": "", "author_url": "", "post_date": "2021-01-05T15:43:58.443000", "content": "", "votes": 0, "replies": [] }, { "id": 1139855, "author_name": "The fearless", "author_url": "", "post_date": "2021-01-05T16:58:49.783000", "content": "

Thank you for sharing!

\n

Is data leakage happening when we are using multilabel stratified k-fold since the ground truth coming from multiple radiologists? @phalanx

", "votes": 1, "replies": [] }, { "id": 1139897, "author_name": "phalanx", "author_url": "", "post_date": "2021-01-05T17:27:37.943000", "content": "

X is unique image_id, so data leakage is not heappening.
\nBut I don't consider rad_id, so distribution of it may be skewed.

", "votes": 3, "replies": [] }, { "id": 1145247, "author_name": "The fearless", "author_url": "", "post_date": "2021-01-09T02:11:08.453000", "content": "

I tried multilabel stratified kfold but it seems to be unstable at all. I reckon your models are stable because of the 2-stage detection approach.

", "votes": 0, "replies": [] }, { "id": 1146609, "author_name": "Manh Lab", "author_url": "", "post_date": "2021-01-09T21:59:42.680000", "content": "

it worked for me. Thanks for that. My model is more stable with this

", "votes": 0, "replies": [] }, { "id": 1165440, "author_name": "tik_boa", "author_url": "", "post_date": "2021-01-23T01:09:10.933000", "content": "

What does M denote?

", "votes": 0, "replies": [] }, { "id": 1173452, "author_name": "adriel cabral", "author_url": "", "post_date": "2021-01-27T22:26:55.860000", "content": "

I know that make CV is very important, but it's no take a long time ? Because, if i understand, u splited your data in 5 folds (train and valid), so u train in 5 diferents datas, how long time it's take ? i'm begginer in Object Detection and no have expericence in make CV in this data type.

", "votes": 0, "replies": [] }, { "id": 1182978, "author_name": "cuongnn", "author_url": "", "post_date": "2021-02-02T17:06:23.843000", "content": "

@phalanx I don't fully understand your idea, can you explained it for me ?

", "votes": 0, "replies": [] } ] }, { "id": 1584285, "author_name": "yuewan zhang", "author_url": "", "post_date": "2021-11-16T12:15:18.143000", "content": "

Hello, sorry to disturb, can you share the source code since the competition has finished? I am a beginner in detection, and I am doing a subject about X-ray detection. Your code would be very helpful for me, it would be very kind of you if you can open source the solution.

", "votes": 0, "replies": [] }, { "id": 1232504, "author_name": "Ryan R", "author_url": "", "post_date": "2021-03-09T20:42:14.107000", "content": "

Thank you for sharing this clean and well documented notebook, great baseline!

", "votes": 0, "replies": [] }, { "id": 1182181, "author_name": "Sylvain L", "author_url": "", "post_date": "2021-02-02T10:44:19.257000", "content": "

Hi, nice baseline !

\n

You're not using non max suppression ?

\n

Thanks

", "votes": 0, "replies": [] }, { "id": 1162102, "author_name": "Bo Peng", "author_url": "", "post_date": "2021-01-21T01:59:44.240000", "content": "

Also are you guys finding better results to do NMS to the training dataset or leaving the training set as is and doing NMS to the predictions.

\n

Thanks,

", "votes": 0, "replies": [] }, { "id": 1162096, "author_name": "Bo Peng", "author_url": "", "post_date": "2021-01-21T01:47:00.347000", "content": "

Are you guys getting better scores after using NMS for the final prediction as a filter? I initially tested using NMS for the final results to remove false positive duplicate boxes and for IOU thresholds for NMS from 0.4-0.8, the LB score is not better.

\n

Thanks!

", "votes": 0, "replies": [ { "id": 1162337, "author_name": "Mohammed Rizin V K", "author_url": "", "post_date": "2021-01-21T05:28:46.703000", "content": "

I improved my score with NMS

", "votes": 2, "replies": [] }, { "id": 1163620, "author_name": "Bo Peng", "author_url": "", "post_date": "2021-01-21T19:59:59.237000", "content": "

FenomeN,

\n

Did you do NMS on the training data and then have predictions, or did you not filter the training data and then do NMS on the predicted data?

\n

Thanks,

", "votes": 0, "replies": [] }, { "id": 1163914, "author_name": "Mohammed Rizin V K", "author_url": "", "post_date": "2021-01-22T03:53:02.497000", "content": "

Doing NMS in training data which lower the score for me both validation and inference.
\nBut doing NMS during inference which predicted multiple boxes on same location will improve your score

", "votes": 2, "replies": [] } ] }, { "id": 1161058, "author_name": "LAZCoder", "author_url": "", "post_date": "2021-01-20T09:50:25.657000", "content": "

Hi @phalanx, may I ask you what is the CV/LB score of the detection model only (i.e. without post-processing predictions with the binary classifier)?

", "votes": 0, "replies": [ { "id": 1161093, "author_name": "phalanx", "author_url": "", "post_date": "2021-01-20T10:24:48.027000", "content": "

detection only: lb 0.178
\nadd classification: +0.027(lb 0.205), num of class 14: 1996
\nmodify detection model, fix classification model: +0.061(lb 0.266), num of class 14: 1996

\n

detection + classification: lb 0.266
\ndetection only: lb 0.240

", "votes": 10, "replies": [] }, { "id": 1164600, "author_name": "Mohammed Rizin V K", "author_url": "", "post_date": "2021-01-22T13:21:28.020000", "content": "

Likewise, this is your result on LB. Can I ask some questions:

\n
    \n
  1. is the model which scores 0.266 on LB going through this process?
    2. \n
  2. Is your MultiStratifiedFold contributes any amount in your score? for ensembling or other cases
    3. \n
  3. Classification is an important task but have a question still: What AUC or Accuracy you achieve in it
    4. \n
  4. Have you tried with Strong or more Augmentations?
    5. \n
  5. You said 5 folds, is the score you posted in discussion topic ie CV: 0.332, LB: 0.155 is ensembled or individual score
    6. \n
  6. fix classification model you mean you had some problem in the network
    7. \n
\n

Sorry for So many questions
\nSorry and Thank you in Advance and Thank you for posting your solution which helps us a lot

", "votes": 0, "replies": [] }, { "id": 1164649, "author_name": "phalanx", "author_url": "", "post_date": "2021-01-22T13:55:33.447000", "content": "
    \n
  1. yes, classification + detection
    2. \n
  2. I don't know. But splitting the data using this technique is a natural choice for me.
    3. \n
  3. AUC: 0.93
    4. \n
  4. I just tried simple augmentation.
    5. \n
  5. yes, ensemble prediction
    6. \n
", "votes": 4, "replies": [] }, { "id": 1164678, "author_name": "Mohammed Rizin V K", "author_url": "", "post_date": "2021-01-22T14:14:31.660000", "content": "

Thank you for Fast Reply.

", "votes": 0, "replies": [] } ] }, { "id": 1247228, "author_name": "", "author_url": "", "post_date": "2021-03-21T14:52:00.897000", "content": "", "votes": 0, "replies": [] }, { "id": 1141195, "author_name": "", "author_url": "", "post_date": "2021-01-06T14:51:49.780000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1139204": "share my solution overview, it is just baseline solution\n```\n[detection]\ndataset\n - no external data\n - image resolution: 512x512\n - use only abnormal images (remove class 14)\nmodel\n - sparse rcnn\n - use this repo: https://github.com/PeizeSun/SparseR-CNN\n - encoder: resnet34(imagenet pretrained)\n - others: random initialize\ntrain\n - single fold\n - augmentation: hflip, scale, shift, random brightness/contrast\n - cls loss: focal loss, regression loss: l1 and giou loss\n - epochs: 60\n - optimizer: Adam\n - scheduler: cosine annealing\n\n[classification]\nclassify abnormal image\ndataset\n - no external data\n - image resolution: 512x512\n - use all images\nmodel\n - resnet50(imagenet pretrained)\n train\n - 5fold\n - augmentation: hflip, scale, shift, rotate, random brightness/contrast\n - loss: bce\n - epochs: 15\n - optimizer: Adam\n - scheduler: cosine annealing\n\nCV: 0.332, LB: 0.155\n```", "1151537": "@phalanx, thank you for your post! What do you think is the best way to handle multiple bboxes per object from different radiologists when setting target bbox? To apply nms, take average or just leave as is? Thx", "1145712": "Thanks for sharing! This is really helpful for beginners like me! ", "1139823": "Thanks for sharing!\nThis is a good start to the competition.\n\np.s. I just remember the GWD competition. @phalanx ", "1139739": "@phalanx How did you make the final prediction? First, classify and then detect only the positives with some threshold?", "1139259": "There are differences in the models, but they are almost identical to mine.", "1150721": "If you were to do this with torchvision detection models would it be one model (which predicts labels and boxes) or 2 models (multi-task learning)?", "1139588": "I also found that the K-fold Strategy is really important. I randomly split data into 5 folds. My model got 0.142 LB with fold 0, while getting just 0.109 with another fold.", "1584285": "Hello, sorry to disturb, can you share the source code since the competition has finished? I am a beginner in detection, and I am doing a subject about X-ray detection. Your code would be very helpful for me, it would be very kind of you if you can open source the solution.", "1232504": "Thank you for sharing this clean and well documented notebook, great baseline!", "1182181": "Hi, nice baseline !\n\nYou're not using non max suppression ?\n\nThanks", "1162102": "Also are you guys finding better results to do NMS to the training dataset or leaving the training set as is and doing NMS to the predictions. \n\nThanks,", "1162096": "Are you guys getting better scores after using NMS for the final prediction as a filter? I initially tested using NMS for the final results to remove false positive duplicate boxes and for IOU thresholds for NMS from 0.4-0.8, the LB score is not better. \n\nThanks!", "1161058": "Hi @phalanx, may I ask you what is the CV/LB score of the detection model only (i.e. without post-processing predictions with the binary classifier)?", "1247228": "", "1141195": "" } }