I am training a Resnet to just predict the classes alone, forget the bounding boxes.
\nUsing Resnet50 pretrained on imagenet1k.
\nIn this competition VinBigData: an image can belong to neither, or any one, or more than one classes in [0, 1, 2, … 13]. Hence using Multi Label Binary targets e.g if the image belongs to class 1 and 3 the target then is [0, 1, 0, 1, 0,0,0,0,0,0,0,0,0,0]
I am using DiceLoss + BCEWithLogitsLoss. Especially using Dice to take care of the sparse and imbalanced classes. Also weighting the classes in the BCEWithLogitsLossloss. I have taken care of things like to feed the BCEWithLogitsLoss with logits and not the sigmoid activations.
\nI trained for over 25 epochs, but still the model's Recall score is 22%, i.e it is still mostly predicting all class labels as 0. Accuracy is 84% high because of sparse binary labels i.e mostly predicting 0. But its of not much use. Cannot use CategoricalCrossEntrolyLoss since its multi binary labels.
How to deal with such a situation ?
", "messages": [ { "id": 1252271, "postDate": "2021-03-25T14:51:21.367Z", "content": "I am training a Resnet to just predict the classes alone, forget the bounding boxes.
\nUsing Resnet50 pretrained on imagenet1k.
\nIn this competition VinBigData: an image can belong to neither, or any one, or more than one classes in [0, 1, 2, … 13]. Hence using Multi Label Binary targets e.g if the image belongs to class 1 and 3 the target then is [0, 1, 0, 1, 0,0,0,0,0,0,0,0,0,0]
I am using DiceLoss + BCEWithLogitsLoss. Especially using Dice to take care of the sparse and imbalanced classes. Also weighting the classes in the BCEWithLogitsLossloss. I have taken care of things like to feed the BCEWithLogitsLoss with logits and not the sigmoid activations.
\nI trained for over 25 epochs, but still the model's Recall score is 22%, i.e it is still mostly predicting all class labels as 0. Accuracy is 84% high because of sparse binary labels i.e mostly predicting 0. But its of not much use. Cannot use CategoricalCrossEntrolyLoss since its multi binary labels.
How to deal with such a situation ?
", "rawMarkdown": "I am training a Resnet to just predict the classes alone, forget the bounding boxes.\nUsing Resnet50 pretrained on imagenet1k.\nIn this competition VinBigData: an image can belong to neither, or any one, or more than one classes in [0, 1, 2, … 13]. Hence using Multi Label Binary targets e.g if the image belongs to class 1 and 3 the target then is [0, 1, 0, 1, 0,0,0,0,0,0,0,0,0,0]\n\nI am using DiceLoss + BCEWithLogitsLoss. Especially using Dice to take care of the sparse and imbalanced classes. Also weighting the classes in the BCEWithLogitsLossloss. I have taken care of things like to feed the BCEWithLogitsLoss with logits and not the sigmoid activations.\nI trained for over 25 epochs, but still the model's Recall score is 22%, i.e it is still mostly predicting all class labels as 0. Accuracy is 84% high because of sparse binary labels i.e mostly predicting 0. But its of not much use. Cannot use CategoricalCrossEntrolyLoss since its multi binary labels.\n\nHow to deal with such a situation ?", "votes": 3 }, { "id": 1255193, "postDate": "2021-03-28T14:52:36.920Z", "content": "I found a workaround: instead of default probability threshold of 0.5, try with small threshold and gradually increase it eg : 0.001, 0.01, 0.1, 0.5. For each threshold calculate F1 score, and select the one which yields max F1 score. I found a quantum jump of F1 from 0.25 to 0.71. And a different threshold for each different class is calculated.
\nHope this helps in classifying yolo outputs also.
You could also try MultiLabelSoftMarginLoss.
", "rawMarkdown": "You could also try MultiLabelSoftMarginLoss.", "votes": 1 }, { "id": 1252502, "postDate": "2021-03-25T18:16:19.327Z", "content": "\n\nCannot use CategoricalCrossEntrolyLoss since its multi binary labels.
\n
You have Multi Label Classification / Multi Target Classification
", "rawMarkdown": "> Cannot use CategoricalCrossEntrolyLoss since its multi binary labels.\n\nYou have Multi Label Classification / Multi Target Classification", "replies": [ { "id": 1252788, "postDate": "2021-03-26T03:50:17.133Z", "content": "This competition VinBigData: an image can belong to neither, or any one, or more than one classes in [0, 1, 2, … 13].
\nHence using Multi Label Binary targets e.g if the image belongs to class 1 and 3, the target then is [0, 1, 0, 1, 0,0,0,0,0,0,0,0,0,0]
Using Resnet50 pretrained on imagenet1k.
", "rawMarkdown": "Using Resnet50 pretrained on imagenet1k." } ], "comments": [ { "id": 1255193, "author_name": "Rishi Chandra", "author_url": "", "post_date": "2021-03-28T14:52:36.920000", "content": "I found a workaround: instead of default probability threshold of 0.5, try with small threshold and gradually increase it eg : 0.001, 0.01, 0.1, 0.5. For each threshold calculate F1 score, and select the one which yields max F1 score. I found a quantum jump of F1 from 0.25 to 0.71. And a different threshold for each different class is calculated.
\nHope this helps in classifying yolo outputs also.
You could also try MultiLabelSoftMarginLoss.
", "votes": 1, "replies": [] }, { "id": 1252502, "author_name": "Mohammed Rizin V K", "author_url": "", "post_date": "2021-03-25T18:16:19.327000", "content": "\n\nCannot use CategoricalCrossEntrolyLoss since its multi binary labels.
\n
You have Multi Label Classification / Multi Target Classification
", "votes": 0, "replies": [ { "id": 1252788, "author_name": "Rishi Chandra", "author_url": "", "post_date": "2021-03-26T03:50:17.133000", "content": "This competition VinBigData: an image can belong to neither, or any one, or more than one classes in [0, 1, 2, … 13].
\nHence using Multi Label Binary targets e.g if the image belongs to class 1 and 3, the target then is [0, 1, 0, 1, 0,0,0,0,0,0,0,0,0,0]
Using Resnet50 pretrained on imagenet1k.
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1252271": "I am training a Resnet to just predict the classes alone, forget the bounding boxes.\nUsing Resnet50 pretrained on imagenet1k.\nIn this competition VinBigData: an image can belong to neither, or any one, or more than one classes in [0, 1, 2, … 13]. Hence using Multi Label Binary targets e.g if the image belongs to class 1 and 3 the target then is [0, 1, 0, 1, 0,0,0,0,0,0,0,0,0,0]\n\nI am using DiceLoss + BCEWithLogitsLoss. Especially using Dice to take care of the sparse and imbalanced classes. Also weighting the classes in the BCEWithLogitsLossloss. I have taken care of things like to feed the BCEWithLogitsLoss with logits and not the sigmoid activations.\nI trained for over 25 epochs, but still the model's Recall score is 22%, i.e it is still mostly predicting all class labels as 0. Accuracy is 84% high because of sparse binary labels i.e mostly predicting 0. But its of not much use. Cannot use CategoricalCrossEntrolyLoss since its multi binary labels.\n\nHow to deal with such a situation ?", "1255193": "I found a workaround: instead of default probability threshold of 0.5, try with small threshold and gradually increase it eg : 0.001, 0.01, 0.1, 0.5. For each threshold calculate F1 score, and select the one which yields max F1 score. I found a quantum jump of F1 from 0.25 to 0.71. And a different threshold for each different class is calculated.\nHope this helps in classifying yolo outputs also.", "1252519": "You could also try MultiLabelSoftMarginLoss.", "1252502": "> Cannot use CategoricalCrossEntrolyLoss since its multi binary labels.\n\nYou have Multi Label Classification / Multi Target Classification", "1252278": "Using Resnet50 pretrained on imagenet1k." } }