{
  "id": 227586,
  "title": "performance of object detection only?",
  "url": "/competitions/vinbigdata-chest-xray-abnormalities-detection/discussion/227586",
  "author_name": "hengck23",
  "post_date": "2021-03-21T09:31:21.112000",
  "votes": 9,
  "comment_count": 11,
  "views": 0,
  "content": "<p>i tried a few models:</p>\n<ul>\n<li>classification (multilabel, 15 class)</li>\n<li>segmentation (multilabel, 14 class)</li>\n<li>detection(multilabel, 14 class)</li>\n</ul>\n<p>It seems that detection alone is pretty bad. I think there are not enough train samples. (or the way i sample the ground truth boxes are not good)</p>\n<ul>\n<li>the big boxes and majority class are always detected. it seems that my detection model detects the chest part rather than the abnormality</li>\n<li>the small box and minority are almost always missed</li>\n</ul>\n<p>if we have used only the detection model (i.e. discard classification filtering, etc ), what is the score you guys are getting?</p>\n<p>what is the performance if you use both +ve and -ve images for training detector?</p>\n<p>Does anyone want to share?</p>\n<hr>\n<p>Interestingly, if we do 15 class classification, we get precision&gt;0.94 and recall&gt;0.65 on local cross-validation. I did not validate segmentation yet, but visual results looks better than the detection</p>\n<p>i am quite puzzled. it seems that there are global information (classification) and local information (segmentation). But somehow, the detection model doesn't work well. Does anyone have any explanation?</p>\n<hr>",
  "messages": [
    {
      "id": 1246960,
      "postDate": "2021-03-21T09:31:21.113Z",
      "content": "<p>i tried a few models:</p>\n<ul>\n<li>classification (multilabel, 15 class)</li>\n<li>segmentation (multilabel, 14 class)</li>\n<li>detection(multilabel, 14 class)</li>\n</ul>\n<p>It seems that detection alone is pretty bad. I think there are not enough train samples. (or the way i sample the ground truth boxes are not good)</p>\n<ul>\n<li>the big boxes and majority class are always detected. it seems that my detection model detects the chest part rather than the abnormality</li>\n<li>the small box and minority are almost always missed</li>\n</ul>\n<p>if we have used only the detection model (i.e. discard classification filtering, etc ), what is the score you guys are getting?</p>\n<p>what is the performance if you use both +ve and -ve images for training detector?</p>\n<p>Does anyone want to share?</p>\n<hr>\n<p>Interestingly, if we do 15 class classification, we get precision&gt;0.94 and recall&gt;0.65 on local cross-validation. I did not validate segmentation yet, but visual results looks better than the detection</p>\n<p>i am quite puzzled. it seems that there are global information (classification) and local information (segmentation). But somehow, the detection model doesn't work well. Does anyone have any explanation?</p>\n<hr>",
      "rawMarkdown": "i tried a few models:\n- classification (multilabel, 15 class)\n- segmentation (multilabel, 14 class)\n- detection(multilabel, 14 class)\n\nIt seems that detection alone is pretty bad. I think there are not enough train samples. (or the way i sample the ground truth boxes are not good)\n- the big boxes and majority class are always detected. it seems that my detection model detects the chest part rather than the abnormality\n- the small box and minority are almost always missed\n\nif we have used only the detection model (i.e. discard classification filtering, etc ), what is the score you guys are getting?\n\nwhat is the performance if you use both +ve and -ve images for training detector?\n\nDoes anyone want to share?\n\n---\n\nInterestingly, if we do 15 class classification, we get precision>0.94 and recall>0.65 on local cross-validation. I did not validate segmentation yet, but visual results looks better than the detection\n\n\ni am quite puzzled. it seems that there are global information (classification) and local information (segmentation). But somehow, the detection model doesn't work well. Does anyone have any explanation?\n\n\n---",
      "votes": 9
    },
    {
      "id": 1247144,
      "postDate": "2021-03-21T13:18:37.903Z",
      "content": "<p>We are using only detection. I did't have any luck with classification. So our current score is only with detection.</p>",
      "rawMarkdown": "We are using only detection. I did't have any luck with classification. So our current score is only with detection.",
      "votes": 4,
      "replies": [
        {
          "id": 1249694,
          "postDate": "2021-03-23T13:08:12.207Z",
          "content": "<p><a href=\"https://www.kaggle.com/tugstugi\" target=\"_blank\">@tugstugi</a>, hey man :) </p>\n<p>Trying to reach you via email. Could you check your inbox? </p>",
          "rawMarkdown": "@tugstugi, hey man :) \n\nTrying to reach you via email. Could you check your inbox? ",
          "votes": 2
        },
        {
          "id": 1249768,
          "postDate": "2021-03-23T14:01:46.190Z",
          "content": "<p><a href=\"https://www.kaggle.com/ivanpan\" target=\"_blank\">@ivanpan</a> thanks for reaching out, unluckily we have too many submissions to merge :D</p>",
          "rawMarkdown": "@ivanpan thanks for reaching out, unluckily we have too many submissions to merge :D",
          "votes": 1
        }
      ]
    },
    {
      "id": 1249810,
      "postDate": "2021-03-23T14:35:02.200Z",
      "content": "<p>I have applied the public notebook classifier on my submissions, the LB drops from 0.326 to 0.303…</p>",
      "rawMarkdown": "I have applied the public notebook classifier on my submissions, the LB drops from 0.326 to 0.303...",
      "votes": 1
    },
    {
      "id": 1247527,
      "postDate": "2021-03-21T19:45:37.267Z",
      "content": "<p>My 14-class multilabel classifier reaches 0.86 AUC on CV. I don't have an explanation either, maybe the boxes are just drawn quite badly?</p>",
      "rawMarkdown": "My 14-class multilabel classifier reaches 0.86 AUC on CV. I don't have an explanation either, maybe the boxes are just drawn quite badly?",
      "votes": 1,
      "replies": [
        {
          "id": 1247642,
          "postDate": "2021-03-21T22:57:17.090Z",
          "content": "<p>That's a strong AUC, mine is much less efficient. May I ask, did your 14 classes classifier help you on LB? </p>",
          "rawMarkdown": "That's a strong AUC, mine is much less efficient. May I ask, did your 14 classes classifier help you on LB? "
        },
        {
          "id": 1247643,
          "postDate": "2021-03-21T22:58:48.723Z",
          "content": "<p>Yes, but just +0.001 :-)</p>",
          "rawMarkdown": "Yes, but just +0.001 :-)",
          "votes": 1
        },
        {
          "id": 1248025,
          "postDate": "2021-03-22T09:22:15.753Z",
          "content": "<p>Our 14-class multilabel classifier reaches 0.87 AUC on CV. never experimented till now</p>",
          "rawMarkdown": "Our 14-class multilabel classifier reaches 0.87 AUC on CV. never experimented till now",
          "votes": 1
        },
        {
          "id": 1249796,
          "postDate": "2021-03-23T14:22:13.730Z",
          "content": "<p>The AUC scores per class are also interesting: [0.90079661 0.83722378 0.76111671 0.94014337 0.91255245 0.8978355 0.86817059 0.82252526 0.83709981 0.73129361 0.93576799 0.79329143<br>\n 0.90918277 0.86189258]<br>\nCalcification and other lesion seem to be also relatively difficult to classify, but ILD gets a relatively high score compared to detection models.</p>",
          "rawMarkdown": "The AUC scores per class are also interesting: [0.90079661 0.83722378 0.76111671 0.94014337 0.91255245 0.8978355 0.86817059 0.82252526 0.83709981 0.73129361 0.93576799 0.79329143\n 0.90918277 0.86189258]\nCalcification and other lesion seem to be also relatively difficult to classify, but ILD gets a relatively high score compared to detection models.",
          "votes": 1
        },
        {
          "id": 1250107,
          "postDate": "2021-03-23T19:45:31.490Z",
          "content": "<p>Hi <a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> , what do you mean by multiclass classifier? Isn'it an object detection challenge? I aknowledge that my question can be silly since I am new in object detection.</p>",
          "rawMarkdown": "Hi @hannes82 , what do you mean by multiclass classifier? Isn'it an object detection challenge? I aknowledge that my question can be silly since I am new in object detection.",
          "votes": 1
        },
        {
          "id": 1250122,
          "postDate": "2021-03-23T19:56:56.960Z",
          "content": "<p>Yes sure, it is an object detection challenge. A multilabel classification is just something else you can do and it may provide some sort of second opinion. </p>",
          "rawMarkdown": "Yes sure, it is an object detection challenge. A multilabel classification is just something else you can do and it may provide some sort of second opinion. "
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 1247144,
      "author_name": "tugstugi",
      "author_url": "",
      "post_date": "2021-03-21T13:18:37.903000",
      "content": "<p>We are using only detection. I did't have any luck with classification. So our current score is only with detection.</p>",
      "votes": 4,
      "replies": [
        {
          "id": 1249694,
          "author_name": "Ivan Panshin",
          "author_url": "",
          "post_date": "2021-03-23T13:08:12.207000",
          "content": "<p><a href=\"https://www.kaggle.com/tugstugi\" target=\"_blank\">@tugstugi</a>, hey man :) </p>\n<p>Trying to reach you via email. Could you check your inbox? </p>",
          "votes": 2,
          "replies": []
        },
        {
          "id": 1249768,
          "author_name": "tugstugi",
          "author_url": "",
          "post_date": "2021-03-23T14:01:46.190000",
          "content": "<p><a href=\"https://www.kaggle.com/ivanpan\" target=\"_blank\">@ivanpan</a> thanks for reaching out, unluckily we have too many submissions to merge :D</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 1249810,
      "author_name": "tugstugi",
      "author_url": "",
      "post_date": "2021-03-23T14:35:02.200000",
      "content": "<p>I have applied the public notebook classifier on my submissions, the LB drops from 0.326 to 0.303…</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1247527,
      "author_name": "Hannes Öhler",
      "author_url": "",
      "post_date": "2021-03-21T19:45:37.267000",
      "content": "<p>My 14-class multilabel classifier reaches 0.86 AUC on CV. I don't have an explanation either, maybe the boxes are just drawn quite badly?</p>",
      "votes": 1,
      "replies": [
        {
          "id": 1247642,
          "author_name": "Matthieu Planté",
          "author_url": "",
          "post_date": "2021-03-21T22:57:17.090000",
          "content": "<p>That's a strong AUC, mine is much less efficient. May I ask, did your 14 classes classifier help you on LB? </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 1247643,
          "author_name": "Hannes Öhler",
          "author_url": "",
          "post_date": "2021-03-21T22:58:48.723000",
          "content": "<p>Yes, but just +0.001 :-)</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1248025,
          "author_name": "Mohammed Rizin V K",
          "author_url": "",
          "post_date": "2021-03-22T09:22:15.753000",
          "content": "<p>Our 14-class multilabel classifier reaches 0.87 AUC on CV. never experimented till now</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1249796,
          "author_name": "Hannes Öhler",
          "author_url": "",
          "post_date": "2021-03-23T14:22:13.730000",
          "content": "<p>The AUC scores per class are also interesting: [0.90079661 0.83722378 0.76111671 0.94014337 0.91255245 0.8978355 0.86817059 0.82252526 0.83709981 0.73129361 0.93576799 0.79329143<br>\n 0.90918277 0.86189258]<br>\nCalcification and other lesion seem to be also relatively difficult to classify, but ILD gets a relatively high score compared to detection models.</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1250107,
          "author_name": "Ulrich G.",
          "author_url": "",
          "post_date": "2021-03-23T19:45:31.490000",
          "content": "<p>Hi <a href=\"https://www.kaggle.com/hannes82\" target=\"_blank\">@hannes82</a> , what do you mean by multiclass classifier? Isn'it an object detection challenge? I aknowledge that my question can be silly since I am new in object detection.</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1250122,
          "author_name": "Hannes Öhler",
          "author_url": "",
          "post_date": "2021-03-23T19:56:56.960000",
          "content": "<p>Yes sure, it is an object detection challenge. A multilabel classification is just something else you can do and it may provide some sort of second opinion. </p>",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1246960": "i tried a few models:\n- classification (multilabel, 15 class)\n- segmentation (multilabel, 14 class)\n- detection(multilabel, 14 class)\n\nIt seems that detection alone is pretty bad. I think there are not enough train samples. (or the way i sample the ground truth boxes are not good)\n- the big boxes and majority class are always detected. it seems that my detection model detects the chest part rather than the abnormality\n- the small box and minority are almost always missed\n\nif we have used only the detection model (i.e. discard classification filtering, etc ), what is the score you guys are getting?\n\nwhat is the performance if you use both +ve and -ve images for training detector?\n\nDoes anyone want to share?\n\n---\n\nInterestingly, if we do 15 class classification, we get precision>0.94 and recall>0.65 on local cross-validation. I did not validate segmentation yet, but visual results looks better than the detection\n\n\ni am quite puzzled. it seems that there are global information (classification) and local information (segmentation). But somehow, the detection model doesn't work well. Does anyone have any explanation?\n\n\n---",
    "1247144": "We are using only detection. I did't have any luck with classification. So our current score is only with detection.",
    "1249810": "I have applied the public notebook classifier on my submissions, the LB drops from 0.326 to 0.303...",
    "1247527": "My 14-class multilabel classifier reaches 0.86 AUC on CV. I don't have an explanation either, maybe the boxes are just drawn quite badly?"
  }
}