{
  "id": 69504,
  "title": "Model & Loss function & Augmutation",
  "url": "/competitions/quickdraw-doodle-recognition/discussion/69504",
  "author_name": "Gary",
  "post_date": "2018-10-24T09:03:07.120000",
  "votes": 10,
  "comment_count": 10,
  "views": 0,
  "content": "<p>Hi, I started this competition recently, and I want to ask you guys here.</p>\n\n<ul>\n<li>What lightweight model did you use, mobilenet, shufflenet or self-designed model?</li>\n<li>What better loss can be used in this competition,  like center loss, Ensemble Soft-Margin \n SoftmaxLoss, Large-Margin Softmax Loss, etc. </li>\n<li>What effective data aug methods did you use?</li>\n</ul>\n\n<p>If you don't mind, you can share the idea here. Thank you, and good luck.</p>",
  "messages": [
    {
      "id": 409418,
      "postDate": "2018-10-24T09:03:07.120Z",
      "content": "<p>Hi, I started this competition recently, and I want to ask you guys here.</p>\n\n<ul>\n<li>What lightweight model did you use, mobilenet, shufflenet or self-designed model?</li>\n<li>What better loss can be used in this competition,  like center loss, Ensemble Soft-Margin \n SoftmaxLoss, Large-Margin Softmax Loss, etc. </li>\n<li>What effective data aug methods did you use?</li>\n</ul>\n\n<p>If you don't mind, you can share the idea here. Thank you, and good luck.</p>",
      "rawMarkdown": "Hi, I started this competition recently, and I want to ask you guys here.\n\n - What lightweight model did you use, mobilenet, shufflenet or self-designed model?\n - What better loss can be used in this competition,  like center loss, Ensemble Soft-Margin \n     SoftmaxLoss, Large-Margin Softmax Loss, etc. \n - What effective data aug methods did you use?\n\nIf you don't mind, you can share the idea here. Thank you, and good luck.",
      "votes": 10
    },
    {
      "id": 411934,
      "postDate": "2018-10-29T08:37:49.513Z",
      "content": "<p>Hi Gary,</p>\n\n<ul>\n<li>I've compared Mobilenet with different, larger networks, and noticed that Mobilenet with default alpha value performs as well as the others. See <a href=\"https://www.kaggle.com/gaborfodor/greyscale-mobilenet-lb-0-892\">this kernel</a> by <a href=\"https://www.kaggle.com/gaborfodor\">beluga</a>, where it is used without much further optimization to achieve a high accuracy.</li>\n<li>Here I cannot tell you much as I still have to go into this. I did implement the truncated top-k entropy loss described in <a href=\"http://openaccess.thecvf.com/content_cvpr_2016/papers/Lapin_Loss_Functions_for_CVPR_2016_paper.pdf\">this paper</a> that optimizes for top-3 accuracy, but noticed it does not perform well when tested with the MAP@3 metric that this competition uses.</li>\n<li>Data augmentation works best when it is well informed, as in designed specifically for the sketch task. In <a href=\"http://homepages.inf.ed.ac.uk/thospeda/papers/yu2016sketchanet.pdf\">this paper</a> you can find a nice overview of augmentation methods on page 6-9.</li>\n</ul>\n\n<p>I hope I've helped you, and maybe someone else can shine their light on the loss functions :)</p>",
      "rawMarkdown": "Hi Gary,\n\n - I've compared Mobilenet with different, larger networks, and noticed that Mobilenet with default alpha value performs as well as the others. See [this kernel][1] by [beluga][2], where it is used without much further optimization to achieve a high accuracy.\n - Here I cannot tell you much as I still have to go into this. I did implement the truncated top-k entropy loss described in [this paper][3] that optimizes for top-3 accuracy, but noticed it does not perform well when tested with the MAP@3 metric that this competition uses.\n - Data augmentation works best when it is well informed, as in designed specifically for the sketch task. In [this paper][4] you can find a nice overview of augmentation methods on page 6-9.\n\nI hope I've helped you, and maybe someone else can shine their light on the loss functions :)\n\n\n  [1]: https://www.kaggle.com/gaborfodor/greyscale-mobilenet-lb-0-892\n  [2]: https://www.kaggle.com/gaborfodor\n  [3]: http://openaccess.thecvf.com/content_cvpr_2016/papers/Lapin_Loss_Functions_for_CVPR_2016_paper.pdf\n  [4]: http://homepages.inf.ed.ac.uk/thospeda/papers/yu2016sketchanet.pdf",
      "votes": 5,
      "replies": [
        {
          "id": 411988,
          "postDate": "2018-10-29T11:10:55.240Z",
          "content": "<p>wow, I will check the paper. Thanks for your favor. You are nice.</p>",
          "rawMarkdown": "wow, I will check the paper. Thanks for your favor. You are nice."
        },
        {
          "id": 412029,
          "postDate": "2018-10-29T12:20:27.123Z",
          "content": "<p>@Kees van Rooijen, can you share Your code of top-k entropy loss?</p>",
          "rawMarkdown": "@Kees van Rooijen, can you share Your code of top-k entropy loss?",
          "votes": 1
        },
        {
          "id": 412035,
          "postDate": "2018-10-29T12:38:20.087Z",
          "content": "<p>official code open souced here: <a href=\"https://github.com/oval-group/smooth-topk\">https://github.com/oval-group/smooth-topk</a></p>",
          "rawMarkdown": "official code open souced here: https://github.com/oval-group/smooth-topk"
        },
        {
          "id": 412047,
          "postDate": "2018-10-29T13:05:27.247Z",
          "content": "<p>But I think the top-k entropy loss is not fit this competition.</p>",
          "rawMarkdown": "But I think the top-k entropy loss is not fit this competition."
        },
        {
          "id": 414319,
          "postDate": "2018-11-02T14:38:54.473Z",
          "content": "<p>@Kees van Rooijen</p>\n\n<p>Did you tried the augmentation method in this paper:</p>\n\n<p><a href=\"http://homepages.inf.ed.ac.uk/thospeda/papers/yu2016sketchanet.pdf\">http://homepages.inf.ed.ac.uk/thospeda/papers/yu2016sketchanet.pdf</a></p>\n\n<p>I found the source code implemented in matlab, it seems like a very complicated augmentation method, did it useful in your model?</p>",
          "rawMarkdown": "@Kees van Rooijen\n\nDid you tried the augmentation method in this paper:\n\nhttp://homepages.inf.ed.ac.uk/thospeda/papers/yu2016sketchanet.pdf\n\nI found the source code implemented in matlab, it seems like a very complicated augmentation method, did it useful in your model?"
        },
        {
          "id": 416076,
          "postDate": "2018-11-06T06:40:25.480Z",
          "content": "<blockquote>\n  <p><strong>wh1te wrote</strong></p>\n  \n  <blockquote>\n    <p>official code open souced here: <a href=\"https://github.com/oval-group/smooth-topk\">https://github.com/oval-group/smooth-topk</a></p>\n  </blockquote>\n</blockquote>\n\n<p>Did you try this loss function?</p>",
          "rawMarkdown": "\n&gt; **wh1te wrote**\n&gt; \n&gt; &gt; official code open souced here: https://github.com/oval-group/smooth-topk\n\nDid you try this loss function?"
        }
      ]
    },
    {
      "id": 415441,
      "postDate": "2018-11-05T05:57:16.147Z",
      "content": "<p>Do we really need augmentation with this amount of data?</p>",
      "rawMarkdown": "Do we really need augmentation with this amount of data?",
      "replies": [
        {
          "id": 415458,
          "postDate": "2018-11-05T07:03:51.763Z",
          "content": "<p>maybe, I haven't done the experiment for 10 days because of the school.</p>",
          "rawMarkdown": "maybe, I haven't done the experiment for 10 days because of the school."
        }
      ]
    },
    {
      "id": 412027,
      "postDate": "2018-10-29T12:16:29.957Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 411934,
      "author_name": "Kees van Rooijen",
      "author_url": "",
      "post_date": "2018-10-29T08:37:49.513000",
      "content": "<p>Hi Gary,</p>\n\n<ul>\n<li>I've compared Mobilenet with different, larger networks, and noticed that Mobilenet with default alpha value performs as well as the others. See <a href=\"https://www.kaggle.com/gaborfodor/greyscale-mobilenet-lb-0-892\">this kernel</a> by <a href=\"https://www.kaggle.com/gaborfodor\">beluga</a>, where it is used without much further optimization to achieve a high accuracy.</li>\n<li>Here I cannot tell you much as I still have to go into this. I did implement the truncated top-k entropy loss described in <a href=\"http://openaccess.thecvf.com/content_cvpr_2016/papers/Lapin_Loss_Functions_for_CVPR_2016_paper.pdf\">this paper</a> that optimizes for top-3 accuracy, but noticed it does not perform well when tested with the MAP@3 metric that this competition uses.</li>\n<li>Data augmentation works best when it is well informed, as in designed specifically for the sketch task. In <a href=\"http://homepages.inf.ed.ac.uk/thospeda/papers/yu2016sketchanet.pdf\">this paper</a> you can find a nice overview of augmentation methods on page 6-9.</li>\n</ul>\n\n<p>I hope I've helped you, and maybe someone else can shine their light on the loss functions :)</p>",
      "votes": 5,
      "replies": [
        {
          "id": 411988,
          "author_name": "Gary",
          "author_url": "",
          "post_date": "2018-10-29T11:10:55.240000",
          "content": "<p>wow, I will check the paper. Thanks for your favor. You are nice.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 412029,
          "author_name": "Svitlana Tarasenko ",
          "author_url": "",
          "post_date": "2018-10-29T12:20:27.123000",
          "content": "<p>@Kees van Rooijen, can you share Your code of top-k entropy loss?</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 412035,
          "author_name": "wh1te",
          "author_url": "",
          "post_date": "2018-10-29T12:38:20.087000",
          "content": "<p>official code open souced here: <a href=\"https://github.com/oval-group/smooth-topk\">https://github.com/oval-group/smooth-topk</a></p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 412047,
          "author_name": "Gary",
          "author_url": "",
          "post_date": "2018-10-29T13:05:27.247000",
          "content": "<p>But I think the top-k entropy loss is not fit this competition.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 414319,
          "author_name": "wh1te",
          "author_url": "",
          "post_date": "2018-11-02T14:38:54.473000",
          "content": "<p>@Kees van Rooijen</p>\n\n<p>Did you tried the augmentation method in this paper:</p>\n\n<p><a href=\"http://homepages.inf.ed.ac.uk/thospeda/papers/yu2016sketchanet.pdf\">http://homepages.inf.ed.ac.uk/thospeda/papers/yu2016sketchanet.pdf</a></p>\n\n<p>I found the source code implemented in matlab, it seems like a very complicated augmentation method, did it useful in your model?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 416076,
          "author_name": "HuyenNguyen",
          "author_url": "",
          "post_date": "2018-11-06T06:40:25.480000",
          "content": "<blockquote>\n  <p><strong>wh1te wrote</strong></p>\n  \n  <blockquote>\n    <p>official code open souced here: <a href=\"https://github.com/oval-group/smooth-topk\">https://github.com/oval-group/smooth-topk</a></p>\n  </blockquote>\n</blockquote>\n\n<p>Did you try this loss function?</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 415441,
      "author_name": "HuyenNguyen",
      "author_url": "",
      "post_date": "2018-11-05T05:57:16.147000",
      "content": "<p>Do we really need augmentation with this amount of data?</p>",
      "votes": 0,
      "replies": [
        {
          "id": 415458,
          "author_name": "Gary",
          "author_url": "",
          "post_date": "2018-11-05T07:03:51.763000",
          "content": "<p>maybe, I haven't done the experiment for 10 days because of the school.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 412027,
      "author_name": "",
      "author_url": "",
      "post_date": "2018-10-29T12:16:29.957000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "409418": "Hi, I started this competition recently, and I want to ask you guys here.\n\n - What lightweight model did you use, mobilenet, shufflenet or self-designed model?\n - What better loss can be used in this competition,  like center loss, Ensemble Soft-Margin \n     SoftmaxLoss, Large-Margin Softmax Loss, etc. \n - What effective data aug methods did you use?\n\nIf you don't mind, you can share the idea here. Thank you, and good luck.",
    "411934": "Hi Gary,\n\n - I've compared Mobilenet with different, larger networks, and noticed that Mobilenet with default alpha value performs as well as the others. See [this kernel][1] by [beluga][2], where it is used without much further optimization to achieve a high accuracy.\n - Here I cannot tell you much as I still have to go into this. I did implement the truncated top-k entropy loss described in [this paper][3] that optimizes for top-3 accuracy, but noticed it does not perform well when tested with the MAP@3 metric that this competition uses.\n - Data augmentation works best when it is well informed, as in designed specifically for the sketch task. In [this paper][4] you can find a nice overview of augmentation methods on page 6-9.\n\nI hope I've helped you, and maybe someone else can shine their light on the loss functions :)\n\n\n  [1]: https://www.kaggle.com/gaborfodor/greyscale-mobilenet-lb-0-892\n  [2]: https://www.kaggle.com/gaborfodor\n  [3]: http://openaccess.thecvf.com/content_cvpr_2016/papers/Lapin_Loss_Functions_for_CVPR_2016_paper.pdf\n  [4]: http://homepages.inf.ed.ac.uk/thospeda/papers/yu2016sketchanet.pdf",
    "415441": "Do we really need augmentation with this amount of data?",
    "412027": ""
  }
}