{
  "id": 73866,
  "title": "CNN with Attention",
  "url": "/competitions/quickdraw-doodle-recognition/discussion/73866",
  "author_name": "Artyom Palvelev",
  "post_date": "2018-12-06T09:23:45.578000",
  "votes": 4,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Everyone knows SE networks are cool. This paper: <a href=\"https://arxiv.org/pdf/1807.06521.pdf\">https://arxiv.org/pdf/1807.06521.pdf</a> suggested that CBAM modules are even better (they tested SE-ResNet50 and CBAM-ResNet50). </p>\n\n<p>We tried a pretrained CBAM-ResNet50 network from this repo: <a href=\"https://github.com/Jongchan/attention-module\">https://github.com/Jongchan/attention-module</a> in this competition. It could achieve 0.936 on both public and private LB and we haven't trained it for very long: it has only seen 40M samples once, no cosine annealing.</p>\n\n<p>We didn't train SE-ResNet50 so it's difficult to compare. One thing I know is, it performed better than Inception-ResNet-V2.</p>\n\n<p>Do you have any better results? Do you know any better implementations? Thanks for reading :)</p>",
  "messages": [
    {
      "id": 434368,
      "postDate": "2018-12-06T09:23:45.580Z",
      "content": "<p>Everyone knows SE networks are cool. This paper: <a href=\"https://arxiv.org/pdf/1807.06521.pdf\">https://arxiv.org/pdf/1807.06521.pdf</a> suggested that CBAM modules are even better (they tested SE-ResNet50 and CBAM-ResNet50). </p>\n\n<p>We tried a pretrained CBAM-ResNet50 network from this repo: <a href=\"https://github.com/Jongchan/attention-module\">https://github.com/Jongchan/attention-module</a> in this competition. It could achieve 0.936 on both public and private LB and we haven't trained it for very long: it has only seen 40M samples once, no cosine annealing.</p>\n\n<p>We didn't train SE-ResNet50 so it's difficult to compare. One thing I know is, it performed better than Inception-ResNet-V2.</p>\n\n<p>Do you have any better results? Do you know any better implementations? Thanks for reading :)</p>",
      "rawMarkdown": "Everyone knows SE networks are cool. This paper: https://arxiv.org/pdf/1807.06521.pdf suggested that CBAM modules are even better (they tested SE-ResNet50 and CBAM-ResNet50). \n\nWe tried a pretrained CBAM-ResNet50 network from this repo: https://github.com/Jongchan/attention-module in this competition. It could achieve 0.936 on both public and private LB and we haven't trained it for very long: it has only seen 40M samples once, no cosine annealing.\n\nWe didn't train SE-ResNet50 so it's difficult to compare. One thing I know is, it performed better than Inception-ResNet-V2.\n\nDo you have any better results? Do you know any better implementations? Thanks for reading :)",
      "votes": 4
    },
    {
      "id": 895028,
      "postDate": "2020-06-21T03:59:26.157Z",
      "rawMarkdown": "",
      "isDeleted": true,
      "replies": [
        {
          "id": 936652,
          "postDate": "2020-07-20T12:11:05.977Z",
          "content": "<p>Hey! I'm afraid I can't guarantee the equivalence. You should compare every line of code with the reference implementation. I posted the link above.</p>",
          "rawMarkdown": "Hey! I'm afraid I can't guarantee the equivalence. You should compare every line of code with the reference implementation. I posted the link above."
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 895028,
      "author_name": "",
      "author_url": "",
      "post_date": "2020-06-21T03:59:26.157000",
      "content": "",
      "votes": 0,
      "replies": [
        {
          "id": 936652,
          "author_name": "Artyom Palvelev",
          "author_url": "",
          "post_date": "2020-07-20T12:11:05.977000",
          "content": "<p>Hey! I'm afraid I can't guarantee the equivalence. You should compare every line of code with the reference implementation. I posted the link above.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "434368": "Everyone knows SE networks are cool. This paper: https://arxiv.org/pdf/1807.06521.pdf suggested that CBAM modules are even better (they tested SE-ResNet50 and CBAM-ResNet50). \n\nWe tried a pretrained CBAM-ResNet50 network from this repo: https://github.com/Jongchan/attention-module in this competition. It could achieve 0.936 on both public and private LB and we haven't trained it for very long: it has only seen 40M samples once, no cosine annealing.\n\nWe didn't train SE-ResNet50 so it's difficult to compare. One thing I know is, it performed better than Inception-ResNet-V2.\n\nDo you have any better results? Do you know any better implementations? Thanks for reading :)",
    "895028": ""
  }
}