{
  "id": 75118,
  "title": "Another approach - All 1d convolutions",
  "url": "/competitions/quickdraw-doodle-recognition/discussion/75118",
  "author_name": "RossWightman",
  "post_date": "2018-12-18T17:08:39.551000",
  "votes": 4,
  "comment_count": 0,
  "views": 0,
  "content": "<p>This dataset is fun so decided to do a few more experiments post competition. Inspired by the 1d conv frontend of my RNN models, I wondered how well a pure 1d convolutional network would perform with a similar seqlen x numchan input where, like RNN channels are normalized [x, y, t, pen up (or down)].</p>\n\n<p>So, quick experiment, I took an SE-ResNext 50 by way of Cadene and Alex Parinov and did an s/2d/1d/g and wala, a fully 1d convolutional net. </p>\n\n<p>I trained for a day and a bit with sequences interpolated and clipped to 224 and hit 0.939/0.940 priv/pub. Certainly not the best single model score, but not bad for one trial. For anyone who is interested in working with data like this in the future, the most interesting consideration here is the speed. During training I hit a throughputs of 1700-1800 sample/sec with two 1080ti. Inference on 1 gpu throughput was over 3000 sample/sec. So compared to RNN and deep 2d conv nets, this is fast! </p>",
  "messages": [
    {
      "id": 441446,
      "postDate": "2018-12-18T17:08:39.550Z",
      "content": "<p>This dataset is fun so decided to do a few more experiments post competition. Inspired by the 1d conv frontend of my RNN models, I wondered how well a pure 1d convolutional network would perform with a similar seqlen x numchan input where, like RNN channels are normalized [x, y, t, pen up (or down)].</p>\n\n<p>So, quick experiment, I took an SE-ResNext 50 by way of Cadene and Alex Parinov and did an s/2d/1d/g and wala, a fully 1d convolutional net. </p>\n\n<p>I trained for a day and a bit with sequences interpolated and clipped to 224 and hit 0.939/0.940 priv/pub. Certainly not the best single model score, but not bad for one trial. For anyone who is interested in working with data like this in the future, the most interesting consideration here is the speed. During training I hit a throughputs of 1700-1800 sample/sec with two 1080ti. Inference on 1 gpu throughput was over 3000 sample/sec. So compared to RNN and deep 2d conv nets, this is fast! </p>",
      "rawMarkdown": "This dataset is fun so decided to do a few more experiments post competition. Inspired by the 1d conv frontend of my RNN models, I wondered how well a pure 1d convolutional network would perform with a similar seqlen x numchan input where, like RNN channels are normalized [x, y, t, pen up (or down)].\n\nSo, quick experiment, I took an SE-ResNext 50 by way of Cadene and Alex Parinov and did an s/2d/1d/g and wala, a fully 1d convolutional net. \n\nI trained for a day and a bit with sequences interpolated and clipped to 224 and hit 0.939/0.940 priv/pub. Certainly not the best single model score, but not bad for one trial. For anyone who is interested in working with data like this in the future, the most interesting consideration here is the speed. During training I hit a throughputs of 1700-1800 sample/sec with two 1080ti. Inference on 1 gpu throughput was over 3000 sample/sec. So compared to RNN and deep 2d conv nets, this is fast! \n\n",
      "votes": 4
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "441446": "This dataset is fun so decided to do a few more experiments post competition. Inspired by the 1d conv frontend of my RNN models, I wondered how well a pure 1d convolutional network would perform with a similar seqlen x numchan input where, like RNN channels are normalized [x, y, t, pen up (or down)].\n\nSo, quick experiment, I took an SE-ResNext 50 by way of Cadene and Alex Parinov and did an s/2d/1d/g and wala, a fully 1d convolutional net. \n\nI trained for a day and a bit with sequences interpolated and clipped to 224 and hit 0.939/0.940 priv/pub. Certainly not the best single model score, but not bad for one trial. For anyone who is interested in working with data like this in the future, the most interesting consideration here is the speed. During training I hit a throughputs of 1700-1800 sample/sec with two 1080ti. Inference on 1 gpu throughput was over 3000 sample/sec. So compared to RNN and deep 2d conv nets, this is fast! \n\n"
  }
}