{ "id": 73641, "title": "What is your best single model?", "url": "/competitions/quickdraw-doodle-recognition/discussion/73641", "author_name": "Strideradu", "post_date": "2018-12-04T14:31:06.749000", "votes": 6, "comment_count": 28, "views": 0, "content": "

Our best single model score is 944, and I am using senet 154 with image size 128.

", "messages": [ { "id": 433063, "postDate": "2018-12-04T16:13:10.743Z", "content": "

953 public, PNASNet5Large, image size 128x128

", "rawMarkdown": "953 public, PNASNet5Large, image size 128x128", "votes": 11, "replies": [ { "id": 433090, "postDate": "2018-12-04T17:21:00.383Z", "content": "

the difference of senet 154 (0.944) and PNASNet5Large (0.953) is large.

\n\n

I suppose the rendering from strokes to image is different for both cases?

\n\n

(Note: both have quite similar performance on image net although PNASNet5Large is better)

", "rawMarkdown": "the difference of senet 154 (0.944) and PNASNet5Large (0.953) is large.\n\nI suppose the rendering from strokes to image is different for both cases?\n\n\n(Note: both have quite similar performance on image net although PNASNet5Large is better)", "votes": 2 }, { "id": 433099, "postDate": "2018-12-04T17:45:14.483Z", "content": "

In my settings, Senet-154 gives 0.952 on LB. So, we will share our approach at the end of the competition :)

", "rawMarkdown": "In my settings, Senet-154 gives 0.952 on LB. So, we will share our approach at the end of the competition :)", "votes": 8 }, { "id": 433385, "postDate": "2018-12-05T02:47:26.783Z", "content": "

Congrats!! & can't wait to see your solution.

", "rawMarkdown": "Congrats!! & can't wait to see your solution." }, { "id": 434124, "postDate": "2018-12-05T23:28:14.310Z", "content": "

my senet 154 wasnt fully tuned but to train senet 154 is too slow

", "rawMarkdown": "my senet 154 wasnt fully tuned but to train senet 154 is too slow" } ] }, { "id": 432972, "postDate": "2018-12-04T14:31:06.750Z", "content": "

Our best single model score is 944, and I am using senet 154 with image size 128.

", "rawMarkdown": "Our best single model score is 944, and I am using senet 154 with image size 128.", "votes": 6 }, { "id": 433314, "postDate": "2018-12-05T01:12:50.133Z", "content": "

LSTM + Attention public LB 0.942

", "rawMarkdown": "LSTM + Attention public LB 0.942", "votes": 2, "replies": [ { "id": 433316, "postDate": "2018-12-05T01:15:28.663Z", "content": "

i am interested in lstm based solution. would you mind posting the code for your model?

\n\n

thanks!

\n\n

do you have results without attention? (i.e. what is the improvement from attention)

", "rawMarkdown": "i am interested in lstm based solution. would you mind posting the code for your model?\n\nthanks!\n\ndo you have results without attention? (i.e. what is the improvement from attention)", "votes": 1 }, { "id": 433326, "postDate": "2018-12-05T01:25:35.740Z", "content": "

I’m interested your lstm solution too, could u please share part of your code?

", "rawMarkdown": "I’m interested your lstm solution too, could u please share part of your code?" }, { "id": 433346, "postDate": "2018-12-05T01:46:07.007Z", "content": "

I'm sorry. I don't plan to post my code now.\nI referred to text classification code.

\n\n

https://github.com/wabyking/TextClassificationBenchmark\nhttps://github.com/prakashpandey9/Text-Classification-Pytorch

\n\n

I did't try to use LSTM without attention. So, I will try and report it !!

", "rawMarkdown": "I'm sorry. I don't plan to post my code now.\nI referred to text classification code.\n\nhttps://github.com/wabyking/TextClassificationBenchmark\nhttps://github.com/prakashpandey9/Text-Classification-Pytorch\n\nI did't try to use LSTM without attention. So, I will try and report it !!" }, { "id": 433388, "postDate": "2018-12-05T02:51:22.913Z", "content": "

thanks for the answer and link!

\n\n

what is the input to lstm? is it the same as originalt tensorflow tutorial? (dx,dy, is_pen_on_paper) or are you using cnn features as input?

\n\n

is there any special pre-processing of the data, e.g. handling of drawing with many points/strokes (unusual long sequence)?

", "rawMarkdown": "thanks for the answer and link!\n\nwhat is the input to lstm? is it the same as originalt tensorflow tutorial? (dx,dy, is_pen_on_paper) or are you using cnn features as input?\n\nis there any special pre-processing of the data, e.g. handling of drawing with many points/strokes (unusual long sequence)?" }, { "id": 433406, "postDate": "2018-12-05T03:28:05.537Z", "content": "

the data process is (dx,dy, ispenon_paper) -> cnn -> lstm -> attention -> linear.\nthis is similar to your pytorch starter kit.\nI did't apply special pre-processing, but I excluded some of the extremely long stroke in the train data.

", "rawMarkdown": "the data process is (dx,dy, ispenon_paper) -> cnn -> lstm -> attention -> linear.\nthis is similar to your pytorch starter kit.\nI did't apply special pre-processing, but I excluded some of the extremely long stroke in the train data.", "votes": 1 }, { "id": 433414, "postDate": "2018-12-05T03:37:34.190Z", "content": "

thanks for the answer. so it is a cnn-lstm approach.

\n\n

i haven't seen any pure lstm (without cnn and using point coordinates as input ) approach that can get good results, which i find it strange. i can get same good local validation score for pure cnn and pure lstm. However, public LB for pure lstm is much worse. (less 0.02). I am suspecting there is some difference between the test and train set \"stroke-wise\"

\n\n

since you have attention weights, you can draw and visualise the more important strokes, which may reveal something interesting in post analysis.

", "rawMarkdown": "thanks for the answer. so it is a cnn-lstm approach.\n\ni haven't seen any pure lstm (without cnn and using point coordinates as input ) approach that can get good results, which i find it strange. i can get same good local validation score for pure cnn and pure lstm. However, public LB for pure lstm is much worse. (less 0.02). I am suspecting there is some difference between the test and train set \"stroke-wise\"\n\nsince you have attention weights, you can draw and visualise the more important strokes, which may reveal something interesting in post analysis." }, { "id": 433687, "postDate": "2018-12-05T10:49:28.667Z", "content": "

I'm also interested in any RNN models! I tried pure LSTM and it scored badly, and I also tried ConvLSTM2D in Keras and it scored 0.885

", "rawMarkdown": "I'm also interested in any RNN models! I tried pure LSTM and it scored badly, and I also tried `ConvLSTM2D` in Keras and it scored 0.885" } ] }, { "id": 433287, "postDate": "2018-12-05T00:27:58.670Z", "content": "

946 public LB DPN92

", "rawMarkdown": "946 public LB DPN92", "votes": 1 }, { "id": 433056, "postDate": "2018-12-04T16:04:00.720Z", "content": "

945 public, SE-resnext50 size 180

", "rawMarkdown": "945 public, SE-resnext50 size 180", "votes": 1 }, { "id": 433156, "postDate": "2018-12-04T19:06:33.120Z", "content": "

0.938 Public LB with MobileNet

", "rawMarkdown": "0.938 Public LB with MobileNet", "votes": 2, "replies": [ { "id": 433377, "postDate": "2018-12-05T02:31:53.913Z", "content": "

I am curious how you achieved that score. The best I could get with MBN was 0.932 public LB.

", "rawMarkdown": "I am curious how you achieved that score. The best I could get with MBN was 0.932 public LB. " }, { "id": 433483, "postDate": "2018-12-05T05:54:20.903Z", "content": "

@HuyenNguyen, MBN with images of size 256, steps_per_epoch=30000 and weights were initialized from MBN which was trained on images of size 128. Model was trained for 100 epochs which took nearly a week of time.

", "rawMarkdown": "@HuyenNguyen, MBN with images of size 256, steps_per_epoch=30000 and weights were initialized from MBN which was trained on images of size 128. Model was trained for 100 epochs which took nearly a week of time. " }, { "id": 433557, "postDate": "2018-12-05T07:16:46.660Z", "content": "

I trained my model on 128 x 128 images to get to 0.932 but when I used it on 224 x 224 images, the score couldn't improve. Maybe I didn't wait for long enough? I only trained it for half a day and then it got stuck and I abandoned it.

", "rawMarkdown": "I trained my model on 128 x 128 images to get to 0.932 but when I used it on 224 x 224 images, the score couldn't improve. Maybe I didn't wait for long enough? I only trained it for half a day and then it got stuck and I abandoned it. " }, { "id": 433688, "postDate": "2018-12-05T10:55:50.587Z", "content": "

I got 0.92359 public LB by using MobileNet with 128x128 grayscale images, trained for 2 epochs, used Beluga's draw_cv2 () to generate grayscale images.

\n\n

I'm curious how you can get such a high score,

\n\n", "rawMarkdown": "I got 0.92359 public LB by using MobileNet with 128x128 grayscale images, trained for 2 epochs, used Beluga's `draw_cv2 ()` to generate grayscale images.\n\nI'm curious how you can get such a high score,\n\n* Do you use grayscale or RGB images?\n* How do you encode temporal information?" }, { "id": 433813, "postDate": "2018-12-05T14:01:24.723Z", "content": "

Using all the train 128x128 greyscale images on MobileNet with batch size 560 and steps per epoch 48500, I was able to achieve 0.931 public as well as private LB score. Ran nearly 8 epochs ...and learning rate 0.001 for first 4 epochs, 0.0001 for next 2 and 0.00001 for last 2...Each epoch took 10.5 hours.

", "rawMarkdown": "Using all the train 128x128 greyscale images on MobileNet with batch size 560 and steps per epoch 48500, I was able to achieve 0.931 public as well as private LB score. Ran nearly 8 epochs ...and learning rate 0.001 for first 4 epochs, 0.0001 for next 2 and 0.00001 for last 2...Each epoch took 10.5 hours. " } ] }, { "id": 434019, "postDate": "2018-12-05T19:22:33.197Z", "content": "

se-resnext50, 128x128, private 0.94617, public 0.94612

\n\n

xception, 160x160, private 0.94555, public 0.94626

", "rawMarkdown": "se-resnext50, 128x128, private 0.94617, public 0.94612\n\nxception, 160x160, private 0.94555, public 0.94626" }, { "id": 433391, "postDate": "2018-12-05T02:54:46.047Z", "content": "

947 public, 946 private, SE-resnext50 size 224

", "rawMarkdown": "947 public, 946 private, SE-resnext50 size 224" }, { "id": 433350, "postDate": "2018-12-05T01:50:17.867Z", "content": "

SEResnet50 w/o tta: public LB: 0.943, private LB: 0.941

", "rawMarkdown": "SEResnet50 w/o tta: public LB: 0.943, private LB: 0.941" }, { "id": 433337, "postDate": "2018-12-05T01:36:41.560Z", "content": "

resnet50 image size 128 0.931

", "rawMarkdown": "resnet50 image size 128 0.931" }, { "id": 433312, "postDate": "2018-12-05T01:09:26.513Z", "content": "

convolutional LSTM public/private LB0.933

", "rawMarkdown": "convolutional LSTM public/private LB0.933" }, { "id": 433283, "postDate": "2018-12-05T00:22:44.967Z", "content": "

ResNet34, image size 256, 0.940

", "rawMarkdown": "ResNet34, image size 256, 0.940" }, { "id": 433311, "postDate": "2018-12-05T01:07:04.727Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 433063, "author_name": "Pavel Ostyakov", "author_url": "", "post_date": "2018-12-04T16:13:10.743000", "content": "

953 public, PNASNet5Large, image size 128x128

", "votes": 11, "replies": [ { "id": 433090, "author_name": "hengck23", "author_url": "", "post_date": "2018-12-04T17:21:00.383000", "content": "

the difference of senet 154 (0.944) and PNASNet5Large (0.953) is large.

\n\n

I suppose the rendering from strokes to image is different for both cases?

\n\n

(Note: both have quite similar performance on image net although PNASNet5Large is better)

", "votes": 2, "replies": [] }, { "id": 433099, "author_name": "Pavel Ostyakov", "author_url": "", "post_date": "2018-12-04T17:45:14.483000", "content": "

In my settings, Senet-154 gives 0.952 on LB. So, we will share our approach at the end of the competition :)

", "votes": 8, "replies": [] }, { "id": 433385, "author_name": "dingwoai", "author_url": "", "post_date": "2018-12-05T02:47:26.783000", "content": "

Congrats!! & can't wait to see your solution.

", "votes": 0, "replies": [] }, { "id": 434124, "author_name": "Strideradu", "author_url": "", "post_date": "2018-12-05T23:28:14.310000", "content": "

my senet 154 wasnt fully tuned but to train senet 154 is too slow

", "votes": 0, "replies": [] } ] }, { "id": 433314, "author_name": "owruby", "author_url": "", "post_date": "2018-12-05T01:12:50.133000", "content": "

LSTM + Attention public LB 0.942

", "votes": 2, "replies": [ { "id": 433316, "author_name": "hengck23", "author_url": "", "post_date": "2018-12-05T01:15:28.663000", "content": "

i am interested in lstm based solution. would you mind posting the code for your model?

\n\n

thanks!

\n\n

do you have results without attention? (i.e. what is the improvement from attention)

", "votes": 1, "replies": [] }, { "id": 433326, "author_name": "wh1te", "author_url": "", "post_date": "2018-12-05T01:25:35.740000", "content": "

I’m interested your lstm solution too, could u please share part of your code?

", "votes": 0, "replies": [] }, { "id": 433346, "author_name": "owruby", "author_url": "", "post_date": "2018-12-05T01:46:07.007000", "content": "

I'm sorry. I don't plan to post my code now.\nI referred to text classification code.

\n\n

https://github.com/wabyking/TextClassificationBenchmark\nhttps://github.com/prakashpandey9/Text-Classification-Pytorch

\n\n

I did't try to use LSTM without attention. So, I will try and report it !!

", "votes": 0, "replies": [] }, { "id": 433388, "author_name": "hengck23", "author_url": "", "post_date": "2018-12-05T02:51:22.913000", "content": "

thanks for the answer and link!

\n\n

what is the input to lstm? is it the same as originalt tensorflow tutorial? (dx,dy, is_pen_on_paper) or are you using cnn features as input?

\n\n

is there any special pre-processing of the data, e.g. handling of drawing with many points/strokes (unusual long sequence)?

", "votes": 0, "replies": [] }, { "id": 433406, "author_name": "owruby", "author_url": "", "post_date": "2018-12-05T03:28:05.537000", "content": "

the data process is (dx,dy, ispenon_paper) -> cnn -> lstm -> attention -> linear.\nthis is similar to your pytorch starter kit.\nI did't apply special pre-processing, but I excluded some of the extremely long stroke in the train data.

", "votes": 1, "replies": [] }, { "id": 433414, "author_name": "hengck23", "author_url": "", "post_date": "2018-12-05T03:37:34.190000", "content": "

thanks for the answer. so it is a cnn-lstm approach.

\n\n

i haven't seen any pure lstm (without cnn and using point coordinates as input ) approach that can get good results, which i find it strange. i can get same good local validation score for pure cnn and pure lstm. However, public LB for pure lstm is much worse. (less 0.02). I am suspecting there is some difference between the test and train set \"stroke-wise\"

\n\n

since you have attention weights, you can draw and visualise the more important strokes, which may reveal something interesting in post analysis.

", "votes": 0, "replies": [] }, { "id": 433687, "author_name": "[he.ai]soulmachine", "author_url": "", "post_date": "2018-12-05T10:49:28.667000", "content": "

I'm also interested in any RNN models! I tried pure LSTM and it scored badly, and I also tried ConvLSTM2D in Keras and it scored 0.885

", "votes": 0, "replies": [] } ] }, { "id": 433287, "author_name": "wh1te", "author_url": "", "post_date": "2018-12-05T00:27:58.670000", "content": "

946 public LB DPN92

", "votes": 1, "replies": [] }, { "id": 433056, "author_name": "Miha Skalic", "author_url": "", "post_date": "2018-12-04T16:04:00.720000", "content": "

945 public, SE-resnext50 size 180

", "votes": 1, "replies": [] }, { "id": 433156, "author_name": "Rajesh Shreedhar", "author_url": "", "post_date": "2018-12-04T19:06:33.120000", "content": "

0.938 Public LB with MobileNet

", "votes": 2, "replies": [ { "id": 433377, "author_name": "HuyenNguyen", "author_url": "", "post_date": "2018-12-05T02:31:53.913000", "content": "

I am curious how you achieved that score. The best I could get with MBN was 0.932 public LB.

", "votes": 0, "replies": [] }, { "id": 433483, "author_name": "Rajesh Shreedhar", "author_url": "", "post_date": "2018-12-05T05:54:20.903000", "content": "

@HuyenNguyen, MBN with images of size 256, steps_per_epoch=30000 and weights were initialized from MBN which was trained on images of size 128. Model was trained for 100 epochs which took nearly a week of time.

", "votes": 0, "replies": [] }, { "id": 433557, "author_name": "HuyenNguyen", "author_url": "", "post_date": "2018-12-05T07:16:46.660000", "content": "

I trained my model on 128 x 128 images to get to 0.932 but when I used it on 224 x 224 images, the score couldn't improve. Maybe I didn't wait for long enough? I only trained it for half a day and then it got stuck and I abandoned it.

", "votes": 0, "replies": [] }, { "id": 433688, "author_name": "[he.ai]soulmachine", "author_url": "", "post_date": "2018-12-05T10:55:50.587000", "content": "

I got 0.92359 public LB by using MobileNet with 128x128 grayscale images, trained for 2 epochs, used Beluga's draw_cv2 () to generate grayscale images.

\n\n

I'm curious how you can get such a high score,

\n\n", "votes": 0, "replies": [] }, { "id": 433813, "author_name": "Abhilash Awasthi", "author_url": "", "post_date": "2018-12-05T14:01:24.723000", "content": "

Using all the train 128x128 greyscale images on MobileNet with batch size 560 and steps per epoch 48500, I was able to achieve 0.931 public as well as private LB score. Ran nearly 8 epochs ...and learning rate 0.001 for first 4 epochs, 0.0001 for next 2 and 0.00001 for last 2...Each epoch took 10.5 hours.

", "votes": 0, "replies": [] } ] }, { "id": 434019, "author_name": "Dinrker", "author_url": "", "post_date": "2018-12-05T19:22:33.197000", "content": "

se-resnext50, 128x128, private 0.94617, public 0.94612

\n\n

xception, 160x160, private 0.94555, public 0.94626

", "votes": 0, "replies": [] }, { "id": 433391, "author_name": "pudae", "author_url": "", "post_date": "2018-12-05T02:54:46.047000", "content": "

947 public, 946 private, SE-resnext50 size 224

", "votes": 0, "replies": [] }, { "id": 433350, "author_name": "cab", "author_url": "", "post_date": "2018-12-05T01:50:17.867000", "content": "

SEResnet50 w/o tta: public LB: 0.943, private LB: 0.941

", "votes": 0, "replies": [] }, { "id": 433337, "author_name": "Yang", "author_url": "", "post_date": "2018-12-05T01:36:41.560000", "content": "

resnet50 image size 128 0.931

", "votes": 0, "replies": [] }, { "id": 433312, "author_name": "luyaxin", "author_url": "", "post_date": "2018-12-05T01:09:26.513000", "content": "

convolutional LSTM public/private LB0.933

", "votes": 0, "replies": [] }, { "id": 433283, "author_name": "杨培文 (Yang Peiwen)", "author_url": "", "post_date": "2018-12-05T00:22:44.967000", "content": "

ResNet34, image size 256, 0.940

", "votes": 0, "replies": [] }, { "id": 433311, "author_name": "", "author_url": "", "post_date": "2018-12-05T01:07:04.727000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "433063": "953 public, PNASNet5Large, image size 128x128", "432972": "Our best single model score is 944, and I am using senet 154 with image size 128.", "433314": "LSTM + Attention public LB 0.942", "433287": "946 public LB DPN92", "433056": "945 public, SE-resnext50 size 180", "433156": "0.938 Public LB with MobileNet", "434019": "se-resnext50, 128x128, private 0.94617, public 0.94612\n\nxception, 160x160, private 0.94555, public 0.94626", "433391": "947 public, 946 private, SE-resnext50 size 224", "433350": "SEResnet50 w/o tta: public LB: 0.943, private LB: 0.941", "433337": "resnet50 image size 128 0.931", "433312": "convolutional LSTM public/private LB0.933", "433283": "ResNet34, image size 256, 0.940", "433311": "" } }