{
  "id": 70816,
  "title": "best rnn/lstm score?",
  "url": "/competitions/quickdraw-doodle-recognition/discussion/70816",
  "author_name": "hengck23",
  "post_date": "2018-11-07T13:54:35.148000",
  "votes": 5,
  "comment_count": 34,
  "views": 0,
  "content": "<p>anyone can share the best score they can get?</p>",
  "messages": [
    {
      "id": 416949,
      "postDate": "2018-11-07T13:54:35.147Z",
      "content": "<p>anyone can share the best score they can get?</p>",
      "rawMarkdown": "anyone can share the best score they can get?",
      "votes": 5
    },
    {
      "id": 417335,
      "postDate": "2018-11-08T05:40:28.343Z",
      "content": "<p><a href=\"https://www.kaggle.com/hengck23\"></a><a href=\"/hengck23\">@hengck23</a></p>\n\n<p>So far I manage to achieve 0.923 in the public LB, with a custom LSTM network, using stroke augmentation during training, and simple TTA.</p>\n\n<p>My training scores are:</p>\n\n<pre><code> - loss:        0.6739   categorical_accuracy:        0.8227   top_3_accuracy:        0.9372 \n - val_loss:  0.6479  val_categorical_accuracy:  0.8299   val_top_3_accuracy: 0.9405\n</code></pre>\n\n<p>I'm having only one fold with 30M training samples and 4M validation. Probably I can still improve my LB score up to 0.93. After that it's time to start training a CNN and a second stage model to combine both. Most likely I will no longer manage to finish that cause I have limited hardware available.</p>",
      "rawMarkdown": "[@hengck23][1]\n\nSo far I manage to achieve 0.923 in the public LB, with a custom LSTM network, using stroke augmentation during training, and simple TTA.\n\nMy training scores are:\n\n     - loss:        0.6739   categorical_accuracy:        0.8227   top_3_accuracy:        0.9372 \n     - val_loss:  0.6479  val_categorical_accuracy:  0.8299   val_top_3_accuracy: 0.9405\n\nI'm having only one fold with 30M training samples and 4M validation. Probably I can still improve my LB score up to 0.93. After that it's time to start training a CNN and a second stage model to combine both. Most likely I will no longer manage to finish that cause I have limited hardware available.\n\n\n  [1]: https://www.kaggle.com/hengck23",
      "votes": 3,
      "replies": [
        {
          "id": 417473,
          "postDate": "2018-11-08T10:43:43.550Z",
          "content": "<p>Nice !</p>\n\n<p>I did not submitt results yet but my LSTM network performs as well as your:\nValidation:  Top1: 0.83 ; Top3: 0.94</p>\n\n<p>I have also a CNN with the following results:\nValidation:  Top1: 0.80 ; Top3: 0.92</p>\n\n<p>Combining both gives:\nValidation:  Top1: 0.84 ; Top3: 0.945</p>",
          "rawMarkdown": "Nice !\n\nI did not submitt results yet but my LSTM network performs as well as your:\nValidation:  Top1: 0.83 ; Top3: 0.94\n\nI have also a CNN with the following results:\nValidation:  Top1: 0.80 ; Top3: 0.92\n\nCombining both gives:\nValidation:  Top1: 0.84 ; Top3: 0.945\n\n\n",
          "votes": 1
        },
        {
          "id": 417548,
          "postDate": "2018-11-08T12:43:50.057Z",
          "content": "<p>Well done! I'm curious to see if your LSTM will score similar to mine on the public LB, cause looking at my local score, I actually expected my LSTM to score higher on the public LB</p>",
          "rawMarkdown": "Well done! I'm curious to see if your LSTM will score similar to mine on the public LB, cause looking at my local score, I actually expected my LSTM to score higher on the public LB"
        },
        {
          "id": 417570,
          "postDate": "2018-11-08T13:21:37.750Z",
          "content": "<p>very impressive. do you use attention?</p>",
          "rawMarkdown": "very impressive. do you use attention?"
        },
        {
          "id": 417590,
          "postDate": "2018-11-08T13:50:56.567Z",
          "content": "<p><a href=\"/jeandebleau\">@jeandebleau</a></p>\n\n<p>is your LSTM using stroke features (e.g. x,y) or CNN extracted features?\nthanks!</p>",
          "rawMarkdown": "@jeandebleau\n\nis your LSTM using stroke features (e.g. x,y) or CNN extracted features?\nthanks!"
        },
        {
          "id": 417600,
          "postDate": "2018-11-08T13:56:08.030Z",
          "content": "<p>Hi, Heng, How did you use TTA this competiton?</p>",
          "rawMarkdown": "Hi, Heng, How did you use TTA this competiton?"
        },
        {
          "id": 417623,
          "postDate": "2018-11-08T14:34:53.477Z",
          "content": "<p>The LSTM is only using stroke features.</p>\n\n<p>The combination of both takes the last hidden layer of the two networks and learns a combination. More or less what was described in a paper that you also mentionned.  </p>\n\n<p>I tried a Convolutional LSTM without much success, also it is extremely slow. I also tried PointNet and PointGrid. These approaches may work but I could not get very good results. </p>\n\n<p>Right now I am trying alternative ideas. </p>",
          "rawMarkdown": "The LSTM is only using stroke features.\n\nThe combination of both takes the last hidden layer of the two networks and learns a combination. More or less what was described in a paper that you also mentionned.  \n\nI tried a Convolutional LSTM without much success, also it is extremely slow. I also tried PointNet and PointGrid. These approaches may work but I could not get very good results. \n\nRight now I am trying alternative ideas. ",
          "votes": 1
        },
        {
          "id": 417665,
          "postDate": "2018-11-08T15:51:56.503Z",
          "content": "<p><a href=\"/huyenvyvy\">@huyenvyvy</a> Yes, I'm using attention. It gave a big boost to my initial LSTM score.\n<a href=\"/hengck23\">@hengck23</a> I'm also only using stroke features and country</p>",
          "rawMarkdown": "@huyenvyvy Yes, I'm using attention. It gave a big boost to my initial LSTM score.\n@hengck23 I'm also only using stroke features and country\n",
          "votes": 1
        },
        {
          "id": 417923,
          "postDate": "2018-11-09T01:46:52.037Z",
          "content": "<p>Did using country info help you compared to only stroke? </p>",
          "rawMarkdown": "Did using country info help you compared to only stroke? "
        },
        {
          "id": 418004,
          "postDate": "2018-11-09T05:16:57.673Z",
          "content": "<p><a href=\"/huyenvyvy\">@huyenvyvy</a> Not really sure, cause I started using country info since the beginning. I might be wrong, but in my opinion the way you draw stuff is strongly related to where you were raised.</p>",
          "rawMarkdown": "@huyenvyvy Not really sure, cause I started using country info since the beginning. I might be wrong, but in my opinion the way you draw stuff is strongly related to where you were raised."
        },
        {
          "id": 418013,
          "postDate": "2018-11-09T05:24:31.567Z",
          "content": "<p>I agree, the intuition seems sound. except it wasn't borne out by the data in my experience :D adding country didn't help at all. </p>",
          "rawMarkdown": "I agree, the intuition seems sound. except it wasn't borne out by the data in my experience :D adding country didn't help at all. "
        },
        {
          "id": 418076,
          "postDate": "2018-11-09T08:55:59.110Z",
          "content": "<p>@Nuno Ferreira, how are you adding the attention layer? when I use it as follows, I get AssertionError.</p>\n\n<blockquote>\n  <p>model.add(Bidirectional(LSTM(128, return_sequences = True)))\n  model.add(AttentionWithContext())</p>\n</blockquote>",
          "rawMarkdown": "@Nuno Ferreira, how are you adding the attention layer? when I use it as follows, I get AssertionError.\n\n&gt; model.add(Bidirectional(LSTM(128, return_sequences = True)))\nmodel.add(AttentionWithContext())"
        },
        {
          "id": 418222,
          "postDate": "2018-11-09T13:54:26.560Z",
          "content": "<p><a href=\"/sheriytm\">@sheriytm</a> I can't see anything wrong in the way you add the attention layer . Do you have any extra logging with the assertion error?</p>",
          "rawMarkdown": "@sheriytm I can't see anything wrong in the way you add the attention layer . Do you have any extra logging with the assertion error?"
        }
      ]
    },
    {
      "id": 416959,
      "postDate": "2018-11-07T14:20:01.303Z",
      "content": "<p>as a reference, i was about to get lb 0.899 in my early experiments</p>",
      "rawMarkdown": "as a reference, i was about to get lb 0.899 in my early experiments\n\n",
      "votes": 3,
      "replies": [
        {
          "id": 417069,
          "postDate": "2018-11-07T17:31:43.727Z",
          "content": "<p>@henkck, Hi Henk,</p>\n\n<p>What do you think, if you train couple of cnn networks and feed the outputs to make new predictions using RNN for the final submission. Do you think it may work? I'm far beginner for this type of completion.</p>",
          "rawMarkdown": "@henkck, Hi Henk,\n\nWhat do you think, if you train couple of cnn networks and feed the outputs to make new predictions using RNN for the final submission. Do you think it may work? I'm far beginner for this type of completion."
        }
      ]
    },
    {
      "id": 419728,
      "postDate": "2018-11-12T13:31:06.057Z",
      "content": "<p>I submitted some results on a lstm network trained on 50k samples per class.</p>\n\n<p>It scores Public LB 0.93\nLocal Top1: 84.7% ; Top3: 94.8%</p>",
      "rawMarkdown": "I submitted some results on a lstm network trained on 50k samples per class.\n\nIt scores Public LB 0.93\nLocal Top1: 84.7% ; Top3: 94.8%\n",
      "votes": 4,
      "replies": [
        {
          "id": 419733,
          "postDate": "2018-11-12T13:40:21.023Z",
          "content": "<p>That's a great score with 50k samples. Are you using just LSTM network or CNN+LSTM?</p>",
          "rawMarkdown": "That's a great score with 50k samples. Are you using just LSTM network or CNN+LSTM?"
        },
        {
          "id": 419741,
          "postDate": "2018-11-12T13:52:27.147Z",
          "content": "<p>This one is only LSTM. </p>",
          "rawMarkdown": "This one is only LSTM. ",
          "votes": 1
        },
        {
          "id": 419745,
          "postDate": "2018-11-12T13:56:53.490Z",
          "content": "<p>How much did you train your model? I trained mine for 10000 steps 20 epochs with all files from train simplified but it didn't perform that well.</p>",
          "rawMarkdown": "How much did you train your model? I trained mine for 10000 steps 20 epochs with all files from train simplified but it didn't perform that well."
        },
        {
          "id": 419747,
          "postDate": "2018-11-12T13:59:47.100Z",
          "content": "<p>I use 50k samples per images. One epoch uses 1000 random images per class, and it is trained for 400 epochs. </p>",
          "rawMarkdown": "I use 50k samples per images. One epoch uses 1000 random images per class, and it is trained for 400 epochs. ",
          "votes": 1
        },
        {
          "id": 419749,
          "postDate": "2018-11-12T14:01:34.393Z",
          "content": "<p>Okay, I think I need to train my model more.</p>",
          "rawMarkdown": "Okay, I think I need to train my model more."
        },
        {
          "id": 419750,
          "postDate": "2018-11-12T14:05:28.093Z",
          "content": "<p>By 1000 random images per class you mean you are using batch size of 1000, right? How many steps you are going per epoch?</p>",
          "rawMarkdown": "By 1000 random images per class you mean you are using batch size of 1000, right? How many steps you are going per epoch?"
        },
        {
          "id": 420168,
          "postDate": "2018-11-13T07:28:56.503Z",
          "content": "<p>Great score!</p>",
          "rawMarkdown": "Great score!"
        },
        {
          "id": 420199,
          "postDate": "2018-11-13T08:52:20.977Z",
          "content": "<p>I define one epoch when 1000 random images per class has been given as input. I use a batch size of 1000 as well. So one epoch is finished when 340 * 1000 images have been analyzed. I do that 400 times. </p>",
          "rawMarkdown": "I define one epoch when 1000 random images per class has been given as input. I use a batch size of 1000 as well. So one epoch is finished when 340 * 1000 images have been analyzed. I do that 400 times. \n",
          "votes": 1
        }
      ]
    },
    {
      "id": 417215,
      "postDate": "2018-11-08T00:24:44.350Z",
      "content": "<p>I achieved 0.829 <a href=\"https://www.kaggle.com/huyenvyvy/bidirectional-lstm-using-data-generator-lb-0-825\">https://www.kaggle.com/huyenvyvy/bidirectional-lstm-using-data-generator-lb-0-825</a></p>",
      "rawMarkdown": "I achieved 0.829 https://www.kaggle.com/huyenvyvy/bidirectional-lstm-using-data-generator-lb-0-825",
      "votes": 4
    },
    {
      "id": 417941,
      "postDate": "2018-11-09T02:45:07.753Z",
      "content": "<p>LB 0.869 on 5k images per class, LSTM</p>",
      "rawMarkdown": "LB 0.869 on 5k images per class, LSTM",
      "votes": 1,
      "replies": [
        {
          "id": 417958,
          "postDate": "2018-11-09T03:53:04.123Z",
          "content": "<p>Did you use any data augmentation? That's a very good result, if you use more data, maybe it'll improve further. </p>",
          "rawMarkdown": "Did you use any data augmentation? That's a very good result, if you use more data, maybe it'll improve further. "
        },
        {
          "id": 418002,
          "postDate": "2018-11-09T05:09:40.913Z",
          "content": "<p>Hi @HuyenNguyen,  I am using stroke augmentations like randomly scaling strokes and randomly removing strokes. I think it may be overfitting, because the validation data size is pretty small. trained on small data size to save GCP credits for later use...</p>",
          "rawMarkdown": "Hi @HuyenNguyen,  I am using stroke augmentations like randomly scaling strokes and randomly removing strokes. I think it may be overfitting, because the validation data size is pretty small. trained on small data size to save GCP credits for later use..."
        },
        {
          "id": 418015,
          "postDate": "2018-11-09T05:30:58.810Z",
          "content": "<p>Hi @luyaxin is there any resource i can look for the augmentations? Thanks.</p>",
          "rawMarkdown": "Hi @luyaxin is there any resource i can look for the augmentations? Thanks."
        },
        {
          "id": 418021,
          "postDate": "2018-11-09T05:46:39.223Z",
          "content": "<p>You can find the stroke augmentations here:\n<a href=\"https://github.com/tensorflow/magenta/tree/master/magenta/models/sketch_rnn\">https://github.com/tensorflow/magenta/tree/master/magenta/models/sketch_rnn</a></p>",
          "rawMarkdown": "You can find the stroke augmentations here:\nhttps://github.com/tensorflow/magenta/tree/master/magenta/models/sketch_rnn",
          "votes": 2
        },
        {
          "id": 418022,
          "postDate": "2018-11-09T05:49:08.540Z",
          "content": "<p>Awesome, Thanks @luyaxin.</p>",
          "rawMarkdown": "Awesome, Thanks @luyaxin."
        }
      ]
    },
    {
      "id": 417342,
      "postDate": "2018-11-08T06:00:27.367Z",
      "content": "<p>I achieved 0.853 using CNN+LSTM using shuffle_csvs.</p>",
      "rawMarkdown": "I achieved 0.853 using CNN+LSTM using shuffle_csvs.",
      "votes": 1
    },
    {
      "id": 416958,
      "postDate": "2018-11-07T14:15:49.090Z",
      "content": "<p>From the public kernels:</p>\n\n<p><a href=\"/kmader\">@kmader</a> 0.55 <a href=\"https://www.kaggle.com/kmader/quickdraw-baseline-lstm-reading-and-submission\">https://www.kaggle.com/kmader/quickdraw-baseline-lstm-reading-and-submission</a></p>\n\n<p><a href=\"/kmader\">@kmader</a> 0.63 <a href=\"https://www.kaggle.com/kmader/quickdraw-with-wavenet-classifier\">https://www.kaggle.com/kmader/quickdraw-with-wavenet-classifier</a></p>\n\n<p><a href=\"/avinashrai\">@avinashrai</a> 0.68 <a href=\"https://www.kaggle.com/avinashrai/quickdraw-with-wavenet-classifier-tunning\">https://www.kaggle.com/avinashrai/quickdraw-with-wavenet-classifier-tunning</a></p>",
      "rawMarkdown": "From the public kernels:\n\n@kmader 0.55 https://www.kaggle.com/kmader/quickdraw-baseline-lstm-reading-and-submission\n\n@kmader 0.63 https://www.kaggle.com/kmader/quickdraw-with-wavenet-classifier\n\n@avinashrai 0.68 https://www.kaggle.com/avinashrai/quickdraw-with-wavenet-classifier-tunning",
      "votes": 2
    },
    {
      "id": 516369,
      "postDate": "2019-04-14T04:08:57.510Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 417335,
      "author_name": "Nuno Ferreira",
      "author_url": "",
      "post_date": "2018-11-08T05:40:28.343000",
      "content": "<p><a href=\"https://www.kaggle.com/hengck23\"></a><a href=\"/hengck23\">@hengck23</a></p>\n\n<p>So far I manage to achieve 0.923 in the public LB, with a custom LSTM network, using stroke augmentation during training, and simple TTA.</p>\n\n<p>My training scores are:</p>\n\n<pre><code> - loss:        0.6739   categorical_accuracy:        0.8227   top_3_accuracy:        0.9372 \n - val_loss:  0.6479  val_categorical_accuracy:  0.8299   val_top_3_accuracy: 0.9405\n</code></pre>\n\n<p>I'm having only one fold with 30M training samples and 4M validation. Probably I can still improve my LB score up to 0.93. After that it's time to start training a CNN and a second stage model to combine both. Most likely I will no longer manage to finish that cause I have limited hardware available.</p>",
      "votes": 3,
      "replies": [
        {
          "id": 417473,
          "author_name": "jeandebleau",
          "author_url": "",
          "post_date": "2018-11-08T10:43:43.550000",
          "content": "<p>Nice !</p>\n\n<p>I did not submitt results yet but my LSTM network performs as well as your:\nValidation:  Top1: 0.83 ; Top3: 0.94</p>\n\n<p>I have also a CNN with the following results:\nValidation:  Top1: 0.80 ; Top3: 0.92</p>\n\n<p>Combining both gives:\nValidation:  Top1: 0.84 ; Top3: 0.945</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 417548,
          "author_name": "Nuno Ferreira",
          "author_url": "",
          "post_date": "2018-11-08T12:43:50.057000",
          "content": "<p>Well done! I'm curious to see if your LSTM will score similar to mine on the public LB, cause looking at my local score, I actually expected my LSTM to score higher on the public LB</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 417570,
          "author_name": "HuyenNguyen",
          "author_url": "",
          "post_date": "2018-11-08T13:21:37.750000",
          "content": "<p>very impressive. do you use attention?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 417590,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2018-11-08T13:50:56.567000",
          "content": "<p><a href=\"/jeandebleau\">@jeandebleau</a></p>\n\n<p>is your LSTM using stroke features (e.g. x,y) or CNN extracted features?\nthanks!</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 417600,
          "author_name": "Gary",
          "author_url": "",
          "post_date": "2018-11-08T13:56:08.030000",
          "content": "<p>Hi, Heng, How did you use TTA this competiton?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 417623,
          "author_name": "jeandebleau",
          "author_url": "",
          "post_date": "2018-11-08T14:34:53.477000",
          "content": "<p>The LSTM is only using stroke features.</p>\n\n<p>The combination of both takes the last hidden layer of the two networks and learns a combination. More or less what was described in a paper that you also mentionned.  </p>\n\n<p>I tried a Convolutional LSTM without much success, also it is extremely slow. I also tried PointNet and PointGrid. These approaches may work but I could not get very good results. </p>\n\n<p>Right now I am trying alternative ideas. </p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 417665,
          "author_name": "Nuno Ferreira",
          "author_url": "",
          "post_date": "2018-11-08T15:51:56.503000",
          "content": "<p><a href=\"/huyenvyvy\">@huyenvyvy</a> Yes, I'm using attention. It gave a big boost to my initial LSTM score.\n<a href=\"/hengck23\">@hengck23</a> I'm also only using stroke features and country</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 417923,
          "author_name": "HuyenNguyen",
          "author_url": "",
          "post_date": "2018-11-09T01:46:52.037000",
          "content": "<p>Did using country info help you compared to only stroke? </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 418004,
          "author_name": "Nuno Ferreira",
          "author_url": "",
          "post_date": "2018-11-09T05:16:57.673000",
          "content": "<p><a href=\"/huyenvyvy\">@huyenvyvy</a> Not really sure, cause I started using country info since the beginning. I might be wrong, but in my opinion the way you draw stuff is strongly related to where you were raised.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 418013,
          "author_name": "HuyenNguyen",
          "author_url": "",
          "post_date": "2018-11-09T05:24:31.567000",
          "content": "<p>I agree, the intuition seems sound. except it wasn't borne out by the data in my experience :D adding country didn't help at all. </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 418076,
          "author_name": "YaGana Sheriff-Hussaini",
          "author_url": "",
          "post_date": "2018-11-09T08:55:59.110000",
          "content": "<p>@Nuno Ferreira, how are you adding the attention layer? when I use it as follows, I get AssertionError.</p>\n\n<blockquote>\n  <p>model.add(Bidirectional(LSTM(128, return_sequences = True)))\n  model.add(AttentionWithContext())</p>\n</blockquote>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 418222,
          "author_name": "Nuno Ferreira",
          "author_url": "",
          "post_date": "2018-11-09T13:54:26.560000",
          "content": "<p><a href=\"/sheriytm\">@sheriytm</a> I can't see anything wrong in the way you add the attention layer . Do you have any extra logging with the assertion error?</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 416959,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2018-11-07T14:20:01.303000",
      "content": "<p>as a reference, i was about to get lb 0.899 in my early experiments</p>",
      "votes": 3,
      "replies": [
        {
          "id": 417069,
          "author_name": "s.s.o",
          "author_url": "",
          "post_date": "2018-11-07T17:31:43.727000",
          "content": "<p>@henkck, Hi Henk,</p>\n\n<p>What do you think, if you train couple of cnn networks and feed the outputs to make new predictions using RNN for the final submission. Do you think it may work? I'm far beginner for this type of completion.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 419728,
      "author_name": "jeandebleau",
      "author_url": "",
      "post_date": "2018-11-12T13:31:06.057000",
      "content": "<p>I submitted some results on a lstm network trained on 50k samples per class.</p>\n\n<p>It scores Public LB 0.93\nLocal Top1: 84.7% ; Top3: 94.8%</p>",
      "votes": 4,
      "replies": [
        {
          "id": 419733,
          "author_name": "Ankit Sati",
          "author_url": "",
          "post_date": "2018-11-12T13:40:21.023000",
          "content": "<p>That's a great score with 50k samples. Are you using just LSTM network or CNN+LSTM?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 419741,
          "author_name": "jeandebleau",
          "author_url": "",
          "post_date": "2018-11-12T13:52:27.147000",
          "content": "<p>This one is only LSTM. </p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 419745,
          "author_name": "Ankit Sati",
          "author_url": "",
          "post_date": "2018-11-12T13:56:53.490000",
          "content": "<p>How much did you train your model? I trained mine for 10000 steps 20 epochs with all files from train simplified but it didn't perform that well.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 419747,
          "author_name": "jeandebleau",
          "author_url": "",
          "post_date": "2018-11-12T13:59:47.100000",
          "content": "<p>I use 50k samples per images. One epoch uses 1000 random images per class, and it is trained for 400 epochs. </p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 419749,
          "author_name": "Ankit Sati",
          "author_url": "",
          "post_date": "2018-11-12T14:01:34.393000",
          "content": "<p>Okay, I think I need to train my model more.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 419750,
          "author_name": "Ankit Sati",
          "author_url": "",
          "post_date": "2018-11-12T14:05:28.093000",
          "content": "<p>By 1000 random images per class you mean you are using batch size of 1000, right? How many steps you are going per epoch?</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 420168,
          "author_name": "Nuno Ferreira",
          "author_url": "",
          "post_date": "2018-11-13T07:28:56.503000",
          "content": "<p>Great score!</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 420199,
          "author_name": "jeandebleau",
          "author_url": "",
          "post_date": "2018-11-13T08:52:20.977000",
          "content": "<p>I define one epoch when 1000 random images per class has been given as input. I use a batch size of 1000 as well. So one epoch is finished when 340 * 1000 images have been analyzed. I do that 400 times. </p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 417215,
      "author_name": "HuyenNguyen",
      "author_url": "",
      "post_date": "2018-11-08T00:24:44.350000",
      "content": "<p>I achieved 0.829 <a href=\"https://www.kaggle.com/huyenvyvy/bidirectional-lstm-using-data-generator-lb-0-825\">https://www.kaggle.com/huyenvyvy/bidirectional-lstm-using-data-generator-lb-0-825</a></p>",
      "votes": 4,
      "replies": []
    },
    {
      "id": 417941,
      "author_name": "luyaxin",
      "author_url": "",
      "post_date": "2018-11-09T02:45:07.753000",
      "content": "<p>LB 0.869 on 5k images per class, LSTM</p>",
      "votes": 1,
      "replies": [
        {
          "id": 417958,
          "author_name": "HuyenNguyen",
          "author_url": "",
          "post_date": "2018-11-09T03:53:04.123000",
          "content": "<p>Did you use any data augmentation? That's a very good result, if you use more data, maybe it'll improve further. </p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 418002,
          "author_name": "luyaxin",
          "author_url": "",
          "post_date": "2018-11-09T05:09:40.913000",
          "content": "<p>Hi @HuyenNguyen,  I am using stroke augmentations like randomly scaling strokes and randomly removing strokes. I think it may be overfitting, because the validation data size is pretty small. trained on small data size to save GCP credits for later use...</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 418015,
          "author_name": "Ankit Sati",
          "author_url": "",
          "post_date": "2018-11-09T05:30:58.810000",
          "content": "<p>Hi @luyaxin is there any resource i can look for the augmentations? Thanks.</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 418021,
          "author_name": "luyaxin",
          "author_url": "",
          "post_date": "2018-11-09T05:46:39.223000",
          "content": "<p>You can find the stroke augmentations here:\n<a href=\"https://github.com/tensorflow/magenta/tree/master/magenta/models/sketch_rnn\">https://github.com/tensorflow/magenta/tree/master/magenta/models/sketch_rnn</a></p>",
          "votes": 2,
          "replies": []
        },
        {
          "id": 418022,
          "author_name": "Ankit Sati",
          "author_url": "",
          "post_date": "2018-11-09T05:49:08.540000",
          "content": "<p>Awesome, Thanks @luyaxin.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 417342,
      "author_name": "Ankit Sati",
      "author_url": "",
      "post_date": "2018-11-08T06:00:27.367000",
      "content": "<p>I achieved 0.853 using CNN+LSTM using shuffle_csvs.</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 416958,
      "author_name": "beluga",
      "author_url": "",
      "post_date": "2018-11-07T14:15:49.090000",
      "content": "<p>From the public kernels:</p>\n\n<p><a href=\"/kmader\">@kmader</a> 0.55 <a href=\"https://www.kaggle.com/kmader/quickdraw-baseline-lstm-reading-and-submission\">https://www.kaggle.com/kmader/quickdraw-baseline-lstm-reading-and-submission</a></p>\n\n<p><a href=\"/kmader\">@kmader</a> 0.63 <a href=\"https://www.kaggle.com/kmader/quickdraw-with-wavenet-classifier\">https://www.kaggle.com/kmader/quickdraw-with-wavenet-classifier</a></p>\n\n<p><a href=\"/avinashrai\">@avinashrai</a> 0.68 <a href=\"https://www.kaggle.com/avinashrai/quickdraw-with-wavenet-classifier-tunning\">https://www.kaggle.com/avinashrai/quickdraw-with-wavenet-classifier-tunning</a></p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 516369,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-04-14T04:08:57.510000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "416949": "anyone can share the best score they can get?",
    "417335": "[@hengck23][1]\n\nSo far I manage to achieve 0.923 in the public LB, with a custom LSTM network, using stroke augmentation during training, and simple TTA.\n\nMy training scores are:\n\n     - loss:        0.6739   categorical_accuracy:        0.8227   top_3_accuracy:        0.9372 \n     - val_loss:  0.6479  val_categorical_accuracy:  0.8299   val_top_3_accuracy: 0.9405\n\nI'm having only one fold with 30M training samples and 4M validation. Probably I can still improve my LB score up to 0.93. After that it's time to start training a CNN and a second stage model to combine both. Most likely I will no longer manage to finish that cause I have limited hardware available.\n\n\n  [1]: https://www.kaggle.com/hengck23",
    "416959": "as a reference, i was about to get lb 0.899 in my early experiments\n\n",
    "419728": "I submitted some results on a lstm network trained on 50k samples per class.\n\nIt scores Public LB 0.93\nLocal Top1: 84.7% ; Top3: 94.8%\n",
    "417215": "I achieved 0.829 https://www.kaggle.com/huyenvyvy/bidirectional-lstm-using-data-generator-lb-0-825",
    "417941": "LB 0.869 on 5k images per class, LSTM",
    "417342": "I achieved 0.853 using CNN+LSTM using shuffle_csvs.",
    "416958": "From the public kernels:\n\n@kmader 0.55 https://www.kaggle.com/kmader/quickdraw-baseline-lstm-reading-and-submission\n\n@kmader 0.63 https://www.kaggle.com/kmader/quickdraw-with-wavenet-classifier\n\n@avinashrai 0.68 https://www.kaggle.com/avinashrai/quickdraw-with-wavenet-classifier-tunning",
    "516369": ""
  }
}