IndRnn\nhttps://github.com/batzner/indrnn
\n\nIt converges faster than lstm\nVery deep sequence could be built that could still converge fast
\n\nGithub:\nhttps://github.com/batzner/indrnn
", "messages": [ { "id": 422317, "postDate": "2018-11-16T04:04:20.157Z", "content": "IndRnn\nhttps://github.com/batzner/indrnn
\n\nIt converges faster than lstm\nVery deep sequence could be built that could still converge fast
\n\nGithub:\nhttps://github.com/batzner/indrnn
", "rawMarkdown": "IndRnn\nhttps://github.com/batzner/indrnn\n\nIt converges faster than lstm\nVery deep sequence could be built that could still converge fast\n\nGithub:\nhttps://github.com/batzner/indrnn\n", "votes": 5 }, { "id": 423532, "postDate": "2018-11-18T13:53:25.850Z", "content": "Can someone explain why RNN makes sense for this competition? I would think that since we are looking at images, convnets would be the best approach. Is it because we have the timing/order for the strokes used to draw the images?
", "rawMarkdown": "Can someone explain why RNN makes sense for this competition? I would think that since we are looking at images, convnets would be the best approach. Is it because we have the timing/order for the strokes used to draw the images?", "votes": 1, "replies": [ { "id": 423538, "postDate": "2018-11-18T14:00:34.010Z", "content": "Yes, integrating timing/order might be helpful and RNNs can be useful for that.
", "rawMarkdown": "Yes, integrating timing/order might be helpful and RNNs can be useful for that.", "votes": 1 }, { "id": 423679, "postDate": "2018-11-18T21:33:03.113Z", "content": "There are at least two way to look at this problem:\n1 - Image classification using images, where CNN is the best choice\n2- sequences of strokes, which consist of sequences of points (x, y) coordinates, where a RNN makes sense
", "rawMarkdown": "There are at least two way to look at this problem:\n1 - Image classification using images, where CNN is the best choice\n2- sequences of strokes, which consist of sequences of points (x, y) coordinates, where a RNN makes sense", "votes": 1 } ] }, { "id": 423511, "postDate": "2018-11-18T12:50:03.670Z", "content": "In my case IndyRNNs were superior to other methods. Problem is that they take forever to train. :/ \nwish there was a CuDNN implementation that worked with ind on dynamic sequences.
", "rawMarkdown": "In my case IndyRNNs were superior to other methods. Problem is that they take forever to train. :/ \nwish there was a CuDNN implementation that worked with ind on dynamic sequences.", "replies": [ { "id": 423524, "postDate": "2018-11-18T13:44:10.210Z", "content": "there is a cuda version
\n\nhttps://github.com/Sunnydreamrain/IndRNN_pytorch
\n\ncuda_IndRNN_onlyrecurrent is the CUDA version. It is much faster than the simple pytorch implementation. For the sequential MNIST example (length 784), it runs over 31 times faster.
", "rawMarkdown": "there is a cuda version\n\nhttps://github.com/Sunnydreamrain/IndRNN_pytorch\n\ncuda_IndRNN_onlyrecurrent is the CUDA version. It is much faster than the simple pytorch implementation. For the sequential MNIST example (length 784), it runs over 31 times faster." }, { "id": 423536, "postDate": "2018-11-18T13:56:26.987Z", "content": "I do my networks in Tensorflow, so I don't know how things work in Pytorch, but for tensorflow you have CUDA-LSTM and CUDNN-LSTM and the latter is much faster. Only the former can be used in dynamic sequences :/
\n\nIndyLSTM has only CUDA version and not CuDNN optimized version
", "rawMarkdown": "I do my networks in Tensorflow, so I don't know how things work in Pytorch, but for tensorflow you have CUDA-LSTM and CUDNN-LSTM and the latter is much faster. Only the former can be used in dynamic sequences :/\n\nIndyLSTM has only CUDA version and not CuDNN optimized version" }, { "id": 423550, "postDate": "2018-11-18T14:16:47.843Z", "content": "one suggestion is to divide the strokes into time slot: e.g. first 0.2 sec, next 0.2 sec ...
\n\nthis will keep your seq fixed length (some padding may be required)
\n\nanother possibility is to divide by first 20%, next 20% of total points/strokes drawn , ... etc
", "rawMarkdown": "one suggestion is to divide the strokes into time slot: e.g. first 0.2 sec, next 0.2 sec ...\n\nthis will keep your seq fixed length (some padding may be required)\n\n\nanother possibility is to divide by first 20%, next 20% of total points/strokes drawn , ... etc", "votes": 2 } ] }, { "id": 423220, "postDate": "2018-11-17T18:11:53.390Z", "rawMarkdown": "", "votes": 2, "isDeleted": true } ], "comments": [ { "id": 423532, "author_name": "Allen", "author_url": "", "post_date": "2018-11-18T13:53:25.850000", "content": "Can someone explain why RNN makes sense for this competition? I would think that since we are looking at images, convnets would be the best approach. Is it because we have the timing/order for the strokes used to draw the images?
", "votes": 1, "replies": [ { "id": 423538, "author_name": "Miha Skalic", "author_url": "", "post_date": "2018-11-18T14:00:34.010000", "content": "Yes, integrating timing/order might be helpful and RNNs can be useful for that.
", "votes": 1, "replies": [] }, { "id": 423679, "author_name": "Nuno Ferreira", "author_url": "", "post_date": "2018-11-18T21:33:03.113000", "content": "There are at least two way to look at this problem:\n1 - Image classification using images, where CNN is the best choice\n2- sequences of strokes, which consist of sequences of points (x, y) coordinates, where a RNN makes sense
", "votes": 1, "replies": [] } ] }, { "id": 423511, "author_name": "Miha Skalic", "author_url": "", "post_date": "2018-11-18T12:50:03.670000", "content": "In my case IndyRNNs were superior to other methods. Problem is that they take forever to train. :/ \nwish there was a CuDNN implementation that worked with ind on dynamic sequences.
", "votes": 0, "replies": [ { "id": 423524, "author_name": "hengck23", "author_url": "", "post_date": "2018-11-18T13:44:10.210000", "content": "there is a cuda version
\n\nhttps://github.com/Sunnydreamrain/IndRNN_pytorch
\n\ncuda_IndRNN_onlyrecurrent is the CUDA version. It is much faster than the simple pytorch implementation. For the sequential MNIST example (length 784), it runs over 31 times faster.
", "votes": 0, "replies": [] }, { "id": 423536, "author_name": "Miha Skalic", "author_url": "", "post_date": "2018-11-18T13:56:26.987000", "content": "I do my networks in Tensorflow, so I don't know how things work in Pytorch, but for tensorflow you have CUDA-LSTM and CUDNN-LSTM and the latter is much faster. Only the former can be used in dynamic sequences :/
\n\nIndyLSTM has only CUDA version and not CuDNN optimized version
", "votes": 0, "replies": [] }, { "id": 423550, "author_name": "hengck23", "author_url": "", "post_date": "2018-11-18T14:16:47.843000", "content": "one suggestion is to divide the strokes into time slot: e.g. first 0.2 sec, next 0.2 sec ...
\n\nthis will keep your seq fixed length (some padding may be required)
\n\nanother possibility is to divide by first 20%, next 20% of total points/strokes drawn , ... etc
", "votes": 2, "replies": [] } ] }, { "id": 423220, "author_name": "", "author_url": "", "post_date": "2018-11-17T18:11:53.390000", "content": "", "votes": 2, "replies": [] } ], "raw_markdown_by_id": { "422317": "IndRnn\nhttps://github.com/batzner/indrnn\n\nIt converges faster than lstm\nVery deep sequence could be built that could still converge fast\n\nGithub:\nhttps://github.com/batzner/indrnn\n", "423532": "Can someone explain why RNN makes sense for this competition? I would think that since we are looking at images, convnets would be the best approach. Is it because we have the timing/order for the strokes used to draw the images?", "423511": "In my case IndyRNNs were superior to other methods. Problem is that they take forever to train. :/ \nwish there was a CuDNN implementation that worked with ind on dynamic sequences.", "423220": "" } }