{ "id": 73710, "title": "Wrap up for useful tricks in Doodle", "url": "/competitions/quickdraw-doodle-recognition/discussion/73710", "author_name": "Yiheng Wang", "post_date": "2018-12-05T02:50:57.805000", "votes": 33, "comment_count": 18, "views": 0, "content": "", "messages": [ { "id": 433386, "postDate": "2018-12-05T02:50:57.807Z", "content": "", "rawMarkdown": " - First of all, you should build **a trustable local validation**, @Outrunner 's post let me know that local valid and public LB is consistent ( https://www.kaggle.com/c/quickdraw-doodle-recognition/discussion/67997#latest-423127 )\n\n - The dataset is too large, and @hengck23 's post let me know I should definitely do experiments with **smaller training set first** ( https://www.kaggle.com/c/quickdraw-doodle-recognition/discussion/69260 )\n\n - How to **draw the image** for training? @hengck23 's post gave me some insights and we can **encode by stroke, points, pause time** and so on ( https://www.kaggle.com/c/quickdraw-doodle-recognition/discussion/70680#latest-427051 ) What's more, let's see @aispiriants 's encoding for different channels, that's amazing! ( https://www.kaggle.com/c/quickdraw-doodle-recognition/discussion/73708 )\n\n - **Which batch size? Which image size?** @hengck23 brings us the answer, which also been proved by many kagglers ( https://www.kaggle.com/c/quickdraw-doodle-recognition/discussion/70558 )\n\n - Which structure to start? @Beluga 's mobilenet is a light, fast and powerful choice ( https://www.kaggle.com/gaborfodor/greyscale-mobilenet-lb-0-892 ), some of the participants get decent results with it.\n\n - **Post processing**? Unfortunately, our team doesn't have enough time to try this trick. But we should definitely use it in the future, according to my own experiences in TGS ( https://www.kaggle.com/c/tgs-salt-identification-challenge/discussion/69047 ), and @firenero 's awesome result here ( https://www.kaggle.com/c/quickdraw-doodle-recognition/discussion/73708 ), you may get **+0.002 in Doodle**. \n\n- **Public and Private Distributions**? Both @hengck23 and @ppleskov found this magic thing. ( https://www.kaggle.com/c/quickdraw-doodle-recognition/discussion/70540#416772 ) Therefore, you can try to post balance the prediction files ( https://www.kaggle.com/c/quickdraw-doodle-recognition/discussion/73738 ) ! \n\n- How to do this post balance? Let's see @pavelost 's code \n( https://github.com/PavelOstyakov/camera_identification )\n\n- **How to ensemble**? Also, @ppleskov tells us the truth. Produce a 1:10 (pos : neg) samples into LGBM! ( https://www.kaggle.com/c/quickdraw-doodle-recognition/discussion/73738 )\n\n - **LSTM+CNN**? Thanks to @rwightman who brings us a 0.941 pure LSTM model ( https://www.kaggle.com/c/quickdraw-doodle-recognition/discussion/73816#434040 )\n\n- Welcome to add any helpful tricks here, and many thanks to these selfless kagglers, and congrats to all of us for our progresses", "votes": 33 }, { "id": 433790, "postDate": "2018-12-05T13:36:22.570Z", "content": "

Really Cool compilation and congrats to you on your Gold win !!

", "rawMarkdown": "Really Cool compilation and congrats to you on your Gold win !!", "votes": 1 }, { "id": 433654, "postDate": "2018-12-05T09:51:11.900Z", "content": "

Brief wrap up of my wrap up:\n1. pseudo labelling -> +0.001 (for one time)\n2. post balance -> + 0.007\n3. ensemble by average/voting -> +0.002 (for our team)\n4. ensemble by a boosting model & blend by weighted average/voting -> +0.003 (?? not sure)

", "rawMarkdown": "Brief wrap up of my wrap up:\n1. pseudo labelling -> +0.001 (for one time)\n2. post balance -> + 0.007\n3. ensemble by average/voting -> +0.002 (for our team)\n4. ensemble by a boosting model & blend by weighted average/voting -> +0.003 (?? not sure)", "votes": 1, "replies": [ { "id": 433669, "postDate": "2018-12-05T10:16:57.980Z", "content": "

How to find the megic 330 for this competition?\nMethod 1: by @hengck23 analyze your prediction file, most confident classes have exact 330\nMethod 2: by @hengck23 submit a prediction, with one class only (like cup, cup, cup). use the result (0.002) * number of rows (112199), and get the lower bound. use the result + 0.999.. (0.0029999) * number of rows, and get the upper bound. 330 belongs to this interval.\nMethod 3: by @ppleskov number of rows +1 / number of classes (340) and you can get 330.\nWe may have to enhance our intuition of the data.

", "rawMarkdown": "How to find the megic 330 for this competition?\nMethod 1: by @hengck23 analyze your prediction file, most confident classes have exact 330\nMethod 2: by @hengck23 submit a prediction, with one class only (like cup, cup, cup). use the result (0.002) * number of rows (112199), and get the lower bound. use the result + 0.999.. (0.0029999) * number of rows, and get the upper bound. 330 belongs to this interval.\nMethod 3: by @ppleskov number of rows +1 / number of classes (340) and you can get 330.\nWe may have to enhance our intuition of the data." } ] }, { "id": 433435, "postDate": "2018-12-05T04:35:08.030Z", "content": "

Nice Job Guys！

", "rawMarkdown": "Nice Job Guys！", "votes": 1 }, { "id": 433426, "postDate": "2018-12-05T04:04:18.503Z", "content": "

Thank you and congrats. We have two post processing methods help us to archive better result (about +0.005). It is coming soon.

", "rawMarkdown": "Thank you and congrats. We have two post processing methods help us to archive better result (about +0.005). It is coming soon.", "votes": 1, "replies": [ { "id": 433434, "postDate": "2018-12-05T04:31:31.387Z", "content": "

Wow, wonderful! Very looking forward to see your solution : )

", "rawMarkdown": "Wow, wonderful! Very looking forward to see your solution : )" } ] }, { "id": 433396, "postDate": "2018-12-05T03:03:29.030Z", "content": "

nice!

", "rawMarkdown": "nice!", "votes": 1 }, { "id": 433393, "postDate": "2018-12-05T02:57:41.707Z", "content": "

Thanks for your sharing, but it seems that many urls are invalidated.

", "rawMarkdown": "Thanks for your sharing, but it seems that many urls are invalidated.\n", "votes": 1, "replies": [ { "id": 433397, "postDate": "2018-12-05T03:05:52.620Z", "content": "

Sorry Gary, I've fixed.

", "rawMarkdown": "Sorry Gary, I've fixed. " }, { "id": 433399, "postDate": "2018-12-05T03:06:55.683Z", "content": "

try removing last \")\" .

", "rawMarkdown": "try removing last \")\" .", "votes": 2 } ] }, { "id": 433475, "postDate": "2018-12-05T05:43:52.317Z", "content": "

Thanks for sharing and congratulation @yiheng! Love your last sentence : “congrat to all of us for our progress”

", "rawMarkdown": "Thanks for sharing and congratulation @yiheng! Love your last sentence : “congrat to all of us for our progress”", "votes": 2, "replies": [ { "id": 433507, "postDate": "2018-12-05T06:13:09.877Z", "content": "

Thanks! kaggle brings us the chance to learn knowledge & practice. The time we dedicated is worthy : )

", "rawMarkdown": "Thanks! kaggle brings us the chance to learn knowledge & practice. The time we dedicated is worthy : )", "votes": 2 } ] }, { "id": 433448, "postDate": "2018-12-05T04:59:32.177Z", "content": "

Dalao

", "rawMarkdown": "Dalao", "votes": 2, "replies": [ { "id": 433452, "postDate": "2018-12-05T05:07:18.223Z", "content": "

Congrats for getting your master tier!

", "rawMarkdown": "Congrats for getting your master tier!" }, { "id": 433454, "postDate": "2018-12-05T05:09:39.783Z", "content": "

Thanks and wish we have chance to team up in the future😀

", "rawMarkdown": "Thanks and wish we have chance to team up in the future😀" }, { "id": 433455, "postDate": "2018-12-05T05:11:21.893Z", "content": "

definitely : )

", "rawMarkdown": "definitely : )" } ] }, { "id": 433433, "postDate": "2018-12-05T04:27:38.083Z", "content": "

Great job！

", "rawMarkdown": "Great job！", "votes": 2 }, { "id": 433735, "postDate": "2018-12-05T12:32:13.177Z", "content": "

That's wonderful! Nice Tricks! Thanks for sharing!

", "rawMarkdown": "That's wonderful! Nice Tricks! Thanks for sharing! ", "votes": 1 } ], "comments": [ { "id": 433790, "author_name": "Vishy", "author_url": "", "post_date": "2018-12-05T13:36:22.570000", "content": "

Really Cool compilation and congrats to you on your Gold win !!

", "votes": 1, "replies": [] }, { "id": 433654, "author_name": "Yiheng Wang", "author_url": "", "post_date": "2018-12-05T09:51:11.900000", "content": "

Brief wrap up of my wrap up:\n1. pseudo labelling -> +0.001 (for one time)\n2. post balance -> + 0.007\n3. ensemble by average/voting -> +0.002 (for our team)\n4. ensemble by a boosting model & blend by weighted average/voting -> +0.003 (?? not sure)

", "votes": 1, "replies": [ { "id": 433669, "author_name": "Yiheng Wang", "author_url": "", "post_date": "2018-12-05T10:16:57.980000", "content": "

How to find the megic 330 for this competition?\nMethod 1: by @hengck23 analyze your prediction file, most confident classes have exact 330\nMethod 2: by @hengck23 submit a prediction, with one class only (like cup, cup, cup). use the result (0.002) * number of rows (112199), and get the lower bound. use the result + 0.999.. (0.0029999) * number of rows, and get the upper bound. 330 belongs to this interval.\nMethod 3: by @ppleskov number of rows +1 / number of classes (340) and you can get 330.\nWe may have to enhance our intuition of the data.

", "votes": 0, "replies": [] } ] }, { "id": 433435, "author_name": "Zuping Wu", "author_url": "", "post_date": "2018-12-05T04:35:08.030000", "content": "

Nice Job Guys！

", "votes": 1, "replies": [] }, { "id": 433426, "author_name": "cab", "author_url": "", "post_date": "2018-12-05T04:04:18.503000", "content": "

Thank you and congrats. We have two post processing methods help us to archive better result (about +0.005). It is coming soon.

", "votes": 1, "replies": [ { "id": 433434, "author_name": "Yiheng Wang", "author_url": "", "post_date": "2018-12-05T04:31:31.387000", "content": "

Wow, wonderful! Very looking forward to see your solution : )

", "votes": 0, "replies": [] } ] }, { "id": 433396, "author_name": "earhian", "author_url": "", "post_date": "2018-12-05T03:03:29.030000", "content": "

nice!

", "votes": 1, "replies": [] }, { "id": 433393, "author_name": "Gary", "author_url": "", "post_date": "2018-12-05T02:57:41.707000", "content": "

Thanks for your sharing, but it seems that many urls are invalidated.

", "votes": 1, "replies": [ { "id": 433397, "author_name": "Yiheng Wang", "author_url": "", "post_date": "2018-12-05T03:05:52.620000", "content": "

Sorry Gary, I've fixed.

", "votes": 0, "replies": [] }, { "id": 433399, "author_name": "kotaro", "author_url": "", "post_date": "2018-12-05T03:06:55.683000", "content": "

try removing last \")\" .

", "votes": 2, "replies": [] } ] }, { "id": 433475, "author_name": "Neuron Engineer", "author_url": "", "post_date": "2018-12-05T05:43:52.317000", "content": "

Thanks for sharing and congratulation @yiheng! Love your last sentence : “congrat to all of us for our progress”

", "votes": 2, "replies": [ { "id": 433507, "author_name": "Yiheng Wang", "author_url": "", "post_date": "2018-12-05T06:13:09.877000", "content": "

Thanks! kaggle brings us the chance to learn knowledge & practice. The time we dedicated is worthy : )

", "votes": 2, "replies": [] } ] }, { "id": 433448, "author_name": "wh1te", "author_url": "", "post_date": "2018-12-05T04:59:32.177000", "content": "

Dalao

", "votes": 2, "replies": [ { "id": 433452, "author_name": "Yiheng Wang", "author_url": "", "post_date": "2018-12-05T05:07:18.223000", "content": "

Congrats for getting your master tier!

", "votes": 0, "replies": [] }, { "id": 433454, "author_name": "wh1te", "author_url": "", "post_date": "2018-12-05T05:09:39.783000", "content": "

Thanks and wish we have chance to team up in the future😀

", "votes": 0, "replies": [] }, { "id": 433455, "author_name": "Yiheng Wang", "author_url": "", "post_date": "2018-12-05T05:11:21.893000", "content": "

definitely : )

", "votes": 0, "replies": [] } ] }, { "id": 433433, "author_name": "SeuTao", "author_url": "", "post_date": "2018-12-05T04:27:38.083000", "content": "

Great job！

", "votes": 2, "replies": [] }, { "id": 433735, "author_name": "Arunkumar Venkataramanan", "author_url": "", "post_date": "2018-12-05T12:32:13.177000", "content": "

That's wonderful! Nice Tricks! Thanks for sharing!

", "votes": 1, "replies": [] } ], "raw_markdown_by_id": { "433386": " - First of all, you should build **a trustable local validation**, @Outrunner 's post let me know that local valid and public LB is consistent ( https://www.kaggle.com/c/quickdraw-doodle-recognition/discussion/67997#latest-423127 )\n\n - The dataset is too large, and @hengck23 's post let me know I should definitely do experiments with **smaller training set first** ( https://www.kaggle.com/c/quickdraw-doodle-recognition/discussion/69260 )\n\n - How to **draw the image** for training? @hengck23 's post gave me some insights and we can **encode by stroke, points, pause time** and so on ( https://www.kaggle.com/c/quickdraw-doodle-recognition/discussion/70680#latest-427051 ) What's more, let's see @aispiriants 's encoding for different channels, that's amazing! ( https://www.kaggle.com/c/quickdraw-doodle-recognition/discussion/73708 )\n\n - **Which batch size? Which image size?** @hengck23 brings us the answer, which also been proved by many kagglers ( https://www.kaggle.com/c/quickdraw-doodle-recognition/discussion/70558 )\n\n - Which structure to start? @Beluga 's mobilenet is a light, fast and powerful choice ( https://www.kaggle.com/gaborfodor/greyscale-mobilenet-lb-0-892 ), some of the participants get decent results with it.\n\n - **Post processing**? Unfortunately, our team doesn't have enough time to try this trick. But we should definitely use it in the future, according to my own experiences in TGS ( https://www.kaggle.com/c/tgs-salt-identification-challenge/discussion/69047 ), and @firenero 's awesome result here ( https://www.kaggle.com/c/quickdraw-doodle-recognition/discussion/73708 ), you may get **+0.002 in Doodle**. \n\n- **Public and Private Distributions**? Both @hengck23 and @ppleskov found this magic thing. ( https://www.kaggle.com/c/quickdraw-doodle-recognition/discussion/70540#416772 ) Therefore, you can try to post balance the prediction files ( https://www.kaggle.com/c/quickdraw-doodle-recognition/discussion/73738 ) ! \n\n- How to do this post balance? Let's see @pavelost 's code \n( https://github.com/PavelOstyakov/camera_identification )\n\n- **How to ensemble**? Also, @ppleskov tells us the truth. Produce a 1:10 (pos : neg) samples into LGBM! ( https://www.kaggle.com/c/quickdraw-doodle-recognition/discussion/73738 )\n\n - **LSTM+CNN**? Thanks to @rwightman who brings us a 0.941 pure LSTM model ( https://www.kaggle.com/c/quickdraw-doodle-recognition/discussion/73816#434040 )\n\n- Welcome to add any helpful tricks here, and many thanks to these selfless kagglers, and congrats to all of us for our progresses", "433790": "Really Cool compilation and congrats to you on your Gold win !!", "433654": "Brief wrap up of my wrap up:\n1. pseudo labelling -> +0.001 (for one time)\n2. post balance -> + 0.007\n3. ensemble by average/voting -> +0.002 (for our team)\n4. ensemble by a boosting model & blend by weighted average/voting -> +0.003 (?? not sure)", "433435": "Nice Job Guys！", "433426": "Thank you and congrats. We have two post processing methods help us to archive better result (about +0.005). It is coming soon.", "433396": "nice!", "433393": "Thanks for your sharing, but it seems that many urls are invalidated.\n", "433475": "Thanks for sharing and congratulation @yiheng! Love your last sentence : “congrat to all of us for our progress”", "433448": "Dalao", "433433": "Great job！", "433735": "That's wonderful! Nice Tricks! Thanks for sharing! " } }