There are two different centers(data_provider) \"Radboud“ and \"Karolinska\".\nIf the data of the new center does not appear in the test set, can we train different models in different centers？\nI try to finetune my model (use both two centers' data) in these two centers separately，I got a gain of about 0.003 QWK in my local valset. (eg. 0.880->0.883, both \"Radboud“ and \"Karolinska\")
", "messages": [ { "id": 914964, "postDate": "2020-07-04T11:45:43.333Z", "content": "There are two different centers(data_provider) \"Radboud“ and \"Karolinska\".\nIf the data of the new center does not appear in the test set, can we train different models in different centers？\nI try to finetune my model (use both two centers' data) in these two centers separately，I got a gain of about 0.003 QWK in my local valset. (eg. 0.880->0.883, both \"Radboud“ and \"Karolinska\")
", "rawMarkdown": "There are two different centers(data_provider) \"Radboud“ and \"Karolinska\".\nIf the data of the new center does not appear in the test set, can we train different models in different centers？\nI try to finetune my model (use both two centers' data) in these two centers separately，I got a gain of about 0.003 QWK in my local valset. (eg. 0.880->0.883, both \"Radboud“ and \"Karolinska\")", "votes": 1 }, { "id": 915712, "postDate": "2020-07-05T03:26:12.007Z", "content": "I trained two models because the provider information is also available in the test set.\nAs a result, Local QWK improved (radboud: 0.8839, karolinska: 0.9112), but lb score worsened (LB: 0.86).
", "rawMarkdown": "I trained two models because the provider information is also available in the test set.\nAs a result, Local QWK improved (radboud: 0.8839, karolinska: 0.9112), but lb score worsened (LB: 0.86).", "replies": [ { "id": 916483, "postDate": "2020-07-05T17:39:04.070Z", "content": "Have you been careful with duplicates ? It's a big problem for Radboud.
", "rawMarkdown": "Have you been careful with duplicates ? It's a big problem for Radboud." }, { "id": 916592, "postDate": "2020-07-05T20:07:47.127Z", "content": "@arroqc did you remove duplicates or consider them as an augmentation ? Did it boost your score ?
", "rawMarkdown": "@arroqc did you remove duplicates or consider them as an augmentation ? Did it boost your score ? " }, { "id": 916640, "postDate": "2020-07-05T21:39:17.860Z", "content": "@alexj21 Even though I've created a split that takes this issue and other into account I haven't yet started to train with it.
", "rawMarkdown": "@alexj21 Even though I've created a split that takes this issue and other into account I haven't yet started to train with it." }, { "id": 916738, "postDate": "2020-07-06T01:27:05.280Z", "content": "@arroqc I have removed duplicates using threshold 0.96 based on kernel here.
\n\n@alexj21 I didn't train separately before removing the duplicates, but there was a jump of 0.86->0.88 LB by removing duplicates in one model training. (1 fold)\nHowever, I can't be sure if it works because LB is so blurry :(
", "rawMarkdown": "@arroqc I have removed duplicates using threshold 0.96 based on kernel [here](https://www.kaggle.com/appian/panda-imagehash-to-detect-duplicate-images).\n\n@alexj21 I didn't train separately before removing the duplicates, but there was a jump of 0.86->0.88 LB by removing duplicates in one model training. (1 fold)\nHowever, I can't be sure if it works because LB is so blurry :(" }, { "id": 917013, "postDate": "2020-07-06T07:11:55.077Z", "content": "Thanks for your answers, it's good to know. \n@tattaka did you observe a lower CV/loss ?
", "rawMarkdown": "Thanks for your answers, it's good to know. \n@tattaka did you observe a lower CV/loss ?" }, { "id": 917056, "postDate": "2020-07-06T07:54:22.757Z", "content": "@alexj21 I got the similar CV compared to before removing the duplicates.
", "rawMarkdown": "@alexj21 I got the similar CV compared to before removing the duplicates." }, { "id": 917061, "postDate": "2020-07-06T07:56:05.160Z", "content": "interesting ! thanks
", "rawMarkdown": "interesting ! thanks" } ] }, { "id": 915566, "postDate": "2020-07-04T20:48:03.013Z", "content": "Are you suggesting to train 2 different models, one for each institute?
", "rawMarkdown": "Are you suggesting to train 2 different models, one for each institute?", "replies": [ { "id": 915657, "postDate": "2020-07-05T01:55:03.307Z", "content": "I don’t know if this method should be chosen because it increases the risk of overfitting and the improvement is extremely limited
", "rawMarkdown": "I don’t know if this method should be chosen because it increases the risk of overfitting and the improvement is extremely limited" } ] } ], "comments": [ { "id": 915712, "author_name": "tattaka", "author_url": "", "post_date": "2020-07-05T03:26:12.007000", "content": "I trained two models because the provider information is also available in the test set.\nAs a result, Local QWK improved (radboud: 0.8839, karolinska: 0.9112), but lb score worsened (LB: 0.86).
", "votes": 0, "replies": [ { "id": 916483, "author_name": "Arnaud Roussel", "author_url": "", "post_date": "2020-07-05T17:39:04.070000", "content": "Have you been careful with duplicates ? It's a big problem for Radboud.
", "votes": 0, "replies": [] }, { "id": 916592, "author_name": "Alex", "author_url": "", "post_date": "2020-07-05T20:07:47.127000", "content": "@arroqc did you remove duplicates or consider them as an augmentation ? Did it boost your score ?
", "votes": 0, "replies": [] }, { "id": 916640, "author_name": "Arnaud Roussel", "author_url": "", "post_date": "2020-07-05T21:39:17.860000", "content": "@alexj21 Even though I've created a split that takes this issue and other into account I haven't yet started to train with it.
", "votes": 0, "replies": [] }, { "id": 916738, "author_name": "tattaka", "author_url": "", "post_date": "2020-07-06T01:27:05.280000", "content": "@arroqc I have removed duplicates using threshold 0.96 based on kernel here.
\n\n@alexj21 I didn't train separately before removing the duplicates, but there was a jump of 0.86->0.88 LB by removing duplicates in one model training. (1 fold)\nHowever, I can't be sure if it works because LB is so blurry :(
", "votes": 0, "replies": [] }, { "id": 917013, "author_name": "Alex", "author_url": "", "post_date": "2020-07-06T07:11:55.077000", "content": "Thanks for your answers, it's good to know. \n@tattaka did you observe a lower CV/loss ?
", "votes": 0, "replies": [] }, { "id": 917056, "author_name": "tattaka", "author_url": "", "post_date": "2020-07-06T07:54:22.757000", "content": "@alexj21 I got the similar CV compared to before removing the duplicates.
", "votes": 0, "replies": [] }, { "id": 917061, "author_name": "Alex", "author_url": "", "post_date": "2020-07-06T07:56:05.160000", "content": "interesting ! thanks
", "votes": 0, "replies": [] } ] }, { "id": 915566, "author_name": "Pasquale", "author_url": "", "post_date": "2020-07-04T20:48:03.013000", "content": "Are you suggesting to train 2 different models, one for each institute?
", "votes": 0, "replies": [ { "id": 915657, "author_name": "seefun", "author_url": "", "post_date": "2020-07-05T01:55:03.307000", "content": "I don’t know if this method should be chosen because it increases the risk of overfitting and the improvement is extremely limited
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "914964": "There are two different centers(data_provider) \"Radboud“ and \"Karolinska\".\nIf the data of the new center does not appear in the test set, can we train different models in different centers？\nI try to finetune my model (use both two centers' data) in these two centers separately，I got a gain of about 0.003 QWK in my local valset. (eg. 0.880->0.883, both \"Radboud“ and \"Karolinska\")", "915712": "I trained two models because the provider information is also available in the test set.\nAs a result, Local QWK improved (radboud: 0.8839, karolinska: 0.9112), but lb score worsened (LB: 0.86).", "915566": "Are you suggesting to train 2 different models, one for each institute?" } }