Hey guys,
\nI have trained a model and would like to submit it. But since this is my very first Kaggle competition, I am not sure what my notebook can/should have in it?
\nI would really appreciate your help!!! :)
", "messages": [ { "id": 2400545, "postDate": "2023-08-21T06:00:20.267Z", "content": "Big question - take a look at the shared notebook that includes answers to your basic questions.
\n\nYou can load all sorts of things from a dataset. For sure your trained model, but also libraries that would need to be downloaded from the internet, other data sets, etc.
\nNot sure why you would need the down-sampled version of the dataset - but that could be added. The 'add data' button in the top right is a powerful tool.
", "rawMarkdown": "Big question - take a look at the shared notebook that includes answers to your basic questions.\n\n[RSNA-ATD: CNN [TPU][Infer]](https://www.kaggle.com/code/awsaf49/rsna-atd-cnn-tpu-infer/input)\n\nYou can load all sorts of things from a dataset. For sure your trained model, but also libraries that would need to be downloaded from the internet, other data sets, etc.\n\nNot sure why you would need the down-sampled version of the dataset - but that could be added. The 'add data' button in the top right is a powerful tool. ", "votes": 1, "replies": [ { "id": 2400620, "postDate": "2023-08-21T06:55:33.630Z", "content": "Thank you for the helpful reply!
\nI have created a down-sampled version of the data for my notebook as a dataset since loading dcm images every time I started the kernel would take a lot of time.
\nI just checked the notebook you shared. I assume the answer to my first question is that we read whatever image is in meta file and test images folder, pass through our model and save output as submission.csv, then upon submission the host will put the rest of the images in the meta data and test images folder. Please correct me if I'm wrong though.
\nThanks for the kind help!
", "rawMarkdown": "Thank you for the helpful reply!\n\nI have created a down-sampled version of the data for my notebook as a dataset since loading dcm images every time I started the kernel would take a lot of time.\n\nI just checked the notebook you shared. I assume the answer to my first question is that we read whatever image is in meta file and test images folder, pass through our model and save output as submission.csv, then upon submission the host will put the rest of the images in the meta data and test images folder. Please correct me if I'm wrong though.\n\nThanks for the kind help!", "replies": [ { "id": 2401986, "postDate": "2023-08-22T00:18:49.563Z", "content": "I understand. It's normal on this type of competition to do the training of the model in a notebook and the prediction in separate notebook. For the training you could use a dataset of the down sampled images.
\nFor a separate prediction notebook those training images would not be needed.
\nYour correct - when you submit for a score kaggle replaces what we can see with a set of similar files that contain the full images, submission file, etc.
", "rawMarkdown": "I understand. It's normal on this type of competition to do the training of the model in a notebook and the prediction in separate notebook. For the training you could use a dataset of the down sampled images.\n\nFor a separate prediction notebook those training images would not be needed.\n\nYour correct - when you submit for a score kaggle replaces what we can see with a set of similar files that contain the full images, submission file, etc. \n", "votes": 1 }, { "id": 2402140, "postDate": "2023-08-22T03:01:27.867Z", "content": "Thank you Jimmy.. I submitted my notebook and is still running after several hours!!
", "rawMarkdown": "Thank you Jimmy.. I submitted my notebook and is still running after several hours!!" }, { "id": 2402155, "postDate": "2023-08-22T03:14:54.630Z", "content": "Looking forward to seeing you on the leader board. I am working local and trying to get my prediction time down - I have been starting the prediction before bed time and it's often still running when I get up, so several hours no surprise.
", "rawMarkdown": "Looking forward to seeing you on the leader board. I am working local and trying to get my prediction time down - I have been starting the prediction before bed time and it's often still running when I get up, so several hours no surprise.", "votes": 1 }, { "id": 2402172, "postDate": "2023-08-22T03:25:53.217Z", "content": "Thank you!! Yeah my first submisison passed 9 hours.. hopefully this one iwll finish in time!
", "rawMarkdown": "Thank you!! Yeah my first submisison passed 9 hours.. hopefully this one iwll finish in time!" } ] } ] }, { "id": 2400518, "postDate": "2023-08-21T05:31:47.843Z", "content": "Hey guys,
\nI have trained a model and would like to submit it. But since this is my very first Kaggle competition, I am not sure what my notebook can/should have in it?
\nI would really appreciate your help!!! :)
", "rawMarkdown": "Hey guys,\n \nI have trained a model and would like to submit it. But since this is my very first Kaggle competition, I am not sure what my notebook can/should have in it? \n\n- How would the system recognize my model and feed the test images to it in my submission file?\n- Can my submission have the trained model loaded as dataset?\n- Can my submission have the down-sampled version of the dataset loaded as well?\n\nI would really appreciate your help!!! :)", "votes": 1 }, { "id": 2401004, "postDate": "2023-08-21T11:30:33.813Z", "content": "How would the system recognize my model and feed the test images to it in my submission file?
\nYou have to create an inference notebook that takes in input from the test_images folder in the dataset. Obviously they only contain 3 images now but they would be replaced with actual test dataset during the time of evaluation. Make sure your notebook saves a submission.csv file in the appropriate format (index, column names etc). This will be used for scoring.
Can my submission have the trained model loaded as dataset?
\nYes, internet has to be disabled for this competition. Which means you have to upload your trained model as a dataset and then use it. Training the model and then using it for inference would create a long run time and is not a feasible option anyway.
\nCan my submission have the down-sampled version of the dataset loaded as well?
\nWhy do you need the dataset at all anyway ? Aren't you just making an inference ? Ideally your submission will have nothing to do with anything except for the test_images folder which is taken care by kaggle.
Loved the complete response.. thank you very much! And true, I won't need the data just for inferring!
", "rawMarkdown": "Loved the complete response.. thank you very much! And true, I won't need the data just for inferring!" } ] } ], "comments": [ { "id": 2400545, "author_name": "PC Jimmmy", "author_url": "", "post_date": "2023-08-21T06:00:20.267000", "content": "Big question - take a look at the shared notebook that includes answers to your basic questions.
\n\nYou can load all sorts of things from a dataset. For sure your trained model, but also libraries that would need to be downloaded from the internet, other data sets, etc.
\nNot sure why you would need the down-sampled version of the dataset - but that could be added. The 'add data' button in the top right is a powerful tool.
", "votes": 1, "replies": [ { "id": 2400620, "author_name": "Parham Mostame", "author_url": "", "post_date": "2023-08-21T06:55:33.630000", "content": "Thank you for the helpful reply!
\nI have created a down-sampled version of the data for my notebook as a dataset since loading dcm images every time I started the kernel would take a lot of time.
\nI just checked the notebook you shared. I assume the answer to my first question is that we read whatever image is in meta file and test images folder, pass through our model and save output as submission.csv, then upon submission the host will put the rest of the images in the meta data and test images folder. Please correct me if I'm wrong though.
\nThanks for the kind help!
", "votes": 0, "replies": [ { "id": 2401986, "author_name": "PC Jimmmy", "author_url": "", "post_date": "2023-08-22T00:18:49.563000", "content": "I understand. It's normal on this type of competition to do the training of the model in a notebook and the prediction in separate notebook. For the training you could use a dataset of the down sampled images.
\nFor a separate prediction notebook those training images would not be needed.
\nYour correct - when you submit for a score kaggle replaces what we can see with a set of similar files that contain the full images, submission file, etc.
", "votes": 1, "replies": [] }, { "id": 2402140, "author_name": "Parham Mostame", "author_url": "", "post_date": "2023-08-22T03:01:27.867000", "content": "Thank you Jimmy.. I submitted my notebook and is still running after several hours!!
", "votes": 0, "replies": [] }, { "id": 2402155, "author_name": "PC Jimmmy", "author_url": "", "post_date": "2023-08-22T03:14:54.630000", "content": "Looking forward to seeing you on the leader board. I am working local and trying to get my prediction time down - I have been starting the prediction before bed time and it's often still running when I get up, so several hours no surprise.
", "votes": 1, "replies": [] }, { "id": 2402172, "author_name": "Parham Mostame", "author_url": "", "post_date": "2023-08-22T03:25:53.217000", "content": "Thank you!! Yeah my first submisison passed 9 hours.. hopefully this one iwll finish in time!
", "votes": 0, "replies": [] } ] } ] }, { "id": 2401004, "author_name": "Harshanand", "author_url": "", "post_date": "2023-08-21T11:30:33.813000", "content": "How would the system recognize my model and feed the test images to it in my submission file?
\nYou have to create an inference notebook that takes in input from the test_images folder in the dataset. Obviously they only contain 3 images now but they would be replaced with actual test dataset during the time of evaluation. Make sure your notebook saves a submission.csv file in the appropriate format (index, column names etc). This will be used for scoring.
Can my submission have the trained model loaded as dataset?
\nYes, internet has to be disabled for this competition. Which means you have to upload your trained model as a dataset and then use it. Training the model and then using it for inference would create a long run time and is not a feasible option anyway.
\nCan my submission have the down-sampled version of the dataset loaded as well?
\nWhy do you need the dataset at all anyway ? Aren't you just making an inference ? Ideally your submission will have nothing to do with anything except for the test_images folder which is taken care by kaggle.
Loved the complete response.. thank you very much! And true, I won't need the data just for inferring!
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "2400545": "Big question - take a look at the shared notebook that includes answers to your basic questions.\n\n[RSNA-ATD: CNN [TPU][Infer]](https://www.kaggle.com/code/awsaf49/rsna-atd-cnn-tpu-infer/input)\n\nYou can load all sorts of things from a dataset. For sure your trained model, but also libraries that would need to be downloaded from the internet, other data sets, etc.\n\nNot sure why you would need the down-sampled version of the dataset - but that could be added. The 'add data' button in the top right is a powerful tool. ", "2400518": "Hey guys,\n \nI have trained a model and would like to submit it. But since this is my very first Kaggle competition, I am not sure what my notebook can/should have in it? \n\n- How would the system recognize my model and feed the test images to it in my submission file?\n- Can my submission have the trained model loaded as dataset?\n- Can my submission have the down-sampled version of the dataset loaded as well?\n\nI would really appreciate your help!!! :)", "2401004": "How would the system recognize my model and feed the test images to it in my submission file?\n\nYou have to create an inference notebook that takes in input from the ```test_images``` folder in the dataset. Obviously they only contain 3 images now but they would be replaced with actual test dataset during the time of evaluation. Make sure your notebook saves a ```submission.csv``` file in the appropriate format (index, column names etc). This will be used for scoring. \n\nCan my submission have the trained model loaded as dataset?\n\nYes, internet has to be disabled for this competition. Which means you have to upload your trained model as a dataset and then use it. Training the model and then using it for inference would create a long run time and is not a feasible option anyway. \n\nCan my submission have the down-sampled version of the dataset loaded as well?\n\nWhy do you need the dataset at all anyway ? Aren't you just making an inference ? Ideally your submission will have nothing to do with anything except for the ```test_images``` folder which is taken care by kaggle. \n " } }