In an effort to save people the trouble to download the 400 Gb of data in order to get started, I've converted the dicom files to png.
\nCode : https://www.kaggle.com/code/theoviel/get-started-quicker-dicom-png-conversion
\nDatasets, 8 parts of ~15 Gb each.
\nCreation is quite painful because of the Kaggle output size limit. Let me know if there are issues, I'll fix them asap.
\nNaming is patient_study_frame.png where frame is retrieved using the z coordinate in the dicom metadata.
Yes, this is a shameless push to try to get to 4x GM =)
\nUpdate 13/08 : I've added rescaling to the preprocessing function, and removed voi_luit. This fixes some contrast issues and should help performances. You might want to re-download the data.
", "messages": [ { "id": 2362121, "postDate": "2023-07-27T20:06:51.817Z", "content": "In an effort to save people the trouble to download the 400 Gb of data in order to get started, I've converted the dicom files to png.
\nCode : https://www.kaggle.com/code/theoviel/get-started-quicker-dicom-png-conversion
\nDatasets, 8 parts of ~15 Gb each.
\nCreation is quite painful because of the Kaggle output size limit. Let me know if there are issues, I'll fix them asap.
\nNaming is patient_study_frame.png where frame is retrieved using the z coordinate in the dicom metadata.
Yes, this is a shameless push to try to get to 4x GM =)
\nUpdate 13/08 : I've added rescaling to the preprocessing function, and removed voi_luit. This fixes some contrast issues and should help performances. You might want to re-download the data.
", "rawMarkdown": "In an effort to save people the trouble to download the 400 Gb of data in order to get started, I've converted the dicom files to png.\n\nCode : https://www.kaggle.com/code/theoviel/get-started-quicker-dicom-png-conversion\n\nDatasets, 8 parts of ~15 Gb each.\n- Part 1 : https://www.kaggle.com/datasets/theoviel/rsna-abdominal-trauma-detection-png-pt1\n- Part 2 : https://www.kaggle.com/datasets/theoviel/rsna-abdominal-trauma-detection-png-pt2\n- Part 3 : https://www.kaggle.com/datasets/theoviel/rsna-2023-abdominal-trauma-detection-pngs-3-8\n- Part 4 : https://www.kaggle.com/datasets/theoviel/rsna-abdominal-trauma-detection-png-pt4\n- Part 5 : https://www.kaggle.com/datasets/theoviel/rsna-abdominal-trauma-detection-png-pt5\n- Part 6 : https://www.kaggle.com/datasets/theoviel/rsna-abdominal-trauma-detection-png-pt6\n- Part 7 : https://www.kaggle.com/datasets/theoviel/rsna-abdominal-trauma-detection-pngs-pt7\n- Part 8 : https://www.kaggle.com/datasets/theoviel/rsna-2023-abdominal-trauma-detection-pngs-18\n\nCreation is quite painful because of the Kaggle output size limit. Let me know if there are issues, I'll fix them asap.\n\nNaming is `patient_study_frame.png` where frame is retrieved using the `z` coordinate in the dicom metadata.\n\nYes, this is a shameless push to try to get to 4x GM =)\n\n\n**Update 13/08 :** I've added rescaling to the preprocessing function, and removed voi_luit. This fixes some contrast issues and should help performances. You might want to re-download the data.", "votes": 174 }, { "id": 2362967, "postDate": "2023-07-28T12:18:04.637Z", "content": "I think there is nothing wrong with this. You are still solving problem for lots of folks.
", "rawMarkdown": "I think there is nothing wrong with this. You are still solving problem for lots of folks.", "votes": 9 }, { "id": 2436244, "postDate": "2023-09-13T12:51:50.697Z", "content": "Hi, I noticed that your data only had windowing, which has some \"Black image\". So, I have created datasets that include basic PNG conversion, windowing, and a new high-contrast scaling. Please upvote it if you find it useful! Thank you!
\nData DICOM to 3 different types of Images (Grayscale)
Very useful, Theo. Upvoted and looking forward to your 4x achievement!
", "rawMarkdown": "Very useful, Theo. Upvoted and looking forward to your 4x achievement!", "votes": 4 }, { "id": 2398338, "postDate": "2023-08-19T15:50:03.683Z", "content": "Very-very useful! I love how Kaggle encourages us helping each-other! Good luck for your 4x GM!
", "rawMarkdown": "Very-very useful! I love how Kaggle encourages us helping each-other! Good luck for your 4x GM!", "votes": 1 }, { "id": 2362219, "postDate": "2023-07-27T22:43:35.780Z", "content": "@theoviel thanks for sharing this information
", "rawMarkdown": "@theoviel thanks for sharing this information", "votes": 1 }, { "id": 2362347, "postDate": "2023-07-28T03:24:25.747Z", "content": "That's a really great way to represent dataset. As the data expands to 460+GB/1.5M Files, it becomes quite difficult to load large chunks of data and preprocess them. As kaggle only provides 73GB of storage, it becomes difficult to store/experiment the data for the people who have less physical resources (me).
This dataset being small (around 10 GB for each part), opens a lot of possible experiments on the data. We can experiment on the small chunks and when we are sure with the pipelines, we can load the larger chunks.
I will be using this datasets as the base of all my learning/experiments/notebooks in this competition
Thanks @theoviel
\nYou will be 4x Grandmaster soon mate :)
", "rawMarkdown": "That's a really `great way` to `represent dataset`. As the data expands to `460+GB`/`1.5M Files`, it becomes `quite difficult` to `load large chunks` of data and `preprocess` them. As kaggle only provides `73GB` of storage, it becomes `difficult` to `store`/`experiment` the data for the people who have `less physical resources` `(me)`.\n\nThis dataset `being small` `(around 10 GB for each part)`, opens a lot of `possible experiments` on the data. We can `experiment` on the `small chunks` and when we are `sure` with the `pipelines`, we can `load` the `larger chunks`. \n\nI will be using this `datasets` as the base of all my `learning`/`experiments`/`notebooks` in this `competition`\n\nThanks @theoviel \n\n**You will be 4x Grandmaster soon mate :)**", "votes": 1 }, { "id": 2363352, "postDate": "2023-07-28T16:06:17.827Z", "content": "Salut Théo,
\nla phrase la plus inspirante de ce précieux Discussion Topic est la dernière. Très peu ont réussi à atteindre votre niveau.
Thanks for the links. I've already been there (in all of them).
\nEt, j'espère que vous aurez bientôt le quatrième X. 🤞(It's a tiny emoji fingers crossed)
\nÀ tout à l'heure,
\nMarília.
thanks for providing data for new commers on this platform, while its very difficult and scary with this large size data
", "rawMarkdown": "thanks for providing data for new commers on this platform, while its very difficult and scary with this large size data" }, { "id": 2447059, "postDate": "2023-09-19T19:43:41.743Z", "content": "Theo, you've done an exceptional job! Keep up the excellent work and continue to excel!
", "rawMarkdown": "Theo, you've done an exceptional job! Keep up the excellent work and continue to excel!" }, { "id": 2444147, "postDate": "2023-09-18T07:32:49.597Z", "content": "Hello, Thank you! I want to clarify one thing: are images from test_images here too?
", "rawMarkdown": "Hello, Thank you! I want to clarify one thing: are images from test_images here too?" }, { "id": 2402714, "postDate": "2023-08-22T09:48:29.473Z", "content": "Thanks @theoviel - you are always helpful and very polite! This makes the competition a bit more accessible (will see anyways).
\nThink people often use datasets but forget to upvote them, have done so and will look for more in future.
\nBest of luck for the GM soon.
Messias!!!!
", "rawMarkdown": "Messias!!!!" }, { "id": 2392770, "postDate": "2023-08-15T21:17:55.280Z", "content": "Thank u for sharing this. This is very useful for me to start this competition. I am very much afraid of looking at the size of the data. It give me confidence to initiate the competition
", "rawMarkdown": "Thank u for sharing this. This is very useful for me to start this competition. I am very much afraid of looking at the size of the data. It give me confidence to initiate the competition" }, { "id": 2371094, "postDate": "2023-08-02T21:29:32.483Z", "content": "Newbie here. Just to be clear, the advantage of the smaller size is primarily to download the data onto your local drive, so that you can use your own hardware/GPU, rather then be limited to Kaggle's hardware. Correct? If so, out of curiosity, what are some IDE's/workflows you all like to use to submit to Kaggle?
", "rawMarkdown": "Newbie here. Just to be clear, the advantage of the smaller size is primarily to download the data onto your local drive, so that you can use your own hardware/GPU, rather then be limited to Kaggle's hardware. Correct? If so, out of curiosity, what are some IDE's/workflows you all like to use to submit to Kaggle? " }, { "id": 2364676, "postDate": "2023-07-29T14:11:48.450Z", "content": "Thanks a lot! It's really useful for me to take the first step!
", "rawMarkdown": "Thanks a lot! It's really useful for me to take the first step!" }, { "id": 2364298, "postDate": "2023-07-29T09:13:20.550Z", "content": "No shame in trying to be a 4x GM and thanks for sharing these files!
", "rawMarkdown": "No shame in trying to be a 4x GM and thanks for sharing these files!" }, { "id": 2363782, "postDate": "2023-07-28T23:36:53.687Z", "content": "Really awesome for the community, Your work is helpful for lot of kagglers. Thanks for sharing @theoviel 👍.
", "rawMarkdown": "Really awesome for the community, Your work is helpful for lot of kagglers. Thanks for sharing @theoviel 👍." }, { "id": 2399222, "postDate": "2023-08-20T08:06:24.100Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2369194, "postDate": "2023-08-01T15:21:34.137Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2480265, "postDate": "2023-10-13T07:41:26.420Z", "content": "Thank you! It is helpful.
", "rawMarkdown": "Thank you! It is helpful." }, { "id": 2443795, "postDate": "2023-09-18T02:06:14.470Z", "content": "Thanks for sharing!
", "rawMarkdown": "Thanks for sharing!" }, { "id": 2426064, "postDate": "2023-09-06T11:29:26.087Z", "content": "Thanks a lot!!
", "rawMarkdown": "Thanks a lot!! " }, { "id": 2424931, "postDate": "2023-09-05T14:51:17.083Z", "content": "thank you so much!!
", "rawMarkdown": "thank you so much!!" }, { "id": 2403911, "postDate": "2023-08-23T00:00:38.537Z", "content": "Thanks Theo!
", "rawMarkdown": "Thanks Theo!" }, { "id": 2401227, "postDate": "2023-08-21T13:56:05.957Z", "content": "thank you my hero
", "rawMarkdown": "thank you my hero" }, { "id": 2397372, "postDate": "2023-08-19T01:43:52.530Z", "content": "Thanks for your help!
", "rawMarkdown": "Thanks for your help!" }, { "id": 2389834, "postDate": "2023-08-14T08:58:41.573Z", "content": "Thanks a lot for your help!
", "rawMarkdown": "Thanks a lot for your help!" }, { "id": 2372260, "postDate": "2023-08-03T15:27:04.417Z", "content": "thank you so much for sharing!
", "rawMarkdown": "thank you so much for sharing!" }, { "id": 2364649, "postDate": "2023-07-29T13:46:23.963Z", "content": "thanks for sharing !
", "rawMarkdown": "thanks for sharing !" } ], "comments": [ { "id": 2362967, "author_name": "Gunes Evitan", "author_url": "", "post_date": "2023-07-28T12:18:04.637000", "content": "I think there is nothing wrong with this. You are still solving problem for lots of folks.
", "votes": 9, "replies": [] }, { "id": 2436244, "author_name": "Minh-Long Pham", "author_url": "", "post_date": "2023-09-13T12:51:50.697000", "content": "Hi, I noticed that your data only had windowing, which has some \"Black image\". So, I have created datasets that include basic PNG conversion, windowing, and a new high-contrast scaling. Please upvote it if you find it useful! Thank you!
\nData DICOM to 3 different types of Images (Grayscale)
Very useful, Theo. Upvoted and looking forward to your 4x achievement!
", "votes": 4, "replies": [] }, { "id": 2398338, "author_name": "Gyula Maloveczky4", "author_url": "", "post_date": "2023-08-19T15:50:03.683000", "content": "Very-very useful! I love how Kaggle encourages us helping each-other! Good luck for your 4x GM!
", "votes": 1, "replies": [] }, { "id": 2362219, "author_name": "DHIRAJ WAGH", "author_url": "", "post_date": "2023-07-27T22:43:35.780000", "content": "@theoviel thanks for sharing this information
", "votes": 1, "replies": [] }, { "id": 2362347, "author_name": "AyushS9020", "author_url": "", "post_date": "2023-07-28T03:24:25.747000", "content": "That's a really great way to represent dataset. As the data expands to 460+GB/1.5M Files, it becomes quite difficult to load large chunks of data and preprocess them. As kaggle only provides 73GB of storage, it becomes difficult to store/experiment the data for the people who have less physical resources (me).
This dataset being small (around 10 GB for each part), opens a lot of possible experiments on the data. We can experiment on the small chunks and when we are sure with the pipelines, we can load the larger chunks.
I will be using this datasets as the base of all my learning/experiments/notebooks in this competition
Thanks @theoviel
\nYou will be 4x Grandmaster soon mate :)
", "votes": 1, "replies": [] }, { "id": 2363352, "author_name": "Marília Prata", "author_url": "", "post_date": "2023-07-28T16:06:17.827000", "content": "Salut Théo,
\nla phrase la plus inspirante de ce précieux Discussion Topic est la dernière. Très peu ont réussi à atteindre votre niveau.
Thanks for the links. I've already been there (in all of them).
\nEt, j'espère que vous aurez bientôt le quatrième X. 🤞(It's a tiny emoji fingers crossed)
\nÀ tout à l'heure,
\nMarília.
thanks for providing data for new commers on this platform, while its very difficult and scary with this large size data
", "votes": 0, "replies": [] }, { "id": 2447059, "author_name": "USMAN SAFDAR", "author_url": "", "post_date": "2023-09-19T19:43:41.743000", "content": "Theo, you've done an exceptional job! Keep up the excellent work and continue to excel!
", "votes": 0, "replies": [] }, { "id": 2444147, "author_name": "Gulnaz Zhambulova", "author_url": "", "post_date": "2023-09-18T07:32:49.597000", "content": "Hello, Thank you! I want to clarify one thing: are images from test_images here too?
", "votes": 0, "replies": [] }, { "id": 2402714, "author_name": "something4kag", "author_url": "", "post_date": "2023-08-22T09:48:29.473000", "content": "Thanks @theoviel - you are always helpful and very polite! This makes the competition a bit more accessible (will see anyways).
\nThink people often use datasets but forget to upvote them, have done so and will look for more in future.
\nBest of luck for the GM soon.
Messias!!!!
", "votes": 0, "replies": [] }, { "id": 2392770, "author_name": "Srikant Nayak", "author_url": "", "post_date": "2023-08-15T21:17:55.280000", "content": "Thank u for sharing this. This is very useful for me to start this competition. I am very much afraid of looking at the size of the data. It give me confidence to initiate the competition
", "votes": 0, "replies": [] }, { "id": 2371094, "author_name": "Ed Izaguirre", "author_url": "", "post_date": "2023-08-02T21:29:32.483000", "content": "Newbie here. Just to be clear, the advantage of the smaller size is primarily to download the data onto your local drive, so that you can use your own hardware/GPU, rather then be limited to Kaggle's hardware. Correct? If so, out of curiosity, what are some IDE's/workflows you all like to use to submit to Kaggle?
", "votes": 0, "replies": [] }, { "id": 2364676, "author_name": "DaydaycodingDaydayup", "author_url": "", "post_date": "2023-07-29T14:11:48.450000", "content": "Thanks a lot! It's really useful for me to take the first step!
", "votes": 0, "replies": [] }, { "id": 2364298, "author_name": "Yassine Alouini", "author_url": "", "post_date": "2023-07-29T09:13:20.550000", "content": "No shame in trying to be a 4x GM and thanks for sharing these files!
", "votes": 0, "replies": [] }, { "id": 2363782, "author_name": "Tariq Mahmood", "author_url": "", "post_date": "2023-07-28T23:36:53.687000", "content": "Really awesome for the community, Your work is helpful for lot of kagglers. Thanks for sharing @theoviel 👍.
", "votes": 0, "replies": [] }, { "id": 2399222, "author_name": "", "author_url": "", "post_date": "2023-08-20T08:06:24.100000", "content": "", "votes": 0, "replies": [] }, { "id": 2369194, "author_name": "", "author_url": "", "post_date": "2023-08-01T15:21:34.137000", "content": "", "votes": 0, "replies": [] }, { "id": 2480265, "author_name": "Naoya", "author_url": "", "post_date": "2023-10-13T07:41:26.420000", "content": "Thank you! It is helpful.
", "votes": 0, "replies": [] }, { "id": 2443795, "author_name": "gunja kumari", "author_url": "", "post_date": "2023-09-18T02:06:14.470000", "content": "Thanks for sharing!
", "votes": 0, "replies": [] }, { "id": 2426064, "author_name": "MinKi Jung", "author_url": "", "post_date": "2023-09-06T11:29:26.087000", "content": "Thanks a lot!!
", "votes": 0, "replies": [] }, { "id": 2424931, "author_name": "huan chen", "author_url": "", "post_date": "2023-09-05T14:51:17.083000", "content": "thank you so much!!
", "votes": 0, "replies": [] }, { "id": 2403911, "author_name": "Chris Deotte", "author_url": "", "post_date": "2023-08-23T00:00:38.537000", "content": "Thanks Theo!
", "votes": 0, "replies": [] }, { "id": 2401227, "author_name": "charles chuang", "author_url": "", "post_date": "2023-08-21T13:56:05.957000", "content": "thank you my hero
", "votes": 0, "replies": [] }, { "id": 2397372, "author_name": "junn", "author_url": "", "post_date": "2023-08-19T01:43:52.530000", "content": "Thanks for your help!
", "votes": 0, "replies": [] }, { "id": 2389834, "author_name": "Yuki_ARL", "author_url": "", "post_date": "2023-08-14T08:58:41.573000", "content": "Thanks a lot for your help!
", "votes": 0, "replies": [] }, { "id": 2372260, "author_name": "golnaz ahmadvand", "author_url": "", "post_date": "2023-08-03T15:27:04.417000", "content": "thank you so much for sharing!
", "votes": 0, "replies": [] }, { "id": 2364649, "author_name": "Alaa Chahboune", "author_url": "", "post_date": "2023-07-29T13:46:23.963000", "content": "thanks for sharing !
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2362121": "In an effort to save people the trouble to download the 400 Gb of data in order to get started, I've converted the dicom files to png.\n\nCode : https://www.kaggle.com/code/theoviel/get-started-quicker-dicom-png-conversion\n\nDatasets, 8 parts of ~15 Gb each.\n- Part 1 : https://www.kaggle.com/datasets/theoviel/rsna-abdominal-trauma-detection-png-pt1\n- Part 2 : https://www.kaggle.com/datasets/theoviel/rsna-abdominal-trauma-detection-png-pt2\n- Part 3 : https://www.kaggle.com/datasets/theoviel/rsna-2023-abdominal-trauma-detection-pngs-3-8\n- Part 4 : https://www.kaggle.com/datasets/theoviel/rsna-abdominal-trauma-detection-png-pt4\n- Part 5 : https://www.kaggle.com/datasets/theoviel/rsna-abdominal-trauma-detection-png-pt5\n- Part 6 : https://www.kaggle.com/datasets/theoviel/rsna-abdominal-trauma-detection-png-pt6\n- Part 7 : https://www.kaggle.com/datasets/theoviel/rsna-abdominal-trauma-detection-pngs-pt7\n- Part 8 : https://www.kaggle.com/datasets/theoviel/rsna-2023-abdominal-trauma-detection-pngs-18\n\nCreation is quite painful because of the Kaggle output size limit. Let me know if there are issues, I'll fix them asap.\n\nNaming is `patient_study_frame.png` where frame is retrieved using the `z` coordinate in the dicom metadata.\n\nYes, this is a shameless push to try to get to 4x GM =)\n\n\n**Update 13/08 :** I've added rescaling to the preprocessing function, and removed voi_luit. This fixes some contrast issues and should help performances. You might want to re-download the data.", "2362967": "I think there is nothing wrong with this. You are still solving problem for lots of folks.", "2436244": "Hi, I noticed that your data only had windowing, which has some \"Black image\". So, I have created datasets that include basic PNG conversion, windowing, and a new high-contrast scaling. Please upvote it if you find it useful! Thank you!\n[Data DICOM to 3 different types of Images (Grayscale)](https://www.kaggle.com/competitions/rsna-2023-abdominal-trauma-detection/discussion/438690)", "2364003": "Very useful, Theo. Upvoted and looking forward to your 4x achievement!", "2398338": "Very-very useful! I love how Kaggle encourages us helping each-other! Good luck for your 4x GM!", "2362219": "@theoviel thanks for sharing this information", "2362347": "That's a really `great way` to `represent dataset`. As the data expands to `460+GB`/`1.5M Files`, it becomes `quite difficult` to `load large chunks` of data and `preprocess` them. As kaggle only provides `73GB` of storage, it becomes `difficult` to `store`/`experiment` the data for the people who have `less physical resources` `(me)`.\n\nThis dataset `being small` `(around 10 GB for each part)`, opens a lot of `possible experiments` on the data. We can `experiment` on the `small chunks` and when we are `sure` with the `pipelines`, we can `load` the `larger chunks`. \n\nI will be using this `datasets` as the base of all my `learning`/`experiments`/`notebooks` in this `competition`\n\nThanks @theoviel \n\n**You will be 4x Grandmaster soon mate :)**", "2363352": "Salut Théo,\nla phrase la plus inspirante de ce précieux Discussion Topic est la dernière. Très peu ont réussi à atteindre votre niveau.\n\nThanks for the links. I've already been there (in all of them). \nEt, j'espère que vous aurez bientôt le quatrième X. 🤞(It's a tiny emoji fingers crossed)\nÀ tout à l'heure,\nMarília.", "2460754": "thanks for providing data for new commers on this platform, while its very difficult and scary with this large size data", "2447059": "Theo, you've done an exceptional job! Keep up the excellent work and continue to excel!", "2444147": "Hello, Thank you! I want to clarify one thing: are images from test_images here too?", "2402714": "Thanks @theoviel - you are always helpful and very polite! This makes the competition a bit more accessible (will see anyways).\nThink people often use datasets but forget to upvote them, have done so and will look for more in future.\nBest of luck for the GM soon. ", "2402040": "Messias!!!!", "2392770": "Thank u for sharing this. This is very useful for me to start this competition. I am very much afraid of looking at the size of the data. It give me confidence to initiate the competition", "2371094": "Newbie here. Just to be clear, the advantage of the smaller size is primarily to download the data onto your local drive, so that you can use your own hardware/GPU, rather then be limited to Kaggle's hardware. Correct? If so, out of curiosity, what are some IDE's/workflows you all like to use to submit to Kaggle? ", "2364676": "Thanks a lot! It's really useful for me to take the first step!", "2364298": "No shame in trying to be a 4x GM and thanks for sharing these files!", "2363782": "Really awesome for the community, Your work is helpful for lot of kagglers. Thanks for sharing @theoviel 👍.", "2399222": "", "2369194": "", "2480265": "Thank you! It is helpful.", "2443795": "Thanks for sharing!", "2426064": "Thanks a lot!! ", "2424931": "thank you so much!!", "2403911": "Thanks Theo!", "2401227": "thank you my hero", "2397372": "Thanks for your help!", "2389834": "Thanks a lot for your help!", "2372260": "thank you so much for sharing!", "2364649": "thanks for sharing !" } }