There are few images in the private test set which the pydicom library is unable to read using dcmread. I have not faced this issue in the training dataset and public test dataset but faced it while submitting the prediction result on the whole test dataset.
Due to this error, my submission notebook loop was breaking in the middle and was unable to create a submission file. Thanks to @yuval6967 for helping me in sorting out this problem with the try and except statement but I don't know exactly for how many images I am doing wrong prediction due to this error.
If you have faced this issue and have a solution to resolve this error it would be great if you can help?
", "messages": [ { "id": 1032878, "postDate": "2020-09-30T13:55:05.553Z", "content": "I followed this guide and installing gdcm, pydicom is able to read all .dcm in the private test set
\nhttps://www.kaggle.com/richardepstein/load-gdcm-in-notebook-without-internet
Thanks!, I am not getting error after installing gdcm.
", "rawMarkdown": "Thanks!, I am not getting error after installing gdcm." } ] }, { "id": 1032781, "postDate": "2020-09-30T12:50:37.273Z", "content": "I'm using pydicom.read_file and I didn't have this issue
", "rawMarkdown": "I'm using pydicom.read_file and I didn't have this issue", "votes": 4, "replies": [ { "id": 1044404, "postDate": "2020-10-09T18:17:06.967Z", "content": "Given below is my inference code that works while submitting but is skipping some images in the private dataset.
\ntest_image_paths = [x for x in pathlib.Path(TEST_IMAGE_DIR).rglob('*.dcm')]\nfor path in test_image_paths:\n image_id = path.name.split('.')[0]\n path = str(path)\n try:\n dicom = pydicom.read_file(path)\n M = float(dicom.RescaleSlope)\n B = float(dicom.RescaleIntercept)\n image = np.asarray(dicom.pixel_array)\n image = image * M\n image = image + B\n image_lung = np.expand_dims(window(image, WL=-600, WW=1500), axis=2)\n image_mediastinal = np.expand_dims(window(image, WL=40, WW=400), axis=2)\n image_pe_specific = np.expand_dims(window(image, WL=100, WW=700), axis=2)\n image = np.concatenate([image_mediastinal, image_pe_specific, image_lung], axis=2)\n rat = MAX_LENGTH / np.max(image.shape)\n image = zoom(image, [rat,rat,1.], prefilter=False, order=1)\n except:\n print('Exception')\n image = np.ones(shape=(256,256,3), dtype=np.float32)\nI am getting a submission error if I don't keep a try and except. So any idea what is the error in try block? I have installed gdcm.
\nThanks
The DICOM images are typically 512x512. Are you scaling your DICOM images to 256x256? Your except: statement creates a 256x256x3 image. Does this match your dicom images?
", "rawMarkdown": "The DICOM images are typically 512x512. Are you scaling your DICOM images to 256x256? Your except: statement creates a 256x256x3 image. Does this match your dicom images?" }, { "id": 1046463, "postDate": "2020-10-11T17:16:27.073Z", "content": "Yes, the zoom function just before except statement makes it to 256x256x3
", "rawMarkdown": "Yes, the zoom function just before except statement makes it to 256x256x3" } ] } ] }, { "id": 1032742, "postDate": "2020-09-30T12:25:28.527Z", "content": "There are few images in the private test set which the pydicom library is unable to read using dcmread. I have not faced this issue in the training dataset and public test dataset but faced it while submitting the prediction result on the whole test dataset.
Due to this error, my submission notebook loop was breaking in the middle and was unable to create a submission file. Thanks to @yuval6967 for helping me in sorting out this problem with the try and except statement but I don't know exactly for how many images I am doing wrong prediction due to this error.
If you have faced this issue and have a solution to resolve this error it would be great if you can help?
", "rawMarkdown": "There are few images in the private test set which the `pydicom` library is unable to read using `dcmread`. I have not faced this issue in the training dataset and public test dataset but faced it while submitting the prediction result on the whole test dataset. \n\nDue to this error, my submission notebook loop was breaking in the middle and was unable to create a submission file. Thanks to @yuval6967 for helping me in sorting out this problem with the `try and except` statement but I don't know exactly for how many images I am doing wrong prediction due to this error.\n\nIf you have faced this issue and have a solution to resolve this error it would be great if you can help?", "votes": 3 }, { "id": 1046387, "postDate": "2020-10-11T15:48:21.170Z", "content": "it seems like i also keep getting this error too when submission, may i ask do you also have \"Submission CSV Not Found\" error when submitting the private test set ?
", "rawMarkdown": "it seems like i also keep getting this error too when submission, may i ask do you also have \"Submission CSV Not Found\" error when submitting the private test set ?", "replies": [ { "id": 1046403, "postDate": "2020-10-11T16:12:14.587Z", "content": "@steaphan install gdcm, there are some files in the private test set which need gdcm to read it using pydicom. Installing gdcm resolve this issue for me [link]
", "rawMarkdown": "@steaphan install gdcm, there are some files in the private test set which need gdcm to read it using pydicom. Installing gdcm resolve this issue for me [[link](https://www.kaggle.com/richardepstein/load-gdcm-in-notebook-without-internet)]", "votes": 1 } ] } ], "comments": [ { "id": 1032878, "author_name": "Marco Stefani", "author_url": "", "post_date": "2020-09-30T13:55:05.553000", "content": "I followed this guide and installing gdcm, pydicom is able to read all .dcm in the private test set
\nhttps://www.kaggle.com/richardepstein/load-gdcm-in-notebook-without-internet
Thanks!, I am not getting error after installing gdcm.
", "votes": 0, "replies": [] } ] }, { "id": 1032781, "author_name": "yuval reina", "author_url": "", "post_date": "2020-09-30T12:50:37.273000", "content": "I'm using pydicom.read_file and I didn't have this issue
", "votes": 4, "replies": [ { "id": 1044404, "author_name": "Jose", "author_url": "", "post_date": "2020-10-09T18:17:06.967000", "content": "Given below is my inference code that works while submitting but is skipping some images in the private dataset.
\ntest_image_paths = [x for x in pathlib.Path(TEST_IMAGE_DIR).rglob('*.dcm')]\nfor path in test_image_paths:\n image_id = path.name.split('.')[0]\n path = str(path)\n try:\n dicom = pydicom.read_file(path)\n M = float(dicom.RescaleSlope)\n B = float(dicom.RescaleIntercept)\n image = np.asarray(dicom.pixel_array)\n image = image * M\n image = image + B\n image_lung = np.expand_dims(window(image, WL=-600, WW=1500), axis=2)\n image_mediastinal = np.expand_dims(window(image, WL=40, WW=400), axis=2)\n image_pe_specific = np.expand_dims(window(image, WL=100, WW=700), axis=2)\n image = np.concatenate([image_mediastinal, image_pe_specific, image_lung], axis=2)\n rat = MAX_LENGTH / np.max(image.shape)\n image = zoom(image, [rat,rat,1.], prefilter=False, order=1)\n except:\n print('Exception')\n image = np.ones(shape=(256,256,3), dtype=np.float32)\nI am getting a submission error if I don't keep a try and except. So any idea what is the error in try block? I have installed gdcm.
\nThanks
The DICOM images are typically 512x512. Are you scaling your DICOM images to 256x256? Your except: statement creates a 256x256x3 image. Does this match your dicom images?
", "votes": 0, "replies": [] }, { "id": 1046463, "author_name": "Jose", "author_url": "", "post_date": "2020-10-11T17:16:27.073000", "content": "Yes, the zoom function just before except statement makes it to 256x256x3
", "votes": 0, "replies": [] } ] } ] }, { "id": 1046387, "author_name": "steaphan", "author_url": "", "post_date": "2020-10-11T15:48:21.170000", "content": "it seems like i also keep getting this error too when submission, may i ask do you also have \"Submission CSV Not Found\" error when submitting the private test set ?
", "votes": 0, "replies": [ { "id": 1046403, "author_name": "SumanSudhir", "author_url": "", "post_date": "2020-10-11T16:12:14.587000", "content": "@steaphan install gdcm, there are some files in the private test set which need gdcm to read it using pydicom. Installing gdcm resolve this issue for me [link]
", "votes": 1, "replies": [] } ] } ], "raw_markdown_by_id": { "1032878": "I followed this guide and installing gdcm, pydicom is able to read all .dcm in the private test set\nhttps://www.kaggle.com/richardepstein/load-gdcm-in-notebook-without-internet", "1032781": "I'm using pydicom.read_file and I didn't have this issue", "1032742": "There are few images in the private test set which the `pydicom` library is unable to read using `dcmread`. I have not faced this issue in the training dataset and public test dataset but faced it while submitting the prediction result on the whole test dataset. \n\nDue to this error, my submission notebook loop was breaking in the middle and was unable to create a submission file. Thanks to @yuval6967 for helping me in sorting out this problem with the `try and except` statement but I don't know exactly for how many images I am doing wrong prediction due to this error.\n\nIf you have faced this issue and have a solution to resolve this error it would be great if you can help?", "1046387": "it seems like i also keep getting this error too when submission, may i ask do you also have \"Submission CSV Not Found\" error when submitting the private test set ?" } }