{"cells":[{"metadata":{},"cell_type":"markdown","source":"# DICOM 2 PNG Converter\n\nThe DICOM images in this competition prove very difficult to use as they are. This notebook walks through the code, process, and output for converting these DICOM images into a usable form that won't completely saturate your hard drive or cloud drive space. "},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nfrom PIL import Image\nimport pydicom as dicom\nimport matplotlib.pylab as plt\nimport cv2\nimport itertools\nimport time\nfrom skimage.transform import resize\n\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\n#for dirname, _, filenames in os.walk('/kaggle/input'):\n#    for filename in filenames:\n#        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_image_file_list = os.listdir('/kaggle/input/vinbigdata-chest-xray-abnormalities-detection/train/')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Each of these files are lung scans formatted in the DICOM image format, an industry standard in healthcare. "},{"metadata":{"trusted":true},"cell_type":"code","source":"# Show original dicom image and dicom pixel array\nimage_path = '/kaggle/input/vinbigdata-chest-xray-abnormalities-detection/train/'+ train_image_file_list[0]\nds = dicom.dcmread(image_path)\n\n_= plt.imshow(ds.pixel_array)    \nprint(ds.pixel_array)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"DICOM files have a lot of associated metadata with them. While it's very useful for healthcare practitioners, none of this metadata is going to be super useful for this classification problem. Converting to the .png format is more ideal for this particular use case. "},{"metadata":{"trusted":true},"cell_type":"code","source":"# Viewing dicom metadata to better understand what dicom is\n\nimage_name = train_image_file_list[0]\n# specify your image path\ninput_path = '/kaggle/input/vinbigdata-chest-xray-abnormalities-detection/train/'\n\noutput_path_stem = '/kaggle/working/converted_train_images/' \n\nds = dicom.dcmread(input_path + image_name)\n\nds","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Creating diretories for the final dump outside of the kaggle/working\nin_dirs = ['/kaggle/input/vinbigdata-chest-xray-abnormalities-detection/train/', '/kaggle/input/vinbigdata-chest-xray-abnormalities-detection/test/']\n\nout_dirs = ['/kaggle/final_output/converted_train_images/', '/kaggle/final_output/converted_test_images/']\nfor thing in out_dirs: os.makedirs(thing)\n    \n# Remove directory\n#import shutil\n#shutil.rmtree('/kaggle/final_output/converted_train_images/')\n#shutil.rmtree('/kaggle/final_output/converted_test_images/')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The following function converts the images into the PNG format. The DICOM images presented two challenges: brightness/contrast, and size. The DICOM images have very large pixel brightnesses which needed to be rescaled to the 0-255 scale. Additionally, the DICOM images are large and needed to be rescaled down to 512 * 512 pixels. These transformations will reduce the quality of the images going into the model, but it seems to be a solid tradeoff given how much more manageable all the data becomes. "},{"metadata":{"trusted":true},"cell_type":"code","source":"import pydicom\n\ndef image_processor(input_dir, output_dir, basewidth = 512, baseheight = 512): \n    \n    \n    file_list = [f for f in os.listdir(input_dir)]\n    \n    \n    for f in file_list:\n        \n        # Read the dicom file\n        ds = pydicom.read_file(input_dir + f, force = True)\n        \n        # Get image array\n        img = ds.pixel_array\n        \n        # Rescaling image array brightness\n        img_rescaled = (img / np.max(img) * 255)\n        \n        # Resizing image array dimensions\n        img_resized = resize(img_rescaled, (512,512), anti_aliasing= True)\n        \n        # Creating new png path\n        new_path = output_dir + f.replace('.dicom', '.png')\n        \n        # Writing image to path\n        cv2.imwrite(new_path ,img_resized) # write png image ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Doing it the least pythonic way possible since I want to eliminate any issues causing my output files to not be generated\nimage_processor(input_dir= in_dirs[0], output_dir= out_dirs[0])\n\nimage_processor(input_dir= in_dirs[1], output_dir= out_dirs[1])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Converted training image sample\npath = '/kaggle/final_output/converted_train_images/4d390e07733ba06e5ff07412f09c0a92.png'\n\nimg = Image.open(path)\nimg.size\n\nImage.open(path)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Converted test image sample\npath = '/kaggle/final_output/converted_test_images/83caa8a85e03606cf57e49147d7ac569.png'\n\nimg = Image.open(path)\nimg.size\n\nImage.open(path)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Checking to make sure all 15,000 training images and 3,000 testing images were converted properly into their respective\n# directories\nconverted_train_list = os.listdir('/kaggle/final_output/converted_train_images/')\nconverted_test_list = os.listdir('/kaggle/final_output/converted_test_images/')\nprint(\"train file number: \" + str(len(converted_train_list)) + \" test file number: \"+ str(len(converted_test_list)))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Hopefully this saves the output upon commit\n!tar -zcf train.tar.gz -C \"/kaggle/final_output/converted_train_images/\" .\n!tar -zcf test.tar.gz -C \"/kaggle/final_output/converted_test_images/\" .","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}