{"cells":[{"metadata":{},"cell_type":"markdown","source":"**Getting Started with Dicom using fastai**\n\nI haven't spent a lot of time using fastai with dicom files and since I'm entering this competition I wanted to use my fastai knowledge to learn more about the dataset and to generally understand everything better with a starter notebook.\n\nThis will be one of a few notebooks. I'll link here to the bounding box version soon after, but first wanted to get a look at the data and follow the [fastai medical imaging tutorial](https://docs.fast.ai/tutorial.medical_imaging.html). "},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"#make sure we have latest build\n! [ -e /content ] && pip install -Uqq fastai ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Import what we will need. Take special note of the fastai.medical. That will give us some great tools to work with dicom files, etc. "},{"metadata":{"trusted":true},"cell_type":"code","source":"from fastai.basics import *\nfrom fastai.callback.all import *\nfrom fastai.vision.all import *\nfrom fastai.medical.imaging import *\n\nimport pydicom,kornia,skimage\nfrom pydicom.dataset import Dataset as DcmDataset\nfrom pydicom.tag import BaseTag as DcmTag\nfrom pydicom.multival import MultiValue as DcmMultiValue\nfrom PIL import Image\n\ntry:\n    import cv2\n    cv2.setNumThreads(0)\nexcept: pass","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Set our paths for files and for our training dicom set. "},{"metadata":{"trusted":true},"cell_type":"code","source":"path = Path('../input/vinbigdata-chest-xray-abnormalities-detection')\ntrain_imgs = path/'train'","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"A quick look to make sure we are on track before we get too far. "},{"metadata":{"trusted":true},"cell_type":"code","source":"fname = train_imgs/'000434271f63a053c4128a0ba6352c7f.dicom'\ndcm = fname.dcmread()\ndcm.show(scale=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Set up our images to get dicom files and read them."},{"metadata":{"trusted":true},"cell_type":"code","source":"items = get_dicom_files(train_imgs)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can now split the set for or training and validation sets"},{"metadata":{"trusted":true},"cell_type":"code","source":"train,val = RandomSplitter()(items)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Pydicom is a python package for parsing DICOM files, making it easier to access the header of the DICOM as well as coverting the raw pixel_data into pythonic structures for easier manipulation. fastai.medical.imaging uses pydicom.dcmread to load the DICOM file.\n\nTo plot an X-ray, we can select an entry in the items list and load the DICOM file with dcmread."},{"metadata":{"trusted":true},"cell_type":"code","source":"patient = 3\nxray_sample = items[patient].dcmread()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now we can view the header meta data within the dicom file. "},{"metadata":{"trusted":true},"cell_type":"code","source":"xray_sample","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"There is a lot of information here and the good news is there is an excellent resource to learn more about these:\n\nhttp://dicom.nema.org/medical/dicom/current/output/chtml/part03/sect_C.7.6.3.html#sect_C.7.6.3.1.4\n\nOne row you will notice is pixel data as an array. We can view this, although in its raw for, isn't very useful. "},{"metadata":{"trusted":true},"cell_type":"code","source":"xray_sample.PixelData[:200]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Because of the complexity in interpreting PixelData, pydicom provides an easy way to get it in a convenient form: pixel_array which returns a numpy.ndarray containing the pixel data:"},{"metadata":{"trusted":true},"cell_type":"code","source":"xray_sample.pixel_array, xray_sample.pixel_array.shape","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We can view the image again."},{"metadata":{"trusted":true},"cell_type":"code","source":"xray_sample.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Remember all the meta data? \nThat data can be pulled into a dataframe.\n\nThanks to [Ben](https://www.kaggle.com/beezus666/chest-x-ray-with-fastai) for finding a solution with the dataframe hanging!"},{"metadata":{"trusted":true},"cell_type":"code","source":"%%time \n# takes 7-8 minutes, so load from pickle\ndicom_dataframe = pd.DataFrame.from_dicoms(items, window=dicom_windows.lungs, px_summ=False)\n\ndicom_dataframe.to_pickle('./dicom_dataframe_pickle.pkl')\ndicom_dataframe.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"dicom_dataframe = pd.read_pickle('./dicom_dataframe_pickle.pkl')\ndicom_dataframe.shape # should be 15k by 29","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"dicom_dataframe.head()","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}