{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Overview\nIn this notebook, we will analyze different methods of reading DICOM files using the pydicom, dicomdl, and SimpleITK packages in Python. We will measure the I/O speed of each method and compare them to find the best one.\n\nhe pydicom package is a pure-Python DICOM library that provides a comprehensive API for reading, writing, and manipulating DICOM files. The dicomdl package is a Python library that uses the libdcmtk library to read DICOM files. The SimpleITK package is a medical image analysis toolkit that also provides a DICOM reader.\n\nTo measure the I/O speed of each method, we will read a large DICOM file and time how long it takes to read the entire file. We will repeat this process for each method and compare the results.","metadata":{}},{"cell_type":"markdown","source":"\n<p align=\"center\" width=\"100%\">\n    <img width=\"100%\" height=\"50%\" src=\"https://medevel.com/content/images/size/w2000/2019/01/Medevel.jpg\">\n</p>","metadata":{}},{"cell_type":"markdown","source":"# Installation\n## dicomdl","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19"}},{"cell_type":"code","source":"!pip install dicomsdl","metadata":{"execution":{"iopub.status.busy":"2023-08-19T22:16:32.223993Z","iopub.execute_input":"2023-08-19T22:16:32.224397Z","iopub.status.idle":"2023-08-19T22:16:45.507956Z","shell.execute_reply.started":"2023-08-19T22:16:32.224365Z","shell.execute_reply":"2023-08-19T22:16:45.506744Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## simpleitk","metadata":{}},{"cell_type":"code","source":"!pip install SimpleITK","metadata":{"execution":{"iopub.status.busy":"2023-08-19T13:38:06.574933Z","iopub.execute_input":"2023-08-19T13:38:06.575281Z","iopub.status.idle":"2023-08-19T13:38:17.698463Z","shell.execute_reply.started":"2023-08-19T13:38:06.575251Z","shell.execute_reply":"2023-08-19T13:38:17.697356Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Import Packages","metadata":{}},{"cell_type":"code","source":"# from nvidia import dali\nimport pydicom as pyd\nimport dicomsdl as dic\nimport SimpleITK as sim\n\nimport time\nfrom tqdm import tqdm\nimport numpy as np\nimport matplotlib.pyplot as plt\n\n# Torch\nimport torch\nfrom torch import nn\nfrom torch.utils.data import Dataset, DataLoader\nimport torchvision.transforms as transforms\n\nfrom pathlib import Path","metadata":{"execution":{"iopub.status.busy":"2023-08-19T22:16:45.510466Z","iopub.execute_input":"2023-08-19T22:16:45.510849Z","iopub.status.idle":"2023-08-19T22:16:49.524384Z","shell.execute_reply.started":"2023-08-19T22:16:45.510812Z","shell.execute_reply":"2023-08-19T22:16:49.523383Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# global vars\nJ2K_SUID = '1.2.840.10008.1.2.4.90'\nJ2K_HEADER = b\"\\x00\\x00\\x00\\x0C\"\nJLL_SUID = '1.2.840.10008.1.2.4.70'\nJLL_HEADER = b\"\\xff\\xd8\\xff\\xe0\"\nSUID2HEADER = {J2K_SUID: J2K_HEADER, JLL_SUID: JLL_HEADER}","metadata":{"execution":{"iopub.status.busy":"2023-08-19T22:16:49.52582Z","iopub.execute_input":"2023-08-19T22:16:49.526463Z","iopub.status.idle":"2023-08-19T22:16:49.531878Z","shell.execute_reply.started":"2023-08-19T22:16:49.52643Z","shell.execute_reply":"2023-08-19T22:16:49.530928Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Get Some Dicom path\nTake **128** samples","metadata":{}},{"cell_type":"code","source":"dcm_paths = list()\nidx = 0\nfor p in Path('/kaggle/input/rsna-2023-abdominal-trauma-detection/train_images').glob('*/*/*.dcm'):\n    if idx>=128:\n        break\n    dcm_paths.append(p)\n    idx+=1","metadata":{"execution":{"iopub.status.busy":"2023-08-19T22:16:49.534906Z","iopub.execute_input":"2023-08-19T22:16:49.535675Z","iopub.status.idle":"2023-08-19T22:16:49.685858Z","shell.execute_reply.started":"2023-08-19T22:16:49.535642Z","shell.execute_reply":"2023-08-19T22:16:49.684838Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Helpers","metadata":{}},{"cell_type":"code","source":"class AbstractDataset(object):\n    def __init__(self,dcm_paths,transform=None):\n        self.dcm_paths = dcm_paths\n        self.transform = transform\n        \n    def __len__(self):\n        return len(self.dcm_paths)\n    \ntransform = transforms.Compose([\n                        transforms.ToTensor(),\n                        transforms.Resize((224, 224),antialias=False),\n                        transforms.RandomHorizontalFlip(),\n                        transforms.Normalize((0.5, ), (0.5, ))])","metadata":{"execution":{"iopub.status.busy":"2023-08-19T22:16:49.687198Z","iopub.execute_input":"2023-08-19T22:16:49.688126Z","iopub.status.idle":"2023-08-19T22:16:49.695751Z","shell.execute_reply.started":"2023-08-19T22:16:49.68809Z","shell.execute_reply":"2023-08-19T22:16:49.694809Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Pydicom","metadata":{}},{"cell_type":"code","source":"class PydicomDataset(AbstractDataset):\n    def __getitem__(self,idx):\n        ds = pyd.dcmread(str(self.dcm_paths[idx]))\n        pixel_img = ds.pixel_array.astype('uint8')\n        if self.transform:\n            pixel_img = self.transform(pixel_img)\n        return pixel_img\npyd_loader = DataLoader(PydicomDataset(dcm_paths, transform), batch_size=64, shuffle=True, num_workers = 2)","metadata":{"execution":{"iopub.status.busy":"2023-08-19T22:16:49.697395Z","iopub.execute_input":"2023-08-19T22:16:49.698141Z","iopub.status.idle":"2023-08-19T22:16:49.706729Z","shell.execute_reply.started":"2023-08-19T22:16:49.698108Z","shell.execute_reply":"2023-08-19T22:16:49.705735Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nfor image in tqdm(pyd_loader):\n    image = image.cuda()","metadata":{"execution":{"iopub.status.busy":"2023-08-19T22:16:49.709241Z","iopub.execute_input":"2023-08-19T22:16:49.710475Z","iopub.status.idle":"2023-08-19T22:16:55.359221Z","shell.execute_reply.started":"2023-08-19T22:16:49.710443Z","shell.execute_reply":"2023-08-19T22:16:55.357361Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## SimpleITK","metadata":{}},{"cell_type":"code","source":"class SimpleITKDataset(AbstractDataset):\n    def __getitem__(self,idx):\n        ds = sim.ReadImage(str(self.dcm_paths[idx]))\n        pixel_img = np.squeeze(sim.GetArrayFromImage(ds),axis=0).astype('float')\n        if self.transform:\n            pixel_img = self.transform(pixel_img)\n        return pixel_img\nsim_loader = DataLoader(SimpleITKDataset(dcm_paths, transform), batch_size=64, shuffle=True, num_workers = 2)","metadata":{"execution":{"iopub.status.busy":"2023-08-19T22:22:38.153687Z","iopub.execute_input":"2023-08-19T22:22:38.154078Z","iopub.status.idle":"2023-08-19T22:22:38.162024Z","shell.execute_reply.started":"2023-08-19T22:22:38.154047Z","shell.execute_reply":"2023-08-19T22:22:38.161069Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nfor image in tqdm(sim_loader):\n    image = image.cuda()","metadata":{"execution":{"iopub.status.busy":"2023-08-19T22:22:38.898823Z","iopub.execute_input":"2023-08-19T22:22:38.899193Z","iopub.status.idle":"2023-08-19T22:22:49.140022Z","shell.execute_reply.started":"2023-08-19T22:22:38.899163Z","shell.execute_reply":"2023-08-19T22:22:49.13888Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Dicomsdl","metadata":{}},{"cell_type":"code","source":"class DicomsdlDataset(AbstractDataset):\n    def __getitem__(self,idx):\n        ds = dic.open(str(self.dcm_paths[idx]))\n        pixel_img = ds.pixelData()\n        if self.transform:\n            pixel_img = self.transform(pixel_img)\n        return pixel_img\ndic_loader = DataLoader(DicomsdlDataset(dcm_paths, transform), batch_size=64, shuffle=True, num_workers = 2)","metadata":{"execution":{"iopub.status.busy":"2023-08-19T22:50:15.558154Z","iopub.execute_input":"2023-08-19T22:50:15.558586Z","iopub.status.idle":"2023-08-19T22:50:15.566942Z","shell.execute_reply.started":"2023-08-19T22:50:15.558549Z","shell.execute_reply":"2023-08-19T22:50:15.565953Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"%%time\nfor image in tqdm(dic_loader):\n    image = image.cuda()","metadata":{"execution":{"iopub.status.busy":"2023-08-19T22:50:19.274932Z","iopub.execute_input":"2023-08-19T22:50:19.275311Z","iopub.status.idle":"2023-08-19T22:50:19.636966Z","shell.execute_reply.started":"2023-08-19T22:50:19.275257Z","shell.execute_reply":"2023-08-19T22:50:19.634332Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Next Updates","metadata":{}},{"cell_type":"markdown","source":"For the next versions, we will use DALI framework to speed up IO.\n\nDALI is a GPU-accelerated data loading library that can be used to load DICOM files much faster than the SimpleITK reader. DALI provides a set of high-performance building blocks for data loading, decoding, and augmentation. It can be integrated with any deep learning framework, such as PyTorch and TensorFlow.","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}