{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Why this notebook?\n\nAfter reading this post about [Pydicom vs DicomSDL (Fast dicom export and processing (1.6-2x faster) 💪)](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/371033), I checked whether DicomSDL had the option to apply a VOI lookup table or windowing operation like Pydicom.<br>\nIt turns out, they don't. They only have this function in their source code 😅\n\n    def apply_window_center():\n        print(\"Hello\")\n        \n\n## Apply_voi_lut\n\nNext, I dug into the source code of Pydicom to see what they do exactly inside the apply_voi_lut method.<br>\nThe code checks several things, of which many don't apply for the images/datasets in the dicom images in this competition.<br>\nFor example, **none** of the datasets in this competition have a valid **VOILUTSequence**.<br>\nThis means the **apply_voi_lut function** will always redirect to the **apply_windowing**.<br>\nTherefore, the next step was to check this function inside Pydicom's code.\n\n## Apply_windowing\n\nHere, the most important thing the code checks is which VOILUTFunction is used in the dataset.<br>\nIn this competition, 22% of the datasets (12139 of 54706) have the Sigmoid function, the rest uses the LINEAR function.<br>\nAs the code is very different for each of these functions, I recreated/copied the necessary code for both functions.<br>\nPlease note that I removed some code for speed, as many checks in the pydicom code is not not applicable in this competition.\nFor example:\n- Pixelrepresentation and ModalityLUTSequence is always NaN\n- Window width/center is always set correctly\n- RescaleSlope & RescaleIntercept is always the same\n\n\n## Results/conclusion\n\nAs you can see in the results below, using the VOILUTFunction is quite important.<br>\nAround 22% of the images uses SIGMOID instead of LINEAR, and the images/colors for Sigmoid are very different, a lot lighter/more yellow.<br>\nBut once you apply the windowing function, they seem completely similar to images with the Linear VOILUTFunction.<br>\n**Therefore**, if you use DicomSDL instead of Pydicom, you should definitely create your own apply_windowing function like in Pydicom!\n","metadata":{}},{"cell_type":"markdown","source":"### ⬇ Libraries","metadata":{}},{"cell_type":"code","source":"# Installs\n!pip install -qU \"python-gdcm\" pydicom pylibjpeg \"opencv-python-headless\"\n!pip install /kaggle/input/dicomsdl/dicomsdl-0.109.1-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl\n\n# Imports\nimport os\nimport time\nimport pydicom\nimport dicomsdl as pysdl\n\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\n\nfrom tqdm import tqdm\nfrom pydicom.pixel_data_handlers.util import apply_voi_lut","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2023-01-10T08:41:51.381378Z","iopub.execute_input":"2023-01-10T08:41:51.381856Z","iopub.status.idle":"2023-01-10T08:42:10.050152Z","shell.execute_reply.started":"2023-01-10T08:41:51.381822Z","shell.execute_reply":"2023-01-10T08:42:10.049247Z"},"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Loading images with Pydicom & DicomSDL","metadata":{}},{"cell_type":"code","source":"def load_image_pydicom(img_path, voi_lut=False):\n    dataset = pydicom.dcmread(img_path)\n    img = dataset.pixel_array\n    if voi_lut:\n        img = apply_voi_lut(img, dataset)\n    if dataset.PhotometricInterpretation == \"MONOCHROME1\":\n        img = np.amax(img) - img\n    return img\n\n\ndef load_image_dicomsdl(img_path, voi_lut=False):\n    dataset = pysdl.open(img_path)\n    img = dataset.pixelData()\n    \n    if voi_lut:\n        # Load only the variables we need\n        center = dataset[\"WindowCenter\"]\n        width = dataset[\"WindowWidth\"]\n        bits_stored = dataset[\"BitsStored\"]\n        voi_lut_function = dataset[\"VOILUTFunction\"]\n\n        # For sigmoid it's a list, otherwise a single value\n        if isinstance(center, list):\n            center = center[0]\n        if isinstance(width, list):\n            width = width[0]\n\n        # Set y_min, max & range\n        y_min = 0\n        y_max = float(2**bits_stored - 1)\n        y_range = y_max\n\n        # Function with default LINEAR (so for Nan, it will use linear)\n        if voi_lut_function == \"SIGMOID\":\n            img = y_range / (1 + np.exp(-4 * (img - center) / width)) + y_min\n        else:\n            # Checks width for < 1 (in our case not necessary, always >= 750)\n            center -= 0.5\n            width -= 1\n\n            below = img <= (center - width / 2)\n            above = img > (center + width / 2)\n            between = np.logical_and(~below, ~above)\n\n            img[below] = y_min\n            img[above] = y_max\n            if between.any():\n                img[between] = (\n                    ((img[between] - center) / width + 0.5) * y_range + y_min\n                )\n    \n    if dataset[\"PhotometricInterpretation\"] == \"MONOCHROME1\":\n        img = np.amax(img) - img\n\n    return img","metadata":{"execution":{"iopub.status.busy":"2023-01-10T10:03:31.883659Z","iopub.execute_input":"2023-01-10T10:03:31.884089Z","iopub.status.idle":"2023-01-10T10:03:31.895513Z","shell.execute_reply.started":"2023-01-10T10:03:31.884062Z","shell.execute_reply":"2023-01-10T10:03:31.894333Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Resulting images with VOILUTFunction -> Linear\n\nAs you can see below, images with VOILUTFunction Linear doesn't change at all.","metadata":{}},{"cell_type":"code","source":"img_path_linear = \"../input/rsna-breast-cancer-detection/train_images/10102/1181635673.dcm\"\nfig, axs = plt.subplots(nrows=1, ncols=4,figsize=(30, 16))\n\nimg = load_image_pydicom(img_path_linear)\nplt.subplot(141)\nplt.title(\"Pydicom - Linear without VOI-LUT\")\nplt.imshow(img)\n\nimg = load_image_dicomsdl(img_path_linear)\nplt.subplot(142)\nplt.title(\"DicomSDL - Linear without VOI-LUT\")\nplt.imshow(img)\n\nimg = load_image_pydicom(img_path_linear, True)\nplt.subplot(143)\nplt.title(\"Pydicom - Linear with VOI-LUT\")\nplt.imshow(img)\n\nimg = load_image_dicomsdl(img_path_linear, True)\nplt.subplot(144)\nplt.title(\"DicomSDL - Linear with VOI-LUT\")\nplt.imshow(img)\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-01-10T10:07:36.535871Z","iopub.execute_input":"2023-01-10T10:07:36.536334Z","iopub.status.idle":"2023-01-10T10:07:42.597597Z","shell.execute_reply.started":"2023-01-10T10:07:36.536297Z","shell.execute_reply":"2023-01-10T10:07:42.596685Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Resulting images with VOILUTFunction -> Sigmoid\n\nFor VOILUTFunction Sigmoid, however, it does have an effect. You can clearly see that the default images for Sigmoid without VOILUT are a lot more light/yellow, and once VOILUT is applied the colors seem more similar to images with VOILUTFunction Linear.","metadata":{}},{"cell_type":"code","source":"img_path_sigmoid = \"../input/rsna-breast-cancer-detection/train_images/10006/1459541791.dcm\"\nfig, axs = plt.subplots(nrows=1, ncols=4,figsize=(30, 16))\n\nimg = load_image_pydicom(img_path_sigmoid)\nplt.subplot(141)\nplt.title(\"Pydicom - Sigmoid without VOI-LUT\")\nplt.imshow(img)\n\nimg = load_image_dicomsdl(img_path_sigmoid)\nplt.subplot(142)\nplt.title(\"DicomSDL - Sigmoid without VOI-LUT\")\nplt.imshow(img)\n\nimg = load_image_pydicom(img_path_sigmoid, True)\nplt.subplot(143)\nplt.title(\"Pydicom - Sigmoid with VOI-LUT\")\nplt.imshow(img)\n\nimg = load_image_dicomsdl(img_path_sigmoid, True)\nplt.subplot(144)\nplt.title(\"DicomSDL - Sigmoid with VOI-LUT\")\nplt.imshow(img)\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-01-10T10:07:18.57917Z","iopub.execute_input":"2023-01-10T10:07:18.579646Z","iopub.status.idle":"2023-01-10T10:07:30.313517Z","shell.execute_reply.started":"2023-01-10T10:07:18.579607Z","shell.execute_reply":"2023-01-10T10:07:30.311958Z"},"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]}]}