{"cells":[{"metadata":{},"cell_type":"markdown","source":"### >>>>>>>>>> for using interactive features please fork it and then run <<<<<<<<<<\n\n![](https://www.osicild.org/uploads/1/2/2/7/122798879/editor/kaggle-v01-clipped_2.png?1569348761)\n## If you want to know details about the Pulmonary Fibrosis Progression, check out my other kernel [Pulmonary Fibrosis for Non-Med People](https://www.kaggle.com/redwankarimsony/pulmonary-fibrosis-for-non-med-people) which contains detailed information about the the disease, symptoms, causes and many more. If you like this kernel, please upvote.. "},{"metadata":{},"cell_type":"markdown","source":"<a id=\"top\"></a>\n\n<div class=\"list-group\" id=\"list-tab\" role=\"tablist\">\n<h3 class=\"list-group-item list-group-item-action active\" data-toggle=\"list\"  role=\"tab\" aria-controls=\"home\">Table of content</h3>\n<font color=\"teal\" size=+1><b>Part 1: EDA</b></font><br>  \n    \n&#9632; [1. Load Dataframes](#1)<br>\n&#9632; [2. Train and Test Dataframe Details](#2)<br>\n&#9632; [3. Number of Patients](#3)<br>\n&#9632; [4. Creating Individual Patient Profiles ](#4)<br>\n&#9632; [5. Gender Distribution](#5)<br>\n&#9632; [6. Age Distribution of the Patients](#6)<br>\n&#9632; [7. Study of SmokingStatus](#7)<br>\n&#9632; [8. Study of Gender vs SmokingStatus](#8)<br>\n&#9632; [9. Age Distribution vs SmokingStatus](#9)<br>\n&#9632; [10. Interactive Patient Lookup (FVC Decay)](#10)<br>\n&#9632; [11. Interactive Patient Lookup (CT Scans)](#11)<br>\n&#9632; [12. Sumarizing the Unique Patient Profile](#12)<br>\n\n    \n<font color=\"teal\" size=+1><b>Part 2: Lung Segmentation</b></font><br>\n&#9632; [13. Segmentation of Lung](#13)"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import os\nimport pandas as pd\nimport numpy as np\nimport seaborn as sns\nimport matplotlib.pyplot as plt\n\n\n#plotly imports\n!pip install chart_studio\nimport plotly.express as px\nimport chart_studio.plotly as py\nimport plotly.graph_objs as go\nfrom plotly.offline import iplot\nimport cufflinks\ncufflinks.go_offline()\ncufflinks.set_config_file(world_readable=True, theme='pearl')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"1\"></a>\n<font color=\"black\" size=+2.5><b>1. Load Dataframes</b></font>\n\n<a href=\"#top\" class=\"btn btn-primary btn-sm\" role=\"button\" aria-pressed=\"true\" style=\"color:white\" data-toggle=\"popover\" title=\"go to Colors\">Go to TOC</a>"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"train_dir = '../input/osic-pulmonary-fibrosis-progression/train/'\ntest_dir = '../input/osic-pulmonary-fibrosis-progression/test/'\n\n\ntrain_df = pd.read_csv('../input/osic-pulmonary-fibrosis-progression/train.csv')\ntest_df = pd.read_csv('../input/osic-pulmonary-fibrosis-progression/test.csv')\nprint('Training Dataframe shape: ', train_df.shape)\n\ntrain_df.head(10)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"2\"></a>\n<font color=\"black\" size=+2.5><b>2. Train and Test Dataframe Details</b></font>\n\n<a href=\"#top\" class=\"btn btn-primary btn-sm\" role=\"button\" aria-pressed=\"true\" style=\"color:white\" data-toggle=\"popover\" title=\"go to Colors\">Go to TOC</a>"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Let's have a look at the detailed info about the dataframes\nprint('Training Dataframe Details: ')\nprint(train_df.info())\n\nprint('\\n\\nTest Dataframe Details: ')\nprint(test_df.info())\n\n\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n<a id=\"3\"></a>\n<font color=\"black\" size=+2.5><b>3. Number of Patients</b></font>\n\n<a href=\"#top\" class=\"btn btn-primary btn-sm\" role=\"button\" aria-pressed=\"true\" style=\"color:white\" data-toggle=\"popover\" title=\"go to Colors\">Go to TOC</a>"},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Number of patients in training set:',\n      len(os.listdir(train_dir)))\nprint('Number of patients in test set:',\n     len(os.listdir(test_dir)))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"4\"></a>\n<font color=\"black\" size=+2.5><b>4. Creating Individual Patient Profiles </b></font>\n\n<a href=\"#top\" class=\"btn btn-primary btn-sm\" role=\"button\" aria-pressed=\"true\" style=\"color:white\" data-toggle=\"popover\" title=\"go to Colors\">Go to TOC</a>\n\nLet's create a dataframe that will contain all the unique patient IDs. "},{"metadata":{"trusted":true},"cell_type":"code","source":"# Creating unique patient lists and their properties. \npatient_ids = os.listdir(train_dir)\npatient_ids = sorted(patient_ids)\n\n#Creating new rows\nno_of_instances = []\nage = []\nsex = []\nsmoking_status = []\n\nfor patient_id in patient_ids:\n    patient_info = train_df[train_df['Patient'] == patient_id].reset_index()\n    no_of_instances.append(len(os.listdir(train_dir + patient_id)))\n    age.append(patient_info['Age'][0])\n    sex.append(patient_info['Sex'][0])\n    smoking_status.append(patient_info['SmokingStatus'][0])\n\n#Creating the dataframe for the patient info    \npatient_df = pd.DataFrame(list(zip(patient_ids, no_of_instances, age, sex, smoking_status)), \n                                 columns =['Patient', 'no_of_instances', 'Age', 'Sex', 'SmokingStatus'])\nprint(patient_df.info())\npatient_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"5\"></a>\n<font color=\"black\" size=+2.5><b>5. Gender Distribution</b></font>\n\n<a href=\"#top\" class=\"btn btn-primary btn-sm\" role=\"button\" aria-pressed=\"true\" style=\"color:white\" data-toggle=\"popover\" title=\"go to Colors\">Go to TOC</a>\n\nIt seems that among the unique patients 78% are Male and rest of them are female. \n"},{"metadata":{"trusted":true},"cell_type":"code","source":"patient_df['Sex'].value_counts(normalize = True).iplot(kind = 'bar', \n                                                        color = 'blue', \n                                                        yTitle = 'Unique patient count',\n                                                        xTitle = 'Gender',\n                                                        title = 'Gender Distribution of the unique patients')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"6\"></a>\n<font color=\"black\" size=+2.5><b>6. Age Distribution of the Patients</b></font>\n\n<a href=\"#top\" class=\"btn btn-primary btn-sm\" role=\"button\" aria-pressed=\"true\" style=\"color:white\" data-toggle=\"popover\" title=\"go to Colors\">Go to TOC</a>\n\nNow let's have a look at the patients age distributions. We observe that the age distribution starts from near 50 years and it tops to 90 years. Therefore, the given data about the pulmonary fibrosis is mainly of the older people.  "},{"metadata":{"trusted":true},"cell_type":"code","source":"import scipy\n\ndata = patient_df.Age.tolist()\nplt.figure(figsize=(18,6))\n# Creating the main histogram\n_, bins, _ = plt.hist(data, 15, density=1, alpha=0.5)\n\n# Creating the best fitting line with mean and standard deviation\nmu, sigma = scipy.stats.norm.fit(data)\nbest_fit_line = scipy.stats.norm.pdf(bins, mu, sigma)\nplt.plot(bins, best_fit_line, color = 'b', linewidth = 3, label = 'fitting curve')\nplt.title(f'Age Distribution [ mean = {\"{:.2f}\".format(mu)}, standard_dev = {\"{:.2f}\".format(sigma)} ]', fontsize = 18)\nplt.xlabel('Age -->')\nplt.show()\n\npatient_df['Age'].iplot(kind='hist',bins=25,color='blue',xTitle='Percent distribution',yTitle='Count')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"let's see the average age of patients based on their sex. However, it is observed that the average age of male and female patients are almost same. However, male patients are mostly sick near the age of 70 "},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(16, 6))\nsns.kdeplot(patient_df.loc[patient_df['Sex'] == 'Male', 'Age'], label = 'Male',shade=True)\nsns.kdeplot(patient_df.loc[patient_df['Sex'] == 'Female', 'Age'], label = 'Female',shade=True)\nplt.xlabel('Age (years)'); plt.ylabel('Density'); plt.title('Distribution of Ages');","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"7\"></a>\n<font color=\"black\" size=+2.5><b>7. Study of SmokingStatus </b></font>\n\n<a href=\"#top\" class=\"btn btn-primary btn-sm\" role=\"button\" aria-pressed=\"true\" style=\"color:white\" data-toggle=\"popover\" title=\"go to Colors\">Go to TOC</a>\n\nIt seems that almost 70% of the patients are ex-smokers (at some point of their lives, they smoked). However, among the patients about 27% of the patients have never smoked in their life. Hilarious thing is that 5% of the patients are still smoking.. LOL"},{"metadata":{"trusted":true},"cell_type":"code","source":"patient_df['SmokingStatus'].value_counts(normalize=True).iplot(kind='bar',\n                                                      yTitle='Percentage', \n                                                      linecolor='black', \n                                                      opacity=0.8,\n                                                      color='blue',\n                                                      theme='pearl',\n                                                      bargap=0.5,\n                                                      title='SmokingStatus Distribution')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"8\"></a>\n<font color=\"black\" size=+2.5><b>8. Study of Gender vs SmokingStatus</b></font>\n\n<a href=\"#top\" class=\"btn btn-primary btn-sm\" role=\"button\" aria-pressed=\"true\" style=\"color:white\" data-toggle=\"popover\" title=\"go to Colors\">Go to TOC</a>"},{"metadata":{"trusted":true},"cell_type":"code","source":"patient_df.groupby(['SmokingStatus', 'Sex']).count()['Patient'].unstack().iplot(kind='bar', \n                                                                                yTitle = 'Unique Patient Count',\n                                                                                title = 'Gender vs SmokingStatus' )","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n<a id=\"9\"></a>\n<font color=\"black\" size=+2.5><b>9. Age Distribution vs Smoking Status</b></font>\n\n<a href=\"#top\" class=\"btn btn-primary btn-sm\" role=\"button\" aria-pressed=\"true\" style=\"color:white\" data-toggle=\"popover\" title=\"go to Colors\">Go to TOC</a>"},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(16, 6))\nsns.kdeplot(patient_df.loc[patient_df['SmokingStatus'] == 'Ex-smoker', 'Age'], label = 'Ex-smoker',shade=True)\nsns.kdeplot(patient_df.loc[patient_df['SmokingStatus'] == 'Never smoked', 'Age'], label = 'Never smoked',shade=True)\nsns.kdeplot(patient_df.loc[patient_df['SmokingStatus'] == 'Currently smokes', 'Age'], label = 'Currently smokes',shade=True)\n# Labeling of plot\nplt.xlabel('Age (years)'); plt.ylabel('Density'); plt.title('Distribution of Ages');","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"10\"></a>\n<font color=\"black\" size=+2.5><b>10. Interactive Patient Lookup (FVC Decay)</b></font>\n\n<a href=\"#top\" class=\"btn btn-primary btn-sm\" role=\"button\" aria-pressed=\"true\" style=\"color:white\" data-toggle=\"popover\" title=\"go to Colors\">Go to TOC</a>\n\nIf you run the following cell, you will get an dropdown menu where you can select any patient ID and it will show you the details present for that patient in the training data frame. It will show you all the information of that patient from the train data frame and also plot the gradual fibrosis progression over time. It plots the decrease of FVC over time.  \n\n#### Observations:\n&#9632;  ***Smoking Status doesn't change over time.*** So smoking status for a single patient is always unique.<br> \n&#9632; ***Age of the patient doesn't change over time*** Whatever the age in the beginning of the diagnosis, even after more than 52 weeks patient's age didn't change in the dataframe. \n"},{"metadata":{"trusted":true},"cell_type":"code","source":"from ipywidgets import interact  #, interactive, IntSlider, ToggleButtons\n\ndef patient_lookup(patient_id):\n    print(train_df[train_df['Patient'] == patient_id])\n    patient_info = train_df[train_df['Patient'] == patient_id].reset_index()\n    fig, (ax1,ax2) = plt.subplots(1,2, figsize = (15, 5))\n    ax1.plot(patient_info['Weeks'].tolist() , patient_info['FVC'].tolist(), marker = '*', linewidth = 3,color = 'r', markeredgecolor = 'b')\n    ax1.set_title('FVC Deterioriation over the Weeks')\n    ax1.set_xlabel('Weeks -->')\n    ax1.set_ylabel('FVC')\n    ax1.grid(True)\n    \n    ax2.plot(patient_info['Weeks'].tolist() , patient_info['Percent'].tolist(),marker = '*', linewidth = 3,\n            color = 'r', markeredgecolor = 'b' )\n    ax2.set_title('Percent change over the weeks')\n    ax2.set_xlabel('Weeks -->')\n    ax2.set_ylabel('Percent(of adult capacity)')\n    ax2.grid(True)\n    fig.suptitle(f'P_ID: {patient_id}', fontsize = 20) \n    \n    \n    \ninteract(patient_lookup, patient_id = patient_ids)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"11\"></a>\n<font color=\"black\" size=+2.5><b>11. Interactive Patient Lookup (CT Scans)</b></font>\n\n<a href=\"#top\" class=\"btn btn-primary btn-sm\" role=\"button\" aria-pressed=\"true\" style=\"color:white\" data-toggle=\"popover\" title=\"go to Colors\">Go to TOC</a><br>\nHere from the dropdown menu, please select any patient id, and it will show you 16 CT-Scans over time for that particular patient. <font color=\"red\">Remember it's an interactive notebook. In order to use the interactive feature, you have to fork it and run it in your own instance in interactive mode.</font>"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"import random\nimport pydicom\ndef explore_dicoms(patient_id, instance):\n    RefDs = pydicom.dcmread('../input/osic-pulmonary-fibrosis-progression/train/' + \n                            patient_id +\n                            '/' + \n                            str(instance) + '.dcm')\n    plt.figure(figsize=(10, 5))\n    plt.imshow(RefDs.pixel_array, cmap='gray');\n    plt.title(f'P_ID: {patient_id}\\nInstance: {instance}')\n    plt.axis('off')\n\n\ndef show_ct_scans(patient_id):\n    no_of_instances = int(patient_df[patient_df['Patient'] == patient_id]['no_of_instances'].values[0])\n    files = sorted(random.sample(range(1, no_of_instances), 9))\n    rows = 3\n    cols = 3\n    fig = plt.figure(figsize=(12,12))\n    for idx in range(1, rows*cols+1):\n        fig.add_subplot(rows, cols, idx)\n        RefDs = pydicom.dcmread(train_dir + patient_id + '/' + str(files[idx-1]) + '.dcm')\n        plt.imshow(RefDs.pixel_array, cmap='gray')\n        plt.title(f'Instance: {files[idx-1]}')\n        plt.axis(False)\n        fig.add_subplot\n    fig.suptitle(f'P_ID: {patient_id}') \n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# show_ct_scans(patient_ids[0])\ninteract(show_ct_scans,patient_id = patient_ids)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<a id=\"12\"></a>\n<font color=\"black\" size=+2.5><b>12. Sumarizing the Unique Patient Profile</b></font>\n\n<a href=\"#top\" class=\"btn btn-primary btn-sm\" role=\"button\" aria-pressed=\"true\" style=\"color:white\" data-toggle=\"popover\" title=\"go to Colors\">Go to TOC</a>"},{"metadata":{"trusted":true},"cell_type":"code","source":"import warnings\nwarnings.filterwarnings('ignore')\nimport pandas_profiling as pdp\nunique_patient_profile  = pdp.ProfileReport(patient_df)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Part 2:\n\n<a id=\"13\"></a>\n* <font color=\"black\" size=+2.5><b>13. Lung Segmentation</b></font>\n\n<a href=\"#top\" class=\"btn btn-primary btn-sm\" role=\"button\" aria-pressed=\"true\" style=\"color:white\" data-toggle=\"popover\" title=\"go to Colors\">Go to TOC</a>\n\nFor the lung segmentation, let's select any patient randomly and then we will see the CT-Scans of that patient. "},{"metadata":{"trusted":true},"cell_type":"code","source":"patient_ids = os.listdir('../input/osic-pulmonary-fibrosis-progression/train/')\npatient_id = 'ID00267637202270790561585'\ndicom_filenames = os.listdir('../input/osic-pulmonary-fibrosis-progression/train/' + patient_id)\ndicom_paths = ['../input/osic-pulmonary-fibrosis-progression/train/' + patient_id + '/'+ file for file in dicom_filenames]\n    \ndef load_scan(paths):\n    slices = [pydicom.read_file(path ) for path in paths]\n    slices.sort(key = lambda x: int(x.InstanceNumber), reverse = True)\n    try:\n        slice_thickness = np.abs(slices[0].ImagePositionPatient[2] - slices[1].ImagePositionPatient[2])\n    except:\n        slice_thickness = np.abs(slices[0].SliceLocation - slices[1].SliceLocation)\n        \n    for s in slices:\n        s.SliceThickness = slice_thickness \n    return slices\n\ndef get_pixels_hu(scans):\n    image = np.stack([s.pixel_array for s in scans])\n    image = image.astype(np.int16)\n    # Set outside-of-scan pixels to 0\n    # The intercept is usually -1024, so air is approximately 0\n    image[image == -2000] = 0\n    \n    # Convert to Hounsfield units (HU)\n    intercept = scans[0].RescaleIntercept\n    slope = scans[0].RescaleSlope\n    if slope != 1:\n        image = slope * image.astype(np.float64)\n        image = image.astype(np.int16)   \n    image += np.int16(intercept)\n    return np.array(image, dtype=np.int16)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# set path and load files \npatient_dicom = load_scan(dicom_paths)\npatient_pixels = get_pixels_hu(patient_dicom)\n#sanity check\nplt.imshow(patient_pixels[46], cmap=plt.cm.bone)\nplt.axis(False)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Lets use some thresholding and morphological operations to segment just the lung from the chest:"},{"metadata":{"trusted":true},"cell_type":"code","source":"# skimage image processing packages\nfrom skimage import measure, morphology\nfrom skimage.morphology import ball, binary_closing\nfrom skimage.measure import label, regionprops\nimport copy\n\n\ndef largest_label_volume(im, bg=-1):\n    vals, counts = np.unique(im, return_counts=True)\n    counts = counts[vals != bg]\n    vals = vals[vals != bg]\n    if len(counts) > 0:\n        return vals[np.argmax(counts)]\n    else:\n        return None\n    \ndef segment_lung_mask(image, fill_lung_structures=True):\n    # not actually binary, but 1 and 2. \n    # 0 is treated as background, which we do not want\n    binary_image = np.array(image >= -700, dtype=np.int8)+1\n    labels = measure.label(binary_image)\n \n    # Pick the pixel in the very corner to determine which label is air.\n    # Improvement: Pick multiple background labels from around the  patient\n    # More resistant to “trays” on which the patient lays cutting the air around the person in half\n    background_label = labels[0,0,0]\n \n    # Fill the air around the person\n    binary_image[background_label == labels] = 2\n \n    # Method of filling the lung structures (that is superior to \n    # something like morphological closing)\n    if fill_lung_structures:\n        # For every slice we determine the largest solid structure\n        for i, axial_slice in enumerate(binary_image):\n            axial_slice = axial_slice - 1\n            labeling = measure.label(axial_slice)\n            l_max = largest_label_volume(labeling, bg=0)\n \n            if l_max is not None: #This slice contains some lung\n                binary_image[i][labeling != l_max] = 1\n    binary_image -= 1 #Make the image actual binary\n    binary_image = 1-binary_image # Invert it, lungs are now 1\n \n    # Remove other air pockets inside body\n    labels = measure.label(binary_image, background=0)\n    l_max = largest_label_volume(labels, bg=0)\n    if l_max is not None: # There are air pockets\n        binary_image[labels != l_max] = 0\n \n    return binary_image","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"By running the code below, you are using skimage functions from above to create a mask that covers the lung. We will use both `fill_lung_structures=True` and `fill_lung_structures=False`, to isolate the lung and the internal structures. Let’s run the code below, and display an example of isolating the lung from the chest:"},{"metadata":{"trusted":true},"cell_type":"code","source":"# get masks \nsegmented_lungs = segment_lung_mask(patient_pixels, fill_lung_structures=False)\nsegmented_lungs_fill = segment_lung_mask(patient_pixels, fill_lung_structures=True)\ninternal_structures = segmented_lungs_fill - segmented_lungs\n\n# isolate lung from chest\ncopied_pixels = copy.deepcopy(patient_pixels)\nfor i, mask in enumerate(segmented_lungs_fill): \n    get_high_vals = mask == 0\n    copied_pixels[i][get_high_vals] = 0\nseg_lung_pixels = copied_pixels\n# sanity check\nf, ax = plt.subplots(1,2, figsize=(10,6))\nax[0].imshow(patient_pixels[46], cmap=plt.cm.bone)\nax[0].axis(False)\nax[0].set_title('Original')\nax[1].imshow(seg_lung_pixels[46], cmap=plt.cm.bone)\nax[1].axis(False)\nax[1].set_title('Segmented')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### 2D Visualizations Techniques\n**Non Interactive:**\nWhen visualizing data, I find very beneficial to visualize each process of your script. This will not only help you understand each step of your code, but it makes for a very nice, clean presentation.\nThe code below will, in essence, visually convey a story on how you segmented your data: \n\n&#9632; original image <br>\n&#9632; the binary mask that covers the lung <br>\n&#9632; highlighting internal structures of the lung using one of the masks <br>"},{"metadata":{"trusted":true},"cell_type":"code","source":"f, ax = plt.subplots(2,2, figsize = (10,10))\n\n# pick random slice \nslice_id = 46\n\nax[0,0].imshow(patient_pixels[slice_id], cmap=plt.cm.bone)\nax[0,0].set_title('Original Dicom')\nax[0,0].axis(False)\n\n\nax[0,1].imshow(segmented_lungs_fill[slice_id], cmap=plt.cm.bone)\nax[0,1].set_title('Lung Mask')\nax[0,1].axis(False)\n\nax[1,0].imshow(seg_lung_pixels[slice_id], cmap=plt.cm.bone)\nax[1,0].set_title('Segmented Lung')\nax[1,0].axis(False)\n\nax[1,1].imshow(seg_lung_pixels[slice_id], cmap=plt.cm.bone)\nax[1,1].imshow(internal_structures[slice_id], cmap='jet', alpha=0.7)\nax[1,1].set_title('Segmentation with \\nInternal Structure')\nax[1,1].axis(False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Interactive(1):\nThere are really only a few ways to display multiple images on jupyter notebook — manually plot each image one by one, or make a plot with multiple columns and rows to display in one figure. Unfortunately, this is not helpful for those that want to scan through each image and get a better understanding of the data. With the code below you create an interactive slide bar that lets you scroll through the images <font color=red>(But the catch is that you have to run the code in **interactive window**. So if you want to use the slider window to browse through the slices, fork the code and run it manually in interactive mode)</font>:"},{"metadata":{"trusted":true},"cell_type":"code","source":"# slide through dicom images using a slide bar \nplt.figure(1)\ndef dicom_animation(x):\n    plt.imshow(patient_pixels[x], cmap = plt.cm.gray)\n    return x\ninteract(dicom_animation, x=(0, len(patient_pixels)-1))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Interactive(2):\nAnother way to visualize the CT-Angiograms in a bit lively fashion is to use gif images. Basically GIFs are a series of images shown at an preconfigured interval automatically. <font color=blue>However, the upper side of GIF is that, it stores images in lossless compression. </blue>. So it is highly efficent and accurate for storing CT scan slides."},{"metadata":{"trusted":true},"cell_type":"code","source":"import imageio\nfrom IPython import display\nprint('Original Image Slices before processing')\nimageio.mimsave(f'./{patient_id}.gif', patient_pixels, duration=0.1)\ndisplay.Image(f'./{patient_id}.gif', format='png')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Lung Segmentation Mask')\nimageio.mimsave(f'./{patient_id}.gif', segmented_lungs_fill, duration=0.1)\ndisplay.Image(f'./{patient_id}.gif', format='png')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Segmented Part of Lung Tissue')\nimageio.mimsave(f'./{patient_id}.gif', seg_lung_pixels, duration=0.1)\ndisplay.Image(f'./{patient_id}.gif', format='png')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"However, among the previous 3 GIFs, one of the most important is the Lung Segmentation Mask. You can see there are several images where we can see that the mask only selects the tissue portion of the lung but it is not considering the air vessels thorugh the lungs i.e. Bronchioles.\n\n<br>\n<font color=blue>(**Bronchioles** are air passages inside the lungs that branch off like tree limbs from the bronchi—the two main air passages into which air flows from the trachea (windpipe) after being inhaled through the nose or mouth. The bronchioles deliver air to tiny sacs called alveoli where oxygen and carbon dioxide are exchanged. They are vulnerable to conditions like asthma, bronchiolitis, cystic fibrosis, and emphysema that can cause constriction and/or obstruction of the airways.) </font>\n\nWe need to close those gaps so that we can segment the whole lung portion. With this view in mind, we can run closing operation so that it will fill up those portions. "},{"metadata":{"trusted":true},"cell_type":"code","source":"from skimage.morphology import opening, closing, binary_dilation\nfrom skimage.morphology import disk\n\ndef plot_comparison(original, filtered, filter_name):\n\n    fig, (ax1, ax2) = plt.subplots(ncols=2, figsize=(8, 4), sharex=True,\n                                   sharey=True)\n    ax1.imshow(original, cmap=plt.cm.gray)\n    ax1.set_title('original')\n    ax1.axis('off')\n    ax2.imshow(filtered, cmap=plt.cm.gray)\n    ax2.set_title(filter_name)\n    ax2.axis('off')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"original = segmented_lungs_fill[46]\n\nrows = 4\ncols = 4\nf, ax = plt.subplots(rows, cols, figsize = (15,12))\n\nfor i in range(rows*cols):\n    if i==0:\n        ax[0,0].imshow(original, cmap = plt.cm.gray)\n        ax[0,0].set_title('Original')\n        ax[0,0].axis(False)\n    else:\n        closed = closing(original, disk(i))\n        ax[int(i/rows),int(i % rows)].set_title(f'closed disk({i})')\n        ax[int(i/rows),int(i % rows)].imshow(closed, cmap = plt.cm.gray)\n        ax[int(i/rows),int(i % rows)].axis('off')\nplt.show()   ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Therefore we can select the desired filters from here and just multiply that with the original image to get lung segmentation from these. Before selecting any particular filter, let's see the segmentation quality of the filters and then we can select the desired filter."},{"metadata":{"trusted":true},"cell_type":"code","source":"original_image = patient_pixels[46]\noriginal = segmented_lungs_fill[46]\nf, ax = plt.subplots(rows, cols, figsize = (15,15))\n\nfor i in range(rows*cols):\n    if i==0:\n        ax[0,0].imshow(original_image, cmap = plt.cm.gray)\n        ax[0,0].set_title('Original')\n        ax[0,0].axis(False)\n    else:\n        closed = closing(original, disk(i))\n        ax[int(i/rows),int(i % rows)].set_title(f'closed with disk({i})')\n        ax[int(i/rows),int(i % rows)].imshow(original_image * closed, cmap = plt.cm.gray)\n        ax[int(i/rows),int(i % rows)].axis('off')\nplt.show()   ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now using the disk size 20 we again we perform closing operation on all the corresponding segmentation mask and produce output in the following  section of code."},{"metadata":{"trusted":true},"cell_type":"code","source":"segmented_output = [image * binary_dilation(closing(mask, disk(20)), disk(4)) \n                    for image, mask in zip(patient_pixels, segmented_lungs_fill )]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Segmented Part of Lung Tissue')\nimageio.mimsave(f'segmented_output_{patient_id}.gif', segmented_output, duration=0.1)\ndisplay.Image(f'segmented_output_{patient_id}.gif', format='png')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-output":true},"cell_type":"code","source":"rows = 4\ncols = 4\nf, ax = plt.subplots(rows, cols, figsize = (15,15))\n\nfor i in range(rows*cols):\n    ax[int(i/rows),int(i % rows)].set_title(f'slice({i*2+25})')\n    ax[int(i/rows),int(i % rows)].imshow(segmented_output[i*2+25], cmap = plt.cm.gray)\n    ax[int(i/rows),int(i % rows)].axis('off')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"unique_patient_profile","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}