{"cells":[{"metadata":{},"cell_type":"markdown","source":"# 👨‍⚕️​🩺​ Pulmonary Embolism EDA\n\n***Problem Statement:*** If every breath is strained and painful, it could be a serious and potentially life-threatening condition. A pulmonary embolism (PE) is caused by an artery blockage in the lung. It is time consuming to confirm a PE and prone to overdiagnosis. Machine learning could help to more accurately identify PE cases, which would make management and treatment more effective for patients.\n\nCurrently, CT pulmonary angiography (CTPA), is the most common type of medical imaging to evaluate patients with suspected PE. These CT scans consist of hundreds of images that require detailed review to identify clots within the pulmonary arteries. As the use of imaging continues to grow, constraints of radiologists’ time may contribute to delayed diagnosis.\n\nThe Radiological Society of North America (RSNA®) has teamed up with the Society of Thoracic Radiology (STR) to help improve the use of machine learning in the diagnosis of PE.\n\nIn this competition, you’ll detect and classify PE cases. In particular, you'll use chest CTPA images (grouped together as studies) and your data science skills to enable more accurate identification of PE. If successful, you'll help reduce human delays and errors in detection and treatment.\n\nWith 60,000-100,000 PE deaths annually in the United States, it is among the most fatal cardiovascular diseases. Timely and accurate diagnosis will help these patients receive better care and may also improve outcomes.\n\n[A full set of acknowledgments can be found on this page.](https://www.kaggle.com/c/rsna-str-pulmonary-embolism-detection/overview/acknowledgments)\n\nPlease upvote and share if you found this useful or have a love one affected by PE ❤️"},{"metadata":{},"cell_type":"markdown","source":"## Table of contents\n\n1. [Example](#example)\n    * [Papers](#papers)\n2. [Prepare to start](#prepare)"},{"metadata":{"trusted":true},"cell_type":"code","source":"import re\nimport gc\nimport os\nimport cv2\nimport glob\nimport keras\nimport shutil\nimport pathlib\nimport PIL\nimport numpy as np\nimport pandas as pd\nimport seaborn as sb\nimport pydicom as dcm\nimport networkx as nx\nimport tensorflow as tf\nimport matplotlib.pyplot as plt\nfrom shutil import copyfile\nfrom datetime import datetime\nfrom packaging import version\nfrom tensorflow import keras as ks\nfrom tensorflow.keras import datasets, layers, models\nfrom kaggle_datasets import KaggleDatasets\nfrom mpl_toolkits.mplot3d import Axes3D\nfrom tqdm import tqdm","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"for dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"# Show current versions\nprint('TensorFlow Version: {}'.format(tf.__version__))\nprint('Eager execution: {}'.format(tf.executing_eagerly()))\nprint('OpenCV Version:{}'.format(cv2.__version__))\nprint('Keras Version:{}'.format(ks.__version__))\nprint('Numpy Version:{}'.format(np.__version__))\nprint('Pandas Version:{}'.format(pd.__version__))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Check the number of GPU's that are ready\nprint(\"Num GPUs Available: \", len(tf.config.list_physical_devices('GPU')))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Check the number of TPU's that are ready\nprint(\"Num TPUs Available: \", len(tf.config.list_physical_devices('TPU')))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"collapsed":true},"cell_type":"code","source":"# Read in CSV\ntrain=pd.read_csv(\"../input/rsna-str-pulmonary-embolism-detection/train.csv\")\nprint(train)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.columns","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Classification labels\ncolumn_names=['StudyInstanceUID', 'SeriesInstanceUID', 'SOPInstanceUID',\n       'pe_present_on_image', 'negative_exam_for_pe', 'qa_motion',\n       'qa_contrast', 'flow_artifact', 'rv_lv_ratio_gte_1', 'rv_lv_ratio_lt_1',\n       'leftsided_pe', 'chronic_pe', 'true_filling_defect_not_pe',\n       'rightsided_pe', 'acute_and_chronic_pe', 'central_pe', 'indeterminate']","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(column_names[3:17])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Lets have a look at the first 3 patients\nfor index, row in train.head(n=20).iterrows():\n    print(index,row)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true,"_kg_hide-output":true,"collapsed":true},"cell_type":"code","source":"# Assign each image a condition\npe_present_on_image=(1,0,0,0,0,0,0,0,0,0,0,0,0,0)\nnegative_exam_for_pe=(0,1,0,0,0,0,0,0,0,0,0,0,0,0)\nleftsided_pe=(1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0)\nchronic_pe=(1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0)\nrightsided_pe=(1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0)\nacute_and_chronic_pe=(1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0)\ncentral_pe=(1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0)\nindeterminate=(1,0,0,0,0,0,0,0,0,0,0,0,0,0,0)\n\n# Count the amount of conditions\npe_present_on_image_count=0\nnegative_exam_for_pe_count=0\nleftsided_pe_count=0\nrightsided_pe_count=0\nacute_and_chronic_pe_count=0\ncentral_pe=0\nindeterminate=0\n\n# Display the condition of each image to console\nfor index, row in train.iterrows():\n    condition=index, row['pe_present_on_image'],row['negative_exam_for_pe'],row['leftsided_pe'],row['rightsided_pe'],row['acute_and_chronic_pe']\n    if condition[3:17]==pe_present_on_image:\n        pe_present_on_image_count+=1\n        print(condition[0]+\"- PE found in images:\",pe_present_on_image)\n    if condition[3:17]==negative_exam_for_pe:\n        negative_exam_for_pe_count+=1\n        print(\"- Negative results:\",negative_exam_for_pe)\n    if condition[3:17]==leftsided_pe:\n        leftsided_pe_count+=1\n        print(\"- Left sided PE:\",leftsided_pe)       \n    if condition[3:17]==rightsided_pe:\n        rightsided_pe_count+=1\n        print(\"- Right sided PE:\",rightsided_pe)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Display the amount of conditions for each category\nprint(\"The amount of suspectible PE found in images:\",pe_present_on_image_count)\nprint(\"The amount of negative results for PE:\",negative_exam_for_pe_count)\nprint(\"The amount of left sided PE:\",leftsided_pe_count)\nprint(\"The amount of right sided PE:\",rightsided_pe_count)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Read in CSV\ntest=pd.read_csv(\"../input/rsna-str-pulmonary-embolism-detection/test.csv\")\nprint(test)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Show the amount of rows & columns\ntest.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test.describe()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Read in CSV\nsample_submission=\"../input/rsna-str-pulmonary-embolism-detection/sample_submission.csv\"\nsubmission=pd.read_csv(sample_submission)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"dcm.dcmread(\"../input/rsna-str-pulmonary-embolism-detection/train/000f7f114264/9f7378c3b2ab/060f829ca995.dcm\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Show the first 25 images in the train folder\nfig, axes = plt.subplots(nrows=5, ncols=5, figsize=(20,20))\nimages = glob.glob(\"../input/rsna-str-pulmonary-embolism-detection/train/000f7f114264/9f7378c3b2ab/*.dcm\")\nfor i, image in enumerate(images):\n    if (i == 25) : break\n    row = i // 5\n    col = i % 5\n    axes[row, col].imshow(dcm.dcmread(image).pixel_array)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# convert the color to grayscale \ngray = dcm.dcmread(image).pixel_array\n\n# resize the image(optional)\ngray = cv2.resize(gray, (200, 200))\n\n# apply smoothing operation\ngray = cv2.blur(gray,(3,3))\n\n# create grid to plot using numpy\nxx, yy = np.mgrid[0:gray.shape[0], 0:gray.shape[1]]\n\n# create the figure\nfig = plt.figure(figsize=(150,150))\nax = fig.gca(projection='3d')\nax.plot_wireframe(xx, yy, gray,rstride=1, cstride=1, cmap=plt.cm.gray,\n linewidth=1)\n\n# rotate 3d plot\nfor angle in range(180, 360):\n    ax.view_init(45, angle)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"\"\"\"\nPractical Computer Vision: Extract Insightful Information from Images Using TensorFlow, Keras, and OpenCV\nBook by Abhinav Dadhich\n\"\"\"\n# convert the color to grayscale \ngray = dcm.dcmread(image).pixel_array\n\n# resize the image(optional)\ngray = cv2.resize(gray, (800, 800))\n\n# apply smoothing operation\ngray = cv2.blur(gray,(3,3))\n\n# create grid to plot using numpy\nxx, yy = np.mgrid[0:gray.shape[0], 0:gray.shape[1]]\n\n# create the figure\nfig = plt.figure(figsize=(50,50))\nax = fig.gca(projection='3d')\nax.contour(xx, yy, gray)\n\n# rotate 3d plot\nfor angle in range(70, 210):\n    ax.view_init(45, angle)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"dcm.dcmread(\"../input/rsna-str-pulmonary-embolism-detection/train/000f7f114264/9f7378c3b2ab/060f829ca995.dcm\").pixel_array","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"images = glob.glob(\"../input/rsna-str-pulmonary-embolism-detection/train/00db04fdae51/bc1f7e2c4087/*.dcm\")\nfor i, image in tqdm(enumerate(images)):\n    #if (i == 3) : break\n    # convert the color to grayscale \n    scan = dcm.dcmread(image).pixel_array\n\n    # resize the image(optional)\n    scan = cv2.resize(scan, (800, 800))\n\n    # apply smoothing operation\n    scan = cv2.blur(scan,(3,3))\n\n    # create grid to plot using numpy\n    x, y = np.mgrid[0:scan.shape[0], 0:scan.shape[1]]\n    \n    # create the figure & 3D Axes\n    fig = plt.figure(figsize=(20,20))\n    ax = fig.gca(projection='3d')\n    # apply contouring\n    ax.contourf(x, y, scan)\n\n    # rotate 3D plot\n    for angle in range(70, 210):\n        ax.view_init(45, angle)\n\n    # turn off axis\n    plt.savefig('./'+str(i)+'animation.png')\n    # Clear the current figure.\n    plt.clf() \n    # Closes all the figure windows.\n    plt.close('all')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from mpl_toolkits.mplot3d.art3d import Poly3DCollection\nimport numpy as np\nfrom skimage import measure\n\ndef plot_3d(image, threshold=700, color=\"navy\"):\n    \n    # Position the scan upright, \n    # so the head of the patient would be at the top facing the camera\n    p = image.transpose(2,1,0)\n    \n    verts, faces,_,_ = measure.marching_cubes_lewiner(p, threshold)\n\n    fig = plt.figure(figsize=(10, 10))\n    ax = fig.add_subplot(111, projection='3d')\n\n    # Fancy indexing: `verts[faces]` to generate a collection of triangles\n    mesh = Poly3DCollection(verts[faces], alpha=0.2)\n    mesh.set_facecolor(color)\n    ax.add_collection3d(mesh)\n\n    ax.set_xlim(0, p.shape[0])\n    ax.set_ylim(0, p.shape[1])\n    ax.set_zlim(0, p.shape[2])\n\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"collapsed":true},"cell_type":"code","source":"plot_3d(image)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import glob\nfrom PIL import Image\n\n# filepaths\nfp_in = \"./*.png\"\nfp_out = \"./pe.gif\"\n\n# https://pillow.readthedocs.io/en/stable/handbook/image-file-formats.html#gif\nimg, *imgs = [Image.open(f) for f in sorted(glob.glob(fp_in))]\nimg.save(fp=fp_out, format='GIF', append_images=imgs,\n         save_all=True, duration=200, loop=0)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"<img src=\"./pe.gif\"  style=\"width:900px;\" />"},{"metadata":{"trusted":true},"cell_type":"code","source":"\"\"\"\nPractical Computer Vision: Extract Insightful Information from Images Using TensorFlow, Keras, and OpenCV\nBook by Abhinav Dadhich\n\"\"\"\n\n# convert the color to grayscale \ngray = dcm.dcmread(image).pixel_array\n\n# resize the image(optional)\ngray = cv2.resize(gray, (800, 800))\n\n# apply smoothing operation\ngray = cv2.blur(gray,(3,3))\n\n# create grid to plot using numpy\nxx, yy = np.mgrid[0:gray.shape[0], 0:gray.shape[1]]\n\n# create the figure\nfig = plt.figure(figsize=(150,150))\nax = fig.gca(projection='3d')\nax.contour(xx, yy, gray, stride=1)\n\n# rotate 3d plot\nfor angle in range(180, 360):\n    ax.view_init(45, angle)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"\"\"\"\nPractical Computer Vision: Extract Insightful Information from Images Using TensorFlow, Keras, and OpenCV\nBook by Abhinav Dadhich\n\"\"\"\n\n# convert the color to grayscale \ngray = dcm.dcmread(image).pixel_array\n\n# resize the image(optional)\ngray = cv2.resize(gray, (100, 100))\n\n# apply smoothing operation\ngray = cv2.blur(gray,(3,3))\n\n# create grid to plot using numpy\nxx, yy = np.mgrid[0:gray.shape[0], 0:gray.shape[1]]\n\n# create the figure\nfig = plt.figure(figsize=(150,150))\nax = fig.gca(projection='3d')\nax.plot_surface(xx, yy, gray,cmap='viridis')\n\n# rotate 3d plot\nfor angle in range(70, 210):\n    ax.view_init(45, angle)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"\"\"\"\nPractical Computer Vision: Extract Insightful Information from Images Using TensorFlow, Keras, and OpenCV\nBook by Abhinav Dadhich\n\"\"\"\n\n# convert the color to grayscale \ngray = dcm.dcmread(image).pixel_array\n\n# resize the image(optional)\ngray = cv2.resize(gray, (200, 200))\n\n# apply smoothing operation\ngray = cv2.blur(gray,(3,3))\n\n# create grid to plot using numpy\nxx, yy = np.mgrid[0:gray.shape[0], 0:gray.shape[1]]\n\n# create the figure\nfig = plt.figure(figsize=(150,150))\nax = fig.gca(projection='3d')\nax.scatter(xx, yy, gray)\n\n# rotate 3d plot\nfor angle in range(180, 360):\n    ax.view_init(90, angle)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}