{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"![Mayo-Strip-AI](https://storage.googleapis.com/kaggle-competitions/kaggle/37333/logos/header.png)\n\nThis is a multi class image classification competition comprising the images that depict clots with an etiology (that is, origin) known to be either CE (Cardioembolic) or LAA (Large Artery Atherosclerosis). There are also supplemental images with either an unknown etiology or an etiology other than CE or LAA.\n\nOur goal is to classify the images into CE or LAA in the test set for each patient.\n\n<h3>Key Terms</h3>\n\n* <b> Ischemic Stroke</b> : An ischemic stroke occurs when the blood supply to part of the brain is interrupted or reduced, preventing brain tissue from getting oxygen and nutrients. Source: <a href=\"https://www.mayoclinic.org/diseases-conditions/stroke/symptoms-causes/syc-20350113#:~:text=An%20ischemic%20stroke%20occurs%20when,brain%20damage%20and%20other%20complications\">mayoclinic.org</a>\n\n* <b> Stroke Etiology</b>: Two main causes of stroke 1. A blocked artery (ischemic stroke) 2. Leaking or bursting of a blood vessel (hemorrhagic stroke) Source: <a href=\"https://www.mayoclinic.org/diseases-conditions/stroke/symptoms-causes/syc-20350113#:~:text=There%20are%20two%20main%20causes,doesn't%20cause%20lasting%20symptoms\">mayoclinic.org</a>\n\n* <b> Whole slide imaging (WSI)</b> : Commonly referred to as “virtual microscopy” refers to scanning of conventional glass slides in order to produce digital slides, is the most recent imaging modality being employed by pathology departments worldwide. WSI consists of two processes. 1. First process is to utilize specialized hardware (scanner) to digitize glass slides to generate a large representative digital image known as digital slide 2. The second process employs specialized software (ie, virtual slide viewer) to view and/or analyze these enormous digital files. Source: <a href=\"https://www.dovepress.com/whole-slide-imaging-in-pathology-advantages-limitations-and-emerging-p-peer-reviewed-fulltext-article-PLMI#:~:text=Whole%20slide%20imaging%20(WSI)%2C,%2C%20educational%2C%20and%20research%20purposes\">dovepress.com</a>","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\n\nimport plotly.graph_objects as go\nfrom plotly.offline import iplot, init_notebook_mode\ninit_notebook_mode(connected=True)\nimport seaborn as sns\nimport matplotlib.pyplot as plt\nfrom plotly.offline import iplot, init_notebook_mode\ninit_notebook_mode(connected=True)\nimport plotly_express as px\nimport plotly.graph_objects as go\nfrom plotly.subplots import make_subplots\nfrom plotly.offline import init_notebook_mode\nimport plotly.io as pio\nfrom plotly.subplots import make_subplots\n# setting default template to plotly_white for all visualizations\npio.templates.default = \"plotly_white\"\n%matplotlib inline\nimport gc\n\nfrom colorama import Fore, Back, Style\n\ny_ = Fore.YELLOW\nr_ = Fore.RED\ng_ = Fore.GREEN\nb_ = Fore.BLUE\nm_ = Fore.MAGENTA\nc_ = Fore.CYAN\nres = Style.RESET_ALL\n\nimport warnings\nwarnings.filterwarnings('ignore')\n#import os\n#for dirname, _, filenames in os.walk('/kaggle/input'):\n    #for filename in filenames:\n        #print(os.path.join(dirname, filename))\n#        pass","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-07-08T14:23:49.616492Z","iopub.execute_input":"2022-07-08T14:23:49.617042Z","iopub.status.idle":"2022-07-08T14:23:49.673633Z","shell.execute_reply.started":"2022-07-08T14:23:49.616998Z","shell.execute_reply":"2022-07-08T14:23:49.672343Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"PATH = '/kaggle/input/mayo-clinic-strip-ai/'\nsubmission = pd.read_csv(PATH + 'sample_submission.csv', index_col=None)\ntrain = pd.read_csv(PATH + 'train.csv', index_col=None)\ntest = pd.read_csv(PATH + 'test.csv', index_col=None)\npd.set_option('display.max_columns', None)  \npd.set_option('display.max_colwidth', None)\nprint(f\"{y_}Train csv shape : {train.shape}{res}\\n{g_}Test csv shape : {test.shape}{res}\")","metadata":{"execution":{"iopub.status.busy":"2022-07-08T14:23:49.79108Z","iopub.execute_input":"2022-07-08T14:23:49.791977Z","iopub.status.idle":"2022-07-08T14:23:49.819604Z","shell.execute_reply.started":"2022-07-08T14:23:49.791937Z","shell.execute_reply":"2022-07-08T14:23:49.817981Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.info()","metadata":{"execution":{"iopub.status.busy":"2022-07-08T14:23:49.993607Z","iopub.execute_input":"2022-07-08T14:23:49.994096Z","iopub.status.idle":"2022-07-08T14:23:50.016584Z","shell.execute_reply.started":"2022-07-08T14:23:49.994059Z","shell.execute_reply":"2022-07-08T14:23:50.015471Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train","metadata":{"execution":{"iopub.status.busy":"2022-07-08T14:23:50.20655Z","iopub.execute_input":"2022-07-08T14:23:50.207025Z","iopub.status.idle":"2022-07-08T14:23:50.227419Z","shell.execute_reply.started":"2022-07-08T14:23:50.20699Z","shell.execute_reply":"2022-07-08T14:23:50.226425Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test","metadata":{"execution":{"iopub.status.busy":"2022-07-08T14:23:50.261482Z","iopub.execute_input":"2022-07-08T14:23:50.262281Z","iopub.status.idle":"2022-07-08T14:23:50.274197Z","shell.execute_reply.started":"2022-07-08T14:23:50.262242Z","shell.execute_reply":"2022-07-08T14:23:50.273149Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def plot_class_pct():\n    label_grp = train.groupby(['label'])['image_id'].count().reset_index()\n    colors = {'CE' : '#DCD427',\n    'LAA' : '#0092CC',\n             }\n    label_map = {'CE' : \"CE (Cardioembolic)\",'LAA' : \"LAA (Large Artery Atherosclerosis)\"}\n    label_grp['color'] = label_grp['label'].apply(lambda x: colors[x])\n    label_grp['lbl'] = label_grp['label'].apply(lambda x: label_map[x])    \n    #label_grp.\n    pio.templates.default = \"plotly_dark\"\n    label_grp['pct'] = round((label_grp['image_id'] / label_grp['image_id'].sum())*100,2)\n    fig = go.Figure(data=[go.Pie(labels=label_grp['lbl'],\n                             values=label_grp['pct'],\n                             hole=.3,\n                             pull=[0.1, 0.1]\n                            )\n                     ]\n               )\n    fig.update_traces(hoverinfo='label+percent', textinfo='percent', textfont_size=16,\n                  marker=dict(colors=label_grp['color'], line=dict(color='#000000', width=2))\n                 )\n    fig.update_layout(title={'text': \"% of labels in training data\",\n        'y':0.9,\n        'x':0.45,\n        'xanchor': 'center',\n        'yanchor': 'top'})\n    fig.show()    ","metadata":{"_kg_hide-input":false,"_kg_hide-output":false,"execution":{"iopub.status.busy":"2022-07-08T14:23:50.451793Z","iopub.execute_input":"2022-07-08T14:23:50.453317Z","iopub.status.idle":"2022-07-08T14:23:50.468141Z","shell.execute_reply.started":"2022-07-08T14:23:50.453209Z","shell.execute_reply":"2022-07-08T14:23:50.466521Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_class_pct()","metadata":{"execution":{"iopub.status.busy":"2022-07-08T14:23:50.661841Z","iopub.execute_input":"2022-07-08T14:23:50.662597Z","iopub.status.idle":"2022-07-08T14:23:50.713898Z","shell.execute_reply.started":"2022-07-08T14:23:50.662554Z","shell.execute_reply":"2022-07-08T14:23:50.71284Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def plot_test_center_pct():\n    test_center = train.groupby(['center_id'])['image_id'].count().reset_index()\n    pio.templates.default = \"plotly_dark\"\n    test_center['pct'] = round((test_center['image_id'] / test_center['image_id'].sum())*100,2)\n    fig = go.Figure(data=[go.Pie(labels=test_center['center_id'],\n                             values=test_center['pct'],\n                             hole=.3,\n                             pull=[0.1, 0.1]\n                            )\n                     ]\n               )\n    fig.update_traces(hoverinfo='label+percent', textinfo='percent', textfont_size=16,\n                  marker=dict(#colors=test_center['color'], \n                              line=dict(color='#000000', width=2))\n                 )\n    fig.update_layout(title={'text': \"% of samples in training data from test centers\",\n        'y':0.9,\n        'x':0.45,\n        'xanchor': 'center',\n        'yanchor': 'top'})\n    fig.show()    ","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-07-08T14:23:50.781542Z","iopub.execute_input":"2022-07-08T14:23:50.782596Z","iopub.status.idle":"2022-07-08T14:23:50.794356Z","shell.execute_reply.started":"2022-07-08T14:23:50.782522Z","shell.execute_reply":"2022-07-08T14:23:50.793144Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plot_test_center_pct()","metadata":{"execution":{"iopub.status.busy":"2022-07-08T14:23:50.936745Z","iopub.execute_input":"2022-07-08T14:23:50.937254Z","iopub.status.idle":"2022-07-08T14:23:50.98383Z","shell.execute_reply.started":"2022-07-08T14:23:50.937216Z","shell.execute_reply":"2022-07-08T14:23:50.9822Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Test center 11 has the highest number of train samples (34%)","metadata":{}},{"cell_type":"code","source":"center_grp = train.groupby(['center_id','label'])['image_id'].count().reset_index()\ncenter_grp.rename(columns={'image_id':'count'},inplace=True)\n\nx = list(center_grp.query(\"label =='CE'\")['center_id'])\n\nfig = go.Figure(data=[\n    go.Bar(name='CE', x=x, y=list(center_grp.query(\"label =='CE'\")['count']), marker=dict(color='#DCD427')\n),\n    go.Bar(name='LAA', x=x, y=list(center_grp.query(\"label =='LAA'\")['count']), marker=dict(color='#0092CC')\n)\n])\n# Change the bar mode\nfig.update_layout(barmode='group')\nfig.update_layout(\n    xaxis = dict(\n        tickmode = 'array',\n        tickvals = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11],\n        title = \"Test Center\"\n    ),\n    yaxis = dict(\n        title = \"Count\"\n    )    \n)\nfig.update_layout(title_text=\"Labels by Test Center\")\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-08T14:23:51.381339Z","iopub.execute_input":"2022-07-08T14:23:51.382506Z","iopub.status.idle":"2022-07-08T14:23:51.420513Z","shell.execute_reply.started":"2022-07-08T14:23:51.38246Z","shell.execute_reply":"2022-07-08T14:23:51.419552Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Test Center 3 has more LAA (Large Artery Atherosclerosis) images than CE (Cardioembolic) images. All other test centers have more CE images. This may not be significant information that influences the classification but an anomaly to note","metadata":{}},{"cell_type":"code","source":"patient_img_count = train.groupby(['patient_id'])['image_id'].count().reset_index().sort_values(by=\"image_id\", ascending=False).reset_index(drop=True)\npatient_img_count.rename(columns={'image_id':'num_images'},inplace=True)\n\nfig = go.Figure(data=[go.Histogram(x=patient_img_count['num_images'], marker=dict(color='#F2A514'))])\nfig.update_layout(\n    xaxis = dict(\n        title = \"Number of images\"\n    ),\n    yaxis = dict(\n        title = \"Count\"\n    )    \n)\nfig.update_layout(title_text=\"Image count per patient - Histogram\")\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-08T14:23:51.962223Z","iopub.execute_input":"2022-07-08T14:23:51.963392Z","iopub.status.idle":"2022-07-08T14:23:51.989513Z","shell.execute_reply.started":"2022-07-08T14:23:51.96334Z","shell.execute_reply":"2022-07-08T14:23:51.988238Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Most patients in the training set have only one image. ","metadata":{}},{"cell_type":"markdown","source":"## OpenSlide \n\nOpenSlide Python is a Python interface to the OpenSlide library.\n\nOpenSlide library provides simple interface for reading whole-slide images, also known as virtual slides, These images can occupy tens of gigabytes when uncompressed, and so cannot be easily read using standard tools or libraries, which are designed for images that can be comfortably uncompressed into RAM. Source : <a href=\"https://openslide.org/api/python/#openslide.OpenSlide\"> openslide.org</a>\n\n#### Reference\nThanks to the author of this <a href=\"https://www.kaggle.com/code/naotous/mayo-eda-whole-slide-images\"> notebook </a> for introducing me to OpenSlide library among others. ","metadata":{}},{"cell_type":"code","source":"from openslide import open_slide\nimport json\nimport openslide\n\nsample_file = PATH + \"/train/3d10be_0.tif\"\n\nslide = open_slide(sample_file)\nslide_props = slide.properties\nprint(f\"{g_}Properties : {json.dumps(dict(slide_props), sort_keys=True, indent=4)}\")\nlide_dims = slide.dimensions\nprint(f\"{m_}Dimension of the sample image{slide.dimensions}\")\nprint(f\"{b_}Number of levels in this image : {len(slide.level_dimensions)}\")\nprint(f\"{g_}Each level is downsampled by : {slide.level_downsamples}\")\n","metadata":{"execution":{"iopub.status.busy":"2022-07-08T14:23:54.351378Z","iopub.execute_input":"2022-07-08T14:23:54.351852Z","iopub.status.idle":"2022-07-08T14:23:54.371634Z","shell.execute_reply.started":"2022-07-08T14:23:54.351791Z","shell.execute_reply":"2022-07-08T14:23:54.370155Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Since the actual file is large, Let's get the thumbnail and display it.","metadata":{}},{"cell_type":"code","source":"slide_thumb = slide.get_thumbnail(size=(600, 600))\n_ = plt.figure(figsize=(6,6))\n_ = plt.imshow(np.array(slide_thumb))    \n_ = plt.title(sample_file)","metadata":{"execution":{"iopub.status.busy":"2022-07-08T14:23:56.053445Z","iopub.execute_input":"2022-07-08T14:23:56.053982Z","iopub.status.idle":"2022-07-08T14:24:11.527106Z","shell.execute_reply.started":"2022-07-08T14:23:56.05394Z","shell.execute_reply":"2022-07-08T14:24:11.525647Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def show_slide(image_id, ax):\n    file = f\"{PATH}/train/{image_id}.tif\"\n    slide = open_slide(file)\n    slide_thumb = slide.get_thumbnail(size=(600, 600))\n    ax.imshow(np.array(slide_thumb))    \n","metadata":{"execution":{"iopub.status.busy":"2022-07-08T14:24:11.529687Z","iopub.execute_input":"2022-07-08T14:24:11.531266Z","iopub.status.idle":"2022-07-08T14:24:11.539185Z","shell.execute_reply.started":"2022-07-08T14:24:11.53119Z","shell.execute_reply":"2022-07-08T14:24:11.537925Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#list(train.query(\"label == 'LAA' and image_num == 0\")['image_id'])","metadata":{"execution":{"iopub.status.busy":"2022-07-08T14:24:11.541205Z","iopub.execute_input":"2022-07-08T14:24:11.542038Z","iopub.status.idle":"2022-07-08T14:24:11.552665Z","shell.execute_reply.started":"2022-07-08T14:24:11.54199Z","shell.execute_reply":"2022-07-08T14:24:11.551171Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def display_sample(samples, title):\n    fig1, ax1 = plt.subplots(1,len(samples), figsize=(18, 5), facecolor='w', edgecolor='b')\n    fig1.subplots_adjust(hspace =.3, wspace=0.3)\n    if len(samples) == 1:\n        show_slide(samples[0],ax1)\n    else:\n        axs = ax1.ravel()\n        for image_id, ax in zip(samples, axs):\n            show_slide(image_id,ax)\n    plt.tight_layout(pad=3.0)\n    plt.subplots_adjust(top=0.91)\n    plt.suptitle(title,fontsize = 20)\n    plt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-08T14:43:22.69925Z","iopub.execute_input":"2022-07-08T14:43:22.699831Z","iopub.status.idle":"2022-07-08T14:43:22.714636Z","shell.execute_reply.started":"2022-07-08T14:43:22.699773Z","shell.execute_reply":"2022-07-08T14:43:22.713329Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ce_samples = ['0ed87f_0']\ndisplay_sample(ce_samples, 'CE Sample')","metadata":{"execution":{"iopub.status.busy":"2022-07-08T14:43:23.863953Z","iopub.execute_input":"2022-07-08T14:43:23.864678Z","iopub.status.idle":"2022-07-08T14:43:38.384915Z","shell.execute_reply.started":"2022-07-08T14:43:23.86464Z","shell.execute_reply":"2022-07-08T14:43:38.383529Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"laa_samples = [ '1db82d_0', '1f018f_0']\ndisplay_sample(laa_samples, 'LAA Samples')","metadata":{"execution":{"iopub.status.busy":"2022-07-08T14:50:10.80884Z","iopub.execute_input":"2022-07-08T14:50:10.809304Z","iopub.status.idle":"2022-07-08T14:52:09.646142Z","shell.execute_reply.started":"2022-07-08T14:50:10.809265Z","shell.execute_reply":"2022-07-08T14:52:09.644953Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Work in progress","metadata":{}}]}