{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.10.10","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":50947,"databundleVersionId":5741538,"sourceType":"competition"},{"sourceId":6074230,"sourceType":"datasetVersion","datasetId":3455489}],"dockerImageVersionId":30513,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"Kaggle medical imaging competitions typically involve three primary tasks: object detection, classification, and segmentation. With the emergence of transformer models, these tasks have witnessed remarkable advancements. Transformers, which revolutionized natural language processing in 2017, have now found applications in medical imaging DL models. Adopting transformers has led to enhanced performance in various medical imaging competitions. In this report, I focus on benchmarking the top Kaggle solutions in classification, object detection, and segmentation for advanced medical imaging modalities such as MRI, CT, and X-rays. Specifically, I will emphasize recent deep learning-based methods for these tasks, highlighting their methodology designs and performances in handling volumetric imaging data. By reviewing and analyzing these top solutions, I aim to provide insights for our community on effectively adapting artificial intelligence (AI) techniques for medical imaging. Overall, this report can serve as a valuable reference for our community to gauge the advancement of Deep Learning in medical imaging competitions in the past five years.","metadata":{}},{"cell_type":"markdown","source":"# 1. Introduction
\n\nArtificial Intelligence (A.I) has been the core of all remarkable developments in our society. Nowadays, A.I has been incorporated into nearly every business with the hope to maximize productivity, efficiency, and accuracy. Over the past few years, I have observed a significant positive impact of A.I, especially Deep Learning (DL) on medical imaging.\n\nMedical imaging is a non-invasive technique and process of imaging the interior of a body for clinical analysis and medical intervention [1]. Different modalities such as Magnetic resonance imaging (MRI), computational tomography (CT), and X-rays can provide versatile information, ranging from structure, morphology to physiological function [2].\n\nThe first part of this report will focus on top DL model types in Kaggle medical imaging competitions. The second part will review some of the top solutions in depth with special attention to the importance of those competitions. \n","metadata":{}},{"cell_type":"code","source":"%matplotlib inline\nimport gc, os, sys, time\nimport pandas as pd, numpy as np\nfrom pathlib import Path\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nfrom matplotlib.ticker import MaxNLocator\nfrom IPython.display import HTML, display\nimport plotly.express as px\nimport plotly.graph_objects as go\nfrom sklearn.preprocessing import StandardScaler\nimport scipy","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2023-07-14T14:44:27.151285Z","iopub.execute_input":"2023-07-14T14:44:27.1517Z","iopub.status.idle":"2023-07-14T14:44:28.788542Z","shell.execute_reply.started":"2023-07-14T14:44:27.151671Z","shell.execute_reply":"2023-07-14T14:44:28.787489Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 2. Data Collection
\n\nData was collected using `meta-kaggle` dataset. `Model detail` and `Model type` was acquired manually from top 10 writeups for each competition. Other columns were based on `meta-kaggle` dataset.\n\nOnly medical imaging competitions were selected. Data cleaning was performed on the meta-data to remove any missing values related to `Model detail`.\n\nIncranial Hemorrage Detection, and Oscis Pulmonary Fibrosis were removed from the meta-data because they are outliers.\n\nData collection, cleaning and extraction were all performed using Excel.","metadata":{}},{"cell_type":"code","source":"medical_competitions = pd.read_csv('/kaggle/input/kaggle-ai-report-23/Meta_data_competitions.csv')\nmetrics = medical_competitions.Metric.unique()\nmedical_competitions.head()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-07-14T14:44:30.4302Z","iopub.execute_input":"2023-07-14T14:44:30.430573Z","iopub.status.idle":"2023-07-14T14:44:30.484065Z","shell.execute_reply.started":"2023-07-14T14:44:30.430546Z","shell.execute_reply":"2023-07-14T14:44:30.483157Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 3. Kaggle Medical Imaging Competitions
\n\nMedical image analysis has become a major research field in biomedical research, with thousands of papers published on various image analysis topics, including classification, object detection, and segmentation [5]. However, validation and evaluation of new methods were based on the authors’ personal data sets, rendering fair and direct comparison of the solutions impossible [6].\n\nTo tackle the fairness and biases in research, there has been a growing interest in organizing biomedical challenges. The first grand challenge in this domain was organized during the International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) in 2007[7]. This competition marked a significant turning point in research practices, leading to changes in how new methods are validated and evaluated.\n \nOver time, research practice began to change, and the number of challenges organized annually has been increasing steadily. Since 2018, Kaggle has become a prestigious platform for hosting many medical imaging competitions with grand prize reward. These competitions often involve classification, object detection, and segmentation using medical images shown in Figure 1. By hosting these competitions on Kaggle, researchers and data scientists are provided with standardized datasets, evaluation metrics, and a common platform for benchmarking their algorithms. This approach enables fair and direct comparisons between different algorithms, fostering advancements in medical image analysis. ","metadata":{}},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"## 3.1 Object Detection
\n\nMedical object detection is the task of identifying medical-based objects within an image. Object detection is an individual classification of the image’s pixels, where the objects present in the image are located. The detection task requires the generation of Regions Of Interest (ROI) containing the objects. In medical imaging, these objects can correspond to anatomical structures (e.g. organs) or anomalies (e.g. pulmonary nodules) [12].\n\n**SIIM-FISABIO-RSNA COVID-19 Detection**\n\n**Goal**: identify and localize COVID-19 abnormalities on chest radiographs.\n\n**Image Modality**: CT scans. A Computerized Tomography scan (CT or CAT scan) uses computers and rotating X-ray machines to create cross-sectional images of the body. These images provide more detailed information than normal X-ray images. They can show the soft tissues, blood vessels, and bones in various parts of the body [[8]](https://www.kaggle.com/code/andradaolteanu/siim-covid-19-box-detect-dcm-metadata).\n\n**Dates**: May 17, 2021 to Aug 9, 2021\n\n**1305** teams, **1786** competitors, **32,307** submissions\n\n**Evaluation**: mAP\n\n**Reward**: $100,000\n\n","metadata":{}},{"cell_type":"markdown","source":"## 3.2 Classification
\n\nMedical Image Classification is a task in medical image analysis that involves classifying medical images, such as X-rays, MRI scans, and CT scans, into different categories based on the type of image or the presence of specific structures or diseases. The goal is to use computer algorithms to automatically identify and classify medical images based on their content, which can help in diagnosis, treatment planning, and disease monitoring [11].\n\n**RSNA Screening Mammography Breast Cancer Detection**\n\n**Goal**: identify breast cancer.\n\n**Image Modality**: low-energy X-rays to examine the human breast for diagnosis and screening\n\n**Dates**: Nov 28, 2022 to Feb 27, 2023\n\n**1,687** teams, **2,146** competitors, **45,911** submissions\n\n**Evaluation**: pF1\n\n**Reward**: $50,000\n\n","metadata":{}},{"cell_type":"markdown","source":"## 3.3 Segmentation
\n\nImage segmentation is a computer vision task in which we label specific regions of pixels in an image with their corresponding classes. Medical image segmentation is a crucial example of this domain and offers numerous benefits for clinical use.\n\nImage segmentation tasks can be classified into two categories: semantic segmentation and instance segmentation [9], [10]. Semantic segmentation is the process of labeling one or more specific regions of interest in an image. This process treats multiple objects within a single category as one entity. In contrast, instance segmentation is the process of detecting and delineating each object of interest in an image. This process is a combination of object detection and semantic segmentation. However, it differs from semantic segmentation because it gives a unique label to every instance of a particular object in the image.","metadata":{}},{"cell_type":"markdown","source":"**UW-Madison GI Tract Image Segmentation**\n\n**Goal**: segment the stomach and intestines on MRI scans.\n\n**Image Modality**: MRI scans. Magnetic resonance imaging (MRI) is a medical imaging technique used in radiology to form pictures of the anatomy and the physiological processes of the body. MRI scanners use strong magnetic fields, magnetic field gradients, and radio waves to generate images of the organs in the body.\n\n**Dates**: Apr 14, 2022 to July 14, 2022\n\n**1,548** teams, **2,078** competitors, **40,956** submissions\n\n**Evaluation**: mean Dice coefficient\n\n**Reward**: $25,000\n\n","metadata":{}},{"cell_type":"markdown","source":"# 4. Deep Learning Models in Medical Imaging
\n\nDeep learning methods have emerged as a dominant force in medical imaging analysis, revolutionizing various aspects of the discipline. Convolutional neural networks (CNNs) have played a pivotal role and remained at the forefront of research and development. Prominent CNN architectures such as ResNets, EfficientNets, and DenseNets have shown great performance in classification. YOLOs and R-CNNs have proven their effectiveness in object detection, while U-Nets dominate segmentation shown in Figure 2, 3. In contrast, RNN architectures like LSTM and GRU have shown great performance in analyzing sequential medical images, such as video or dynamic medical imaging modalities like functional MRI (fMRI) shown in Figure 2, 3. Although Vision Transformers have recently emerged and replaced convolutions with a complex attention mechanism, which has exceeded the performance of CNNs in many tasks, they need enormous amounts of training data, even more than CNNs. Due to the lack of medical image data, they are not commonly used in medical imaging competitions shown in Figure 3. Nevertheless, the potential of Vision Transformers in medical imaging analysis remains promising. As the availability of annotated medical image datasets increases, these models may become more prevalent and demonstrate their full capabilities in solving challenging medical imaging tasks.\n\n","metadata":{}},{"cell_type":"code","source":"# Figure 1: Distribution of models used in top solutions\nmodels = medical_competitions['Model type'].value_counts().sort_values(ascending=False).reset_index()\nmodels.columns = ['Model type','Count']\nmodels['Count'] = np.round(models['Count']/models['Count'].sum()*100,2)\n\nfig = px.bar(models, x='Model type', y='Count', \n text='Count',\n color_discrete_sequence=['#00b4d8'],\n height=500, width=900,\n title='Figure 3: Overview of Top Model Types in Medical Imaging Competitions',\n template='plotly_white')\nfig.update_traces(textposition='outside')\nfig.update_xaxes(title='Model Type')\nfig.update_yaxes(title='Percentage %')\nfig.update_traces(width=0.50,marker_line_color = 'black', marker_line_width = 2)\nfig.update_layout(title_y=0.02, title_x=0.1)\nfig.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-07-14T14:44:38.098054Z","iopub.execute_input":"2023-07-14T14:44:38.098668Z","iopub.status.idle":"2023-07-14T14:44:40.083643Z","shell.execute_reply.started":"2023-07-14T14:44:38.09863Z","shell.execute_reply":"2023-07-14T14:44:40.082476Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Over the past 5 years, CNN and UNet models have emerged as powerful tools in numerous medical competitions, consistently delivering remarkable results, as shown in Figure 4, 5. Specifically, CNNs have primarily been employed for classification and object detection tasks since they are excellent feature extractors. Therefore, it can be utilized to classify medical images and avoid complicated and expensive feature engineering [3]. In contrast, UNet, the most widespread image segmentation architecture, has achieved tremendous attention from academic and industrial researchers [4]. As a result, UNets have emerged as highly effective tools for segmentation tasks across various medical image modalities, as shown in Figure 5.","metadata":{}},{"cell_type":"code","source":"# Figure 1: Distribution of models used in top solutions\nmodels = medical_competitions.groupby(['Model type','Year'], as_index=False)['Model type'].value_counts()\nmodels.columns = ['Model type','Year','Count']\nmodels['Count'] = np.round(models['Count']/models['Count'].sum()*100,2)\n\n\nfig = px.bar(models, x='Year', y='Count', color='Model type',\n barmode='group',color_discrete_sequence=['#00b4d8','#edede9','#d6ccc2',\n '#f5ebe0','#fb8b24' ],\n height=500, width=800,opacity=0.9,\n title='Figure 4: Distribution of top model types used in top 10% solutions from 2018 to 2023',\n template='plotly_white')\nfig.update_xaxes(title='Year')\nfig.update_yaxes(title='Percentage %')\nfig.update_traces(width=0.12,marker_line_color = 'black', marker_line_width = 2)\nfig.update_layout(bargap=0.4,\n legend_title_text='',\n legend=dict(orientation='h',\n yanchor=\"top\",\n y=1.1,\n xanchor=\"left\",\n x=0.0\n ),\n title_y=0.02, title_x=0.01\n\n )\nfig.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-07-14T14:44:40.85699Z","iopub.execute_input":"2023-07-14T14:44:40.857389Z","iopub.status.idle":"2023-07-14T14:44:41.026809Z","shell.execute_reply.started":"2023-07-14T14:44:40.857361Z","shell.execute_reply":"2023-07-14T14:44:41.025809Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Figure 1: Distribution of models used in top solutions\nmodels = medical_competitions.groupby(['Model type','Task'], as_index=False)['Model type'].value_counts()\nmodels.columns = ['Model type','Task','Count']\nmodels['Count'] = np.round(models['Count']/models['Count'].sum()*100,2)\n\nfig = px.bar(models, x='Task', y='Count', color='Model type',\n barmode='group',color_discrete_sequence=['#00b4d8','#edede9','#d6ccc2',\n '#f5ebe0','#fb8b24' ],\n height=500, width=800,opacity=0.9,\n title='Figure 5: Distribution of top model types used in top 10% solutions in different tasks',\n template='plotly_white')\n\nfig.update_xaxes(title='Model type')\nfig.update_yaxes(title='Percentage %')\n\nfig.update_traces(width=0.12,marker_line_color = 'black', marker_line_width = 2)\nfig.update_layout(bargap=0.4,\n legend_title_text='',\n legend=dict(orientation='h',\n yanchor=\"top\",\n y=1.1,\n xanchor=\"left\",\n x=0.0\n ),\n title_y=0.02, title_x=0.01\n )\nfig.show()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-07-14T14:44:43.849424Z","iopub.execute_input":"2023-07-14T14:44:43.850118Z","iopub.status.idle":"2023-07-14T14:44:43.989972Z","shell.execute_reply.started":"2023-07-14T14:44:43.850082Z","shell.execute_reply":"2023-07-14T14:44:43.988998Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 5. Conclusion
\n\nIn this report, I presented an overview of the advancement of AI in Kaggle medical imaging competitions. Over the past few years, Kaggle has provided a great platform with various real-life medical image datasets to benchmark algorithms. Deep learning methods, particularly convolutional neural networks (CNNs) and UNet models, have shown their effectiveness in dealing with various medical image modalities such as MRI, CT scans, and X-rays in these competitions. Recently, the emergence of Vision Transformers has shown some promising applications, including medical image analysis. Nevertheless, despite their potentials, Vision Transformers are not widely used in medical imaging competitions.\n\n","metadata":{}},{"cell_type":"markdown","source":"# References
\n\n[[1] Medical Imaging](https://en.wikipedia.org/wiki/Medical_imaging)\n\n[[2] Review and Prospect: Artificial Intelligence in Advanced Medical Imaging](https://www.frontiersin.org/articles/10.3389/fradi.2021.781868/full)\n\n[[3] Deep convolutional neural network based medical image classification for disease diagnosis](https://journalofbigdata.springeropen.com/articles/10.1186/s40537-019-0276-2)\n\n[[4] Medical Image Segmentation Review: The Success of U-Net](https://arxiv.org/pdf/2211.14830.pdf)\n\n[[5] 20th anniversary of the Medical Image Analysis journal (MedIA)](https://pubmed.ncbi.nlm.nih.gov/27503079/)\n\n[[6] Anything you can do, I can do better (No you can't)](https://www.sciencedirect.com/science/article/abs/pii/0734189X86900836?via%3Dihub)\n\n[[7] Comparison and evaluation of methods for liver segmentation from CT datasets](https://pubmed.ncbi.nlm.nih.gov/19211338/)\n\n[[8] 😷SIIM Covid-19: Box Detect & .dcm metadata](https://www.kaggle.com/code/andradaolteanu/siim-covid-19-box-detect-dcm-metadata)\n\n[[9] Deep Semantic Segmentation of Natural and Medical Images: A Review](https://arxiv.org/abs/1910.07655)\n\n[[10] Image Segmentation Using Deep Learning: A Survey](https://arxiv.org/abs/2001.05566)\n\n[[11] Medical Image Classification](https://paperswithcode.com/task/medical-image-classification#:~:text=Medical%20Image%20Classification%20is%20a,of%20specific%20structures%20or%20diseases.)\n\n[[12] Detection and segmentation in medical imaging: types of deep learning models](https://www.imaios.com/en/resources/blog/introduction-to-deep-learning-model-types-for-object-detection-in-medical-imaging)","metadata":{}},{"cell_type":"code","source":"# Submission\nsubmission_df = pd.read_csv(\"/kaggle/input/2023-kaggle-ai-report/sample_submission.csv\")\nsubmission_df.loc[0]['value']='Kaggle Competitions'\nsubmission_df.loc[1]['value']='https://www.kaggle.com/code/nghihuynh/kaggle-ai-report-medical-imaging-competitions'\nsubmission_df.loc[2]['value']='https://www.kaggle.com/code/ahsuna123/kaggle-ai-report-healthcare-surge/comments#2336493'\nsubmission_df.loc[3]['value']='https://www.kaggle.com/code/bnzn261029/kaggle-competitions-reflect-the-development-of-ai/comments#2344474'\nsubmission_df.loc[4]['value']='https://www.kaggle.com/code/omarrajaa/image-data-report/comments#2344544'\n\nsubmission_df.to_csv('submission.csv', index=False)\nsubmission_df.head()","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-07-14T15:11:28.922977Z","iopub.execute_input":"2023-07-14T15:11:28.923417Z","iopub.status.idle":"2023-07-14T15:11:28.943539Z","shell.execute_reply.started":"2023-07-14T15:11:28.923387Z","shell.execute_reply":"2023-07-14T15:11:28.942166Z"},"trusted":true},"execution_count":null,"outputs":[]}]}