{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"#

📊 RSNA Analysis | EDA Guidelines 📊

\n
\n\n

\n\n

\n \n\n INFORMATION\n\n⚫️ site_id = ID code for the source hospital.\n\n⚫️ patient_id = ID code for the patient.\n\n\n⚫️ image_id = ID code for the image.\n \n⚫️ laterality = Whether the image is of the left or right breast.\n \n⚫️ view = The orientation of the image. The default for a screening exam is to capture two views per breast.\n \n⚫️ age = The patient's age in years.\n \n⚫️ implant = Whether or not the patient had breast implants. Site 1 only provides breast implant information at the patient level, not at the breast level..\n\n⚫️ density = A rating for how dense the breast tissue is, with A being the least dense and D being the most dense. Extremely dense tissue can make diagnosis more difficult. Only provided for train.\n\n \n⚫️ machine_id = An ID code for the imaging device.\n \n⚫️ cancer = Whether or not the breast was positive for cancer. The target value. Only provided for train..\n \n⚫️ biopsy = Whether or not a follow-up biopsy was performed on the breast. Only provided for train.. \n \n⚫️ invasive = If the breast is positive for cancer, whether or not the cancer proved to be invasive. Only provided for train.. \n\n \n⚫️ BIRADS = 0 if the breast required follow-up, 1 if the breast was rated as negative for cancer, and 2 if the breast was rated as normal. Only provided for train.. \n \n⚫️ prediction_id = The ID for the matching submission row. Multiple images will share the same prediction ID. Test only.. \n \n⚫️ difficult_negative_case = True if the case was unusually difficult. Only provided for train.. \n

\n \n","metadata":{}},{"cell_type":"markdown","source":"\n

Table Of Content

\n\n * [1. IMPORTING LIBRARIES](#1)\n \n* [2. LOADING DATASET](#2)\n \n* [3. EXPLORATORY SOME INFORMATION ABOUT DATASET](#3)\n\n* [4. DATA VISUALIZATION](#4)\n\n* [5. RESULT](#5)\n\n* [6. AUTHOR MESSAGE](#6)\n\n\n![](https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcTn9Loy1OmNg2RQ_Wa2yBME-soZEZPDXimZYg&usqp=CAU)\n","metadata":{}},{"cell_type":"markdown","source":" Let's get started! ","metadata":{}},{"cell_type":"markdown","source":"\n#

IMPORTING LIBRARIES

","metadata":{}},{"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport seaborn as sns \nimport matplotlib.pyplot as plt\nimport missingno as msno\nimport plotly.graph_objs as go\nimport plotly.express as px\nplt.style.use('seaborn-dark')\nplt.style.context('grayscale')\n%matplotlib inline\nfrom wordcloud import WordCloud, STOPWORDS","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-12-09T14:50:44.357794Z","iopub.execute_input":"2022-12-09T14:50:44.358193Z","iopub.status.idle":"2022-12-09T14:50:44.367172Z","shell.execute_reply.started":"2022-12-09T14:50:44.358153Z","shell.execute_reply":"2022-12-09T14:50:44.366021Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n#

LOADING DATASET

","metadata":{}},{"cell_type":"code","source":"df=pd.read_csv(\"/kaggle/input/rsna-breast-cancer-detection/train.csv\")\ndf.head()","metadata":{"execution":{"iopub.status.busy":"2022-12-09T14:50:44.38932Z","iopub.execute_input":"2022-12-09T14:50:44.389722Z","iopub.status.idle":"2022-12-09T14:50:44.492179Z","shell.execute_reply.started":"2022-12-09T14:50:44.389688Z","shell.execute_reply":"2022-12-09T14:50:44.491274Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n#

EXPLORATORY SOME INFORMATION ABOUT DATASET

","metadata":{}},{"cell_type":"markdown","source":"# There are 7 main BI-RADS scores, or categories:\n# 0 - Need additional imaging evaluation\n# 1 - Negative\n# 2 - Benign\n# 3 - Probably Benign\n# 4 - Suspicious\n# 5 - Highly Suggestive of Malignancy\n# 6 - Known Biopsy-Proven Malignancy\n# Screening mammograms can only be assigned a BI-RADS score of 0, 1, or 2. This is why the dataset only contains 3 of the BI-RADS categories.","metadata":{}},{"cell_type":"markdown","source":" \n\n\n\n\n\n\n\n","metadata":{}},{"cell_type":"code","source":"df.info()","metadata":{"execution":{"iopub.status.busy":"2022-12-09T14:50:44.493653Z","iopub.execute_input":"2022-12-09T14:50:44.494683Z","iopub.status.idle":"2022-12-09T14:50:44.518805Z","shell.execute_reply.started":"2022-12-09T14:50:44.494642Z","shell.execute_reply":"2022-12-09T14:50:44.517691Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df.describe()","metadata":{"execution":{"iopub.status.busy":"2022-12-09T14:50:44.520185Z","iopub.execute_input":"2022-12-09T14:50:44.520508Z","iopub.status.idle":"2022-12-09T14:50:44.575776Z","shell.execute_reply.started":"2022-12-09T14:50:44.520476Z","shell.execute_reply":"2022-12-09T14:50:44.574593Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df.columns","metadata":{"execution":{"iopub.status.busy":"2022-12-09T14:50:44.577898Z","iopub.execute_input":"2022-12-09T14:50:44.578222Z","iopub.status.idle":"2022-12-09T14:50:44.585783Z","shell.execute_reply.started":"2022-12-09T14:50:44.578191Z","shell.execute_reply":"2022-12-09T14:50:44.584515Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df.isnull().sum()","metadata":{"execution":{"iopub.status.busy":"2022-12-09T14:50:44.587076Z","iopub.execute_input":"2022-12-09T14:50:44.587568Z","iopub.status.idle":"2022-12-09T14:50:44.609456Z","shell.execute_reply.started":"2022-12-09T14:50:44.587523Z","shell.execute_reply":"2022-12-09T14:50:44.608026Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n#

DATA VISUALIZATION

","metadata":{}},{"cell_type":"code","source":"msno.bar(df, figsize = (16,5),color = \"magenta\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-12-09T14:50:44.807742Z","iopub.execute_input":"2022-12-09T14:50:44.808129Z","iopub.status.idle":"2022-12-09T14:50:45.699881Z","shell.execute_reply.started":"2022-12-09T14:50:44.808097Z","shell.execute_reply":"2022-12-09T14:50:45.698665Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":" Let's get started! ","metadata":{}},{"cell_type":"markdown","source":"# ⚫️ Used visualization to show missing data \n# ⚫️ There is a lack of birads category and density categories.\n","metadata":{}},{"cell_type":"code","source":"df['laterality'].value_counts().plot(kind='bar',figsize = (10, 5))\nplt.title('laterality')\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2022-12-09T15:04:18.172645Z","iopub.execute_input":"2022-12-09T15:04:18.17391Z","iopub.status.idle":"2022-12-09T15:04:18.314156Z","shell.execute_reply.started":"2022-12-09T15:04:18.173856Z","shell.execute_reply":"2022-12-09T15:04:18.312984Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df['density'].value_counts().plot(kind='bar',figsize = (10, 5),color=\"lightblue\")\nplt.title('density')\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2022-12-09T15:05:31.593456Z","iopub.execute_input":"2022-12-09T15:05:31.593898Z","iopub.status.idle":"2022-12-09T15:05:31.748459Z","shell.execute_reply.started":"2022-12-09T15:05:31.593855Z","shell.execute_reply":"2022-12-09T15:05:31.747599Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df['machine_id'].value_counts().plot(kind='bar',figsize = (10, 5),color=\"yellow\")\nplt.title('machine_id')\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2022-12-09T15:06:42.177828Z","iopub.execute_input":"2022-12-09T15:06:42.178252Z","iopub.status.idle":"2022-12-09T15:06:42.374477Z","shell.execute_reply.started":"2022-12-09T15:06:42.178213Z","shell.execute_reply":"2022-12-09T15:06:42.373291Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df['view'].value_counts().plot(kind='bar',figsize = (10, 5),color=\"black\")\nplt.title('view')\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2022-12-09T15:08:00.600082Z","iopub.execute_input":"2022-12-09T15:08:00.600498Z","iopub.status.idle":"2022-12-09T15:08:00.775312Z","shell.execute_reply.started":"2022-12-09T15:08:00.600462Z","shell.execute_reply":"2022-12-09T15:08:00.774417Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"df['difficult_negative_case'].value_counts().plot(kind='bar',figsize = (10, 5),color=\"pink\")\nplt.title('difficult_negative_case')\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2022-12-09T15:11:15.317363Z","iopub.execute_input":"2022-12-09T15:11:15.318432Z","iopub.status.idle":"2022-12-09T15:11:15.450233Z","shell.execute_reply.started":"2022-12-09T15:11:15.318389Z","shell.execute_reply":"2022-12-09T15:11:15.44911Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sns.displot(df, x=\"BIRADS\", hue=\"biopsy\", multiple=\"dodge\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-12-09T14:50:45.702425Z","iopub.execute_input":"2022-12-09T14:50:45.703275Z","iopub.status.idle":"2022-12-09T14:50:46.283518Z","shell.execute_reply.started":"2022-12-09T14:50:45.703228Z","shell.execute_reply":"2022-12-09T14:50:46.282306Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"labels = df[\"biopsy\"].value_counts().index\nsizes = df[\"biopsy\"].value_counts()\ncolors = ['magenta','cyan',\"orange\",\"yellow\"]\nplt.figure(figsize = (8,8))\nplt.pie(sizes, labels=labels, rotatelabels=False, autopct='%1.1f%%',colors=colors,shadow=True, startangle=90)\nplt.title('Biopsy',color = 'green',fontsize = 15)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-12-09T14:50:46.284889Z","iopub.execute_input":"2022-12-09T14:50:46.285224Z","iopub.status.idle":"2022-12-09T14:50:46.447799Z","shell.execute_reply.started":"2022-12-09T14:50:46.285191Z","shell.execute_reply":"2022-12-09T14:50:46.446108Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.style.use(\"fivethirtyeight\")\nplt.figure(figsize=(10,10))\nplt.title(\"Battery power by price\")\nsns.set(font_scale=1)\nsns.barplot(data=df, y=\"cancer\",x=\"BIRADS\",palette=\"viridis\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-12-09T14:50:46.451938Z","iopub.execute_input":"2022-12-09T14:50:46.452962Z","iopub.status.idle":"2022-12-09T14:50:47.064269Z","shell.execute_reply.started":"2022-12-09T14:50:46.45289Z","shell.execute_reply":"2022-12-09T14:50:47.063048Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"age_distribution = df.query(\"cancer==1\").groupby(['age','cancer'])['cancer'].count().reset_index(name='count')\nage_distribution['cancer'] = age_distribution.cancer.map({0:'No Cancer',1:'Cancer'})\nfig = px.bar(age_distribution, x=\"age\", y=\"count\", color=\"age\",\n pattern_shape=\"age\", pattern_shape_sequence=[\".\", \"x\", \"\\\\\"],width=1000)\nfig.update_coloraxes(showscale=False)\nfig.update_layout(showlegend=False)\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-12-09T14:50:47.065836Z","iopub.execute_input":"2022-12-09T14:50:47.066212Z","iopub.status.idle":"2022-12-09T14:50:47.356885Z","shell.execute_reply.started":"2022-12-09T14:50:47.066179Z","shell.execute_reply":"2022-12-09T14:50:47.355404Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.subplots(figsize=(6, 6))\nsns.set_style('darkgrid')\nsns.set_palette('Pastel1')\n\ndata = [\n df[(df[\"cancer\"] == 0) & (df[\"BIRADS\"] == 0.0)][\"image_id\"].count(),\n df[(df[\"cancer\"] == 0) & (df[\"BIRADS\"] == 1.0)][\"image_id\"].count(),\n df[(df[\"cancer\"] == 0) & (df[\"BIRADS\"] == 2.0)][\"image_id\"].count(),\n]\ncounts = pd.DataFrame()\n_ = plt.pie(\n data, labels=[\"BIRADS 0\", \"BIRADS 1\", \"BIRADS 2\"],\n autopct=lambda x: \"{:,.0f} = {:.2f}%\".format(x * sum(data)/100, x),\n explode=[0.05] * 3, \n pctdistance=0.5, \n colors=sns.color_palette(\"Pastel1\")[0:3],\n)\n","metadata":{"execution":{"iopub.status.busy":"2022-12-09T14:50:47.358704Z","iopub.execute_input":"2022-12-09T14:50:47.35924Z","iopub.status.idle":"2022-12-09T14:50:47.505971Z","shell.execute_reply.started":"2022-12-09T14:50:47.359193Z","shell.execute_reply":"2022-12-09T14:50:47.504359Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"a,ax=plt.subplots(figsize=(12,12))\nsns.countplot(data=df,x=\"BIRADS\", saturation=.75, dodge=False,ax=ax,palette=\"terrain\")\nplt.xticks(rotation=90)\nplt.xlabel(\"BIRADS\",fontsize=20,color=\"blue\")\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-12-09T14:50:47.508241Z","iopub.execute_input":"2022-12-09T14:50:47.509812Z","iopub.status.idle":"2022-12-09T14:50:47.789546Z","shell.execute_reply.started":"2022-12-09T14:50:47.509707Z","shell.execute_reply":"2022-12-09T14:50:47.788445Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"\nfig = px.histogram(df, x=\"cancer\", y=\"density\", color=\"cancer\",barmode=\"relative\",\n marginal=\"violin\")\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-12-09T14:50:47.792077Z","iopub.execute_input":"2022-12-09T14:50:47.792506Z","iopub.status.idle":"2022-12-09T14:50:48.287849Z","shell.execute_reply.started":"2022-12-09T14:50:47.792474Z","shell.execute_reply":"2022-12-09T14:50:48.286957Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n","metadata":{}},{"cell_type":"code","source":"fig = px.scatter(df, x=\"age\", y=\"biopsy\")\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-12-09T14:50:48.2892Z","iopub.execute_input":"2022-12-09T14:50:48.289889Z","iopub.status.idle":"2022-12-09T14:50:48.365638Z","shell.execute_reply.started":"2022-12-09T14:50:48.289854Z","shell.execute_reply":"2022-12-09T14:50:48.364653Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = px.histogram(df, x = \"age\", marginal = \"box\", color_discrete_sequence = [\"#ff00ff\"],\n title = \"Age of women histogram ♀\")\n\nfig.update_layout(plot_bgcolor = \"#000000\", paper_bgcolor = \"#000000\", bargap = 0.3, font = dict(family = \"PT Sans\", size = 14, color = \"#FFFFFF\"))\nfig.update_traces(marker = dict(line = dict(width = 1.5, color = \"#FFFFFF\")))\nfig.update_yaxes(showgrid = True, gridwidth = 0.25, gridcolor = \"#646464\")\nfig.show()","metadata":{"execution":{"iopub.status.busy":"2022-12-09T15:14:56.131388Z","iopub.execute_input":"2022-12-09T15:14:56.132496Z","iopub.status.idle":"2022-12-09T15:14:56.273122Z","shell.execute_reply.started":"2022-12-09T15:14:56.132432Z","shell.execute_reply":"2022-12-09T15:14:56.271956Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize= (16, 8))\nsns.heatmap(df.corr(), annot = True, cmap= 'gnuplot2_r', fmt= '.1f');","metadata":{"execution":{"iopub.status.busy":"2022-12-09T15:26:56.131828Z","iopub.execute_input":"2022-12-09T15:26:56.132267Z","iopub.status.idle":"2022-12-09T15:26:57.125517Z","shell.execute_reply.started":"2022-12-09T15:26:56.132229Z","shell.execute_reply":"2022-12-09T15:26:57.124397Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n#

RESULT

\n\n# 🔵 In this data set, the number of people for whom biopsy is requested with the suspicion of cancer is in the minority.\n\n# 🔵 In BIRADS 1-2, we see that the possibility of cancer is almost nonexistent.\n\n# 🔵 We see that the risk of cancer increases in the age range of 50-70 years.\n\n# 🔵 We see that this data set mostly consists of BI RADS-1.\n\n# 🔵 Since the density b and c are too much in the dataset, the cancer rate seems higher, but it may not be right to say this because the dataset contains NaN values ​​too much.\n\n# 🔵 A patient under 30 years of age who underwent biopsy is not possible for this data set.","metadata":{}},{"cell_type":"markdown","source":"\n#

AUTHOR MESSAGE

\n","metadata":{}},{"cell_type":"markdown","source":" ⚪️ If you liked this Notebook, please do upvote. If you have any questions, feel free to comment! Best Wishes ⚪️ ","metadata":{}},{"cell_type":"markdown","source":"# https://www.linkedin.com/in/melike-dilekci-727224204","metadata":{}}]}