{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# <font color='313187'>Notebook Index<font><a class='anchor' id='top'></a>    \n## - [Libraries and plotting presets](#Libraries)\n    \n    \n## - [EDA: About tabular data](#Data)\n#### - [Patients diagnosis by age](#Cancer)\n#### - [Patients invasivity cancer by age](#Invasivity)\n    \n    \n## - [EDA: About image data](#DataImages)\n#### - [Data processing and plot function](#Processing)    \n#### - [Image of a Pacient: No cancer diagnosis with implants](#NoCI)\n#### - [Image of a Pacient: No cancer diagnose without implants](#NoCNoI)\n#### - [Image of a Pacient: Non invasive cancer diagnose with implants](#CNoII)\n#### - [Image of a Pacient: Non invasive cancer diagnose without implants](#CNoINoI)\n#### - [Image of a Pacient: Invasive cancer diagnose with implants](#CII)\n#### - [Image of a Pacient: Invasive cancer diagnose without implants](#CINoI)","metadata":{}},{"cell_type":"markdown","source":"# <font color='313187'>Libraries and plotting presets<font><a class='anchor' id='Libraries'></a> [↑](#top)","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport matplotlib.pyplot as plt\nimport numpy as np\nimport os\nimport glob\nimport cv2","metadata":{"execution":{"iopub.status.busy":"2022-12-01T15:45:17.596853Z","iopub.execute_input":"2022-12-01T15:45:17.59718Z","iopub.status.idle":"2022-12-01T15:45:17.603629Z","shell.execute_reply.started":"2022-12-01T15:45:17.597155Z","shell.execute_reply":"2022-12-01T15:45:17.602239Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"colors = [\"#FFEDE9\", \"#FFE8EF\", \"#E73193\", \"#E70090\", \"#E70067\"]\n\nbackGround = colors[0]\nbackGround2 = colors[1]\n_c1 = colors[2]\n_c2 = colors[4]\nfont = colors[3]","metadata":{"execution":{"iopub.status.busy":"2022-12-01T15:45:17.611776Z","iopub.execute_input":"2022-12-01T15:45:17.613687Z","iopub.status.idle":"2022-12-01T15:45:17.620408Z","shell.execute_reply.started":"2022-12-01T15:45:17.613632Z","shell.execute_reply":"2022-12-01T15:45:17.619368Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# <font color='313187'>EDA: About data<font><a class='anchor' id='Data'></a> [↑](#top)\n    \n#### Reading data (tabular data)","metadata":{}},{"cell_type":"code","source":"original_data = pd.read_csv(\"/kaggle/input/rsna-breast-cancer-detection/train.csv\")\noriginal_data.head(6)","metadata":{"execution":{"iopub.status.busy":"2022-12-01T15:45:17.631094Z","iopub.execute_input":"2022-12-01T15:45:17.63147Z","iopub.status.idle":"2022-12-01T15:45:17.695345Z","shell.execute_reply.started":"2022-12-01T15:45:17.631442Z","shell.execute_reply":"2022-12-01T15:45:17.693494Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"Shape of original data: \\n - {} rows \\n - {} columns\".format(original_data.shape[0], original_data.shape[1]))","metadata":{"execution":{"iopub.status.busy":"2022-12-01T15:45:17.696715Z","iopub.execute_input":"2022-12-01T15:45:17.696989Z","iopub.status.idle":"2022-12-01T15:45:17.704197Z","shell.execute_reply.started":"2022-12-01T15:45:17.696965Z","shell.execute_reply":"2022-12-01T15:45:17.702301Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### A little description of values *(you can found it on data description on competition site)*\n\n- `site_id` - ID code for the source hospital. **It seems irrelevant**\n- `patient_id` - ID code for the patient.\n- `image_id` - ID code for the image.\n- `laterality` - Whether the image is of the left or right breast.\n- `view` - The orientation of the image. The default for a screening exam is to capture two views per breast.\n- `age` - The patient's age in years.\n- `implant` - Whether or not the patient had breast implants. Site 1 only provides breast implant information at the patient level, not at the breast level.\n- `density` - A rating for how dense the breast tissue is, with A being the least dense and D being the most dense. Extremely dense tissue can make diagnosis more difficult.\n- `machine_id` - An ID code for the imaging device.\n- `cancer` - The target value. Only provided for train.\n- `biopsy` - Whether or not a follow-up biopsy was performed on the breast. \n- `invasive` - If the breast is positive for cancer, whether or not the cancer proved to be invasive.\n- `BIRADS` - 0 if the breast required follow-up, 1 if the breast was rated as negative for cancer, and 2 if the breast was rated as normal.\n- `difficult_negative_case` - True if the case was unusually difficult. Only provided for train.\n","metadata":{}},{"cell_type":"code","source":"print(\"Hospital ID unique values: {}\".format( original_data[\"site_id\"].unique())    )\nprint(\"View unique values: {}\".format( original_data[\"view\"].unique())              )\nprint(\"Machine ID unique values: {}\".format( original_data[\"machine_id\"].unique())  )","metadata":{"execution":{"iopub.status.busy":"2022-12-01T15:45:17.706597Z","iopub.execute_input":"2022-12-01T15:45:17.706956Z","iopub.status.idle":"2022-12-01T15:45:17.731166Z","shell.execute_reply.started":"2022-12-01T15:45:17.706928Z","shell.execute_reply":"2022-12-01T15:45:17.729203Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Drop duplicates to analyze by unique patient not duplicated recorded rows","metadata":{}},{"cell_type":"code","source":"data = original_data.drop_duplicates(['patient_id'],keep='first')\ndata.head(6)","metadata":{"execution":{"iopub.status.busy":"2022-12-01T15:45:17.734073Z","iopub.execute_input":"2022-12-01T15:45:17.734423Z","iopub.status.idle":"2022-12-01T15:45:17.763284Z","shell.execute_reply.started":"2022-12-01T15:45:17.734368Z","shell.execute_reply":"2022-12-01T15:45:17.761897Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"Shape of data after drop duplicated values: \\n - {} rows\".format(data.shape[0]))","metadata":{"execution":{"iopub.status.busy":"2022-12-01T15:45:17.765172Z","iopub.execute_input":"2022-12-01T15:45:17.765564Z","iopub.status.idle":"2022-12-01T15:45:17.772355Z","shell.execute_reply.started":"2022-12-01T15:45:17.765537Z","shell.execute_reply":"2022-12-01T15:45:17.771308Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# <font color='313187'>Patients diagnosis by age<font><a class='anchor' id='Cancer'></a> [↑](#top)","metadata":{}},{"cell_type":"code","source":"CancerDF = data[data[\"cancer\"] == 1]\nnoCancerDF = data[data[\"cancer\"] == 0]\n\ngrouped_Cancer = CancerDF.groupby('age')['cancer','patient_id'].count()\ngrouped_noCancer = noCancerDF.groupby('age')['cancer','patient_id'].count()","metadata":{"execution":{"iopub.status.busy":"2022-12-01T15:45:17.774121Z","iopub.execute_input":"2022-12-01T15:45:17.77462Z","iopub.status.idle":"2022-12-01T15:45:17.799467Z","shell.execute_reply.started":"2022-12-01T15:45:17.774583Z","shell.execute_reply":"2022-12-01T15:45:17.797431Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, ax= plt.subplots(figsize= (20, 25))\n\nfig.patch.set_facecolor(backGround)\nax.set_facecolor(backGround)\nax.spines.left.set_visible(False)\nax.spines.right.set_visible(False)\nax.spines.top.set_visible(False)\n\nplt.barh(grouped_noCancer.index, grouped_noCancer[\"cancer\"].values, color=_c1, edgecolor='black')\nplt.barh(grouped_Cancer.index, grouped_Cancer[\"cancer\"].values,  color=_c2, edgecolor='black')\n\nplt.grid(axis='x', alpha=0.2)\nplt.xlim(0, 600)\nplt.xticks([100, 200, 300, 400, 500, 600])\nplt.yticks(grouped_noCancer.index)\nplt.margins(x=0.5, y=0.025)\n\nplt.legend(['Patients with NO cancer diagnosis', 'Patients with cancer diagnosis'], fontsize=12, facecolor=backGround2, loc=5)\nplt.title(\"\\nPatients diagnosis by age\\n\", fontsize=25, pad=20, color=font, family=\"monospace\") \nplt.text(420, 86, \"Total Patients on analysis: {}\".format(len(data[\"patient_id\"].unique())), fontsize=16, family=\"sans-serif\", alpha=0.9)\nplt.text(420, 85, \" - Patients with NO cancer diagnosis: {}\".format(grouped_noCancer[\"cancer\"].values.sum()), fontsize=14, family=\"sans-serif\", alpha=0.7)\nplt.text(420, 84, \" - Patients with cancer diagnosis: {}\".format(grouped_Cancer[\"cancer\"].values.sum()), fontsize=14, family=\"sans-serif\", alpha=0.7)\nplt.text(360, 25, \"Ismael Alvariño - RSNA Screening Mammography Breast Cancer Detection\", fontsize=12, family=\"sans-serif\", alpha=0.5)\n\nplt.show()\n\n#if savefig:\n#    fig.savefig(\"Name.png\", dpi=1500)","metadata":{"_kg_hide-input":false,"execution":{"iopub.status.busy":"2022-12-01T15:45:17.80088Z","iopub.execute_input":"2022-12-01T15:45:17.801519Z","iopub.status.idle":"2022-12-01T15:45:18.572056Z","shell.execute_reply.started":"2022-12-01T15:45:17.801492Z","shell.execute_reply":"2022-12-01T15:45:18.571081Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# <font color='313187'>Patients invasivity cancer by age<font><a class='anchor' id='Invasivity'></a> [↑](#top)","metadata":{}},{"cell_type":"code","source":"InvasiveDF = CancerDF[CancerDF[\"invasive\"] == 1]\nnoInvasiveDF = CancerDF[CancerDF[\"invasive\"] == 0]\n\ngrouped_InvasiveDF = InvasiveDF.groupby('age')['invasive'].count()\ngrouped_noInvasiveDF = noInvasiveDF.groupby('age')['invasive'].count()","metadata":{"execution":{"iopub.status.busy":"2022-12-01T15:45:18.573339Z","iopub.execute_input":"2022-12-01T15:45:18.573798Z","iopub.status.idle":"2022-12-01T15:45:18.582352Z","shell.execute_reply.started":"2022-12-01T15:45:18.57377Z","shell.execute_reply":"2022-12-01T15:45:18.58136Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, ax= plt.subplots(figsize= (20, 21))\n\nfig.patch.set_facecolor(backGround)\nax.set_facecolor(backGround)\nax.spines.left.set_visible(False)\nax.spines.right.set_visible(False)\nax.spines.top.set_visible(False)\n\nplt.barh(grouped_InvasiveDF.index, grouped_InvasiveDF.values, color=_c1, edgecolor='black')\nplt.barh(grouped_noInvasiveDF.index, grouped_noInvasiveDF.values,  color=_c2, edgecolor='black')\n\nplt.grid(axis='x', alpha=0.2)\nplt.xlim(0, 20)\nplt.xticks([5, 10, 15, 20])\nplt.yticks(grouped_InvasiveDF.index)\nplt.margins(x=0.5, y=0.025)\n\nplt.legend(['Patients with NO invasive cancer diagnosis', 'Patients with invasive cancer diagnosis'], fontsize=12, facecolor=backGround, loc=5)\nplt.title(\"\\nPatients invasivity cancer by age\\n\", fontsize=25, pad=20, color=font, family=\"sans-serif\") \nplt.text(14, 86, \"Total cancer diagnosed patients: {}\".format(len(CancerDF[\"patient_id\"].unique())), fontsize=16, family=\"sans-serif\", alpha=0.9)\nplt.text(14, 85, \" - Patients with NO cancer diagnosis: {}\".format(grouped_noInvasiveDF.values.sum()), fontsize=14, family=\"sans-serif\", alpha=0.7)\nplt.text(14, 84, \" - Patients with cancer diagnosis: {}\".format(grouped_InvasiveDF.values.sum()), fontsize=14, family=\"sans-serif\", alpha=0.7)\nplt.text(12, 37, \"Ismael Alvariño - RSNA Screening Mammography Breast Cancer Detection\", fontsize=12, family=\"sans-serif\", alpha=0.5)\n\nplt.show()\n\n#if savefig:\n#    fig.savefig(\"Name.png\", dpi=1500)","metadata":{"execution":{"iopub.status.busy":"2022-12-01T15:45:18.584526Z","iopub.execute_input":"2022-12-01T15:45:18.585257Z","iopub.status.idle":"2022-12-01T15:45:19.12Z","shell.execute_reply.started":"2022-12-01T15:45:18.585219Z","shell.execute_reply":"2022-12-01T15:45:19.119002Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# <font color='313187'>EDA: About image data<font><a class='anchor' id='DataImages'></a> [↑](#top)\n## <font color='313187'>Data processing and plot function<font><a class='anchor' id='Processing'></a> [↑](#top)","metadata":{}},{"cell_type":"code","source":"non_cancer = data[data[\"cancer\"] == 0][[\"patient_id\", \"cancer\", \"invasive\", \"implant\"]]\n\nnon_cancer_implants = non_cancer[non_cancer[\"implant\"] == 1]\nnon_cancer_NOimplants = non_cancer[non_cancer[\"implant\"] == 0]\n\n\ncancer = data[data[\"cancer\"] == 1]\n\nnon_invasive = cancer[cancer[\"invasive\"] == 0][[\"patient_id\", \"cancer\", \"invasive\", \"implant\"]]\nnon_invasive_implants = non_invasive[non_invasive[\"implant\"] == 1]\nnon_invasive_NOimplants = non_invasive[non_invasive[\"implant\"] == 0]\n\ninvasive = cancer[cancer[\"invasive\"] == 1][[\"patient_id\", \"cancer\", \"invasive\", \"implant\"]]\ninvasive_implants = invasive[invasive[\"implant\"] == 1]\ninvasive_NOimplants = invasive[invasive[\"implant\"] == 0]","metadata":{"execution":{"iopub.status.busy":"2022-12-01T15:45:19.12256Z","iopub.execute_input":"2022-12-01T15:45:19.123427Z","iopub.status.idle":"2022-12-01T15:45:19.140557Z","shell.execute_reply.started":"2022-12-01T15:45:19.123355Z","shell.execute_reply":"2022-12-01T15:45:19.139057Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"No cancer diagnose with implants: \\n{}\\n\".format(non_cancer_implants.head(3) ))\nprint(\"No cancer diagnose without implants: \\n{}\\n\".format(non_cancer_NOimplants.head(3) ))\nprint(\"Non invasive cancer diagnose with implants: \\n{}\\n\".format(non_invasive_implants.head(3) ))\nprint(\"Non invasive cancer diagnose without implants: \\n{}\\n\".format(non_invasive_NOimplants.head(3) ))\nprint(\"Invasive cancer diagnose with implants: \\n{}\\n\".format(invasive_implants.head(3) ))\nprint(\"Invasive cancer diagnose without implants: \\n{}\\n\".format(invasive_NOimplants.head(3) ))","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-12-01T15:45:19.142818Z","iopub.execute_input":"2022-12-01T15:45:19.143162Z","iopub.status.idle":"2022-12-01T15:45:19.163407Z","shell.execute_reply.started":"2022-12-01T15:45:19.143135Z","shell.execute_reply":"2022-12-01T15:45:19.162699Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def show_images_for_patient(patient_id):\n    P_DATA = data[data[\"patient_id\"] == patient_id]\n    \n    patient_dir = os.path.join('/kaggle/input/rsna-mammography-images-as-pngs/images_as_pngs_1024/train_images_processed_1024', str(patient_id))\n    num_images = len(glob.glob(f\"{patient_dir}/*\"))\n    \n    fig, axs = plt.subplots(2, 2, figsize=(15,15))\n    \n    plt.suptitle(\"Ismael Alvariño - RSNA Screening Mammography Breast Cancer Detection\", fontsize=12, family=\"sans-serif\", alpha=0.35, color=backGround)\n    fig.text(0.1,0.9, \"\\n Patient: {}\\n\\n  -- Age: {}\\n  -- Cancer: {}\\n  -- Invasive: {}\\n  -- Implant: {}\".format(int(P_DATA[\"patient_id\"]), int(P_DATA[\"age\"]), int(P_DATA[\"cancer\"]), int(P_DATA[\"invasive\"]), int(P_DATA[\"implant\"])) , \n             fontsize=15, color=backGround, ha='left', family=\"monospace\", alpha=0.8)\n    \n    axs = axs.flatten()\n    for i, img_path in zip( range(4), os.listdir(patient_dir) ):\n        \n        ds = cv2.imread(os.path.join( patient_dir, img_path) )\n        \n        fig.patch.set_facecolor(\"#000000\")\n        axs[i].set_facecolor(\"#000000\")\n        axs[i].spines.left.set_visible(False)\n        axs[i].spines.right.set_visible(False)\n        axs[i].spines.top.set_visible(False)\n        axs[i].spines.bottom.set_visible(False)\n        axs[i].get_xaxis().set_visible(False)\n        axs[i].get_yaxis().set_visible(False)\n\n        axs[i].imshow(cv2.cvtColor(ds,  cv2.COLOR_BGR2RGB))","metadata":{"execution":{"iopub.status.busy":"2022-12-01T15:45:28.235784Z","iopub.execute_input":"2022-12-01T15:45:28.236242Z","iopub.status.idle":"2022-12-01T15:45:28.252217Z","shell.execute_reply.started":"2022-12-01T15:45:28.236203Z","shell.execute_reply":"2022-12-01T15:45:28.250369Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## <font color='313187'>Image of a Pacient: No cancer diagnosis with implants<font><a class='anchor' id='NoCI'></a> [↑](#top)","metadata":{}},{"cell_type":"code","source":"try:\n    show_images_for_patient( non_cancer_implants[\"patient_id\"].tolist()[-1] )\nexcept:\n    print(\"There's no patients with thes features\")","metadata":{"execution":{"iopub.status.busy":"2022-12-01T15:45:28.254495Z","iopub.execute_input":"2022-12-01T15:45:28.25493Z","iopub.status.idle":"2022-12-01T15:45:29.419812Z","shell.execute_reply.started":"2022-12-01T15:45:28.254865Z","shell.execute_reply":"2022-12-01T15:45:29.418299Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## <font color='313187'>Image of a Pacient: No cancer diagnose without implants<font><a class='anchor' id='NoCNoI'></a> [↑](#top)","metadata":{}},{"cell_type":"code","source":"try:\n    show_images_for_patient( non_cancer_NOimplants[\"patient_id\"].tolist()[-2] )\nexcept:\n    print(\"There's no patients with thes features\")","metadata":{"execution":{"iopub.status.busy":"2022-12-01T15:45:29.421131Z","iopub.execute_input":"2022-12-01T15:45:29.421451Z","iopub.status.idle":"2022-12-01T15:45:30.618562Z","shell.execute_reply.started":"2022-12-01T15:45:29.421425Z","shell.execute_reply":"2022-12-01T15:45:30.617031Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## <font color='313187'>Image of a Pacient: Non invasive cancer diagnose with implants<font><a class='anchor' id='CNoII'></a> [↑](#top)","metadata":{}},{"cell_type":"code","source":"try:\n    show_images_for_patient( non_invasive_implants[\"patient_id\"].tolist()[-1] )\nexcept:\n    print(\"There's no patients with thes features\")","metadata":{"execution":{"iopub.status.busy":"2022-12-01T15:45:30.620338Z","iopub.execute_input":"2022-12-01T15:45:30.620718Z","iopub.status.idle":"2022-12-01T15:45:30.628078Z","shell.execute_reply.started":"2022-12-01T15:45:30.620691Z","shell.execute_reply":"2022-12-01T15:45:30.626469Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## <font color='313187'>Image of a Pacient: Non invasive cancer diagnose without implants <font><a class='anchor' id='CNoINoI'></a> [↑](#top)","metadata":{}},{"cell_type":"code","source":"try:\n    show_images_for_patient( non_invasive_NOimplants[\"patient_id\"].tolist()[-1] )\nexcept:\n    print(\"There's no patients with thes features\")","metadata":{"execution":{"iopub.status.busy":"2022-12-01T15:45:30.629757Z","iopub.execute_input":"2022-12-01T15:45:30.6301Z","iopub.status.idle":"2022-12-01T15:45:31.747672Z","shell.execute_reply.started":"2022-12-01T15:45:30.630064Z","shell.execute_reply":"2022-12-01T15:45:31.74653Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## <font color='313187'>Image of a Pacient: Invasive cancer diagnose with implants <font><a class='anchor' id='CII'></a> [↑](#top)","metadata":{}},{"cell_type":"code","source":"try:\n    show_images_for_patient( invasive_implants[\"patient_id\"].tolist()[-1] )\nexcept:\n    print(\"There's no patients with thes features\")","metadata":{"execution":{"iopub.status.busy":"2022-12-01T15:45:31.748883Z","iopub.execute_input":"2022-12-01T15:45:31.74978Z","iopub.status.idle":"2022-12-01T15:45:32.808187Z","shell.execute_reply.started":"2022-12-01T15:45:31.749745Z","shell.execute_reply":"2022-12-01T15:45:32.80684Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## <font color='313187'>Image of a Pacient: Invasive cancer diagnose without implants<font><a class='anchor' id='CINoI'></a> [↑](#top)","metadata":{}},{"cell_type":"code","source":"try:\n    show_images_for_patient( invasive_NOimplants[\"patient_id\"].tolist()[-1] )\nexcept:\n    print(\"There's no patients with thes features\")","metadata":{"execution":{"iopub.status.busy":"2022-12-01T15:45:32.809685Z","iopub.execute_input":"2022-12-01T15:45:32.810033Z","iopub.status.idle":"2022-12-01T15:45:33.921958Z","shell.execute_reply.started":"2022-12-01T15:45:32.81Z","shell.execute_reply":"2022-12-01T15:45:33.920941Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### That's all now for this third version, in future versions I want to continue exploring the tabular data, image data and why not try to get a good psition applying deep learning to predict results.\n\n#### **Upvote if you liked for more updates!**\n#### **Thanks for the feedBack!**","metadata":{}}]}