{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"

If you liked the notebook please upvote it.

","metadata":{}},{"cell_type":"markdown","source":"

About Dataset

\n

site_id - ID code for the source hospital.

\n

patient_id - ID code for the patient.

\n

pimage_id - ID code for the image.

\n

laterality - Whether the image is of the left or right breast.

\n

view - The orientation of the image. The default for a screening exam is to capture two views per breast.

\n

age - The patient's age in years.

\n

implant - Whether or not the patient had breast implants. Site 1 only provides breast implant information at the patient level, not at the breast level.

\n

density - A rating for how dense the breast tissue is, with A being the least dense and D being the most dense. Extremely dense tissue can make diagnosis more difficult. Only provided for train.

\n

machine_id - An ID code for the imaging device.

\n

cancer - Whether or not the breast was positive for malignant cancer. The target value. Only provided for train.

\n

biopsy - Whether or not a follow-up biopsy was performed on the breast. Only provided for train.

\n

invasive - If the breast is positive for cancer, whether or not the cancer proved to be invasive. Only provided for train.

\n

BIRADS - 0 if the breast required follow-up, 1 if the breast was rated as negative for cancer, and 2 if the breast was rated as normal. Only provided for train.

\n

prediction_id - The ID for the matching submission row. Multiple images will share the same prediction ID. Test only.

\n

difficult_negative_case - True if the case was unusually difficult. Only provided for train.

","metadata":{}},{"cell_type":"code","source":"# Importing the necessary librarues\nimport pandas as pd\nimport numpy as np\nimport warnings \nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport missingno as msno\nwarnings.filterwarnings(\"ignore\")","metadata":{"execution":{"iopub.status.busy":"2023-02-17T03:31:48.97989Z","iopub.execute_input":"2023-02-17T03:31:48.980252Z","iopub.status.idle":"2023-02-17T03:31:48.985808Z","shell.execute_reply.started":"2023-02-17T03:31:48.98022Z","shell.execute_reply":"2023-02-17T03:31:48.984677Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#Loading the data\ntrain=pd.read_csv(\"/kaggle/input/rsna-breast-cancer-detection/train.csv\")\ntest=pd.read_csv(\"/kaggle/input/rsna-breast-cancer-detection/test.csv\")\nsubmission=pd.read_csv(\"/kaggle/input/rsna-breast-cancer-detection/sample_submission.csv\")","metadata":{"execution":{"iopub.status.busy":"2023-02-17T03:31:49.267431Z","iopub.execute_input":"2023-02-17T03:31:49.267752Z","iopub.status.idle":"2023-02-17T03:31:49.376237Z","shell.execute_reply.started":"2023-02-17T03:31:49.267724Z","shell.execute_reply":"2023-02-17T03:31:49.375264Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# showing the first five rows of the training data\ntrain.head()","metadata":{"execution":{"iopub.status.busy":"2023-02-17T03:31:49.772744Z","iopub.execute_input":"2023-02-17T03:31:49.773883Z","iopub.status.idle":"2023-02-17T03:31:49.798254Z","shell.execute_reply.started":"2023-02-17T03:31:49.773842Z","shell.execute_reply":"2023-02-17T03:31:49.797301Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(\"The shape of the training data is:\",train.shape)\nprint(\"Total number of unique patients:\",train['patient_id'].nunique())","metadata":{"execution":{"iopub.status.busy":"2023-02-17T03:31:50.025324Z","iopub.execute_input":"2023-02-17T03:31:50.026711Z","iopub.status.idle":"2023-02-17T03:31:50.040081Z","shell.execute_reply.started":"2023-02-17T03:31:50.026654Z","shell.execute_reply":"2023-02-17T03:31:50.039126Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Training data is having 54706 observations but unique patients are 11913 it means lot of patients having more than one observation in the data.","metadata":{}},{"cell_type":"code","source":"msno.bar(train,sort=\"ascending\",figsize=(16,8),color='pink')","metadata":{"execution":{"iopub.status.busy":"2023-02-17T03:31:50.457119Z","iopub.execute_input":"2023-02-17T03:31:50.457474Z","iopub.status.idle":"2023-02-17T03:31:51.459035Z","shell.execute_reply.started":"2023-02-17T03:31:50.457444Z","shell.execute_reply":"2023-02-17T03:31:51.45813Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Bridas and density is having more than50% null values","metadata":{}},{"cell_type":"code","source":"#Adding new feature in the data which is having absolute path of the images\npath=\"/kaggle/input/rsna-breast-cancer-detection/train_images/\"\nli=[]\nfor i in range(len(train)):\n li.append(path+str(train[\"patient_id\"][i])+\"/\"+str(train[\"image_id\"][i])+\".dcm\")\ntrain['img_path']=li","metadata":{"execution":{"iopub.status.busy":"2023-02-17T03:31:51.461839Z","iopub.execute_input":"2023-02-17T03:31:51.462585Z","iopub.status.idle":"2023-02-17T03:31:52.159956Z","shell.execute_reply.started":"2023-02-17T03:31:51.462554Z","shell.execute_reply":"2023-02-17T03:31:52.158971Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import matplotlib.pyplot as plt\n\ndef pie_chart(data, variable, title):\n fig, ax = plt.subplots(figsize=(10, 8))\n fig.suptitle(title, fontsize=20)\n \n # Get the labels and sizes for the pie chart\n labels = list(data[variable].value_counts().index)\n sizes = data[variable].value_counts().values\n \n # Customize the pie chart's appearance\n colors = plt.cm.Set3.colors[:len(labels)]\n explode = [0.05] * len(labels)\n ax.pie(sizes, labels=labels, colors=colors, explode=explode, pctdistance=0.7, autopct=\"%1.1f%%\")\n circle = plt.Circle((0, 0), 0.4, fc=\"white\")\n ax.add_artist(circle)\n \n # Add a legend and customize its appearance\n ax.legend(labels, loc=\"best\", fontsize=12)\n for text in ax.legend().get_texts():\n text.set_text(text.get_text().capitalize())\n \n # Customize the plot's appearance\n ax.axis('equal')\n plt.tight_layout()\n \n # Display the plot\n plt.show()\n","metadata":{"execution":{"iopub.status.busy":"2023-02-17T03:31:52.16136Z","iopub.execute_input":"2023-02-17T03:31:52.161835Z","iopub.status.idle":"2023-02-17T03:31:52.17172Z","shell.execute_reply.started":"2023-02-17T03:31:52.161797Z","shell.execute_reply":"2023-02-17T03:31:52.170654Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pie_chart(train,\"site_id\",\"site\")","metadata":{"execution":{"iopub.status.busy":"2023-02-17T03:31:52.174326Z","iopub.execute_input":"2023-02-17T03:31:52.174876Z","iopub.status.idle":"2023-02-17T03:31:52.499232Z","shell.execute_reply.started":"2023-02-17T03:31:52.17484Z","shell.execute_reply":"2023-02-17T03:31:52.49792Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The site feature is having 54% observations from site 1 and 46% observations from site2","metadata":{}},{"cell_type":"code","source":"pie_chart(train,\"laterality\",\"laterality\")","metadata":{"execution":{"iopub.status.busy":"2023-02-17T03:31:52.501058Z","iopub.execute_input":"2023-02-17T03:31:52.501772Z","iopub.status.idle":"2023-02-17T03:31:52.818859Z","shell.execute_reply.started":"2023-02-17T03:31:52.501732Z","shell.execute_reply":"2023-02-17T03:31:52.817581Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Laterality is having almost half half data distributed between R and L.","metadata":{}},{"cell_type":"code","source":"pie_chart(train,\"view\",\"view\")","metadata":{"execution":{"iopub.status.busy":"2023-02-17T03:31:52.820814Z","iopub.execute_input":"2023-02-17T03:31:52.821527Z","iopub.status.idle":"2023-02-17T03:31:53.245002Z","shell.execute_reply.started":"2023-02-17T03:31:52.821486Z","shell.execute_reply":"2023-02-17T03:31:53.243989Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"In view feature MLO and CC is having the most number of observations.","metadata":{}},{"cell_type":"code","source":"pie_chart(train,\"cancer\",\"cancer\")","metadata":{"execution":{"iopub.status.busy":"2023-02-17T03:31:53.247685Z","iopub.execute_input":"2023-02-17T03:31:53.248063Z","iopub.status.idle":"2023-02-17T03:31:53.610855Z","shell.execute_reply.started":"2023-02-17T03:31:53.248025Z","shell.execute_reply":"2023-02-17T03:31:53.609751Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Data is hugely imbalanced only 2.1% observations is having cancer only.","metadata":{}},{"cell_type":"code","source":"pie_chart(train,\"biopsy\",\"biopsy\")","metadata":{"execution":{"iopub.status.busy":"2023-02-17T03:31:53.893702Z","iopub.execute_input":"2023-02-17T03:31:53.894052Z","iopub.status.idle":"2023-02-17T03:31:54.239218Z","shell.execute_reply.started":"2023-02-17T03:31:53.894023Z","shell.execute_reply":"2023-02-17T03:31:54.238232Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"only 5.4% patients went through the biopsy.","metadata":{}},{"cell_type":"code","source":"pie_chart(train,\"invasive\",\"invasive\")","metadata":{"execution":{"iopub.status.busy":"2023-02-17T03:31:54.810805Z","iopub.execute_input":"2023-02-17T03:31:54.811526Z","iopub.status.idle":"2023-02-17T03:31:55.159308Z","shell.execute_reply.started":"2023-02-17T03:31:54.811487Z","shell.execute_reply":"2023-02-17T03:31:55.158281Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"1.5% patients having invasive category cancer.","metadata":{}},{"cell_type":"code","source":"pie_chart(train,\"implant\",\"implant\")","metadata":{"execution":{"iopub.status.busy":"2023-02-17T03:31:55.813476Z","iopub.execute_input":"2023-02-17T03:31:55.81384Z","iopub.status.idle":"2023-02-17T03:31:56.15869Z","shell.execute_reply.started":"2023-02-17T03:31:55.81381Z","shell.execute_reply":"2023-02-17T03:31:56.157707Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"2.7% patients having breast implants","metadata":{}},{"cell_type":"code","source":"# ((train.groupby('implant')['cancer'].sum()/train['implant'].value_counts())*100).plot(kind='bar')\nimport matplotlib.pyplot as plt\n\n# Calculate the percentage of cancer cases for each implant type\ncancer_rate = (train.groupby('implant')['cancer'].sum() / train['implant'].value_counts()) * 100\n\n# Create a bar plot with customized settings\nplt.figure(figsize=(10, 6)) # set the figure size\nax = cancer_rate.plot(kind='bar', color='orange') # set the color of the bars\nax.set_title('Percentage of Cancer Cases by Implant Type', fontsize=16) # set the title and font size\nax.set_xlabel('Implant Type', fontsize=14) # set the x-axis label and font size\nax.set_ylabel('Cancer Rate (%)', fontsize=14) # set the y-axis label and font size\nax.tick_params(axis='x', labelrotation=0, labelsize=12) # customize the x-axis tick labels\nax.tick_params(axis='y', labelsize=12) # customize the y-axis tick labels\nplt.grid(axis='y', linestyle='--') # add grid lines to the y-axis\nplt.show() # di","metadata":{"execution":{"iopub.status.busy":"2023-02-17T03:31:57.063358Z","iopub.execute_input":"2023-02-17T03:31:57.064612Z","iopub.status.idle":"2023-02-17T03:31:57.259286Z","shell.execute_reply.started":"2023-02-17T03:31:57.064568Z","shell.execute_reply":"2023-02-17T03:31:57.258292Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Around 2% individuals having cancer who didn't go through breast implant whereas less than 1% individuals having cancer who went through breast implant.","metadata":{}},{"cell_type":"code","source":"train['age_bin']=pd.qcut(train['age'], 4,retbins=False, precision=3, duplicates='raise')\n\n# Calculate the percentage of cancer cases accroding to the age bin\ncancer_rate = (train.groupby('age_bin')['cancer'].sum() / train['age_bin'].value_counts()) * 100\n\n# Create a bar plot with customized settings\nplt.figure(figsize=(10, 6)) # set the figure size\nax = cancer_rate.plot(kind='bar', color='pink') # set the color of the bars\nax.set_title('Percentage of Cancer Cases by age group', fontsize=16) # set the title and font size\nax.set_xlabel('age group', fontsize=14) # set the x-axis label and font size\nax.set_ylabel('Cancer Rate (%)', fontsize=14) # set the y-axis label and font size\nax.tick_params(axis='x', labelrotation=0, labelsize=12) # customize the x-axis tick labels\nax.tick_params(axis='y', labelsize=12) # customize the y-axis tick labels\nplt.grid(axis='y', linestyle='--') # add grid lines to the y-axis\nplt.show() # di","metadata":{"execution":{"iopub.status.busy":"2023-02-17T03:31:59.357092Z","iopub.execute_input":"2023-02-17T03:31:59.357478Z","iopub.status.idle":"2023-02-17T03:31:59.634311Z","shell.execute_reply.started":"2023-02-17T03:31:59.357444Z","shell.execute_reply":"2023-02-17T03:31:59.633277Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"so age is playing role age_group(66-89) is having maximum 4% cancer rate whereas age group(25-51) is having least 1% cancer rate.","metadata":{}},{"cell_type":"code","source":"import seaborn as sns\nimport matplotlib.pyplot as plt\n\n# Set the figure size and theme\nplt.figure(figsize=(10,10))\nsns.set_theme(context=\"notebook\")\n\n# Create the distribution plot\nsns.distplot(train['age'], color='green', kde=True)\n\n# Add a vertical line for the mean value\nplt.axvline(train['age'].mean(), color='r')\n\n# Add text to display the mean value\nplt.text(train['age'].mean()+1, 0.03, 'Mean = {:.2f}'.format(train['age'].mean()), fontsize=14, color='r')\n\n# Customize the plot's appearance\nplt.title('Age Distribution', fontsize=18)\nplt.xlabel('Age', fontsize=14)\nplt.ylabel('Density', fontsize=14)\nplt.xticks(fontsize=12)\nplt.yticks(fontsize=12)\nsns.despine()\n\n# Display the plot\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-02-17T03:32:01.910842Z","iopub.execute_input":"2023-02-17T03:32:01.91121Z","iopub.status.idle":"2023-02-17T03:32:02.5252Z","shell.execute_reply.started":"2023-02-17T03:32:01.911176Z","shell.execute_reply":"2023-02-17T03:32:02.524278Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.head()","metadata":{"execution":{"iopub.status.busy":"2023-02-17T03:33:12.930025Z","iopub.execute_input":"2023-02-17T03:33:12.930544Z","iopub.status.idle":"2023-02-17T03:33:12.960424Z","shell.execute_reply.started":"2023-02-17T03:33:12.9305Z","shell.execute_reply":"2023-02-17T03:33:12.959442Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from pydicom.pixel_data_handlers import pillow_handler\npillow_handler.PillowJPEGTransferSyntaxes.append('1.2.840.10008.1.2.4.70')","metadata":{"execution":{"iopub.status.busy":"2023-02-17T03:41:14.871277Z","iopub.execute_input":"2023-02-17T03:41:14.872059Z","iopub.status.idle":"2023-02-17T03:41:14.877235Z","shell.execute_reply.started":"2023-02-17T03:41:14.872018Z","shell.execute_reply":"2023-02-17T03:41:14.876155Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import pydicom\nimport matplotlib.pyplot as plt\nfrom PIL import Image\n\nfig, axs = plt.subplots(2, 5, figsize=(12,8))\naxs = axs.flatten()\n\nfor k, path in enumerate(train['img_path'][:10]):\n axs[k].set_title(\"Images\", fontsize = 16, weight='bold')\n\n img = pydicom.dcmread(path).pixel_array\n img = Image.fromarray(img).resize((224,224))\n axs[k].imshow(img, cmap=\"turbo\")\n axs[k].axis(\"off\")\n\nplt.tight_layout()\nplt.show()\n","metadata":{"execution":{"iopub.status.busy":"2023-02-17T04:28:43.637362Z","iopub.execute_input":"2023-02-17T04:28:43.638092Z","iopub.status.idle":"2023-02-17T04:28:52.712228Z","shell.execute_reply.started":"2023-02-17T04:28:43.638056Z","shell.execute_reply":"2023-02-17T04:28:52.711333Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from PIL import Image\n\ndef view_image(train, variable,view, image_size=(224, 224)):\n fig, axs = plt.subplots(1, 5, figsize=(12,8))\n axs = axs.flatten()\n img_path = train[train[variable] == view][:5]\n for k, path in enumerate(img_path['img_path']):\n axs[k].set_title(\"Images\", fontsize=16, weight='bold')\n img = pydicom.dcmread(path).pixel_array\n img = Image.fromarray(img).resize(image_size)\n axs[k].imshow(img, cmap=\"turbo\", aspect=\"equal\")\n axs[k].axis(\"off\")\n plt.tight_layout()\n plt.show()\n","metadata":{"execution":{"iopub.status.busy":"2023-02-17T04:34:00.156806Z","iopub.execute_input":"2023-02-17T04:34:00.157167Z","iopub.status.idle":"2023-02-17T04:34:00.164303Z","shell.execute_reply.started":"2023-02-17T04:34:00.157136Z","shell.execute_reply":"2023-02-17T04:34:00.163349Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"view_image(train,\"view\",\"CC\")","metadata":{"execution":{"iopub.status.busy":"2023-02-17T04:34:07.632019Z","iopub.execute_input":"2023-02-17T04:34:07.632398Z","iopub.status.idle":"2023-02-17T04:34:11.710227Z","shell.execute_reply.started":"2023-02-17T04:34:07.632345Z","shell.execute_reply":"2023-02-17T04:34:11.709319Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"view_image(train,\"view\",\"MLO\")","metadata":{"execution":{"iopub.status.busy":"2023-02-17T04:34:14.29136Z","iopub.execute_input":"2023-02-17T04:34:14.291745Z","iopub.status.idle":"2023-02-17T04:34:19.818252Z","shell.execute_reply.started":"2023-02-17T04:34:14.291712Z","shell.execute_reply":"2023-02-17T04:34:19.817292Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"

Work in Progress

","metadata":{}}]}