{"cells":[{"metadata":{},"cell_type":"markdown","source":"# **What Exactly is Cancer ?**\n\n\nCancer refers to any one of a large number of diseases characterized by the development of abnormal cells that divide uncontrollably and have the ability to infiltrate and destroy normal body tissue. Cancer often has the ability to spread throughout your body.\n\nCancer is the second-leading cause of death in the world. But survival rates are improving for many types of cancer, thanks to improvements in cancer screening and cancer treatment.\n\n![](http://cancernz.org.nz/assets/Cancer-information/Bowel-cancer-booklet/Bowel-Cancer2x-Page-05-Image-0002.jpg)\n\n\nNo Worries from Corona Now , Cancer is more Lethal \n\n\n\n**Symptoms**\n\nSigns and symptoms caused by cancer will vary depending on what part of the body is affected.\n\nSome general signs and symptoms associated with, but not specific to, cancer, include:\n\n*     Fatigue\n*     Lump or area of thickening that can be felt under the skin\n*     Weight changes, including unintended loss or gain\n*     Skin changes, such as yellowing, darkening or redness of the skin, sores that won't heal, or changes to existing moles\n*     Changes in bowel or bladder habits\n*     Persistent cough or trouble breathing\n*     Difficulty swallowing\n*     Hoarseness\n*     Persistent indigestion or discomfort after eating\n*     Persistent, unexplained muscle or joint pain\n*     Persistent, unexplained fevers or night sweats\n*     Unexplained bleeding or bruising\n\n\n**Causes**\n\nCancer is caused by changes (mutations) to the DNA within cells. The DNA inside a cell is packaged into a large number of individual genes, each of which contains a set of instructions telling the cell what functions to perform, as well as how to grow and divide. Errors in the instructions can cause the cell to stop its normal function and may allow a cell to become cancerous.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"\n\n\nfrom IPython.display import YouTubeVideo\nYouTubeVideo(\"WPgJafGz4fg\", height=500, width=700)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"After Knowing about Cancer In general Let's jump to our Very own Problem of Prostrate Cancer\n\n\n\n![](https://static.wixstatic.com/media/130396_1f6ada7a691f4ba8839b05c49e6a1604~mv2.jpg/v1/fill/w_626,h_290/130396_1f6ada7a691f4ba8839b05c49e6a1604~mv2.jpg) ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"markdown","source":"\n\nThe prostate is a small walnut shaped gland in the pelvis of men. It is located next to the bladder and can be examined by getting a digital rectal exam. Prostate cancer is a form of cancer that develops in the prostate gland. It is the second-leading cause of cancer deaths for men in the U.S. About 1 in 9 men will be diagnosed with prostate cancer in their lifetime. This year, nearly 191,000 men will be diagnosed with prostate cancer.\n\nGrowths in the prostate can be benign (not cancer) or malignant (cancer).\n\nBenign growths (like benign prostatic hyperplasia (BPH):\n\n*    Are rarely a threat to life\n*    Don't invade the tissues around them\n*    Don't spread to other parts of the body\n*    Can be removed and can grow back very slowly (but usually don't grow back)\n \n**Malignant growths (prostate cancer):**\n\n*    May sometimes be a threat to life\n*    Can spread to nearby organs and tissues (such as the bladder or rectum)\n*    Can spread (metastasize) to other parts of the body (like lymph nodes or bone) \n*    Often can be removed but sometimes grow back\n\nProstate cancer cells can spread by breaking away from a prostate tumor. They can travel through blood vessels or lymph nodes to reach other parts of the body. After spreading, cancer cells may attach to other tissues and grow to form new tumors, causing damage where they land.\n\nWhen prostate cancer spreads from its original place to another part of the body, the new tumor has the same kind of abnormal cells and the same name as the primary (original) tumor. For example, if prostate cancer spreads to the bones, the cancer cells in the bones are actually prostate cancer cells. The disease is metastatic prostate cancer, not bone cancer. For that reason, it's treated as prostate cancer in bone.\n\nTo understand prostate cancer, it helps to know how the prostate normally works.\n### The Prostate\n\nThe prostate   and seminal vesicles are part of the male reproductive system. The prostate is about the size of a walnut and weighs about one ounce. The seminal vesicles are two much smaller paired glands. These glands are attached to each side of the prostate. Some have said that the seminal vesicles look like rabbit ears attached to the prostate. The prostate is below the bladder and in front of the rectum. The prostate surrounds the urethra. The urethra is a tube that carries urine from the bladder out through the penis. This is why men with an enlarged prostate have difficulty urinating. It can disrupt the flow of urine from the bladder.\n\nThe main job of the prostate and seminal vesicles is to make fluid to bathe semen. During ejaculation, sperm is made in the testicles, and then moves to the urethra. At the same time, fluid from the prostate and the seminal vesicles also moves into the urethra. This mixture of semen and fluid from the prostate and seminal vesicles forms the ejaculate that passes through the urethra and out of the penis.\n\nWhen prostate cancer occurs, it starts in the prostate gland and occasionally spreads to the seminal vesicles.\n\n### Symptoms\n\nIn its early stages, prostate cancer often has no symptoms. When symptoms do occur, they can be like those of an enlarged prostate or BPH. Prostate cancer can also cause symptoms unrelated to BPH. If you have urinary problems, talk with your healthcare provider about them.\n\nSymptoms of prostate cancer can be:\n\n*    Dull pain in the lower pelvic area\n*    Frequent urinating\n*    Trouble urinating, pain, burning, or weak urine flow\n*    Blood in the urine (Hematuria) \n*    Painful ejaculation\n*    Pain in the lower back, hips or upper thighs\n*    Loss of appetite\n*    Loss of weight\n*    Bone pain\n\n\n###  Causes\n\nNo one knows why or how prostate cancer starts. Autopsy studies show 1 in 3 men over the age of 50 have some cancer cells in the prostate. Eight out of ten \"autopsy cancers\" found are small, with tumors that are not harmful.\n\nEven though there is no known reason for prostate cancer, there are many risks associated with the disease.\nWhat Are The Risk Factors for Prostate Cancer?\nAge\n\nAs men age, their risk of getting prostate cancer goes up. It is rarely found in men younger than age 40. Damage to the genetic material (DNA) of prostate cells is more likely for men over the age of 55. Damaged or abnormal prostate cells can begin to grow out of control and form tumors.\n\nAge is a well-known risk factor for prostate cancer. But, smoking and being overweight are more closely linked with dying from prostate cancer.\nEthnicity\n\nAfrican American men have, by far, the highest incidence of the disease. One in six African American men will get prostate cancer. African American men are more likely to get prostate cancer at an earlier age. They are also more like to have aggressive tumors that grow quickly, spread and cause death. The reason why prostate cancer is more prevalent in African American men is unclear yet it may be due to socioeconomic, environmental, diet or other factors. Other ethnicities, such as Hispanic and Asian men, are less likely to get prostate cancer.\n\n\n#### What Are The Survival Rates For Prostate Cancer?\n\nMany men with prostate cancer will not die from it; they will die from other causes. For men who are diagnosed, it is better if it is caught early.\n\nSurvival rates for men with prostate cancer have increased over the years, thanks to better screening and treatment options. Today, 99% of men with prostate cancer will live for at least 5 years after diagnosis. Many men having treatment are cured. Most prostate cancer is slow-growing and takes many years to progress. One out of three men will survive after five years, even if the cancer has spread to other parts of the body.\n\n\n### Treatment\n\nSome cancers grow so slowly that treatment may not be needed at all. Others grow fast and are life-threatening so treatment is usually necessary. Deciding what treatment you should get can be complex. Talk with your healthcare team about your options. Your treatment plan will depend on:\n\n*    The stage and grade of the cancer (Gleason score and TNM stage) \n*    Your risk category (whether the cancer is low, intermediate or high risk)\n*    Your age and health\n*    Your preferences with respect to side effects, long-term effects and treatment goals \n\nResults from other diagnostic tests will help your provider understand if the cancer can spread or recur (return) after treatment.\n\nBefore you decide what to do, you should consider how immediate and long-term side effects from treatment will affect your life, and what you're willing to tolerate. Also, you should consider that you may try different things over time.\n\nIf you have time before you start treatment, consider your range of options. Get a second opinion from different prostate cancer experts. You may need to see another urologist, oncologist or radiation oncologist. Consider the expertise of your doctor before you begin. With more experienced surgeons, the risk of permanent side effects (like incontinence) is lower. Also, it helps to talk with other survivors and learn from their experiences.\n\nIn addition, try and get or stay healthy. Eating a well-balanced diet, maintaining a healthy weight, exercising and not smoking are all important factors when fighting prostate cancer..\n\nMoreover, don't ignore your emotions. Think about how you're coping with this diagnosis. Many men who have prostate cancer feel worried, stressed and angry. You and those that care about you may need to consider professional counseling.\n\nTreatment choices for prostate cancer include:\nSurveillance\n\n1.    Active Surveillance\n2.    Watchful Waiting\n\nLocalized Therapy\n\n1.    Surgery\n2.    Radiation Therapy\n3.    Cryotherapy\n4.   Focal Therapy\n\nSystemic Therapy \n\n1.    Hormonal Therapy\n2.    Chemotherapy\n3.    Immunotherapy\n\n","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"from IPython.display import YouTubeVideo\nYouTubeVideo(\"1Q7ERNtLcvk\", height=500, width=700)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt; import seaborn as sns\nplt.style.use('seaborn-whitegrid')\nimport openslide\nimport os\nimport cv2\nimport torch\ntrain = pd.read_csv('../input/prostate-cancer-grade-assessment/train.csv')\ngpu = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\ngpu","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"BASE_FOLDER='/kaggle/input/prostate-cancer-grade-assessment/'","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train = pd.read_csv(BASE_FOLDER+\"train.csv\")\ntest = pd.read_csv(BASE_FOLDER+\"test.csv\")\nsub = pd.read_csv(BASE_FOLDER+\"sample_submission.csv\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"markdown","source":"\n\n\nWe will see the distribution of data providers:","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(10, 7))\nsns.countplot(train.data_provider)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(10, 7))\nsns.countplot(train.isup_grade);","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print('Total samples       :',train.shape[0])\nfor i in train.columns:\n  print(\"Total No. of Unique values in Column {}  : {}\".format(i,len(train[i].unique())))\n  if len(train[i].unique()) <20:\n    print(train[i].unique())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"PATH = \"../input/prostate-cancer-grade-assessment/\"\n\ndf_train = pd.read_csv(f'{PATH}train.csv')\ndf_test = pd.read_csv(f'{PATH}test.csv')\n\ndf_train.head().style.set_caption('Quick Overview of train.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(f\"Number of training data: {len(df_train)}\\n\")\n\nprint(f\"Unique data_providers: {df_train.data_provider.unique()}\\n\")\nprint(f\"Unique isup_grade: {df_train.isup_grade.unique()}\\n\")\nprint(f\"Unique gleason_score: {df_train.gleason_score.unique()}\\n\")\n\nprint(f\"Missing data:\\n{df_train.isna().any()}\\n\")\n\nmasks = os.listdir(PATH + 'train_label_masks/')\nimages = os.listdir(PATH + 'train_images/')\n\ndf_masks = pd.Series(masks).to_frame()\ndf_masks.columns = ['mask_file_name']\ndf_masks['image_id'] = df_masks.mask_file_name.apply(lambda x: x.split('_')[0])\ndf_train = pd.merge(df_train, df_masks, on='image_id', how='outer')\ndel df_masks\nprint(f\"There are {len(df_train[df_train.mask_file_name.isna()])} images without a mask.\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\nGleason Score and ISUP Grade\n\n The grading process consists of finding and classifying cancer tissue into so-called Gleason patterns (3, 4, or 5)[...]. After the biopsy is assigned a Gleason score, it is converted into an ISUP grade on a 1-5 scale. [...] However, the system suffers from significant inter-observer variability between pathologists, limiting its usefulness for individual patients. This variability in ratings could lead to unnecessary treatment, or worse, missing a severe diagnosis.\n\n","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"print(f\"Train data shape before reduction: {len(df_train)}\")\ndf_train_red = df_train[~df_train.mask_file_name.isna()]\nprint(f\"Train data shape after reduction: {len(df_train_red)}\")\n\nno_masks = df_train[df_train.mask_file_name.isna()][['image_id']]\nno_masks['Suspicious_because'] = 'No Mask'","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train_red.groupby('isup_grade').gleason_score.unique().to_frame().style.set_caption('Mapping of ISUP Grade to Gleason Score')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n### One Mislabeled Image?\n\nIn the above dataframe it looks like one image might have been converted to a wrong ISUP grade.\n","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"df_train_red[(df_train_red.isup_grade == 2) & (df_train_red.gleason_score != '3+4')]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n### Differences Between Data Providers\n\nThere are two data providers.\n\n    They used different scanners with slightly different maximum microscope resolutions and worked with different pathologists for labeling their images.\n\n","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"providers = df_train_red.data_provider.unique()\n\nfig = plt.figure(figsize=(6,4))\nax = sns.countplot(x=\"isup_grade\", hue=\"data_provider\", data=df_train_red)\nplt.title(\"ISUP Grade Count by Data Provider\", fontsize=14)\nplt.xlabel(\"ISUP Grade\", fontsize=14)\nplt.ylabel(\"Count\", fontsize=14)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"collapsed":true},"cell_type":"code","source":"\n\ndf_train_red[\"height\"] = 0\ndf_train_red[\"width\"] = 0\ndf_train_red[0] = 0\ndf_train_red[1] = 0\ndf_train_red[2] = 0\ndf_train_red[3] = 0\ndf_train_red[4] = 0\ndf_train_red[5] = 0\n\ndef get_image_data(row):\n    biopsy = skimage.io.MultiImage(PATH + 'train_label_masks/' + row.image_id + '_mask.tiff')\n    temp = biopsy[-1][:, :, 0]\n    counts = pd.Series(temp.reshape(-1)).value_counts()\n    row.height = temp.shape[0]\n    row.width = temp.shape[1]\n    row.update(counts)\n    return row\n\n\n\n\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import skimage.io\n\ndf_train_red = df_train_red.apply(lambda row: get_image_data(row), axis=1)\ndf_train_red['pixels'] = df_train_red.height * df_train_red.width","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, [ax1, ax2] = plt.subplots(nrows=1, ncols=2, figsize=(16, 6))\n\n\"\"\"\nInspired by something similiar I saw here https://www.kaggle.com/dhananjay3/panda-eda-all-you-need-to-know\n\"\"\"\nsns.scatterplot(data=df_train_red, x='width', y='height', marker='.',hue='data_provider', ax=ax1)\nax1.set_title(\"Image Sizes by Data Provider\", fontsize=14)\nax1.set_xlabel(\"Image Width\", fontsize=14)\nax1.set_ylabel(\"Image Height\", fontsize=14)\n\nsns.kdeplot(df_train_red[df_train_red.data_provider == 'karolinska'].pixels, label='karolinska', ax=ax2)\nsns.kdeplot(df_train_red[df_train_red.data_provider == 'radboud'].pixels, label= 'radboud', ax=ax2)\n\nax2.set_title(\"Image Sizes by Data Provider\", fontsize=14)\nax2.set_ylabel(\"Pixels per Image\", fontsize=14)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"This kernel would be updated soon , just performed the preliminary EDA\n\nDetailed EDA and model architechture to be explained in no time ","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}