\n\n---\n"},{"metadata":{},"cell_type":"markdown","source":"0 IMPORTS"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# PIP Installs\n!/opt/conda/bin/python3.7 -m pip install -q --upgrade pip # Upgrade PIP\n!pip install -q pylibjpeg pylibjpeg-libjpeg pylibjpeg-openjpeg # Install/Upgrade PyDicom Dependencies\n\n# Machine Learning and Data Science Imports\nimport tensorflow_probability as tfp\nimport tensorflow_datasets as tfds\nimport tensorflow_addons as tfa\nimport tensorflow_hub as hub\nfrom skimage import exposure\nimport pandas as pd; pd.options.mode.chained_assignment = None\nimport numpy as np\nimport scipy\n\n# Built In Imports\nfrom datetime import datetime\nfrom glob import glob\nimport warnings\nimport IPython\nimport urllib\nimport zipfile\nimport pickle\nimport shutil\nimport string\nimport math\nimport tqdm\nimport time\nimport os\nimport gc\nimport re\n\n# Visualization Imports\nfrom matplotlib.colors import ListedColormap\nimport matplotlib.patches as patches\nimport plotly.graph_objects as go\nimport matplotlib.pyplot as plt\nimport plotly.express as px\nimport seaborn as sns\nfrom PIL import Image\nimport matplotlib\nimport plotly\nimport PIL\nimport cv2\n\n# PRESETS\nFIG_FONT = dict(family=\"Helvetica, Arial\", size=14, color=\"#7f7f7f\")\nLABEL_COLORS = [px.colors.label_rgb(px.colors.convert_to_RGB_255(x)) for x in sns.color_palette(\"Spectral\", 15)]\nLABEL_COLORS_WOUT_NO_FINDING = LABEL_COLORS[:8]+LABEL_COLORS[9:]\n\n# Other Imports\nfrom pydicom.pixel_data_handlers.util import apply_voi_lut\nfrom tqdm.notebook import tqdm\nimport pydicom\n\nprint(\"\\n... IMPORTS COMPLETE ...\\n\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"1 BACKGROUND INFORMATION"},{"metadata":{},"cell_type":"markdown","source":"
1.1 THE DATA
\n\n---\n\nBACKGROUND INFORMATION\n\nIn this competition, we are classifying common thoracic lung diseases and localizing critical findings. **This is an object detection and classification problem.**\n\nFor each test image, you will be predicting a bounding box and class for all findings. If you predict that there are no findings, you should create a prediction of **`14 1 0 0 1 1`** *(14 is the class ID for no finding, and this provides a one-pixel bounding box with a confidence of 1.0)*\n\nNote that the images are in **DICOM** format, which means they contain additional data that might be useful for visualizing and classifying.\n\n\n\n \n\nDATASET INFORMATION\n\nThe dataset comprises **`18,000`** postero-anterior (PA) CXR scans in DICOM format, which were de-identified to protect patient privacy. \n\nAll images were labeled by a panel of experienced radiologists for the presence of **14** critical radiographic findings as listed below:\n\n> **`0`** - Aortic enlargement \n**`1`** - Atelectasis \n**`2`** - Calcification \n**`3`** - Cardiomegaly \n**`4`** - Consolidation \n**`5`** - ILD \n**`6`** - Infiltration \n**`7`** - Lung Opacity \n**`8`** - Nodule/Mass \n**`9`** - Other lesion \n**`10`** - Pleural effusion \n**`11`** - Pleural thickening \n**`12`** - Pneumothorax \n**`13`** - Pulmonary fibrosis \n**`14`** - \"No finding\" observation was intended to capture the absence of all findings above\n\nNote that a key part of this competition is working with ground truth from multiple radiologists. That means that the same image will have multiple ground-truth labels as annotated by different radiologists.\n\n \n\nDATA FILES\n> **`train.csv`** - the train set metadata, with one row for each object, including a class and a bounding box (multiple rows per image possible) \n**`sample_submission.csv`** - a sample submission file in the correct format\n\n \n\nTRAIN COLUMNS\n> **`image_id`** - unique image identifier \n**`class_name`** - the name of the class of detected object (or \"No finding\") \n**`class_id`** - the ID of the class of detected object \n**`rad_id`** - the ID of the radiologist that made the observation \n**`x_min`** - minimum X coordinate of the object's bounding box \n**`y_min`** - minimum Y coordinate of the object's bounding box \n**`x_max`** - maximum X coordinate of the object's bounding box \n**`y_max`** - maximum Y coordinate of the object's bounding box"},{"metadata":{},"cell_type":"markdown","source":"
1.2 THE GOAL
\n\n---\n\nIn this competition, you’ll automatically localize and classify **`14`** types of thoracic abnormalities from chest radiographs. You'll work with a dataset consisting of **`18,000`** scans that have been annotated by experienced radiologists. You can train your model with **`15,000`** independently-labeled images and will be evaluated on a test set of **`3,000`** images. These annotations were collected via VinBigData's web-based platform, VinLab. Details on building the dataset can be found in our recent paper “VinDr-CXR: An open dataset of chest X-rays with radiologist's annotations”."},{"metadata":{},"cell_type":"markdown","source":"
1.3 ADDITIONAL INFORMATION ON ABNORMALITIES
\n\n
Much Of The Content For This Markdown Cell Comes From This Notebook Written By The Talented User ANZ
\n\n---\n\n Aortic enlargement\n* Aortic enlargement is known as a sign of an aortic aneurysm. This condition often occurs in the ascending aorta. \n* In general, the term aneurysm is used when the axial diameter is >5.0 cm for the ascending aorta and >4.0 cm for the descending aorta.\n\n Atelectasis\n* Atelectasis is a condition where there is no air in part or all of the lungs and they have collapsed. \n* A common cause of atelectasis is obstruction of the bronchi.\n* In atelectasis, there is an increase in density on chest x-ray (usually whiter; black on black-and-white inversion images).\n\n Calcification\n* Calcium (calcification) may be deposited in areas where previous inflammation of the lungs or pleura has healed. \n* Many diseases or conditions can cause calcification on chest x-ray. \n* Calcification may occur in the Aorta (as with atherosclerosis) or it may occur in mediastinal lymph nodes (as with previous infection, tuberculosis, or histoplasmosis).\n\n Cardiomegaly\n* Cardiomegaly is usually diagnosed when the ratio of the heart's width to the width of the chest is more than 50%. This diagnostic criterion may be an essential basis for this competition.\n* Cardiomegaly can be caused by many conditions, including hypertension, coronary artery disease, infections, inherited disorders, and cardiomyopathies.\n* The heart-to-lung ratio criterion for the diagnosis of cardiomegaly is a ratio of greater than 0.5. However, this is only valid if the XRay is performed while the patient is standing. If the patient is sitting or in bed, this criterion cannot be used. To determine whether a patient is sitting or standing (and consequently whether this criteron is valid), we will detect the presence of air in the stomach (if there is no air in it, the patient is not standing and the criterion cannot be used)\n\n Consolidation\n* Consolidation is a decrease in lung permeability due to infiltration of fluid, cells, or tissue replacing the air-containing spaces in the alveoli.\n* Consolidation is officially referred to as air space consolidation. \n* On X-rays displaying air space consolidation, the lung field's density is increased, and pulmonary blood vessels are not seen, but black bronchi can be seen in the white background, which is called \"air bronchogram\". Since air remains in the bronchial tubes, they do not absorb X-rays and appear black, and the black and white are reversed from normal lung fields.\n\n ILD\n* ILD stands for \"Interstitial Lung Disease\".\n* Interstitial Lung Disease is a general term for many conditions in which the interstitial space is injured. \n* The interstitial space refers to the walls of the alveoli (air sacs in the lungs) and the space around the blood vessels and small airways.\n* Chest radiographic findings include ground-glass opacities (i.e., an area of hazy opacification), linear reticular shadows, and granular shadows.\n\n Infiltration\n* The infiltration of some fluid component into the alveoli causes an infiltrative shadow (Infiltration).\n* It is difficult to distinguish from consolidation and, in some cases, impossible to distinguish. Please see [this link](https://allnurses.com/consolidation-vs-infiltrate-vs-opacity-t483538/) for more information.\n\n Lung Opacity\n* Lung opacity is a loose term with many potential interpretations/meanings. Please see this [kaggle discussion](https://www.kaggle.com/zahaviguy/what-are-lung-opacities) for more information.\n* Lung opacity can often be identified as any area in the chest radiograph that is more white than it should be.\n\n Nodule/Mass\n* Nodules and masses are seen primarily in lung cancer, and metastasis from other parts of the body such as colon cancer and kidney cancer, tuberculosis, pulmonary mycosis, non-tuberculous mycobacterium, obsolete pneumonia, and benign tumors.\n* A nodule/mass is a round shade (typically less than 3 cm in diameter – resulting in much smaller than average bounding boxes) that appears on a chest X-ray image. \n\n Other lesion\n* Others include all abnormalities that do not fall into any other category. This includes bone penetrating images, fractures, subcutaneous emphysema, etc.\n\n Pleural effusion\n* Pleural effusion is the accumulation of water outside the lungs in the chest cavity. \n* The outside of the lungs is covered by a thin membrane consisting of two layers known as the pleura. Fluid accumulation between these two layers (chest-wall/parietal-pleura and the lung-tissue/visceral-pleura) is called pleural effusion.\n* The findings of pleural effusion vary widely and vary depending on whether the radiograph is taken in the upright or supine position.\n* The most common presentation of pleural effusion is elevation of the diaphragm on one side, flattening the diaphragm, or blunting the angle between rib and diaphragm (typically more than 30 degrees)\n\n Pleural thickening\n* The pleura is the membrane that covers the lungs, and the change in the thickness of the pleura is called pleural thickening. \n* It is often seen in the uppermost part of the lung field (the apex of the lung).\n\n Pneumothorax\n* A pneumothorax is a condition in which air leaks from the lungs and accumulates in the chest cavity. \n* When air leaks and accumulates in the chest, it cannot expand outward like a balloon due to the ribs' presence. Instead, the lungs are pushed by the air and become smaller. In other words, a pneumothorax is a situation where air leaks from the lungs and the lungs become smaller (collapsed).\n* In a chest radiograph of a pneumothorax, the collapsed lung is whiter than normal, and the area where the lung is gone is uniformly black. Besides, the edges of the lung may appear linear.\n\n Pulmonary fibrosis\n* Pulmonary Fibrosis is inflammation of the lung interstitium due to various causes, resulting in thickening and hardening of the walls, fibrosis, and scarring.\n* The fibrotic areas lose their air content, which often results in dense cord shadows or granular shadows.\n\n No finding\n* There are no findings on x-ray images. This is the normal image and is the baseline image needed to differentiate from the abnormal image."},{"metadata":{},"cell_type":"markdown","source":"2 NOTEBOOK SETUP"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Define the root data directory\nDATA_DIR = \"/kaggle/input/vinbigdata-chest-xray-abnormalities-detection\"\n\n# Define the paths to the training and testing dicom folders respectively\nTRAIN_DIR = os.path.join(DATA_DIR, \"train\")\nTEST_DIR = os.path.join(DATA_DIR, \"test\")\n\n# Capture all the relevant full train/test paths\nTRAIN_DICOM_PATHS = [os.path.join(TRAIN_DIR, f_name) for f_name in os.listdir(TRAIN_DIR)]\nTEST_DICOM_PATHS = [os.path.join(TEST_DIR, f_name) for f_name in os.listdir(TEST_DIR)]\nprint(f\"\\n... The number of training files is {len(TRAIN_DICOM_PATHS)} ...\")\nprint(f\"... The number of testing files is {len(TEST_DICOM_PATHS)} ...\")\n\n# Define paths to the relevant csv files\nTRAIN_CSV = os.path.join(DATA_DIR, \"train.csv\")\nSS_CSV = os.path.join(DATA_DIR, \"sample_submission.csv\")\n\n# Create the relevant dataframe objects\ntrain_df = pd.read_csv(TRAIN_CSV)\nss_df = pd.read_csv(SS_CSV)\n\nprint(\"\\n\\nTRAIN DATAFRAME\\n\\n\")\ndisplay(train_df.head(3))\n\nprint(\"\\n\\nSAMPLE SUBMISSION DATAFRAME\\n\\n\")\ndisplay(ss_df.head(3))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"3 HELPER FUNCTIONS"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"def dicom2array(path, voi_lut=True, fix_monochrome=True):\n \"\"\" Convert dicom file to numpy array \n \n Args:\n path (str): Path to the dicom file to be converted\n voi_lut (bool): Whether or not VOI LUT is available\n fix_monochrome (bool): Whether or not to apply monochrome fix\n \n Returns:\n Numpy array of the respective dicom file \n \n \"\"\"\n # Use the pydicom library to read the dicom file\n dicom = pydicom.read_file(path)\n \n # VOI LUT (if available by DICOM device) is used to \n # transform raw DICOM data to \"human-friendly\" view\n if voi_lut:\n data = apply_voi_lut(dicom.pixel_array, dicom)\n else:\n data = dicom.pixel_array\n \n # The XRAY may look inverted\n # - If we want to fix this we can\n if fix_monochrome and dicom.PhotometricInterpretation == \"MONOCHROME1\":\n data = np.amax(data) - data\n \n # Normalize the image array and return\n data = data - np.min(data)\n data = data / np.max(data)\n data = (data * 255).astype(np.uint8)\n return data\n\ndef plot_image(img, title=\"\", figsize=(8,8), cmap=None):\n \"\"\" Function to plot an image to save a bit of time \"\"\"\n plt.figure(figsize=figsize)\n \n if cmap:\n plt.imshow(img, cmap=cmap)\n else:\n img\n plt.imshow(img)\n \n plt.title(title, fontweight=\"bold\")\n plt.axis(False)\n plt.show()\n \ndef get_image_id(path):\n \"\"\" Function to return the image-id from a path \"\"\"\n return path.rsplit(\"/\", 1)[1].rsplit(\".\", 1)[0]\n\ndef create_fractional_bbox_coordinates(row):\n \"\"\" Function to return bbox coordiantes as fractions from DF row \"\"\"\n frac_x_min = row[\"x_min\"]/row[\"img_width\"]\n frac_x_max = row[\"x_max\"]/row[\"img_width\"]\n frac_y_min = row[\"y_min\"]/row[\"img_height\"]\n frac_y_max = row[\"y_max\"]/row[\"img_height\"]\n return frac_x_min, frac_x_max, frac_y_min, frac_y_max\n\ndef draw_bboxes(img, tl, br, rgb, label=\"\", label_location=\"tl\", opacity=0.1, line_thickness=0):\n \"\"\" TBD \n \n Args:\n TBD\n \n Returns:\n TBD \n \"\"\"\n rect = np.uint8(np.ones((br[1]-tl[1], br[0]-tl[0], 3))*rgb)\n sub_combo = cv2.addWeighted(img[tl[1]:br[1],tl[0]:br[0],:], 1-opacity, rect, opacity, 1.0) \n img[tl[1]:br[1],tl[0]:br[0],:] = sub_combo\n\n if line_thickness>0:\n img = cv2.rectangle(img, tuple(tl), tuple(br), rgb, line_thickness)\n \n if label:\n # DEFAULTS\n FONT = cv2.FONT_HERSHEY_SIMPLEX\n FONT_SCALE = 1.666\n FONT_THICKNESS = 3\n FONT_LINE_TYPE = cv2.LINE_AA\n \n if type(label)==str:\n LABEL = label.upper().replace(\" \", \"_\")\n else:\n LABEL = f\"CLASS_{label:02}\"\n \n text_width, text_height = cv2.getTextSize(LABEL, FONT, FONT_SCALE, FONT_THICKNESS)[0]\n \n label_origin = {\"tl\":tl, \"br\":br, \"tr\":(br[0],tl[1]), \"bl\":(tl[0],br[1])}[label_location]\n label_offset = {\n \"tl\":np.array([0, -10]), \"br\":np.array([-text_width, text_height+10]), \n \"tr\":np.array([-text_width, -10]), \"bl\":np.array([0, text_height+10])\n }[label_location]\n img = cv2.putText(img, LABEL, tuple(label_origin+label_offset), \n FONT, FONT_SCALE, rgb, FONT_THICKNESS, FONT_LINE_TYPE)\n \n return img","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"4 TABULAR DATA\n\nRECALL THAT THESE ARE THE TRAIN COLUMNS\n> **`image_id`** - unique image identifier \n**`class_name`** - the name of the class of detected object (or \"No finding\") \n**`class_id`** - the ID of the class of detected object \n**`rad_id`** - the ID of the radiologist that made the observation \n**`x_min`** - minimum X coordinate of the object's bounding box \n**`y_min`** - minimum Y coordinate of the object's bounding box \n**`x_max`** - maximum X coordinate of the object's bounding box \n**`y_max`** - maximum Y coordinate of the object's bounding box"},{"metadata":{},"cell_type":"markdown","source":"
4.1 IMAGE_ID COLUMN EXPLORATION
\n\n---\n\nThe **`image_id`** column contains a **U**nique **ID**entifier (**UID**) that indicates which patient the respective row (object) relates to.\n\nAs there can be up to three radiologists annotating the same image and potentially multiple objects/bboxes per image, it is possible for a single image UID to occur many times. However, please note that we know from the competition data details that there exists ***only one image for one patient***. This means that if a specific image_id appears 12 times, that there are 4 objects in the image, and each object was annotated by all three radiologists.\n\n*SIDE-NOTE* – Due to the ***one image to one patient*** rule, the column name **`image_id`** could be replaced with **`patient_id`** and it would mean exactly the same thing.\n\n
\n\nTOTAL OBJECT ANNOTATIONS PER IMAGE\n\nLet's count the distribution of the amount of annotations per unique **`image_id`** value. Note that we use a log-axis for the count axis to handle the large number of values present at 3 annotations (a single object annotated similarily by 3 radiologists)\n\n---\n\n**From the histogram plotted below we can ascertain the following information:**\n* Images contain at least 3 annotations (1 distinct object annotation by 3 radiologists)\n* Images contain at most 57 annotations (19 distinct object annotations by 3 radiologists)\n* The vast majority of images only have 3 annotations (~11,000 out of 15,000 images)\n* The distribution has a heavy skew (**`value=3.8687`** **`# FROM --> scipy.stats.skew(train_df.image_id.value_counts().values)`**). Remember that a perfectly symetrical distribution would have a skew value of **`0`**."},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = px.histogram(train_df.image_id.value_counts(), \n log_y=True, color_discrete_sequence=['indianred'], opacity=0.7,\n labels={\"value\":\"Number of Annotations Per Image\"},\n title=\"DISTRIBUTION OF # OF ANNOTATIONS PER PATIENT \" \\\n \"(Log Scale for Y-Axis)\",\n )\nfig.update_layout(showlegend=False,\n xaxis_title=\"Number of Unique Images\",\n yaxis_title=\"Count of All Object Annotations\",\n font=FIG_FONT,)\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"UNIQUE OBJECT ANNOTATIONS PER IMAGE\n\nLet's count the distribution of **UNIQUE** object-label annotations per unique **`image_id`** value. This means if a radiologist identifies 8 nodules in an image, we count that as 1 unique object annotation. The goal of this is to determine the distributions of different diseases occuring within the same patient.\n\nNote that we use a log-axis for the count axis to handle the large number of values present at 1 unique abnormality\n\n---\n\n**From the histogram plotted below we can ascertain the following information:**\n* Images contain no more than 10 unique abnormalities (out of a possible 14)\n* The more unique abnormalities present in an image, the rarer it is."},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = px.histogram(train_df.groupby('image_id')[\"class_name\"].unique().apply(lambda x: len(x)), \n log_y=True, color_discrete_sequence=['skyblue'], opacity=0.7,\n labels={\"value\":\"Number of Unique Abnormalities\"},\n title=\"DISTRIBUTION OF # OF ANNOTATIONS PER PATIENT \" \\\n \"(Log Scale for Y-Axis)\",\n )\nfig.update_layout(showlegend=False,\n xaxis_title=\"Number of Unique Abnormalities\",\n yaxis_title=\"Count of Unique Patients\",\n font=FIG_FONT,)\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"
4.2 CLASS_NAME COLUMN EXPLORATION
\n\n---\n\nThe **`class_name`** column indicates the label as a string for the respective object/annotation (each row is for one object/annotation). \n
\n\nANNOTATIONS PER CLASS\n\nWe know there are 15 different possible **`class_name`**s (including **`No finding`**). To identify the distribution of counts across the labels we will use a bar-chart."},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = px.bar(train_df.class_name.value_counts().sort_index(), \n color=train_df.class_name.value_counts().sort_index().index, opacity=0.85,\n color_discrete_sequence=LABEL_COLORS, log_y=True,\n labels={\"y\":\"Annotations Per Class\", \"x\":\"\"},\n title=\"Annotations Per Class\",)\nfig.update_layout(legend_title=None,\n font=FIG_FONT,\n xaxis_title=\"\",\n yaxis_title=\"Annotations Per Class\")\n\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"
4.3 CLASS_ID COLUMN EXPLORATION
\n\n---\n\nThe **`class_id`** column indicates the label encoded as a number the respective object/annotation (each row is for one object/annotation). Knowing this, we will remove the previous **class_name** column, as we would rather work with a numeric representation. Prior to removal we will generate a map that will allow us to translate the numeric labels back into their respective string represntations."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Create dictionary mappings\nint_2_str = {i:train_df[train_df[\"class_id\"]==i].iloc[0][\"class_name\"] for i in range(15)}\nstr_2_int = {v:k for k,v in int_2_str.items()}\nint_2_clr = {str_2_int[k]:LABEL_COLORS[i] for i,k in enumerate(sorted(str_2_int.keys()))}\n\nprint(\"\\n... Dictionary Mapping Class Integer to Class String Representation [int_2_str]...\\n\")\ndisplay(int_2_str)\n\nprint(\"\\n... Dictionary Mapping Class String to Class Integer Representation [str_2_int]...\\n\")\ndisplay(str_2_int)\n\nprint(\"\\n... Dictionary Mapping Class Integer to Color Representation [str_2_clr]...\\n\")\ndisplay(int_2_clr)\n\nprint(\"\\n... Head of Train Dataframe After Dropping The Class Name Column...\\n\")\ntrain_df.drop(columns=[\"class_name\"], inplace=True)\ndisplay(train_df.head(5))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"
4.4 RAD_ID COLUMN EXPLORATION
\n\n---\n\nThe **`rad_id`** column indicates the the ID of the radiologist that made the observation. Remember, three radiologists will annotate a given image out of a pool of seventeen possible radiologists, where the radiologist ID is encoded from R1 to R17.\n
\n\nANNOTATIONS PER RADIOLOGIST\n\nWe know there are 17 possible radiologists (**`rad_id`**s). To identify the distribution of annotations performed across the radiologists we will use a historgram.\n\n---\n\n**From the histogram plotted below we can ascertain the following information**\n* 3 of the radiologists (R9, R10, & R8 in that order) are responsible for the vast majority of annotations (~40-50% of all annotations)\n* Among the other 11 radiologists there is some variation around the number of annotations made, however, these 11 radiologists all made between 3121 annotations and 812 annotations with the vast majority annotating 1800-2200 objects."},{"metadata":{"trusted":true},"cell_type":"code","source":"fig = px.histogram(train_df, x=\"rad_id\", color=\"rad_id\",opacity=0.85,\n labels={\"rad_id\":\"Radiologist ID\"},\n title=\"DISTRIBUTION OF # OF ANNOTATIONS PER RADIOLOGIST\",\n ).update_xaxes(categoryorder=\"total descending\")\nfig.update_layout(legend_title=\"RADIOLOGIST ID\",\n xaxis_title=\"Radiologist ID\",\n yaxis_title=\"Number of Annotations Made\",\n font=FIG_FONT,)\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"ANNOTATIONS PER RADIOLOGIST SEPERATED BY CLASS LABEL\n\nWe have already identified that three of the radiologists are responsible for almost 50% of all of the annotations. We would now like to identify if all of the radiologists were able to see and annotate all 15 classes. If so, can we identify any additional skew or problems that might arise?\n\n---\n\n**From the first histogram plotted below we can ascertain the following information**\n* 3 of the radiologists (R9, R10, & R8 in that order) are responsible for the vast majority of annotations (~40-50% of all annotations)\n* Among the other 11 radiologists there is some variation around the number of annotations made, however, these 11 radiologists all made between 3121 annotations and 812 annotations with the vast majority annotating 1800-2200 objects.\n\n---\n\n**From the second histogram plotted below we can ascertain the following information**\n* Among the other 11 radiologists, 7 of them (R1 through R7) have only ever annotated images as **`No finding`**\n* The other 4 radiologists are also heavily skewed towards the **`No finding`** label when compared to the main 3 radiologists (R8 through R10). This seems to actually be closer to the overall distribution, however it might allow us to estimate that radiologists other than R8, R9, and R10, are much more likely to annotate images as **`No finding`**.\n* The downside to this distribution, is that if we include this information in the model than the model will learn that 7 of the radiologists classify images as **`No finding`** 100% of the time!\n\n Note that this second plot could have been generated by interacting with the first histogram as plotly has this functionality built-in"},{"metadata":{"trusted":true},"cell_type":"code","source":"# #################################################################### #\n# TO DO - NORMALIZE RADIOLOGIST COUNTS BASED ON ANNOTATION PER IMAGE #\n# #################################################################### #\n\nfig = go.Figure()\n\nfor i in range(15):\n fig.add_trace(go.Histogram(\n x=train_df[train_df[\"class_id\"]==i][\"rad_id\"],\n marker_color=int_2_clr[i],\n name=f\"{int_2_str[i]}\"))\n\nfig.update_xaxes(categoryorder=\"total descending\")\nfig.update_layout(title=\"DISTRIBUTION OF CLASS LABEL ANNOTATIONS BY RADIOLOGIST\",\n barmode='stack',\n xaxis_title=\"Radiologist ID\",\n yaxis_title=\"Number of Annotations Made\",\n font=FIG_FONT,)\nfig.show()\n\nfig = go.Figure()\nfor i in range(15):\n fig.add_trace(go.Histogram(\n x=train_df[(train_df[\"class_id\"]==i) & (~train_df[\"rad_id\"].isin([\"R8\",\"R9\",\"R10\"]))][\"rad_id\"],\n marker_color=int_2_clr[i],\n name=f\"{int_2_str[i]}\"))\n\nfig.update_xaxes(categoryorder=\"total descending\")\nfig.update_layout(title=\"DISTRIBUTION OF CLASS LABEL ANNOTATIONS BY RADIOLOGIST \" \\\n \"(EXCLUDING TOP 3 RADIOLOGISTS --> R8, R9 & R10)\",\n barmode='stack',\n xaxis_title=\"Radiologist ID\",\n yaxis_title=\"Number of Annotations Made\",\n font=FIG_FONT,)\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"
4.5 EXPLORATION OF BBOX COORDINATE COLUMNS
\n\n---\n\nThe **`x_min`**, **`y_min`**, **`x_max`**, and **`y_max`** columns indicate the location of the annotated object bounding box, where the top-left corner is represented by the tuple (**`x_min`**, **`y_min`**) and the bottom-right corner is represented by the tuple (**`x_max`**, **`y_max`**).\n\nA value of **`NaN`** coincides with a label 14 (**`No finding`**) and means that there is nothing to annotate (healthy x-ray). \nFor the purpose of examining these columns we will only be examining rows where the objects have been annotated with a bounding box \n(i.e. All rows with a label of **`No finding`** will be discarded)
\n\nPLOT HEATMAP REPRESENTING BOUNDING BOXES FOR VARIOUS CLASSES\n\nThere's a lot to digest within these plots. The important thing to focus on will be identifying for each class the approximate range of locations the annotations are found in and the intensity of the locations within the heatmap.\n\n---\n\n**From the heatmaps plotted below we can ascertain the following information**\n* Regarding Aortic Enlargement (CLASS-ID: 0)\n * Heatmap distribution is slightly oval (vertical) and is very tight and intense, located in the centre of the image (slight drift to the top-right).\n* Regarding Atelectasis (CLASS-ID: 1)\n * Heatmap distribution is lung shaped and relatively diffuse with a circular focus on the upper-left part of the left lung.\n* Regarding Calcification (CLASS-ID: 2)\n * Heatmap distribution is lung shaped and relatively diffuse with a oval (vertical) focus on the top-left edge of the right lung.\n* Regarding Cardiomegaly (CLASS-ID: 3)\n * Heatmap distribution is rectangular and is very tight and intense, located in the bottom-centre (to bottom-centre-right) of the image.\n* Regarding Consolidation (CLASS-ID: 4)\n * Heatmap distribution is lung shaped and relatively diffuse, the focus of the distribution covers the entire left lung.\n* Regarding ILD (CLASS-ID: 5)\n * Heatmap distribution is lung shaped and relatively diffuse, the focus leans a little towards the centre of the lungs.\n* Regarding Infiltration (CLASS-ID: 6)\n * Heatmap distribution is lung shaped and relatively diffuse, the focus of the distribution covers the entire left lung.\n* Regarding Lung Opacity (CLASS-ID: 7)\n * Heatmap distribution is lung shaped and relatively diffuse, the focus of the distribution covers the entire left lung.\n* Regarding Nodule/Mass (CLASS-ID: 8)\n * Heatmap distribution is lung shaped and relatively diffuse, the focus leans a little towards the centre of the lungs. (NOTE: The diffusion pattern looks patchy... probably due to smaller bounding boxes)\n* Regarding Other Lesion (CLASS-ID: 9)\n * Heatmap distribution is incredibly diffuse and covers most of the image, the focus is towards a vertical-strip in the centre of the image.\n* Regarding Pleural Effusion (CLASS-ID: 10)\n * Heatmap distribution is lung shaped (slightly more rectangular?) and relatively diffuse, the focus is towards the bottom of the lungs and although both lungs are covered, the left lung has a stronger focus.\n* Regarding Pleural Thickening (CLASS-ID: 11)\n * Heatmap distribution is vaguely lung shaped (patches near top and focus trails down exterior lung edge fading as it goes), the focus is towards the top of the lungs is oval (horizontal).\n* Regarding Pneumothorax (CLASS-ID: 12)\n * Heatmap distribution is lung shaped (more rectangular), the focus is on the entire left lung however the right lung has some diffuse coverage.\n* Regarding Pulmonary Fibrosis (CLASS-ID: 13)\n * Heatmap distribution is vaguely lung shaped (patches near top and focus trails down lung fading as it goes), the focus is towards the top of the lung and it is oval.\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Get paths to images where bboxes exist `class_id!=14`\nbbox_df = train_df[train_df.class_id!=14].reset_index(drop=True)\nBBOX_PATHS = [\n os.path.join(TRAIN_DIR, name+\".dicom\") \\\n for name in bbox_df.image_id.unique()\n]\n\n# Initalize our map for image sizes\nsizes_of_images_w_bboxes = {}\n\n# ############################################################### #\n# ############## THIS STEP WILL TAKE 15-30 MINUTES ############## #\n# ############################################################### #\n#\n# Get the image sizes so we can resize the bboxes all based on a static size\n# so that we can generate a heatmap that is representative of the actual\n# locations of annotations\nfor path in tqdm(BBOX_PATHS, total=len(BBOX_PATHS)):\n dicom = pydicom.read_file(path)\n sizes_of_images_w_bboxes[path[:-6].rsplit(\"/\", 1)[1]] = \\\n (dicom.Rows, dicom.Columns)\n# ############################################################### #\n\n# Create new dataframe columns for the source image width and height\nbbox_df[\"img_height\"] = bbox_df[\"image_id\"].map(lambda x: sizes_of_images_w_bboxes[x][0])\nbbox_df[\"img_width\"] = bbox_df[\"image_id\"].map(lambda x: sizes_of_images_w_bboxes[x][1])\n\n# Create new dataframe columns for the bboxes that is a \n# percentage of the respective source image width and height\n# -- i.e. if x_min is 100 and the image width is 1000 than frac_x_min is 0.1\n# -- i.e. if y_max is 28 and the image height is 900 than frac_y_max is 0.031\n#\n# This will allow us to pick a heat-map size and make sure that we can use\n# all of the bounding boxes and scale them appropriately\n# -- NOTE: We will most likely default the heatmap to the average\n# image shape so that there is as little distortion as possible\nbbox_df[\"frac_x_min\"], bbox_df[\"frac_x_max\"], bbox_df[\"frac_y_min\"], bbox_df[\"frac_y_max\"] = \\\n zip(*bbox_df.apply(create_fractional_bbox_coordinates, axis=1))\n\n# # Record some important values for later\nave_src_img_height = np.mean([size[0] for size in sizes_of_images_w_bboxes.values()], dtype=np.int32)\nave_src_img_width = np.mean([size[1] for size in sizes_of_images_w_bboxes.values()], dtype=np.int32)\n\n# # Preview the dataframe\nbbox_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# DEFAULT\nHEATMAP_SIZE = (ave_src_img_height, ave_src_img_width, 14)\n\n# Initialize\nheatmap = np.zeros((HEATMAP_SIZE), dtype=np.int16)\nbbox_np = bbox_df[[\"class_id\", \"frac_x_min\", \"frac_x_max\", \"frac_y_min\", \"frac_y_max\"]].to_numpy()\nbbox_np[:, 1:3] *= ave_src_img_width\nbbox_np[:, 3:5] *= ave_src_img_height\nbbox_np = np.floor(bbox_np).astype(np.int16)\n\n# Color map stuff\ncustom_cmaps = [\n matplotlib.colors.LinearSegmentedColormap.from_list(\n colors=[(0.,0.,0.), c, (0.95,0.95,0.95)], \n name=f\"custom_{i}\") for i,c in enumerate(sns.color_palette(\"Spectral\", 15))\n]\ncustom_cmaps.pop(8) # Remove No-Finding\n\nfor row in tqdm(bbox_np, total=bbox_np.shape[0]):\n heatmap[row[3]:row[4]+1, row[1]:row[2]+1, row[0]] += 1\n \nfig = plt.figure(figsize=(20,25))\nplt.suptitle(\"Heatmaps Showing Bounding Box Placement\\n \", fontweight=\"bold\", fontsize=16)\nfor i in range(15):\n plt.subplot(4, 4, i+1)\n if i==0:\n plt.imshow(heatmap.mean(axis=-1), cmap=\"bone\")\n plt.title(f\"Average of All Classes\", fontweight=\"bold\")\n else:\n plt.imshow(heatmap[:, :, i-1], cmap=custom_cmaps[i-1])\n plt.title(f\"{int_2_str[i-1]} – ({i})\", fontweight=\"bold\")\n \n plt.axis(False)\nfig.tight_layout(rect=[0, 0.03, 1, 0.97])\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"INVESTIGATE THE SIZES OF BOUNDING BOXES AND THE IMPACT OF CLASS\n\nAs we wish to examine the average, as well as the upper and lower limits for various class-based bounding box statistics, we will use a box plot to investigate. To make things easier to understand let us consider the following basic buckets.\n\nBounding Box Area - Median\n* Under 0.01 –– Smallest\n* 0.01 to 0.02 –– Small\n* 0.02 to 0.04 –– Medium\n* 0.04 to 0.06 –– Large\n* Above 0.06 –– Largest\n\nBounding Box Area - Quartile Range\n* Under 0.0075 –– Smallest\n* 0.0075 to 0.0125 –– Small\n* 0.0125 to 0.0250 –– Medium\n* 0.0250 to 0.0500 –– Large\n* Above 0.0500 –– Largest\n\n---\n\n**From the boxplot plotted below we can ascertain the following information**\n* Regarding Aortic Enlargement Box Plot (CLASS-ID: 0)\n * Median Value is Small ––– Quartile Range is Smallest\n* Regarding Atelectasis Box Plot (CLASS-ID: 1)\n * Median Value is Medium ––– Quartile Range is Large\n* Regarding Calcification Box Plot (CLASS-ID: 2)\n * Median Value is Smallest ––– Quartile Range is Medium\n* Regarding Cardiomegaly Box Plot (CLASS-ID: 3)\n * Median Value is Large ––– Quartile Range is Large\n* Regarding Consolidation Box Plot (CLASS-ID: 4)\n * Median Value is Medium ––– Quartile Range is Large\n* Regarding ILD Box Plot (CLASS-ID: 5)\n * Median Value is Largest ––– Quartile Range is Largest\n* Regarding Infiltration Box Plot (CLASS-ID: 6)\n * Median Value is Medium ––– Quartile Range is Large\n* Regarding Lung Opacity Box Plot (CLASS-ID: 7)\n * Median Value is Medium ––– Quartile Range is Large\n* Regarding Nodule/Mass Box Plot (CLASS-ID: 8)\n * Median Value is Smallest ––– Quartile Range is Smallest\n* Regarding Other Lesion Box Plot (CLASS-ID: 9)\n * Median Value is Small ––– Quartile Range is Large\n* Regarding Pleural Effusion Box Plot (CLASS-ID: 10)\n * Median Value is Smallest ––– Quartile Range is Large\n* Regarding Pleural Thickening Box Plot (CLASS-ID: 11)\n * Median Value is Smallest ––– Quartile Range is Smallest\n* Regarding Pneumothorax Box Plot (CLASS-ID: 12)\n * Median Value is Largest ––– Quartile Range is Largest\n* Regarding Pulmonary Fibrosis Box Plot (CLASS-ID: 13)\n * Median Value is Small ––– Quartile Range is Medium\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Update bbox dataframe to make this easier\nbbox_df[\"frac_bbox_area\"] = (bbox_df[\"frac_x_max\"]-bbox_df[\"frac_x_min\"])*(bbox_df[\"frac_y_max\"]-bbox_df[\"frac_y_min\"])\nbbox_df[\"class_id_as_str\"] = bbox_df[\"class_id\"].map(int_2_str)\ndisplay(bbox_df.head())\n\nfig = px.box(bbox_df.sort_values(by=\"class_id_as_str\"), x=\"class_id_as_str\", y=\"frac_bbox_area\", color=\"class_id_as_str\", \n color_discrete_sequence=LABEL_COLORS_WOUT_NO_FINDING, notched=True,\n labels={\"class_id_as_str\":\"Class Name\", \"frac_bbox_area\":\"BBox Area (%)\"},\n title=\"DISTRIBUTION OF BBOX AREAS AS % OF SOURCE IMAGE AREA \" \\\n \"(Some Upper Outliers Excluded For Better Visualization)\")\n\nfig.update_layout(showlegend=True,\n yaxis_range=[-0.025,0.4],\n legend_title_text=None,\n xaxis_title=\"\",\n yaxis_title=\"Bounding Box Area %\",\n font=FIG_FONT,)\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"INVESTIGATE THE ASPECT RATIO OF BOUNDING BOXES AND THE IMPACT OF CLASS\n\nWe want to understand the average shape (wide-narrow, square, etc.) of the bouning-boxes associated with each class, and to do this we will use a bar chart with some pre-drawn lines.\n\n---\n\n**From the bar chart plotted below we can ascertain the following information:**\n* The average size of bounding-boxes by class is usually close to square (usually on the horizontal rectangle size of square).\n* Cardiomegaly has, on average, very thin, rectangular, horizontal boxes (mean width is ~2.9x larger than mean height).\n* Pleural Thickening has, on average, thin, rectangular, horizontal boxes (mean width is ~1.9x larger than mean height).\n* ILD has, on average, somewhat thin, rectangular, vertical boxes (mean height is ~1.6x larger than mean width)\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Aspect Ratio is Calculated as Width/Height\nbbox_df[\"aspect_ratio\"] = (bbox_df[\"x_max\"]-bbox_df[\"x_min\"])/(bbox_df[\"y_max\"]-bbox_df[\"y_min\"])\n\n# Display average means for each class_id so we can examine the newly created Aspect Ratio Column\ndisplay(bbox_df.groupby(\"class_id\").mean())\n\n# Generate the bar plot\nfig = px.bar(x=[int_2_str[x] for x in range(14)], y=bbox_df.groupby(\"class_id\").mean()[\"aspect_ratio\"], \n color=[int_2_str[x] for x in range(14)], opacity=0.85,\n color_discrete_sequence=LABEL_COLORS_WOUT_NO_FINDING, \n labels={\"x\":\"Class Name\", \"y\":\"Aspect Ratio (W/H)\"},\n title=\"Aspect Ratios For Bounding Boxes By Class\",)\nfig.update_layout(font=FIG_FONT,\n yaxis_title=\"Aspect Ratio (W/H)\",\n xaxis_title=None,\n legend_title_text=None)\nfig.add_hline(y=1, line_width=2, line_dash=\"dot\", \n annotation_font_size=10, \n annotation_text=\"SQUARE ASPECT RATIO\", \n annotation_position=\"bottom left\", \n annotation_font_color=\"black\")\nfig.add_hrect(y0=0, y1=0.5, line_width=0, fillcolor=\"red\", opacity=0.125,\n annotation_text=\">2:1 VERTICAL RECTANGLE REGION\", \n annotation_position=\"bottom right\", \n annotation_font_size=10,\n annotation_font_color=\"red\")\nfig.add_hrect(y0=2, y1=3.5, line_width=0, fillcolor=\"green\", opacity=0.04,\n annotation_text=\">2:1 HORIZONTAL RECTANGLE REGION\", \n annotation_position=\"top right\", \n annotation_font_size=10,\n annotation_font_color=\"green\")\nfig.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"5 IMAGE DATA\n\nRecall that the image data is stored in DICOM format and the annotations are stored in our **`train_df`** Dataframe."},{"metadata":{},"cell_type":"markdown","source":"
5.1 CLASS TO HELP PARSE THE DATA
\n\n---\n\nClasses are very useful when we need to bind data to functionality. In this case, I have created a class (unwieldy as it may be currently in it's initial version) to help with that called **`TrainData`**.\n\nI will get into the details of how the methods work at a later time... and for today I will simply generate the outputs using each method to show their functionality."},{"metadata":{"trusted":true},"cell_type":"code","source":"class TrainData():\n def __init__(self, df, train_dir, cmap=\"Spectral\"):\n # Initialize\n self.df = df\n self.train_dir = train_dir\n \n # Visualization\n self.cmap = cmap\n self.pal = [tuple([int(x) for x in np.array(c)*(255,255,255)]) for c in sns.color_palette(cmap, 15)]\n self.pal.pop(8)\n \n # Store df components in individual numpy arrays for easy access based on index\n tmp_numpy = self.df.to_numpy()\n image_ids = tmp_numpy[0]\n class_ids = tmp_numpy[1]\n rad_ids = tmp_numpy[2]\n bboxes = tmp_numpy[3:]\n \n self.img_annotations = self.get_annotations(get_all=True)\n \n # Clean-Up\n del tmp_numpy; gc.collect();\n \n \n def get_annotations(self, get_all=False, image_ids=None, class_ids=None, rad_ids=None, index=None):\n \"\"\" TBD \n \n Args:\n get_all (bool, optional): TBD\n image_ids (list of strs, optional): TBD\n class_ids (list of ints, optional): TBD\n rad_ids (list of strs, optional): TBD\n index (int, optional):\n \n Returns:\n \n \n \"\"\"\n if not get_all and image_ids is None and class_ids is None and rad_ids is None and index is None:\n raise ValueError(\"Expected one of the following arguments to be passed:\" \\\n \"\\n\\t\\t– `get_all`, `image_id`, `class_id`, `rad_id`, or `index`\")\n # Initialize\n tmp_df = self.df.copy()\n \n if not get_all:\n if image_ids is not None:\n tmp_df = tmp_df[tmp_df.image_id.isin(image_ids)]\n if class_ids is not None:\n tmp_df = tmp_df[tmp_df.class_id.isin(class_ids)]\n if rad_ids is not None:\n tmp_df = tmp_df[tmp_df.rad_id.isin(rad_ids)]\n if index is not None:\n tmp_df = tmp_df.iloc[index]\n \n annotations = {image_id:[] for image_id in tmp_df.image_id.to_list()}\n for row in tmp_df.to_numpy():\n \n # Update annotations dictionary\n annotations[row[0]].append(dict(\n img_path=os.path.join(self.train_dir, row[0]+\".dicom\"),\n image_id=row[0],\n class_id=int(row[1]),\n rad_id=int(row[2][1:]),\n ))\n \n # Catch to convert float array to integer array\n if row[1]==14:\n annotations[row[0]][-1][\"bbox\"]=row[3:]\n else:\n annotations[row[0]][-1][\"bbox\"]=row[3:].astype(np.int32)\n return annotations\n \n def get_annotated_image(self, image_id, annots=None, plot=False, plot_size=(18,25), plot_title=\"\"):\n if annots is None:\n annots = self.img_annotations.copy()\n \n if type(annots) != list:\n image_annots = annots[image_id]\n else:\n image_annots = annots\n \n img = cv2.cvtColor(dicom2array(image_annots[0][\"img_path\"]),cv2.COLOR_GRAY2RGB)\n for ann in image_annots:\n if ann[\"class_id\"] != 14:\n img = draw_bboxes(img, \n ann[\"bbox\"][:2], ann[\"bbox\"][-2:], \n rgb=self.pal[ann[\"class_id\"]], \n label=int_2_str[ann[\"class_id\"]], \n opacity=0.08, line_thickness=4)\n if plot:\n plot_image(img, title=plot_title, figsize=plot_size)\n \n return img\n \n def plot_image_ids(self, image_id_list, height_multiplier=6, verbose=True):\n annotations = self.get_annotations(image_ids=image_id_list)\n annotated_imgs = []\n n = len(image_id_list)\n \n plt.figure(figsize=(20, height_multiplier*n))\n for i, (image_id, annots) in enumerate(annotations.items()):\n if i >= n:\n break\n if verbose:\n print(f\".\", end=\"\")\n plt.subplot(n//2,2,i+1)\n plt.imshow(self.get_annotated_image(image_id, annots))\n plt.axis(False)\n plt.title(f\"Image ID – {image_id}\")\n plt.tight_layout(rect=[0, 0.03, 1, 0.97])\n plt.show()\n \n def plot_classes(self, class_list, n=4, height_multiplier=6, verbose=True):\n annotations = self.get_annotations(class_ids=class_list)\n annotated_imgs = []\n\n plt.figure(figsize=(20, height_multiplier*n))\n for i, (image_id, annots) in enumerate(annotations.items()):\n if i >= n:\n break\n if verbose:\n print(f\".\", end=\"\")\n plt.subplot(n//2,2,i+1)\n plt.imshow(self.get_annotated_image(image_id, annots))\n plt.axis(False)\n plt.title(f\"Image ID – {image_id}\")\n plt.tight_layout(rect=[0, 0.03, 1, 0.97])\n plt.show()\n\n def plot_radiologists(self, rad_id_list, n=4, height_multiplier=6, verbose=True):\n annotations = self.get_annotations(rad_ids=rad_id_list)\n annotated_imgs = []\n\n plt.figure(figsize=(20, height_multiplier*n))\n for i, (image_id, annots) in enumerate(annotations.items()):\n if i >= n:\n break\n if verbose:\n print(f\".\", end=\"\")\n plt.subplot(n//2,2,i+1)\n plt.imshow(self.get_annotated_image(image_id, annots))\n plt.axis(False)\n plt.title(f\"Image ID – {image_id}\")\n plt.tight_layout(rect=[0, 0.03, 1, 0.97])\n plt.show()\n\ntrain_data = TrainData(train_df, TRAIN_DIR)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"PLOT IMAGES FROM THE CORRESPONDING IMAGE IDS\n\nSummary to be done later\n\n---\n\nMore detail to come later"},{"metadata":{"trusted":true},"cell_type":"code","source":"IMAGE_ID_LIST = train_df[train_df.class_id!=14].image_id[25:29].to_list()\ntrain_data.plot_image_ids(image_id_list=IMAGE_ID_LIST, verbose=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"PLOT IMAGES CONTAINING A SINGLE CLASS\n\nSummary to be done later... \n\n**NOTE: Only the bounding boxes for the specified classes will be drawn... probably a TBD in the future as a possible arg**\n\n---\n\nMore detail to come later"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data.plot_classes(class_list=[7,], n=2, verbose=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"PLOT IMAGES CONTAINING ONE OR MORE CLASSES FROM A LIST\n\nSummary to be done later... \n\n**NOTE: Images need not contain ALL the classes... potential future improvement or option.**\n\n---\n\nMore detail to come later"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data.plot_classes(class_list=[5,8,11], n=4, verbose=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"PLOT IMAGES ANNOTATED BY A SINGLE OR MULTIPLE RADIOLOGIST(S)\n\nSummary to be done later... \n\n**NOTE: Same caveat as plotting based on class. Only bounding boxes annotated by the specified radiologist will be plotted** \n**NOTE: As radiologists often annotate `No tissue`, images may not contain ANY bounding boxes**\n\n---\n\nMore detail to come later"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data.plot_radiologists(rad_id_list=[\"R8\"], verbose=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"
5.2 VISUALIZE EACH ABNORMALITY
\n\n---\n\nWe will leverage our created class to visualize 4 examples of every class...\n\n\n* Aortic Enlargement (CLASS-ID: 0)\n* Atelectasis (CLASS-ID: 1)\n* Calcification (CLASS-ID: 2)\n* Cardiomegaly (CLASS-ID: 3)\n* Consolidation (CLASS-ID: 4)\n* ILD (CLASS-ID: 5)\n* Infiltration (CLASS-ID: 6)\n* Lung Opacity (CLASS-ID: 7)\n* Nodule/Mass (CLASS-ID: 8)\n* Other Lesion (CLASS-ID: 9)\n* Pleural Effusion (CLASS-ID: 10)\n* Pleural Thickening (CLASS-ID: 11)\n* Pneumothorax (CLASS-ID: 12)\n* Pulmonary Fibrosis (CLASS-ID: 13)\n* No Tissue Present (CLASS-ID: 14)"},{"metadata":{},"cell_type":"markdown","source":"AORTIC ENLARGMENT - (0)\n\nSummary to be done later... \n\n---\n\nMore detail to come later"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data.plot_classes(class_list=[0,], n=4, verbose=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"ATELECTASIS - (1)\n\nSummary to be done later... \n\n---\n\nMore detail to come later"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data.plot_classes(class_list=[1,], n=4, verbose=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"CALCIFICATION - (2)\n\nSummary to be done later... \n\n---\n\nMore detail to come later"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data.plot_classes(class_list=[2,], n=4, verbose=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"CARDIOMEGALY - (3)\n\nSummary to be done later... \n\n---\n\nMore detail to come later"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data.plot_classes(class_list=[3,], n=4, verbose=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"CONSOLIDATION - (4)\n\nSummary to be done later... \n\n---\n\nMore detail to come later"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data.plot_classes(class_list=[4,], n=4, verbose=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"ILD - (5)\n\nSummary to be done later... \n\n---\n\nMore detail to come later"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data.plot_classes(class_list=[5,], n=4, verbose=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"INFILTRATION - (6)\n\nSummary to be done later... \n\n---\n\nMore detail to come later"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data.plot_classes(class_list=[6,], n=4, verbose=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"LUNG OPACITY - (7)\n\nSummary to be done later... \n\n---\n\nMore detail to come later"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data.plot_classes(class_list=[7,], n=4, verbose=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"NODULE/MASS - (8)\n\nSummary to be done later... \n\n---\n\nMore detail to come later"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data.plot_classes(class_list=[8,], n=4, verbose=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"OTHER LESION - (9)\n\nSummary to be done later... \n\n---\n\nMore detail to come later"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data.plot_classes(class_list=[9,], n=4, verbose=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"PLEURAL EFFUSION - (10)\n\nSummary to be done later... \n\n---\n\nMore detail to come later"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data.plot_classes(class_list=[10,], n=4, verbose=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"PLEURAL THICKENING - (11)\n\nSummary to be done later... \n\n---\n\nMore detail to come later"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data.plot_classes(class_list=[11,], n=4, verbose=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"PNEUMOTHORAX - (12)\n\nSummary to be done later... \n\n---\n\nMore detail to come later"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data.plot_classes(class_list=[12,], n=4, verbose=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"PULMONARY FIBROSIS - (13)\n\nSummary to be done later... \n\n---\n\nMore detail to come later"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data.plot_classes(class_list=[13,], n=4, verbose=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"NO TISSUE - (14)\n\nSummary to be done later... \n\n---\n\nMore detail to come later"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data.plot_classes(class_list=[14,], n=4, verbose=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"6 COMBINING/MERGING OVERLAPPING ANNOTATIONS (WIP)\n\nEXPLANATION COMING SOON – NOT SURE ABOUT THE HANDELING OF NODULES AND OTHER SMALL BBOXES THAT GET ENGULFED BY LARGER SIMILAR ANNOTATIONS"},{"metadata":{"trusted":true},"cell_type":"code","source":"def calc_iou(bbox_1, bbox_2):\n # determine the coordinates of the intersection rectangle\n x_left = max(bbox_1[0], bbox_2[0])\n y_top = max(bbox_1[1], bbox_2[1])\n x_right = min(bbox_1[2], bbox_2[2])\n y_bottom = min(bbox_1[3], bbox_2[3])\n\n # Check if bboxes overlap at all (if not return 0)\n if x_right < x_left or y_bottom < y_top:\n return 0.0\n \n # The intersection of two axis-aligned bounding boxes is always an\n # axis-aligned bounding box\n else:\n intersection_area = (x_right - x_left) * (y_bottom - y_top)\n \n # compute the area of both AABBs\n bbox_1_area = (bbox_1[2] - bbox_1[0]) * (bbox_1[3] - bbox_1[1])\n bbox_2_area = (bbox_2[2] - bbox_2[0]) * (bbox_2[3] - bbox_2[1])\n\n # compute the intersection over union by taking the intersection\n # area and dividing it by the sum of prediction + ground-truth\n # areas - the interesection area\n iou = intersection_area / float(bbox_1_area + bbox_2_area - intersection_area)\n return iou\n\ndef redux_bboxes(annots):\n def _get_inner_box(bboxes):\n xmin = max([box[0] for box in bboxes])\n ymin = max([box[1] for box in bboxes])\n xmax = min([box[2] for box in bboxes])\n ymax = min([box[3] for box in bboxes])\n if (xmax<=xmin) or (ymax<=ymin):\n return None\n else:\n return [xmin, ymin, xmax, ymax]\n \n valid_list_indices = [] \n new_bboxes = []\n new_class_ids = []\n new_rad_ids = []\n \n for i, (class_id, rad_id, bbox) in enumerate(zip(annots[\"class_id\"], annots[\"rad_id\"], annots[\"bbox\"])):\n intersecting_boxes = [bbox,]\n other_bboxes = [x for j,x in enumerate(annots[\"bbox\"]) if j!=i]\n other_classes = [x for j,x in enumerate(annots[\"class_id\"]) if j!=i]\n for j, (other_class_id, other_bbox) in enumerate(zip(other_classes, other_bboxes)):\n if class_id==other_class_id:\n iou = calc_iou(bbox, other_bbox)\n if iou>0.:\n intersecting_boxes.append(other_bbox)\n\n if len(intersecting_boxes)>1:\n inner_box = _get_inner_box(intersecting_boxes)\n if inner_box and inner_box not in new_bboxes:\n new_bboxes.append(inner_box)\n new_class_ids.append(class_id)\n new_rad_ids.append(rad_id) \n\n annots[\"bbox\"] = new_bboxes\n annots[\"rad_id\"] = new_rad_ids\n annots[\"class_id\"] = new_class_ids\n \n return annots\n\n# Make GT Dataframe\ngt_df = train_df[train_df.class_id!=14]\n\n# Apply Manipulations and Merger Functions\ngt_df[\"bbox\"] = gt_df.loc[:, [\"x_min\",\"y_min\",\"x_max\",\"y_max\"]].values.tolist()\ngt_df.drop(columns=[\"x_min\",\"y_min\",\"x_max\",\"y_max\"], inplace=True)\ngt_df = gt_df.groupby([\"image_id\"]).agg({k:list for k in gt_df.columns if k !=\"image_id\"}).reset_index()\ngt_df = gt_df.apply(redux_bboxes, axis=1)\n\n# Recreate the Original Dataframe Style\ngt_df = gt_df.apply(pd.Series.explode).reset_index(drop=True).dropna()\ngt_df[\"x_min\"] = gt_df[\"bbox\"].apply(lambda x: x[0])\ngt_df[\"y_min\"] = gt_df[\"bbox\"].apply(lambda x: x[1])\ngt_df[\"x_max\"] = gt_df[\"bbox\"].apply(lambda x: x[2])\ngt_df[\"y_max\"] = gt_df[\"bbox\"].apply(lambda x: x[3])\ngt_df.drop(columns=[\"bbox\"], inplace=True)\n\n# Add back in NaN Rows As A Single Annotation\ngt_df = pd.concat([\n gt_df, train_df.loc[train_df['class_id'] == 14].drop_duplicates(subset=[\"image_id\"])\n]).reset_index(drop=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"gt_data = TrainData(gt_df, TRAIN_DIR)\nIMAGE_ID_LIST = gt_df[gt_df.class_id!=14].groupby(\"image_id\") \\\n .count() \\\n .sort_values(by=\"class_id\", ascending=False) \\\n .index[0:100:20]\n\nfor i, IMAGE_ID in enumerate(IMAGE_ID_LIST):\n train_data.get_annotated_image(IMAGE_ID, annots=None, plot=True, plot_size=(18,22), plot_title=f\"ORIGINAL – IMG #{i+1}\")\n gt_data.get_annotated_image(IMAGE_ID, annots=None, plot=True, plot_size=(18,22), plot_title=f\"REDUX VERSION – IMG #{i+1}\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}