{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# **VinBigData Chest X-ray Abnormalities Detection**","metadata":{}},{"cell_type":"markdown","source":"# Exploratory Data Analysis + 2 class classifier pipeline using EfficientNet-B0","metadata":{}},{"cell_type":"markdown","source":"*In this competition, you’ll automatically localize and classify 14 types of thoracic abnormalities from chest radiographs. You'll work with a dataset consisting of 18,000 scans that have been annotated by experienced radiologists. You can train your model with 15,000 independently-labeled images and will be evaluated on a test set of 3,000 images. These annotations were collected via VinBigData's web-based platform, VinLab. Details on building the dataset can be found in our recent paper “[VinDr-CXR: An open dataset of chest X-rays with radiologist's annotations](https://arxiv.org/pdf/2012.15029.pdf)”.*","metadata":{}},{"cell_type":"markdown","source":"The EDA comes from different notebooks. You can refer to these to have more details:\n* [https://www.kaggle.com/dschettler8845/visual-in-depth-eda-vinbigdata-competition-data](https://www.kaggle.com/dschettler8845/visual-in-depth-eda-vinbigdata-competition-data)\n* [https://www.kaggle.com/bjoernholzhauer/eda-dicom-reading-vinbigdata-chest-x-ray](https://www.kaggle.com/bjoernholzhauer/eda-dicom-reading-vinbigdata-chest-x-ray)\n* [https://www.kaggle.com/bryanb/vinbigdata-chest-x-ray-eda-fusing-boxes](https://www.kaggle.com/bryanb/vinbigdata-chest-x-ray-eda-fusing-boxes)","metadata":{}},{"cell_type":"markdown","source":"# IMPORTS  ","metadata":{}},{"cell_type":"code","source":"# PIP Installs\n!/opt/conda/bin/python3.7 -m pip install -q --upgrade pip      # Upgrade PIP\n!pip install -q pylibjpeg pylibjpeg-libjpeg pylibjpeg-openjpeg # Install/Upgrade PyDicom Dependencies\n\n# Machine Learning and Data Science Imports\nimport tensorflow_probability as tfp\nimport tensorflow_datasets as tfds\nimport tensorflow_addons as tfa\nimport tensorflow_hub as hub\nfrom skimage import exposure\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torch.utils.data import Dataset,DataLoader\nfrom torch import optim\nimport sklearn\n\nimport pandas as pd; pd.options.mode.chained_assignment = None\nimport numpy as np\nimport scipy\n\n# Built In Imports\nfrom datetime import datetime\nfrom glob import glob\nimport warnings\nimport IPython\nimport urllib\nimport zipfile\nimport pickle\nimport shutil\nimport string\nimport math\nimport tqdm\nimport time\nimport os\nimport gc\nimport re\n\n# Visualization Imports\nimport albumentations as A\nfrom albumentations.pytorch import ToTensorV2\nfrom matplotlib.colors import ListedColormap\nimport matplotlib.patches as patches\nimport plotly.graph_objects as go\nimport matplotlib.pyplot as plt\nimport plotly.express as px\nimport seaborn as sns\nfrom PIL import Image\nimport matplotlib\nimport plotly\nimport PIL\nimport cv2\n\n# PRESETS\nFIG_FONT = dict(family=\"Helvetica, Arial\", size=14, color=\"#7f7f7f\")\nLABEL_COLORS = [px.colors.label_rgb(px.colors.convert_to_RGB_255(x)) for x in sns.color_palette(\"Spectral\", 15)]\nLABEL_COLORS_WOUT_NO_FINDING = LABEL_COLORS[:8]+LABEL_COLORS[9:]\n\n# Other Imports\nfrom pydicom.pixel_data_handlers.util import apply_voi_lut\nfrom tqdm.notebook import tqdm\nimport pydicom\nfrom tqdm.notebook import tqdm\nimport warnings\nwarnings.filterwarnings('ignore')\ntimm_path = \"../input/timm-pytorch-image-models/pytorch-image-models-master\"\nimport sys\nsys.path.append(timm_path)\nimport timm\n\nprint(\"\\n... IMPORTS COMPLETE ...\\n\")\n\n\ndevice = torch.device('cuda' if torch.cuda.is_available() else 'cpu')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# EDA","metadata":{}},{"cell_type":"markdown","source":"## NOTEBOOK SETUP","metadata":{}},{"cell_type":"code","source":"# Define the root data directory\nDATA_DIR = \"/kaggle/input/vinbigdata-chest-xray-abnormalities-detection\"\n\n# Define the paths to the training and testing dicom folders respectively\nTRAIN_DIR = os.path.join(DATA_DIR, \"train\")\nTEST_DIR = os.path.join(DATA_DIR, \"test\")\n\n# Capture all the relevant full train/test paths\nTRAIN_DICOM_PATHS = [os.path.join(TRAIN_DIR, f_name) for f_name in os.listdir(TRAIN_DIR)]\nTEST_DICOM_PATHS = [os.path.join(TEST_DIR, f_name) for f_name in os.listdir(TEST_DIR)]\nprint(f\"\\n... The number of training files is {len(TRAIN_DICOM_PATHS)} ...\")\nprint(f\"... The number of testing files is {len(TEST_DICOM_PATHS)} ...\")\n\n# Define paths to the relevant csv files\nTRAIN_CSV = os.path.join(DATA_DIR, \"train.csv\")\nSS_CSV = os.path.join(DATA_DIR, \"sample_submission.csv\")\n\n# Create the relevant dataframe objects\ntrain_df = pd.read_csv(TRAIN_CSV)\nss_df = pd.read_csv(SS_CSV)\n\nprint(\"\\n\\nTRAIN DATAFRAME\\n\\n\")\ndisplay(train_df.head(3))\n\nprint(\"\\n\\nSAMPLE SUBMISSION DATAFRAME\\n\\n\")\ndisplay(ss_df.head(3))\n\nOUTPUT_DIR = './'\nif not os.path.exists(OUTPUT_DIR):\n    os.makedirs(OUTPUT_DIR)\n\n\n\ntimm_path = \"../input/timm-pytorch-image-models/pytorch-image-models-master\"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_df = pd.read_csv(f'{DATA_DIR}/sample_submission.csv')\ntest_df.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df[['class_id', 'class_name', 'rad_id']].groupby(['class_id', 'class_name']).count().rename(columns={'rad_id': 'Number of records'})","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"So, what do these categories mean? This is actually explained in the paper linked on the competition homepage:\n\n**Aortic enlargement:** \"An abnormal bulge that occurs in the wall of the major blood vessel.\" \n\n**Atelectasis:** \"Collapse of a part of the lung due to a decrease in the amount of air in the alveoli resulting in volume loss and increased density.\" (see also Merck Manual)\n\n**Calcification:** \"Deposition of calcium salts in the lung.\" - one article I looked at says that \"[...] calcifications occur in a damaged lung following an inflammatory process such as infection (tuberculosis, histoplasmosis, Pneumocystis carnii), bleeding or pulmonary infarction\" (Bendayan et al. 2000)\n\n**Cardiomegaly:** \"Enlargement of the heart, occurs when the heart of an adult patient is larger than normal and the cardiothoracic ratio is greater than 0.5.\"\n\n**Consolidation:** \"Any pathologic process that fills the alveoli with fluid, pus, blood, cells (including tumor cells) or other substances resulting in lobar, diffuse or multifocal ill-defined opacities.\"\n\n**ILD:** \"Interstitial lung disease (ILD) Involvement of the supporting tissue of the lung parenchyma resulting in fine or coarse reticular opacities or small nodules.\" (see also Merck Manual)\n\n**Infiltration:** \"An abnormal substance that accumulates gradually within cells or body tissues or any substance or type of cell that occurs within or spreads as through the interstices (interstitium and/or alveoli) of the lung, that is foreign to the lung, or that accumulates in greater than normal quantity within it.\"\n\n**Lung Opacity:** \"Any abnormal focal or generalized opacity or opacities in lung fields (blanket tag including but not limited to consolidation, cavity, fibrosis, nodule, mass, calcification, interstitial thickening, etc.).\" \n\n**Nodule/Mass:** \"Any space occupying lesion either solitary or multiple.\"\n\n**Other lesion:** \"Other lesions that are not on the list of findings or abnormalities mentioned above.\"\n\n**Pleural effusion:** \"Abnormal accumulations of fluid within the pleural space.\" (see also Merck Manual)\n\n**Pleural thickening:** \"Any form of thickening involving either the parietal or visceral pleura.\"\n\n**Pneumothorax:** \"The presence of gas (air) in the pleural space.\" (see also the Merck Manual)\n\n**Pulmonary fibrosis:** \"An excess of fibrotic tissue in the lung.\"\n\n**No finding:** Kind of self-explanatory (there were no findings).","metadata":{}},{"cell_type":"markdown","source":"## ANNOTATIONS PER CLASS\n \nWe know there are 15 different possible class_names (including No finding). To identify the distribution of counts across the labels we will use a bar-chart.","metadata":{}},{"cell_type":"code","source":"fig = px.bar(train_df.class_name.value_counts().sort_index(), \n             color=train_df.class_name.value_counts().sort_index().index, opacity=0.85,\n             color_discrete_sequence=LABEL_COLORS, log_y=True,\n             labels={\"y\":\"Annotations Per Class\", \"x\":\"\"},\n             title=\"<b>Annotations Per Class</b>\",)\nfig.update_layout(legend_title=None,\n                  font=FIG_FONT,\n                  xaxis_title=\"\",\n                  yaxis_title=\"<b>Annotations Per Class</b>\")\n\nfig.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## RAD_ID COLUMN EXPLORATION\nThe rad_id column indicates the the ID of the radiologist that made the observation. Remember, three radiologists will annotate a given image out of a pool of seventeen possible radiologists, where the radiologist ID is encoded from R1 to R17.\n\n\n#### ANNOTATIONS PER RADIOLOGIST\n\nWe know there are 17 possible radiologists (rad_ids). To identify the distribution of annotations performed across the radiologists we will use a historgram.\n\nFrom the histogram plotted below we can ascertain the following information\n\n3 of the radiologists (R9, R10, & R8 in that order) are responsible for the vast majority of annotations (~40-50% of all annotations)\nAmong the other 14 radiologists there is some variation around the number of annotations made, however, these 14 radiologists all made between 3121 annotations and 812 annotations with the vast majority annotating 1800-2200 objects.","metadata":{}},{"cell_type":"code","source":"fig = px.histogram(train_df, x=\"rad_id\", color=\"rad_id\",opacity=0.85,\n                   labels={\"rad_id\":\"Radiologist ID\"},\n                   title=\"<b>DISTRIBUTION OF # OF ANNOTATIONS PER RADIOLOGIST</b>\",\n                   ).update_xaxes(categoryorder=\"total descending\")\nfig.update_layout(legend_title=\"<b>RADIOLOGIST ID</b>\",\n                  xaxis_title=\"<b>Radiologist ID</b>\",\n                  yaxis_title=\"<b>Number of Annotations Made</b>\",\n                  font=FIG_FONT,)\nfig.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### ANNOTATIONS PER RADIOLOGIST SEPERATED BY CLASS LABEL\n\nWe have already identified that three of the radiologists are responsible for almost 50% of all of the annotations. We would now like to identify if all of the radiologists were able to see and annotate all 15 classes. If so, can we identify any additional skew or problems that might arise?\n\nFrom the first histogram plotted below we can ascertain the following information\n\n3 of the radiologists (R9, R10, & R8 in that order) are responsible for the vast majority of annotations (~40-50% of all annotations)\nAmong the other 11 radiologists there is some variation around the number of annotations made, however, these 11 radiologists all made between 3121 annotations and 812 annotations with the vast majority annotating 1800-2200 objects.\nFrom the second histogram plotted below we can ascertain the following information\n\nAmong the other 11 radiologists, 7 of them (R1 through R7) have only ever annotated images as No finding\nThe other 4 radiologists are also heavily skewed towards the No finding label when compared to the main 3 radiologists (R8 through R10). This seems to actually be closer to the overall distribution, however it might allow us to estimate that radiologists other than R8, R9, and R10, are much more likely to annotate images as No finding.\nThe downside to this distribution, is that if we include this information in the model than the model will learn that 7 of the radiologists classify images as No finding 100% of the time!","metadata":{}},{"cell_type":"code","source":"# Create dictionary mappings\nint_2_str = {i:train_df[train_df[\"class_id\"]==i].iloc[0][\"class_name\"] for i in range(15)}\nstr_2_int = {v:k for k,v in int_2_str.items()}\nint_2_clr = {str_2_int[k]:LABEL_COLORS[i] for i,k in enumerate(sorted(str_2_int.keys()))}\n\n###### #\n#  TO DO - NORMALIZE RADIOLOGIST COUNTS BASED ON ANNOTATION PER IMAGE  #\n# #################################################################### #\n\nfig = go.Figure()\n\nfor i in range(15):\n    fig.add_trace(go.Histogram(\n        x=train_df[train_df[\"class_id\"]==i][\"rad_id\"],\n        marker_color=int_2_clr[i],\n        name=f\"<b>{int_2_str[i]}</b>\"))\n\nfig.update_xaxes(categoryorder=\"total descending\")\nfig.update_layout(title=\"<b>DISTRIBUTION OF CLASS LABEL ANNOTATIONS BY RADIOLOGIST</b>\",\n                  barmode='stack',\n                  xaxis_title=\"<b>Radiologist ID</b>\",\n                  yaxis_title=\"<b>Number of Annotations Made</b>\",\n                  font=FIG_FONT,)\nfig.show()\n\nfig = go.Figure()\nfor i in range(15):\n    fig.add_trace(go.Histogram(\n        x=train_df[(train_df[\"class_id\"]==i) & (~train_df[\"rad_id\"].isin([\"R8\",\"R9\",\"R10\"]))][\"rad_id\"],\n        marker_color=int_2_clr[i],\n        name=f\"<b>{int_2_str[i]}</b>\"))\n\nfig.update_xaxes(categoryorder=\"total descending\")\nfig.update_layout(title=\"<b>DISTRIBUTION OF CLASS LABEL ANNOTATIONS BY RADIOLOGIST   \" \\\n                  \"<i><sub>(EXCLUDING TOP 3 RADIOLOGISTS --> R8, R9 & R10)</sub></i></b>\",\n                  barmode='stack',\n                  xaxis_title=\"<b>Radiologist ID</b>\",\n                  yaxis_title=\"<b>Number of Annotations Made</b>\",\n                  font=FIG_FONT,)\nfig.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## CORRELATION BETWEEN CLASSES","metadata":{}},{"cell_type":"markdown","source":"How is the occurence of the different classes correlated? Unsurprisingly, no finding is negatively correlated with all findings, while the various findings are slightly positively correlated, except for a few exceptions with strong correlations such as '0: Aortic enlargement' and '3: Cardiomegaly'.","metadata":{}},{"cell_type":"code","source":"tmpdf = train_df[['class_id', 'image_id', 'rad_id']].groupby(['class_id', 'image_id']).count().reset_index()\ntmpdf['rad_id'] = np.minimum(tmpdf['rad_id'].values, 1)\ncorr  = tmpdf.pivot(index='image_id', columns='class_id', values='rad_id').fillna(0).reset_index(drop=True).corr()\ncorr.style.background_gradient(cmap='coolwarm', vmin=-1.0, vmax=1.0).set_precision(2)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Where do the different findings tend to be?\nAs we can see below, certain findings tend to concentrate in certain area - we'll ignore that images might not all be the same size. E.g. aortic enlargment is where the aorta is, cardiomegaly is around the heart, etc., which may be useful for checking the plausibility of model outputs/knowledge we might try to explicitly provide to networks (that of course assumes that all images show upper bodies from a similar distance).","metadata":{}},{"cell_type":"code","source":"locations = np.zeros((14, int(np.ceil(np.max(train_df.y_max)/10)*10), int(np.ceil(np.max(train_df.x_max)/10)*10)))\nfor index, row in tqdm(train_df.iterrows(), total=train_df.shape[0]):\n    if row['class_id']<14:\n        locations[row['class_id'], int(row['y_min']):int(row['y_max']+1), int(row['x_min']):int(row['x_max']+1)] += 1","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"classes = train_df[['class_id', 'class_name', 'rad_id']].groupby(['class_id', 'class_name']).count().rename(columns={'rad_id': 'Number of records'}).reset_index()\n\nfor index, row in classes.iterrows():\n    if index==0:\n        label_dict = {row['class_id']: row['class_name']}\n    else:\n        label_dict.update({row['class_id']: row['class_name']})","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"f, axs = plt.subplots(5, 3, sharey=True, sharex=True, figsize=(16,28));\n\nfor class_id in range(14):\n    axs[class_id // 3, class_id - 3*(class_id // 3)].imshow(locations[class_id], cmap='inferno', interpolation='nearest');\n    axs[class_id // 3, class_id - 3*(class_id // 3)].set_title(str(class_id) + ': ' + label_dict[class_id])\n    \nplt.show();","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## INVESTIGATE THE ASPECT RATIO OF BOUNDING BOXES AND THE IMPACT OF CLASS\n\nWe want to understand the average shape (wide-narrow, square, etc.) of the bouning-boxes associated with each class, and to do this we will use a bar chart with some pre-drawn lines.\n\nFrom the bar chart plotted below we can ascertain the following information: \n\nThe average size of bounding-boxes by class is usually close to square (usually on the horizontal rectangle size of square). \\\n**Cardiomegaly** has, on average, very thin, rectangular, horizontal boxes (mean width is ~2.9x larger than mean height). \\\n**Pleural Thickening** has, on average, thin, rectangular, horizontal boxes (mean width is ~1.9x larger than mean height). \\\n**ILD** has, on average, somewhat thin, rectangular, vertical boxes (mean height is ~1.6x larger than mean width)","metadata":{}},{"cell_type":"code","source":"# Get paths to images where bboxes exist `class_id!=14`\nbbox_df = train_df[train_df.class_id!=14].reset_index(drop=True)\n\n# Aspect Ratio is Calculated as Width/Height\nbbox_df[\"aspect_ratio\"] = (bbox_df[\"x_max\"]-bbox_df[\"x_min\"])/(bbox_df[\"y_max\"]-bbox_df[\"y_min\"])\n\n# Display average means for each class_id so we can examine the newly created Aspect Ratio Column\ndisplay(bbox_df.groupby(\"class_id\").mean())\n\n# Generate the bar plot\nfig = px.bar(x=[int_2_str[x] for x in range(14)], y=bbox_df.groupby(\"class_id\").mean()[\"aspect_ratio\"], \n             color=[int_2_str[x] for x in range(14)], opacity=0.85,\n             color_discrete_sequence=LABEL_COLORS_WOUT_NO_FINDING, \n             labels={\"x\":\"Class Name\", \"y\":\"Aspect Ratio (W/H)\"},\n             title=\"<b>Aspect Ratios For Bounding Boxes By Class</b>\",)\nfig.update_layout(font=FIG_FONT,\n                  yaxis_title=\"<b>Aspect Ratio (W/H)</b>\",\n                  xaxis_title=None,\n                  legend_title_text=None)\nfig.add_hline(y=1, line_width=2, line_dash=\"dot\", \n              annotation_font_size=10, \n              annotation_text=\"<b>SQUARE ASPECT RATIO</b>\", \n              annotation_position=\"bottom left\", \n              annotation_font_color=\"black\")\nfig.add_hrect(y0=0, y1=0.5, line_width=0, fillcolor=\"red\", opacity=0.125,\n              annotation_text=\"<b>>2:1 VERTICAL RECTANGLE REGION</b>\", \n              annotation_position=\"bottom right\", \n              annotation_font_size=10,\n              annotation_font_color=\"red\")\nfig.add_hrect(y0=2, y1=3.5, line_width=0, fillcolor=\"green\", opacity=0.04,\n              annotation_text=\"<b>>2:1 HORIZONTAL RECTANGLE REGION</b>\", \n              annotation_position=\"top right\", \n              annotation_font_size=10,\n              annotation_font_color=\"green\")\nfig.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## PLOT IMAGES FROM THE CORRESPONDING IMAGE IDS","metadata":{}},{"cell_type":"code","source":"# Create dictionary mappings\nint_2_str = {i:train_df[train_df[\"class_id\"]==i].iloc[0][\"class_name\"] for i in range(15)}\nstr_2_int = {v:k for k,v in int_2_str.items()}\nint_2_clr = {str_2_int[k]:LABEL_COLORS[i] for i,k in enumerate(sorted(str_2_int.keys()))}\ntrain_df.drop(columns=[\"class_name\"], inplace=True)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def dicom2array(path, voi_lut=True, fix_monochrome=True):\n    \"\"\" Convert dicom file to numpy array \n    \n    Args:\n        path (str): Path to the dicom file to be converted\n        voi_lut (bool): Whether or not VOI LUT is available\n        fix_monochrome (bool): Whether or not to apply monochrome fix\n        \n    Returns:\n        Numpy array of the respective dicom file \n        \n    \"\"\"\n    # Use the pydicom library to read the dicom file\n    dicom = pydicom.read_file(path)\n    \n    # VOI LUT (if available by DICOM device) is used to \n    # transform raw DICOM data to \"human-friendly\" view\n    if voi_lut:\n        data = apply_voi_lut(dicom.pixel_array, dicom)\n    else:\n        data = dicom.pixel_array\n        \n    # The XRAY may look inverted\n    #   - If we want to fix this we can\n    if fix_monochrome and dicom.PhotometricInterpretation == \"MONOCHROME1\":\n        data = np.amax(data) - data\n    \n    # Normalize the image array and return\n    data = data - np.min(data)\n    data = data / np.max(data)\n    data = (data * 255).astype(np.uint8)\n    return data\n\ndef plot_image(img, title=\"\", figsize=(8,8), cmap=None):\n    \"\"\" Function to plot an image to save a bit of time \"\"\"\n    plt.figure(figsize=figsize)\n    \n    if cmap:\n        plt.imshow(img, cmap=cmap)\n    else:\n        img\n        plt.imshow(img)\n        \n    plt.title(title, fontweight=\"bold\")\n    plt.axis(False)\n    plt.show()\n    \ndef get_image_id(path):\n    \"\"\" Function to return the image-id from a path \"\"\"\n    return path.rsplit(\"/\", 1)[1].rsplit(\".\", 1)[0]\n\ndef create_fractional_bbox_coordinates(row):\n    \"\"\" Function to return bbox coordiantes as fractions from DF row \"\"\"\n    frac_x_min = row[\"x_min\"]/row[\"img_width\"]\n    frac_x_max = row[\"x_max\"]/row[\"img_width\"]\n    frac_y_min = row[\"y_min\"]/row[\"img_height\"]\n    frac_y_max = row[\"y_max\"]/row[\"img_height\"]\n    return frac_x_min, frac_x_max, frac_y_min, frac_y_max\n\ndef draw_bboxes(img, tl, br, rgb, label=\"\", label_location=\"tl\", opacity=0.1, line_thickness=0):\n    \"\"\" TBD \n    \n    Args:\n        TBD\n        \n    Returns:\n        TBD \n    \"\"\"\n    rect = np.uint8(np.ones((br[1]-tl[1], br[0]-tl[0], 3))*rgb)\n    sub_combo = cv2.addWeighted(img[tl[1]:br[1],tl[0]:br[0],:], 1-opacity, rect, opacity, 1.0)    \n    img[tl[1]:br[1],tl[0]:br[0],:] = sub_combo\n\n    if line_thickness>0:\n        img = cv2.rectangle(img, tuple(tl), tuple(br), rgb, line_thickness)\n        \n    if label:\n        # DEFAULTS\n        FONT = cv2.FONT_HERSHEY_SIMPLEX\n        FONT_SCALE = 1.666\n        FONT_THICKNESS = 3\n        FONT_LINE_TYPE = cv2.LINE_AA\n        \n        if type(label)==str:\n            LABEL = label.upper().replace(\" \", \"_\")\n        else:\n            LABEL = f\"CLASS_{label:02}\"\n        \n        text_width, text_height = cv2.getTextSize(LABEL, FONT, FONT_SCALE, FONT_THICKNESS)[0]\n        \n        label_origin = {\"tl\":tl, \"br\":br, \"tr\":(br[0],tl[1]), \"bl\":(tl[0],br[1])}[label_location]\n        label_offset = {\n            \"tl\":np.array([0, -10]), \"br\":np.array([-text_width, text_height+10]), \n            \"tr\":np.array([-text_width, -10]), \"bl\":np.array([0, text_height+10])\n        }[label_location]\n        img = cv2.putText(img, LABEL, tuple(label_origin+label_offset), \n                          FONT, FONT_SCALE, rgb, FONT_THICKNESS, FONT_LINE_TYPE)\n    \n    return img","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class TrainData():\n    def __init__(self, df, train_dir, cmap=\"Spectral\"):\n        # Initialize\n        self.df = df\n        self.train_dir = train_dir\n        \n        # Visualization\n        self.cmap = cmap\n        self.pal = [tuple([int(x) for x in np.array(c)*(255,255,255)]) for c in sns.color_palette(cmap, 15)]\n        self.pal.pop(8)\n        \n        # Store df components in individual numpy arrays for easy access based on index\n        tmp_numpy = self.df.to_numpy()\n        image_ids = tmp_numpy[0]\n        class_ids = tmp_numpy[1]\n        rad_ids = tmp_numpy[2]\n        bboxes = tmp_numpy[3:]\n        \n        self.img_annotations = self.get_annotations(get_all=True)\n        \n        # Clean-Up\n        del tmp_numpy; gc.collect();\n        \n        \n    def get_annotations(self, get_all=False, image_ids=None, class_ids=None, rad_ids=None, index=None):\n        \"\"\" TBD \n        \n        Args:\n            get_all (bool, optional): TBD\n            image_ids (list of strs, optional): TBD\n            class_ids (list of ints, optional): TBD\n            rad_ids (list of strs, optional): TBD\n            index (int, optional):\n        \n        Returns:\n        \n        \n        \"\"\"\n        if not get_all and image_ids is None and class_ids is None and rad_ids is None and index is None:\n            raise ValueError(\"Expected one of the following arguments to be passed:\" \\\n                             \"\\n\\t\\t– `get_all`, `image_id`, `class_id`, `rad_id`, or `index`\")\n        # Initialize\n        tmp_df = self.df.copy()\n        \n        if not get_all:\n            if image_ids is not None:\n                tmp_df = tmp_df[tmp_df.image_id.isin(image_ids)]\n            if class_ids is not None:\n                tmp_df = tmp_df[tmp_df.class_id.isin(class_ids)]\n            if rad_ids is not None:\n                tmp_df = tmp_df[tmp_df.rad_id.isin(rad_ids)]\n            if index is not None:\n                tmp_df = tmp_df.iloc[index]\n            \n        annotations = {image_id:[] for image_id in tmp_df.image_id.to_list()}\n        for row in tmp_df.to_numpy():\n            \n            # Update annotations dictionary\n            annotations[row[0]].append(dict(\n                img_path=os.path.join(self.train_dir, row[0]+\".dicom\"),\n                image_id=row[0],\n                class_id=int(row[1]),\n                rad_id=int(row[2][1:]),\n            ))\n            \n            # Catch to convert float array to integer array\n            if row[1]==14:\n                annotations[row[0]][-1][\"bbox\"]=row[3:]\n            else:\n                annotations[row[0]][-1][\"bbox\"]=row[3:].astype(np.int32)\n        return annotations\n    \n    def get_annotated_image(self, image_id, annots=None, plot=False, plot_size=(18,25), plot_title=\"\"):\n        if annots is None:\n            annots = self.img_annotations.copy()\n        \n        if type(annots) != list:\n            image_annots = annots[image_id]\n        else:\n            image_annots = annots\n            \n        img = cv2.cvtColor(dicom2array(image_annots[0][\"img_path\"]),cv2.COLOR_GRAY2RGB)\n        for ann in image_annots:\n            if ann[\"class_id\"] != 14:\n                img = draw_bboxes(img, \n                                ann[\"bbox\"][:2], ann[\"bbox\"][-2:], \n                                rgb=self.pal[ann[\"class_id\"]], \n                                label=int_2_str[ann[\"class_id\"]], \n                                opacity=0.08, line_thickness=4)\n        if plot:\n            plot_image(img, title=plot_title, figsize=plot_size)\n        \n        return img\n    \n    def plot_image_ids(self, image_id_list, height_multiplier=6, verbose=True):\n        annotations = self.get_annotations(image_ids=image_id_list)\n        annotated_imgs = []\n        n = len(image_id_list)\n        \n        plt.figure(figsize=(20, height_multiplier*n))\n        for i, (image_id, annots) in enumerate(annotations.items()):\n            if i >= n:\n                break\n            if verbose:\n                print(f\".\", end=\"\")\n            plt.subplot(n//2,2,i+1)\n            plt.imshow(self.get_annotated_image(image_id, annots))\n            plt.axis(False)\n            plt.title(f\"Image ID – {image_id}\")\n        plt.tight_layout(rect=[0, 0.03, 1, 0.97])\n        plt.show()\n        \n    def plot_classes(self, class_list, n=4, height_multiplier=6, verbose=True):\n        annotations = self.get_annotations(class_ids=class_list)\n        annotated_imgs = []\n\n        plt.figure(figsize=(20, height_multiplier*n))\n        for i, (image_id, annots) in enumerate(annotations.items()):\n            if i >= n:\n                break\n            if verbose:\n                print(f\".\", end=\"\")\n            plt.subplot(n//2,2,i+1)\n            plt.imshow(self.get_annotated_image(image_id, annots))\n            plt.axis(False)\n            plt.title(f\"Image ID – {image_id}\")\n        plt.tight_layout(rect=[0, 0.03, 1, 0.97])\n        plt.show()\n\n    def plot_radiologists(self, rad_id_list, n=4, height_multiplier=6, verbose=True):\n        annotations = self.get_annotations(rad_ids=rad_id_list)\n        annotated_imgs = []\n\n        plt.figure(figsize=(20, height_multiplier*n))\n        for i, (image_id, annots) in enumerate(annotations.items()):\n            if i >= n:\n                break\n            if verbose:\n                print(f\".\", end=\"\")\n            plt.subplot(n//2,2,i+1)\n            plt.imshow(self.get_annotated_image(image_id, annots))\n            plt.axis(False)\n            plt.title(f\"Image ID – {image_id}\")\n        plt.tight_layout(rect=[0, 0.03, 1, 0.97])\n        plt.show()\n\ntrain_data = TrainData(train_df, TRAIN_DIR)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"IMAGE_ID_LIST = train_df[train_df.class_id!=14].image_id[25:29].to_list()\ntrain_data.plot_image_ids(image_id_list=IMAGE_ID_LIST, verbose=False)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## EDA WITH METADATA\n\nLet's consider we extracted all metadata, you can find the data here: https://www.kaggle.com/bryanb/vinbigdata-chestxray-metadata\n\nI chose to keep only few information: sex, age, height and weight","metadata":{}},{"cell_type":"code","source":"# Load full metadata set\ntrain_metadata = pd.read_csv(\"../input/vinbigdata-chestxray-metadata/train_dicom_metadata.csv\")\ntest_metadata = pd.read_csv(\"../input/vinbigdata-chestxray-metadata/test_dicom_metadata.csv\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Keep some columns only\ntrain_metadata_filtered = train_metadata[[\"Patient's Sex\", \"Patient's Age\", \"Patient's Size\", \"Patient's Weight\"]]\ntest_metadata_filtered = test_metadata[[\"Patient's Sex\", \"Patient's Age\", \"Patient's Size\", \"Patient's Weight\"]]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(train_metadata_filtered[\"Patient's Size\"].value_counts())\nprint(train_metadata_filtered[\"Patient's Weight\"].value_counts())\n\nprint(test_metadata_filtered[\"Patient's Size\"].value_counts())\nprint(test_metadata_filtered[\"Patient's Weight\"].value_counts())\n\ntrain_metadata_filtered = train_metadata_filtered.drop([\"Patient's Size\", \"Patient's Weight\"], axis=1)\ntest_metadata_filtered = test_metadata_filtered.drop([\"Patient's Size\", \"Patient's Weight\"], axis=1)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def get_first_el(row):\n    resu = 'NA'\n    if len(str(row))>5:\n        resu = re.search(r\"(?<=\\[)(.*?)(?=\\,)\", row).group()\n    return resu","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_metadata_filtered[\"Patient's Sex\"] = train_metadata_filtered[\"Patient's Sex\"].fillna(\"NA\")\ntrain_metadata_filtered.loc[train_metadata_filtered[\"Patient's Sex\"]==\"O\"] = np.nan\n\ntrain_metadata_filtered[\"Patient's Age\"] = train_metadata_filtered[\"Patient's Age\"].fillna(\"0\")\ntrain_metadata_filtered[\"Patient's Age\"] = train_metadata_filtered[\"Patient's Age\"].apply(lambda x:re.search(r\"\\d*\", str(x)).group())\ntrain_metadata_filtered.loc[train_metadata_filtered[\"Patient's Age\"]== '']= np.nan\ntrain_metadata_filtered[\"Patient's Age\"] = train_metadata_filtered[\"Patient's Age\"].astype(float)\ntest_metadata_filtered[\"Patient's Sex\"] = test_metadata_filtered[\"Patient's Sex\"].fillna(\"NA\")\ntest_metadata_filtered.loc[test_metadata_filtered[\"Patient's Sex\"]==\"O\"] = np.nan\n\ntest_metadata_filtered[\"Patient's Age\"] = test_metadata_filtered[\"Patient's Age\"].fillna(\"0\")\ntest_metadata_filtered[\"Patient's Age\"] = test_metadata_filtered[\"Patient's Age\"].apply(lambda x:re.search(r\"\\d*\", str(x)).group())\ntest_metadata_filtered.loc[test_metadata_filtered[\"Patient's Age\"]== '']= np.nan\ntest_metadata_filtered[\"Patient's Age\"] = test_metadata_filtered[\"Patient's Age\"].astype(float)\n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_metadata_filtered.to_csv(\"train_dicom_metadata_filtered.csv\", index=False)\ntest_metadata_filtered.to_csv(\"test_dicom_metadata_filtered.csv\", index=False)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_metadata_age = train_metadata_filtered.loc[(train_metadata_filtered[\"Patient's Age\"] > 0) & \n                                                 (train_metadata_filtered[\"Patient's Age\"] < 100), :]\n\nfig = px.histogram(train_metadata_age, x=\"Patient's Age\",\n                   marginal=\"box\",\n                   hover_data=train_metadata_age.columns)\n\nfig.update_layout(\n    title=\"Age distribution (train)\")\n\nfig.show()\n\ndel(train_metadata_age)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_metadata_counts = list(train_metadata_filtered.loc[train_metadata_filtered[\"Patient's Sex\"] != \"NA\", \"Patient's Sex\"].value_counts())\ntrain_metadata_labels = list(train_metadata_filtered.loc[train_metadata_filtered[\"Patient's Sex\"] != \"NA\", \"Patient's Sex\"].value_counts().index)\n\nfig = go.Figure(data=[go.Pie(labels=train_metadata_labels, \n                             values=train_metadata_counts, \n                             hole=.3,\n                             title_text=\"Sex distribution (train)\")])\nfig.show()\n\ndel(train_metadata_counts, train_metadata_labels)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data = train_metadata_filtered.loc[(train_metadata_filtered[\"Patient's Sex\"] != \"NA\") &\n                                   (train_metadata_filtered[\"Patient's Age\"] > 0) &\n                                   (train_metadata_filtered[\"Patient's Age\"] < 100), :]\n\nfig = px.histogram(data, \n                   x=\"Patient's Age\", \n                   color=\"Patient's Sex\", \n                   marginal=\"box\",\n                   hover_data=data.columns,\n                   histnorm = \"probability\")\n\nfig.update_layout(\n    title=\"Age distribution by sex (train)\")\n\nfig.show()\n\ndel(data)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 2 CLASS CLASSIFIER PIPELINE USING EFFICIENTNET","metadata":{}},{"cell_type":"markdown","source":"To have a better understanding of the following code, you can refer to the following notebook: \n* [https://www.kaggle.com/mrinath/2-class-classifier-pipeline-using-effnet](https://www.kaggle.com/mrinath/2-class-classifier-pipeline-using-effnet)\n\nI decided to use the same method than the notebook linked above but tried with a different model.","metadata":{}},{"cell_type":"code","source":"#train  =  pd.read_csv(\"../input/vinbigdata-chest-xray-abnormalities-detection/train.csv\") \nis_normal_df = train_df.groupby(\"image_id\")[\"class_id\"].agg(lambda s: (s == 14).sum()).reset_index().rename({\"class_id\": \"num_normal_annotations\"}, axis=1)\nis_normal_df.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def change(x):\n    if (x==3):\n        x=1\n    return x\nis_normal_df['target'] = is_normal_df['num_normal_annotations'].apply(lambda x: change(x))\ndf = is_normal_df[[\"image_id\",\"target\"]]\ndf.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## SPLITTING","metadata":{}},{"cell_type":"code","source":"from sklearn.model_selection import StratifiedKFold\nskf  =  StratifiedKFold(n_splits = 5, random_state = 42,shuffle = True)\nfolds = df.copy()\nfor f,(tr_idx,val_idx) in enumerate(skf.split(folds,folds.target)):\n    folds.loc[val_idx,'fold'] = int(f)\nfolds['fold'] = folds['fold'].astype(int)    \n","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"folds.image_id=folds.image_id+\".png\"\nimg_path = \"../input/vinbigdata-chest-xray-resized-png-1024x1024/train\"\ndf_paths = [os.path.join(img_path,x) for x in folds.image_id]\nfolds['path'] = df_paths\nfolds.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## TRANSFORMS","metadata":{}},{"cell_type":"code","source":"train_aug = A.Compose(\n    [  \n\n        A.Resize(300,300,p=1.0),\n        A.CLAHE(clip_limit=4.0, p=0.85),\n\n     A.Normalize(\n            p=1.0),\n        ToTensorV2(p=1.0)\n    ]\n)\nval_aug = A.Compose(\n    [\n         A.Resize(300,300,p=1.0),\n        A.HorizontalFlip(p=0.5),\n         A.Normalize(\n            p=1.0),\n        ToTensorV2(p=1.0),\n    ]\n)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class Xray(Dataset):\n    def __init__(self,df,augs=None):\n        self.df = df\n        self.augs = augs\n    def __len__(self):\n        return(len(self.df))\n    def __getitem__(self,idx):\n        img_src = self.df.loc[idx,'path']\n        image = cv2.imread(img_src)\n        image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB).astype(np.uint8)\n        \n        target = self.df.loc[idx,'target']\n        \n        if (self.augs):\n            transformed = self.augs(image=image)\n            image = transformed['image']\n        \n        return image,torch.tensor(target) ","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data = Xray(folds,augs = train_aug)\nload = DataLoader(data,batch_size = 1)\nimg,target = next(iter(load))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.imshow(img.squeeze(0).permute(1,2,0))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## EFFICIENTNET-B0 MODEL","metadata":{}},{"cell_type":"code","source":"model=timm.create_model('efficientnet_b0', pretrained=False) # set pretrained=True to use the pretrained weights\nnum_features = model.classifier.in_features\nmodel.classifier = nn.Linear(num_features, 1)\nfor param in model.parameters():\n    param.requires_grad_(True)\n    \n#for param in model.parameters():\n#    param.requires_grad_(False)\n#ct = 0\n#for child in model.children():\n#    ct += 1\n#    if ct > 8:\n#        for param in child.parameters():\n#            param.requires_grad = True\n#model","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ss=F.sigmoid(model(torch.randn(3,3,300,300)))\nss","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## HELPING FUNCTIONS","metadata":{}},{"cell_type":"code","source":"class AverageMeter(object):\n    \"\"\"Computes and stores the average and current value\"\"\"\n    def __init__(self):\n        self.reset()\n\n    def reset(self):\n        self.val = 0\n        self.avg = 0\n        self.sum = 0\n        self.count = 0\n\n    def update(self, val, n=1):\n        self.val = val\n        self.sum += val * n\n        self.count += n\n        self.avg = self.sum / self.count","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def train_one_epoch(train_loader,model,optimizer,criterion,e,epochs):\n    losses = AverageMeter()\n    scores = AverageMeter()\n    model.train()\n    global_step = 0\n    loop = tqdm(enumerate(train_loader),total = len(train_loader))\n    \n    for step,(image,labels) in loop:\n        image = image.to(device)\n        labels = labels.unsqueeze(1)\n        labels= labels.to(device)\n        output = model(image)\n        batch_size = labels.size(0)\n        loss = criterion(output,labels.float())\n        \n        out = F.sigmoid(output)\n        outputs = out.cpu().detach().numpy()\n        targets = labels.cpu().detach().numpy()\n        try:\n            auc = sklearn.metrics.roc_auc_score(targets, outputs)\n            losses.update(loss.item(), batch_size)\n            scores.update(auc.item(), batch_size)\n            loss.backward()\n            optimizer.step()\n            optimizer.zero_grad()\n            global_step += 1\n        \n            loop.set_description(f\"Epoch {e+1}/{epochs}\")\n            loop.set_postfix(loss = loss.item(), auc = auc.item(), stage = 'train')\n        \n            \n        except ValueError:\n            pass\n        \n        \n       \n        \n    return losses.avg,scores.avg","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def val_one_epoch(loader,model,optimizer,criterion):\n    losses = AverageMeter()\n    scores = AverageMeter()\n    model.eval()\n    global_step = 0\n    loop = tqdm(enumerate(loader),total = len(loader))\n    \n    for step,(image,labels) in loop:\n        image = image.to(device)\n        labels = labels.unsqueeze(1)\n        labels = labels.to(device)\n        batch_size = labels.size(0)\n        with torch.no_grad():\n            output = model(image)\n        loss = criterion(output,labels.float())\n        \n        out = F.sigmoid(output)\n        outputs = out.cpu().detach().numpy()\n        targets = labels.cpu().detach().numpy()\n        try:\n            auc = sklearn.metrics.roc_auc_score(targets, outputs)\n            losses.update(loss.item(), batch_size)\n            scores.update(auc.item(), batch_size)\n            loop.set_postfix(loss = loss.item(), auc = auc.item(), stage = 'valid')\n            optimizer.step()\n            optimizer.zero_grad()\n            global_step += 1\n        except ValueError:\n            pass\n        \n        \n        \n        \n        \n    \n        \n    return losses.avg,scores.avg","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## TRAINING LOOP","metadata":{}},{"cell_type":"code","source":"def fit(model,fold_n,training_batch_size=32,validation_batch_size=64):\n    \n    train_data=folds[folds.fold != fold_n]\n    val_data=folds[folds.fold == fold_n]\n    train_data= Xray(train_data.reset_index(drop=True),augs = train_aug)\n    val_data= Xray(val_data.reset_index(drop=True),augs = val_aug)\n    \n    \n    train_loader = DataLoader(train_data,\n                             shuffle=True,\n                        num_workers=0,\n                        batch_size=training_batch_size)\n    valid_loader = DataLoader(val_data,\n                             shuffle=False,\n                        num_workers=0,\n                        batch_size=validation_batch_size)\n    model = model\n    model.to(device)\n    criterion=nn.BCEWithLogitsLoss()\n    optimizer = optim.Adam(model.parameters(), lr=0.001)\n    scheduler = optim.lr_scheduler.ReduceLROnPlateau(optimizer,patience = 3,verbose = True)\n    epochs= 5\n    \n    best_acc = 0\n    \n    loop = range(epochs)\n    for e in loop:\n        \n        train_loss,train_auc = train_one_epoch(train_loader,model,optimizer,criterion,e,epochs)\n         #scheduling step if given\n    \n        #scheduler.step()\n        \n        print(f'For epoch {e+1}/{epochs}')\n        print(f'average train_loss {train_loss}')\n        print(f'average train_auc {train_auc}' )\n        \n        val_loss,val_auc = val_one_epoch(valid_loader,model,optimizer,criterion)\n        \n        scheduler.step(val_loss)\n        \n        print(f'avarage val_loss { val_loss }')\n        print(f'avarage val_auc {val_auc}')\n        \n        \n        \n        \n        if (val_auc>best_acc):\n            best_acc =val_auc\n            print(f'saving model for {best_acc}')\n            torch.save(model.state_dict(),OUTPUT_DIR+ f'Fold {fold_n} model with val_acc {best_acc}.pth')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fit(model,0)","metadata":{"trusted":true},"execution_count":null,"outputs":[]}]}