{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Fracture Detection - Baseline #3 3D CNN Classification\n\n![Competition image](https://i.imgur.com/3iLsS6i.png)\n\n- *Author: Mariusz Wiśniewski*\n- *Competition: [RSNA 2022 Cervical Spine Fracture Detection](https://www.kaggle.com/competitions/rsna-2022-cervical-spine-fracture-detection)*\n\n## Overview\n\nIn this notebook, we will train a neural network to detect fractures of *C1-C7* cervical vertebrae from computer tomography (CT) scans.\n\n### Libraries Used\n\n- [PyTorch 🔥](https://pytorch.org)\n- [YOLOv5 🚀](https://github.com/ultralytics/yolov5)\n- [Weights&Biases 📈](https://wandb.ai/)\n\n### References\n\n- [[RSNA_22] Dicom to NumPy 3D 📓](https://www.kaggle.com/code/vmuzhichenko/rsna-22-dicom-to-numpy-3d)\n- [Yolov5 Cervical Spine (Neck) Fracture Detection 📓](https://www.kaggle.com/code/vbookshelf/yolov5-cervical-spine-neck-fracture-detection/notebook)\n- [YOLOv4: Optimal Speed and Accuracy of Object Detection 📃](https://arxiv.org/abs/2004.10934v1)","metadata":{"execution":{"iopub.execute_input":"2022-08-06T08:00:38.103969Z","iopub.status.busy":"2022-08-06T08:00:38.103499Z","iopub.status.idle":"2022-08-06T08:00:38.109681Z","shell.execute_reply":"2022-08-06T08:00:38.108025Z","shell.execute_reply.started":"2022-08-06T08:00:38.103929Z"},"papermill":{"duration":0.096947,"end_time":"2021-07-18T05:37:05.065223","exception":false,"start_time":"2021-07-18T05:37:04.968276","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"# About the Task\n\nIn this notebook we will train the *YOLOv5* object detection model to automatically detect fractures on CT scans of the cervical spine.\n\n- **Input:** CT scans of cervical vetebrae in *PNG* format.\n- **Output:** Bounding box coordinates, and a score that indicates the probability that a fracture is present.\n\nThis model does not identify which vertebra is fractured. The predictions are only for the `patient_overall` class.","metadata":{}},{"cell_type":"markdown","source":"# Project Setup","metadata":{}},{"cell_type":"markdown","source":"## Import Statements","metadata":{}},{"cell_type":"code","source":"import gc\nimport itertools\nimport os\nimport random\nimport shutil\nimport time\nimport warnings\nfrom pprint import pprint\n\nimport albumentations as albu\nimport cv2\nimport matplotlib.pyplot as plt\nimport numpy as np\nimport pandas as pd\nimport seaborn as sn\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nimport torch.optim as optim\nimport torchvision\nimport torchvision.transforms as transforms\nimport wandb\nimport yaml\nfrom albumentations import Compose\nfrom kaggle_secrets import UserSecretsClient\nfrom numpy import asarray\nfrom PIL import Image\nfrom skimage.transform import resize\nfrom sklearn import metrics, model_selection\nfrom sklearn.metrics import (classification_report, confusion_matrix,\n                             jaccard_score)\nfrom sklearn.model_selection import StratifiedShuffleSplit\nfrom sklearn.utils import shuffle\nfrom torch.utils.data import DataLoader, Dataset\nfrom torchvision import transforms, utils\nfrom torchmetrics.detection.mean_ap import MeanAveragePrecision\nfrom tqdm.notebook import tqdm\n\nwarnings.filterwarnings('ignore')","metadata":{"papermill":{"duration":11.55616,"end_time":"2021-07-18T05:38:38.821538","exception":false,"start_time":"2021-07-18T05:38:27.265378","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-01-02T01:32:20.7589Z","iopub.execute_input":"2023-01-02T01:32:20.75938Z","iopub.status.idle":"2023-01-02T01:32:20.800336Z","shell.execute_reply.started":"2023-01-02T01:32:20.759339Z","shell.execute_reply":"2023-01-02T01:32:20.79923Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Random Seed","metadata":{}},{"cell_type":"code","source":"# Random seed for reproducibility\nseed = 27\n\nos.environ['PYTHONHASHSEED'] = str(seed)\nrandom.seed(seed)\nnp.random.seed(seed)\ntorch.manual_seed(seed)\ntorch.cuda.manual_seed_all(seed)\ntorch.backends.cudnn.deterministic = True","metadata":{"papermill":{"duration":0.227737,"end_time":"2021-07-18T05:38:39.265366","exception":false,"start_time":"2021-07-18T05:38:39.037629","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-01-01T23:29:06.270003Z","iopub.execute_input":"2023-01-01T23:29:06.272735Z","iopub.status.idle":"2023-01-01T23:29:06.282907Z","shell.execute_reply.started":"2023-01-01T23:29:06.272691Z","shell.execute_reply":"2023-01-01T23:29:06.281817Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Weights & Biases Setup","metadata":{}},{"cell_type":"code","source":"user_secrets = UserSecretsClient()\nwandb.login(key=user_secrets.get_secret('WANDB_API_KEY'))","metadata":{"execution":{"iopub.status.busy":"2023-01-01T23:29:07.210208Z","iopub.execute_input":"2023-01-01T23:29:07.211064Z","iopub.status.idle":"2023-01-01T23:29:08.109789Z","shell.execute_reply.started":"2023-01-01T23:29:07.211029Z","shell.execute_reply":"2023-01-01T23:29:08.10847Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Downloading Saved Weights","metadata":{}},{"cell_type":"code","source":"api = wandb.Api()\nrun = 'run_1bvetl83_model:v44'\nartifact = api.artifact(f'nexer8/FD_Object_Detection/{run}', type='model')\nartifact.download()","metadata":{"execution":{"iopub.status.busy":"2023-01-01T23:29:12.163527Z","iopub.execute_input":"2023-01-01T23:29:12.163948Z","iopub.status.idle":"2023-01-01T23:29:14.079684Z","shell.execute_reply.started":"2023-01-01T23:29:12.163911Z","shell.execute_reply":"2023-01-01T23:29:14.078611Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!mv 'artifacts/run_1bvetl83_model:v44/best.pt' best.pt\n!rm -rf artifacts/","metadata":{"execution":{"iopub.status.busy":"2023-01-01T23:29:23.092526Z","iopub.execute_input":"2023-01-01T23:29:23.092917Z","iopub.status.idle":"2023-01-01T23:29:25.281604Z","shell.execute_reply.started":"2023-01-01T23:29:23.092888Z","shell.execute_reply":"2023-01-01T23:29:25.279493Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Project Configuration","metadata":{}},{"cell_type":"code","source":"EPOCHS = 45\nBATCH_SIZE = 32\nIMAGE_SIZE = 512\n\nNUM_CORES = os.cpu_count()\nprint(f'Number of cores: {NUM_CORES}')","metadata":{"papermill":{"duration":0.225834,"end_time":"2021-07-18T05:38:40.577433","exception":false,"start_time":"2021-07-18T05:38:40.351599","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-01-01T23:29:28.420483Z","iopub.execute_input":"2023-01-01T23:29:28.420939Z","iopub.status.idle":"2023-01-01T23:29:28.427803Z","shell.execute_reply.started":"2023-01-01T23:29:28.420899Z","shell.execute_reply":"2023-01-01T23:29:28.426484Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Setting up YOLOv5\n\nThe YOLOv5 model being used here needs to have the internet turned for training to work. However, it does not need to have the internet on during inference.","metadata":{"papermill":{"duration":0.215149,"end_time":"2021-07-18T05:38:41.008781","exception":false,"start_time":"2021-07-18T05:38:40.793632","status":"completed"},"tags":[]}},{"cell_type":"code","source":"!git clone https://github.com/ultralytics/yolov5\n%cd yolov5/\n%pip install -qr requirements.txt\n%cd ../","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2023-01-01T23:29:33.092358Z","iopub.execute_input":"2023-01-01T23:29:33.092733Z","iopub.status.idle":"2023-01-01T23:29:48.740249Z","shell.execute_reply.started":"2023-01-01T23:29:33.092703Z","shell.execute_reply":"2023-01-01T23:29:48.738592Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Data Preparation","metadata":{"papermill":{"duration":0.215286,"end_time":"2021-07-18T05:38:44.189361","exception":false,"start_time":"2021-07-18T05:38:43.974075","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"## About the Dataset\n\nThe training data for the competition is made up of 2019 patient volumetric CT scans. Each scan comprises numerous cervical spine images (slices) in *DICOM* format. There are 7217 images with bounding boxes annotations indicating the precise location of fractures.\n\nAll images that have bounding box annotations were chosen to build the dataset. In addition, 7217 scans with no fractures were chosen at random. All *DICOM* images were converted to *PNG* format without any resizing ([link to the notebook](https://www.kaggle.com/code/mariuszwisniewski/rsna-fd-data-preparation-for-object-detection).","metadata":{}},{"cell_type":"markdown","source":"## Loading and Preprocessing","metadata":{}},{"cell_type":"code","source":"def create_study_slice(row):\n    return f'{row[\"StudyInstanceUID\"]}_{row[\"slice_number\"]}'","metadata":{"execution":{"iopub.status.busy":"2023-01-01T23:29:48.743296Z","iopub.execute_input":"2023-01-01T23:29:48.744182Z","iopub.status.idle":"2023-01-01T23:29:48.750049Z","shell.execute_reply.started":"2023-01-01T23:29:48.74413Z","shell.execute_reply":"2023-01-01T23:29:48.7488Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"TRAIN_DATA_CSV_PATH = '../input/rsna-fracture-detection-bbox-train/train.csv'\nTRAIN_IMG_PATH = '../input/rsna-fracture-detection-bbox-train/train_images'\ndata_df = pd.read_csv(TRAIN_DATA_CSV_PATH)\ndata_df['study_slice'] = data_df.apply(create_study_slice, axis=1)\ndata_df.head()","metadata":{"papermill":{"duration":0.415195,"end_time":"2021-07-18T05:38:44.823811","exception":false,"start_time":"2021-07-18T05:38:44.408616","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-01-01T23:29:48.752195Z","iopub.execute_input":"2023-01-01T23:29:48.752684Z","iopub.status.idle":"2023-01-01T23:29:49.070056Z","shell.execute_reply.started":"2023-01-01T23:29:48.752636Z","shell.execute_reply":"2023-01-01T23:29:49.068832Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"data_df['label'].value_counts()","metadata":{"execution":{"iopub.status.busy":"2023-01-01T23:29:49.0727Z","iopub.execute_input":"2023-01-01T23:29:49.073335Z","iopub.status.idle":"2023-01-01T23:29:49.08529Z","shell.execute_reply.started":"2023-01-01T23:29:49.073298Z","shell.execute_reply":"2023-01-01T23:29:49.08403Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Adding a column with bbox data","metadata":{}},{"cell_type":"code","source":"BBOX_TRAIN_CSV_PATH = '../input/rsna-2022-cervical-spine-fracture-detection/train_bounding_boxes.csv'\nbbox_data_df = pd.read_csv(BBOX_TRAIN_CSV_PATH)\nbbox_data_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-01-01T23:29:54.252165Z","iopub.execute_input":"2023-01-01T23:29:54.252615Z","iopub.status.idle":"2023-01-01T23:29:54.289702Z","shell.execute_reply.started":"2023-01-01T23:29:54.252578Z","shell.execute_reply":"2023-01-01T23:29:54.288498Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"bbox_data_df['study_slice'] = bbox_data_df.apply(create_study_slice, axis=1)\nbbox_data_df = bbox_data_df.set_index('study_slice')\nbbox_data_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-01-01T23:29:57.897704Z","iopub.execute_input":"2023-01-01T23:29:57.898128Z","iopub.status.idle":"2023-01-01T23:29:58.03536Z","shell.execute_reply.started":"2023-01-01T23:29:57.898096Z","shell.execute_reply":"2023-01-01T23:29:58.034077Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Verify that there is only 1 bbox annotation per slice.\ncases = sorted(list(bbox_data_df['StudyInstanceUID'].unique()))\n\nfor case in cases:\n    slice_per_instance_df = bbox_data_df[bbox_data_df['StudyInstanceUID'] == case][[\n        'slice_number']]\n    assert slice_per_instance_df.count(\n    ).values[0] == slice_per_instance_df.nunique().values[0]","metadata":{"execution":{"iopub.status.busy":"2023-01-01T23:30:02.506785Z","iopub.execute_input":"2023-01-01T23:30:02.507249Z","iopub.status.idle":"2023-01-01T23:30:03.183831Z","shell.execute_reply.started":"2023-01-01T23:30:02.507197Z","shell.execute_reply":"2023-01-01T23:30:03.18266Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"study_slices = list(data_df['study_slice'])\nbbox_list = []\nfor i in range(len(data_df)):\n    target = data_df.loc[i, 'label']\n    study_slice = data_df.loc[i, 'study_slice']\n\n    if target == 1:\n        x = bbox_data_df.loc[study_slice, 'x']\n        y = bbox_data_df.loc[study_slice, 'y']\n        width = bbox_data_df.loc[study_slice, 'width']\n        height = bbox_data_df.loc[study_slice, 'height']\n\n        bbox_list.append({\n            'x': x,\n            'y': y,\n            'width': width,\n            'height': height\n        })\n\n    else:\n        bbox_list.append('none')\n\n\ndata_df['bboxes'] = bbox_list\ndata_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-01-01T23:30:05.601198Z","iopub.execute_input":"2023-01-01T23:30:05.601693Z","iopub.status.idle":"2023-01-01T23:30:06.255593Z","shell.execute_reply.started":"2023-01-01T23:30:05.601652Z","shell.execute_reply":"2023-01-01T23:30:06.254313Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Build Train, Validation, and Test Datasets\n\nThe data was split into 3 separate sets: 60% of all images for training, 20% for validation, and 20% for testing using stratified split by class (0 - `normal`, 1 - `fracture`).","metadata":{"papermill":{"duration":0.215665,"end_time":"2021-07-18T05:38:57.522433","exception":false,"start_time":"2021-07-18T05:38:57.306768","status":"completed"},"tags":[]}},{"cell_type":"code","source":"train_indices, test_indices = next(\n    StratifiedShuffleSplit(1, train_size=0.8, random_state=seed).split(\n        data_df['study_slice'], data_df['label']\n    )\n)\n\ntmp_labels = [data_df['label'].values[idx] for idx in train_indices]\ntmp_paths = [data_df['study_slice'].values[idx] for idx in train_indices]\n\ntmp_train_indices, tmp_val_indices = next(\n    StratifiedShuffleSplit(1, train_size=0.75, random_state=seed).split(\n        tmp_paths, tmp_labels\n    )\n)\n\ntmp_train_paths = [tmp_paths[idx] for idx in tmp_train_indices]\ntmp_val_paths = [tmp_paths[idx] for idx in tmp_val_indices]\n\ntrain_indices = [data_df.index[data_df['study_slice']\n                               == path].values[0] for path in tmp_train_paths]\nval_indices = [data_df.index[data_df['study_slice'] == path].values[0]\n               for path in tmp_val_paths]\n\n# Check if there are no common indices\nassert not bool(set(train_indices) & set(val_indices))\nassert not bool(set(train_indices) & set(test_indices))\nassert not bool(set(val_indices) & set(test_indices))\n\nprint(\n    'Number of samples:\\n'\n    f'train: {len(train_indices)} ({round(len(train_indices) / len(data_df) * 100)}% of the dataset)\\n'\n    f'validation: {len(val_indices)} ({round(len(val_indices) / len(data_df) * 100)}% of the dataset)\\n'\n    f'test: {len(test_indices)} ({round(len(test_indices) / len(data_df) * 100)}% of the dataset)'\n)","metadata":{"execution":{"iopub.status.busy":"2023-01-01T23:30:19.670886Z","iopub.execute_input":"2023-01-01T23:30:19.671921Z","iopub.status.idle":"2023-01-01T23:30:33.984579Z","shell.execute_reply.started":"2023-01-01T23:30:19.671873Z","shell.execute_reply":"2023-01-01T23:30:33.983329Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Create a correct directory structure\n\nWe need to create a directory structure inside the `yolov5` folder that will look as follows.\n\n```\nBASE_DIR\n├── images\n│   ├── train\n│   └── validation\n└── labels\n    ├── train\n    └── validation\n```","metadata":{"papermill":{"duration":0.214448,"end_time":"2021-07-18T05:39:00.717004","exception":false,"start_time":"2021-07-18T05:39:00.502556","status":"completed"},"tags":[]}},{"cell_type":"code","source":"os.chdir('/kaggle/working/yolov5')\nBASE_DIR = 'base_dir'\nos.makedirs(f'{BASE_DIR}/images/train', exist_ok=True)\nos.makedirs(f'{BASE_DIR}/images/validation', exist_ok=True)\nos.makedirs(f'{BASE_DIR}/images/test', exist_ok=True)\nos.makedirs(f'{BASE_DIR}/labels/train', exist_ok=True)\nos.makedirs(f'{BASE_DIR}/labels/validation', exist_ok=True)\nos.makedirs(f'{BASE_DIR}/labels/test', exist_ok=True)\n\n# Display the folder structure\n!tree {BASE_DIR}","metadata":{"papermill":{"duration":1.026715,"end_time":"2021-07-18T05:39:02.410223","exception":false,"start_time":"2021-07-18T05:39:01.383508","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-01-01T23:30:33.987032Z","iopub.execute_input":"2023-01-01T23:30:33.988011Z","iopub.status.idle":"2023-01-01T23:30:35.109361Z","shell.execute_reply.started":"2023-01-01T23:30:33.987962Z","shell.execute_reply":"2023-01-01T23:30:35.107953Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Create the Datasets","metadata":{"papermill":{"duration":0.376731,"end_time":"2021-07-18T05:39:03.198014","exception":false,"start_time":"2021-07-18T05:39:02.821283","status":"completed"},"tags":[]}},{"cell_type":"code","source":"os.chdir('/kaggle/working/')","metadata":{"papermill":{"duration":0.227543,"end_time":"2021-07-18T05:39:03.850619","exception":false,"start_time":"2021-07-18T05:39:03.623076","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-01-01T23:30:35.1111Z","iopub.execute_input":"2023-01-01T23:30:35.112436Z","iopub.status.idle":"2023-01-01T23:30:35.11949Z","shell.execute_reply.started":"2023-01-01T23:30:35.112347Z","shell.execute_reply":"2023-01-01T23:30:35.118137Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df = data_df[data_df.index.isin(train_indices)]\nval_df = data_df[data_df.index.isin(val_indices)]\ntest_df = data_df[data_df.index.isin(test_indices)]\n\nprint(f'Training set:\\n{train_df[\"label\"].value_counts()}')\nprint(f'Validation set:\\n{val_df[\"label\"].value_counts()}')\nprint(f'Testing set:\\n{test_df[\"label\"].value_counts()}')","metadata":{"execution":{"iopub.status.busy":"2023-01-01T23:30:35.121947Z","iopub.execute_input":"2023-01-01T23:30:35.122563Z","iopub.status.idle":"2023-01-01T23:30:35.149536Z","shell.execute_reply.started":"2023-01-01T23:30:35.122526Z","shell.execute_reply":"2023-01-01T23:30:35.148122Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# YOLOv5\n\nThe model expects the bounding box data to be in range [0, 1]. Furthermore, it requires the label data to be saved as *.txt* file with it contents formatted as follows: `class x-center y-center width height`, e.g. `0 0.1 0.2 200 300`.","metadata":{}},{"cell_type":"markdown","source":"## Processing Data for the Model","metadata":{}},{"cell_type":"code","source":"# Iterate through each row in the dataframe\n\n# We run the function below separately for\n# the train and val sets.\n# Remember that each image gets it's own text file\n# containing the info for all bboxes on that image.\n\n# For each image:\n# 1- get the info for each bounding box\n# 2- write the bounding box info to a txt file\n# 3- save the txt file in the correct folder\n# 4- copy the image to the correct folder\n\n# Note on bboxes:\n# For each image we have a list of dictionaries. Each dict\n# contains the coords one one bbox on that image.\n# We don't need to do anything if the image does not have any bboxes.\ndef process_data_for_yolo(df, data_type='train'):\n    for _, row in tqdm(df.iterrows(), total=len(df)):\n        # Get the target\n        target = row['label']\n        # Create the image file name\n        study_slice = row['study_slice']\n        fname = f'{study_slice}.png'\n\n        # Only create txt files for class 1 images\n        if target == 1:\n            # Get the list of bboxes on the image.\n            # Each item in the list is a dict containing the image coords.\n            bbox_dict = row['bboxes']\n            # put the coords into a list\n            bbox_list = [bbox_dict]\n            # These are the original image sizes.\n            # If we have resized the images then this must be changed to\n            # the new sizes. We will then also be using resized bbox coords.\n            image_width = row['width']\n            image_height = row['height']\n\n            # Convert into the Yolo input format\n            # ...................................\n            yolo_data = []\n            # We only have one class i.e. opacity\n            # We will set the class_id to 0 for all images.\n            # Class numbers must start from 0.\n            class_id = target\n            # row by row\n            for coord_dict in bbox_list:\n                xmin = int(coord_dict['x'])\n                ymin = int(coord_dict['y'])\n                bbox_w = int(coord_dict['width'])\n                bbox_h = int(coord_dict['height'])\n\n                x_center = xmin + (bbox_w/2)\n                y_center = ymin + (bbox_h/2)\n\n                # Normalize\n                # Yolo expects the dimensions to be normalized i.e.\n                # all values between 0 and 1.\n                x_center = x_center/image_width\n                y_center = y_center/image_height\n                bbox_w /= image_width\n                bbox_h /= image_height\n\n                # [class_id, x-center, y-center, width, height]\n                yolo_list = [class_id, x_center, y_center, bbox_w, bbox_h]\n                yolo_data.append(yolo_list)\n\n            # convert to nump array\n            yolo_data = np.array(yolo_data)\n\n            # Write the image bbox info to a txt file\n            #image_id = image_name.split('.')[0]\n            np.savetxt(os.path.join(f'yolov5/{BASE_DIR}',\n                                    f\"labels/{data_type}/{study_slice}.txt\"),\n                       yolo_data,\n                       fmt=[\"%d\", \"%f\", \"%f\", \"%f\", \"%f\"]\n                       )  # fmt means format the columns\n\n        # Copy the image to images\n        # Set the path to the images here.\n        shutil.copyfile(\n            f'{TRAIN_IMG_PATH}/{fname}',\n            os.path.join(f'yolov5/{BASE_DIR}', f'images/{data_type}/{fname}')\n        )\n\n\nprocess_data_for_yolo(train_df, data_type='train')\nprocess_data_for_yolo(val_df, data_type='validation')\nprocess_data_for_yolo(test_df, data_type='test')","metadata":{"papermill":{"duration":3.555252,"end_time":"2021-07-18T05:39:11.818383","exception":false,"start_time":"2021-07-18T05:39:08.263131","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-01-01T23:30:43.104346Z","iopub.execute_input":"2023-01-01T23:30:43.104733Z","iopub.status.idle":"2023-01-01T23:33:03.363672Z","shell.execute_reply.started":"2023-01-01T23:30:43.104702Z","shell.execute_reply":"2023-01-01T23:33:03.362457Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Check that the files have been created\nprint(len(os.listdir(f'yolov5/{BASE_DIR}/images/train')))\nprint(len(os.listdir(f'yolov5/{BASE_DIR}/images/validation')))\nprint(len(os.listdir(f'yolov5/{BASE_DIR}/images/test')))\n\nprint(len(os.listdir(f'yolov5/{BASE_DIR}/labels/train')))\nprint(len(os.listdir(f'yolov5/{BASE_DIR}/labels/validation')))\nprint(len(os.listdir(f'yolov5/{BASE_DIR}/labels/test')))","metadata":{"papermill":{"duration":0.237175,"end_time":"2021-07-18T05:39:12.278949","exception":false,"start_time":"2021-07-18T05:39:12.041774","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-01-01T23:33:07.740764Z","iopub.execute_input":"2023-01-01T23:33:07.741146Z","iopub.status.idle":"2023-01-01T23:33:07.763984Z","shell.execute_reply.started":"2023-01-01T23:33:07.741115Z","shell.execute_reply":"2023-01-01T23:33:07.762834Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Create the Dataset File\nYolo requires that we also create a yaml file inside the yolov5 folder.","metadata":{"papermill":{"duration":0.222359,"end_time":"2021-07-18T05:39:14.419466","exception":false,"start_time":"2021-07-18T05:39:14.197107","status":"completed"},"tags":[]}},{"cell_type":"code","source":"data = {\n    'train': f'{BASE_DIR}/images/train',\n    'val': f'{BASE_DIR}/images/validation',\n    'nc': 2,\n    'names': ['normal', 'fracture']\n}\n\n\nwith open(r'yolov5/data.yaml', 'w') as file:\n    documents = yaml.dump(data, file)","metadata":{"papermill":{"duration":0.245553,"end_time":"2021-07-18T05:39:14.886293","exception":false,"start_time":"2021-07-18T05:39:14.64074","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Create a Custom Hyperameter/Augmentation File","metadata":{"papermill":{"duration":0.223307,"end_time":"2021-07-18T05:39:17.039924","exception":false,"start_time":"2021-07-18T05:39:16.816617","status":"completed"},"tags":[]}},{"cell_type":"code","source":"hyperparams = {\n    'lr0': 0.01,  # initial learning rate (SGD=1E-2, Adam=1E-3)\n    'lrf': 0.032,  # final OneCycleLR learning rate (lr0 * lrf)\n    'momentum': 0.937,  # SGD momentum/Adam beta1\n    'weight_decay': 0.0005,  # optimizer weight decay 5e-4\n    'warmup_epochs': 3.0,  # warmup epochs (fractions ok)\n    'warmup_momentum': 0.8,  # warmup initial momentum\n    'warmup_bias_lr': 0.1,  # warmup initial bias lr\n    'box': 0.1,  # box loss gain\n    'cls': 1.0,  # cls loss gain\n    'cls_pw': 0.5,  # cls BCELoss positive_weight\n    'obj': 2.0,  # obj loss gain (scale with pixels)\n    'obj_pw': 0.5,  # obj BCELoss positive_weight\n    'iou_t': 0.20,  # IoU training threshold\n    'anchor_t': 4.0,  # anchor-multiple threshold\n    'anchors': 0,  # anchors per output layer (0 to ignore)\n    'fl_gamma': 0.0,  # focal loss gamma (efficientDet default gamma=1.5)\n    'hsv_h': 0,  # image HSV-Hue augmentation (fraction)\n    'hsv_s': 0,  # image HSV-Saturation augmentation (fraction)\n    'hsv_v': 0,  # image HSV-Value augmentation (fraction)\n    'degrees': 30.0,  # image rotation (+/- deg)\n    'translate': 0.2,  # image translation (+/- fraction)\n    'scale': 0.3,  # image scale (+/- gain)\n    'shear': 0.0,  # image shear (+/- deg)\n    'perspective': 0.0,  # image perspective (+/- fraction), range 0-0.001\n    'flipud': 0.2,  # image flip up-down (probability)\n    'fliplr': 0.5,  # image flip left-right (probability)\n    'mosaic': 0.8,  # image mosaic (probability)\n    'mixup': 0.0,  # image mixup (probability)\n    'copy_paste': 0.0\n}\n\n\nwith open(r'yolov5/hyperparams.yaml', 'w') as file:\n    documents = yaml.dump(hyperparams, file)","metadata":{"papermill":{"duration":0.239227,"end_time":"2021-07-18T05:39:17.506221","exception":false,"start_time":"2021-07-18T05:39:17.266994","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Model Training","metadata":{"papermill":{"duration":0.239099,"end_time":"2021-07-18T05:39:19.65831","exception":false,"start_time":"2021-07-18T05:39:19.419211","status":"completed"},"tags":[]}},{"cell_type":"code","source":"os.chdir('/kaggle/working/yolov5/')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!python train.py --img {IMAGE_SIZE} \\\n    --batch {BATCH_SIZE} \\\n    --epochs {EPOCHS} \\\n    --data data.yaml \\\n    --hyp hyperparams.yaml \\\n    --weights /kaggle/working/best.pt \\\n    --save-period 1 \\\n    --project FD_Object_Detection \\\n    --name 0-mw-yolov5-baseline-round4 \\\n    > /kaggle/working/model_training.log 2>&1","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Copy the trained model\n\nWe will now copy the best trained model to the *Kaggle* working directory. This will make the model easier to access.","metadata":{"papermill":{"duration":3.382839,"end_time":"2021-07-18T06:53:20.977817","exception":false,"start_time":"2021-07-18T06:53:17.594978","status":"completed"},"tags":[]}},{"cell_type":"code","source":"train_run = 'FD_Object_Detection'\nexperiment = '0-mw-yolov5-baseline-round4'","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"shutil.copyfile(\n    f'/kaggle/working/yolov5/{train_run}/{experiment}/weights/best.pt',\n    '/kaggle/working/best.pt'\n)","metadata":{"papermill":{"duration":4.267281,"end_time":"2021-07-18T06:53:43.461979","exception":false,"start_time":"2021-07-18T06:53:39.194698","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Get the Last Experiment\n\n*YOLOv5* saves every training run as an experiment.","metadata":{"papermill":{"duration":3.532048,"end_time":"2021-07-18T06:53:51.514652","exception":false,"start_time":"2021-07-18T06:53:47.982604","status":"completed"},"tags":[]}},{"cell_type":"code","source":"exp_list = os.listdir(f'/kaggle/working/yolov5/{train_run}')\nexp = exp_list[0]\nprint(exp)","metadata":{"papermill":{"duration":4.55418,"end_time":"2021-07-18T06:53:59.475607","exception":false,"start_time":"2021-07-18T06:53:54.921427","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"os.listdir(f'{train_run}/{exp}')","metadata":{"papermill":{"duration":3.352582,"end_time":"2021-07-18T06:54:27.864267","exception":false,"start_time":"2021-07-18T06:54:24.511685","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Visualizing Training History\n\n- All the training curves are stored as one *.png* file.\n- The summary displayed at the end of training presents the results for the **last** epoch, which does not necessarily correspond to the **best** one. The results for each epoch are inside `results.txt`. We will load this file to get the best epoch as well as the corresponding *mAP* score.\n- Images showing the true and predicted bounding boxes for each validation bach are saved as well.","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize = (15, 15))\nplt.imshow(plt.imread(f'{train_run}/{exp}/results.png'))","metadata":{"papermill":{"duration":4.331596,"end_time":"2021-07-18T06:55:04.665328","exception":false,"start_time":"2021-07-18T06:55:00.333732","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Get the Best Epoch and mAP","metadata":{"papermill":{"duration":3.325504,"end_time":"2021-07-18T06:55:11.376077","exception":false,"start_time":"2021-07-18T06:55:08.050573","status":"completed"},"tags":[]}},{"cell_type":"code","source":"full_results_df = pd.read_csv(f'{train_run}/{exp}/results.csv')\nfull_results_df.columns = full_results_df.columns.str.replace(' ', '')\nfull_results_df.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# choose only the columns we want\ncol_names = [\n    'epoch',\n    'metrics/precision',\n    'metrics/recall',\n    'metrics/mAP_0.5',\n    'metrics/mAP_0.5:0.95'\n]\n# filter out specific columns\nresults_df = full_results_df[col_names]\n# results_df.columns = col_names\nresults_df.head(10)","metadata":{"papermill":{"duration":3.75076,"end_time":"2021-07-18T06:55:48.88542","exception":false,"start_time":"2021-07-18T06:55:45.13466","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Get the best map0.5\nbest_map = results_df['metrics/mAP_0.5'].max()\nprint('---------------------')\nprint(f'Best mAP_0.5:{best_map}\\n')\n\n# print the row that contains the best map0.5\nfull_results_df = results_df[results_df['metrics/mAP_0.5'] == best_map]\nprint(full_results_df.head())\nprint('---------------------')","metadata":{"papermill":{"duration":3.438387,"end_time":"2021-07-18T06:55:55.694216","exception":false,"start_time":"2021-07-18T06:55:52.255829","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Display a Batch of Training Images","metadata":{"execution":{"iopub.execute_input":"2021-07-05T08:19:53.683161Z","iopub.status.busy":"2021-07-05T08:19:53.682523Z","iopub.status.idle":"2021-07-05T08:19:53.68887Z","shell.execute_reply":"2021-07-05T08:19:53.687579Z","shell.execute_reply.started":"2021-07-05T08:19:53.683105Z"},"papermill":{"duration":3.572099,"end_time":"2021-07-18T06:56:03.775207","exception":false,"start_time":"2021-07-18T06:56:00.203108","status":"completed"},"tags":[]}},{"cell_type":"code","source":"plt.figure(figsize=(15, 15))\nplt.imshow(plt.imread(f'{train_run}/{exp}/train_batch0.jpg'))","metadata":{"papermill":{"duration":4.375391,"end_time":"2021-07-18T06:56:11.494857","exception":false,"start_time":"2021-07-18T06:56:07.119466","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Display True and Predicted BBoxes\n\nHere we will display the true and predicted bboxes for two validation batches.","metadata":{"papermill":{"duration":3.378424,"end_time":"2021-07-18T06:56:18.287695","exception":false,"start_time":"2021-07-18T06:56:14.909271","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# BATCH 0 - TRUE BBOXES\nplt.figure(figsize=(15, 15))\nplt.imshow(plt.imread(f'{train_run}/{exp}/val_batch0_labels.jpg'))","metadata":{"papermill":{"duration":3.956924,"end_time":"2021-07-18T06:56:25.962776","exception":false,"start_time":"2021-07-18T06:56:22.005852","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# BATCH 0 - PREDICTED BBOXES\nplt.figure(figsize=(15, 15))\nplt.imshow(plt.imread(f'{train_run}/{exp}/val_batch0_pred.jpg'))","metadata":{"papermill":{"duration":4.657357,"end_time":"2021-07-18T06:56:34.292266","exception":false,"start_time":"2021-07-18T06:56:29.634909","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# BATCH 1 - TRUE BBOXES\nplt.figure(figsize=(15, 15))\nplt.imshow(plt.imread(f'{train_run}/{exp}/val_batch1_labels.jpg'))","metadata":{"papermill":{"duration":4.052771,"end_time":"2021-07-18T06:56:41.741357","exception":false,"start_time":"2021-07-18T06:56:37.688586","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# BATCH 1 - PREDICTED BBOXES\nplt.figure(figsize=(15, 15))\nplt.imshow(plt.imread(f'{train_run}/{exp}/val_batch1_pred.jpg'))","metadata":{"papermill":{"duration":4.138354,"end_time":"2021-07-18T06:56:49.649222","exception":false,"start_time":"2021-07-18T06:56:45.510868","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Model Testing\n\nNow we will evaluate our model by running the detection on all the testing images. Keep in mind that a *.txt* file will be created only if the model is able to detect some objects on an image.","metadata":{"papermill":{"duration":4.04488,"end_time":"2021-07-18T06:57:04.446831","exception":false,"start_time":"2021-07-18T06:57:00.401951","status":"completed"},"tags":[]}},{"cell_type":"code","source":"test_images_path = f'{BASE_DIR}/images/test'\nbest_model_path = '/kaggle/working/best.pt'\n\n!python detect.py --source {test_images_path} \\\n    --weights {best_model_path} \\\n    --img {IMAGE_SIZE} \\\n    --save-txt \\\n    --save-conf \\\n    --exist-ok \\\n    > /kaggle/working/model_testing.log 2>&1 ","metadata":{"_kg_hide-output":true,"papermill":{"duration":41.782806,"end_time":"2021-07-18T06:58:29.775248","exception":false,"start_time":"2021-07-18T06:57:47.992442","status":"completed"},"tags":[],"_kg_hide-input":false,"execution":{"iopub.status.busy":"2023-01-01T23:34:48.082956Z","iopub.execute_input":"2023-01-01T23:34:48.083447Z","iopub.status.idle":"2023-01-02T00:09:06.75099Z","shell.execute_reply.started":"2023-01-01T23:34:48.083404Z","shell.execute_reply":"2023-01-02T00:09:06.747094Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Processing Labels","metadata":{}},{"cell_type":"code","source":"BBOX_LABELS_PATH = f'{BASE_DIR}/labels/test'\nbbox_label_files = os.listdir(BBOX_LABELS_PATH)\ncols = ['class', 'x-center', 'y-center',\n        'bbox_width', 'bbox_height']\n\nfor i, bbox_file in enumerate(bbox_label_files):\n    # set the path\n    path = f'{BBOX_LABELS_PATH}/{bbox_file}'\n\n    # put the file contents into a dataframe\n    df = pd.read_csv(path, sep=\" \", header=None)\n\n    # add the column names to the datafrae\n    df.columns = cols\n\n    fname = bbox_file.replace('txt', 'png')\n\n    # add a new column with the fname\n    df['file_name'] = fname\n\n    # stack the dataframes for each txt file\n    test_true_df = df if i == 0 else pd.concat([test_true_df, df], axis=0)\ntest_true_df.head()","metadata":{"execution":{"iopub.status.busy":"2023-01-02T01:31:22.520938Z","iopub.execute_input":"2023-01-02T01:31:22.521493Z","iopub.status.idle":"2023-01-02T01:31:26.829623Z","shell.execute_reply.started":"2023-01-02T01:31:22.521453Z","shell.execute_reply":"2023-01-02T01:31:26.828275Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Processing Predictions","metadata":{"papermill":{"duration":3.642808,"end_time":"2021-07-18T06:58:37.370203","exception":false,"start_time":"2021-07-18T06:58:33.727395","status":"completed"},"tags":[]}},{"cell_type":"code","source":"txt_files_list = os.listdir('runs/detect/exp/labels')\nprint(len(txt_files_list))\nprint(txt_files_list[0])","metadata":{"papermill":{"duration":4.536142,"end_time":"2021-07-18T06:58:45.507384","exception":false,"start_time":"2021-07-18T06:58:40.971242","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-01-02T01:31:26.83178Z","iopub.execute_input":"2023-01-02T01:31:26.83229Z","iopub.status.idle":"2023-01-02T01:31:26.840381Z","shell.execute_reply.started":"2023-01-02T01:31:26.832224Z","shell.execute_reply":"2023-01-02T01:31:26.839268Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Put the info inside all the txt files into one dataframe.\ntxt_files_list = os.listdir('runs/detect/exp/labels')\n# create a list of column names\ncols = ['class', 'x-center', 'y-center',\n        'bbox_width', 'bbox_height', 'conf-score']\n\nfor i, txt_file in enumerate(txt_files_list):\n    # set the path\n    path = f'runs/detect/exp/labels/{txt_file}'\n\n    # put the file contents into a dataframe\n    df = pd.read_csv(path, sep=\" \", header=None)\n\n    # add the column names to the datafrae\n    df.columns = cols\n\n    fname = txt_file.replace('txt', 'png')\n\n    # add a new column with the fname\n    df['file_name'] = fname\n\n    # stack the dataframes for each txt file\n    test_preds_df = df if i == 0 else pd.concat([test_preds_df, df], axis=0)\ntest_preds_df.head()","metadata":{"papermill":{"duration":5.758562,"end_time":"2021-07-18T06:58:54.922543","exception":false,"start_time":"2021-07-18T06:58:49.163981","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-01-02T01:31:28.03096Z","iopub.execute_input":"2023-01-02T01:31:28.031443Z","iopub.status.idle":"2023-01-02T01:31:32.339356Z","shell.execute_reply.started":"2023-01-02T01:31:28.031404Z","shell.execute_reply":"2023-01-02T01:31:32.33803Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Add the predictions to test_df\n# reset the index\ntest_df = test_df.reset_index(drop=True)\n\n# create a new column called 'file_name'\ntest_df['file_name'] = test_df.apply(lambda row: f'{row[\"study_slice\"]}.png', axis=1)\n\ntest_preds = []\npred_list = list(test_preds_df['file_name'])\n\nfor i in range(len(test_df)):\n    fname = test_df.loc[i, 'file_name']\n    # The fname will only be in the pred list if Yolo created a txt file for the val image.\n    if fname in pred_list:\n        test_preds.append(1)\n    else:\n        test_preds.append(0)\n\ntest_df['preds'] = test_preds\n# Check the distribution of the predicted classes\ntest_df['preds'].value_counts()","metadata":{"papermill":{"duration":3.732707,"end_time":"2021-07-18T06:59:02.462832","exception":false,"start_time":"2021-07-18T06:58:58.730125","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-01-02T01:40:55.55437Z","iopub.execute_input":"2023-01-02T01:40:55.554811Z","iopub.status.idle":"2023-01-02T01:40:55.698615Z","shell.execute_reply.started":"2023-01-02T01:40:55.554777Z","shell.execute_reply":"2023-01-02T01:40:55.69782Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Confusion Matrix","metadata":{"papermill":{"duration":3.706424,"end_time":"2021-07-18T06:59:10.094022","exception":false,"start_time":"2021-07-18T06:59:06.387598","status":"completed"},"tags":[]}},{"cell_type":"code","source":"CLASS_NAMES = ['Normal', 'Fracture']\ny_test = list(test_df['label'])\ny_pred = list(test_df['preds'])\n\ncfsn_matrix = confusion_matrix(y_test, y_pred)\ndf_cm = pd.DataFrame(cfsn_matrix, index=range(2), columns=CLASS_NAMES)\nplt.figure(figsize=(15, 6))\nsn.heatmap(df_cm, annot=True, linewidths=0.5, fmt='d')","metadata":{"papermill":{"duration":4.773196,"end_time":"2021-07-18T06:59:18.504453","exception":false,"start_time":"2021-07-18T06:59:13.731257","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-01-02T01:41:20.131551Z","iopub.execute_input":"2023-01-02T01:41:20.132003Z","iopub.status.idle":"2023-01-02T01:41:20.379501Z","shell.execute_reply.started":"2023-01-02T01:41:20.131964Z","shell.execute_reply":"2023-01-02T01:41:20.37802Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Classification Report","metadata":{"papermill":{"duration":3.649494,"end_time":"2021-07-18T06:59:33.878606","exception":false,"start_time":"2021-07-18T06:59:30.229112","status":"completed"},"tags":[]}},{"cell_type":"code","source":"classification_report(y_test, y_pred, target_names=CLASS_NAMES)","metadata":{"papermill":{"duration":4.01076,"end_time":"2021-07-18T06:59:41.630028","exception":false,"start_time":"2021-07-18T06:59:37.619268","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2023-01-02T01:42:57.880531Z","iopub.execute_input":"2023-01-02T01:42:57.88102Z","iopub.status.idle":"2023-01-02T01:42:57.899392Z","shell.execute_reply.started":"2023-01-02T01:42:57.880983Z","shell.execute_reply":"2023-01-02T01:42:57.898096Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Object Detection Metrics","metadata":{}},{"cell_type":"code","source":"metric = MeanAveragePrecision(box_format='cxcywh')\nmetric.persistent(True)","metadata":{"execution":{"iopub.status.busy":"2023-01-02T01:33:23.14653Z","iopub.execute_input":"2023-01-02T01:33:23.146978Z","iopub.status.idle":"2023-01-02T01:33:23.154485Z","shell.execute_reply.started":"2023-01-02T01:33:23.146939Z","shell.execute_reply":"2023-01-02T01:33:23.1529Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"test_preds_fnames = test_preds_df['file_name'].values\ntest_true_fnames = test_true_df['file_name'].values\ntest_fnames = np.unique(np.concatenate((test_preds_fnames, test_true_fnames), axis=0))","metadata":{"execution":{"iopub.status.busy":"2023-01-02T01:33:25.585055Z","iopub.execute_input":"2023-01-02T01:33:25.585482Z","iopub.status.idle":"2023-01-02T01:33:25.595531Z","shell.execute_reply.started":"2023-01-02T01:33:25.585447Z","shell.execute_reply":"2023-01-02T01:33:25.593801Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def get_bbox_for_evaluation(row):\n    if not row.empty:\n        bbox = row[['x-center', 'y-center', 'bbox_width', 'bbox_height']].values\n        y = [\n            dict(\n                boxes=torch.tensor(bbox),\n                labels=torch.tensor([1] * len(bbox)),\n                scores=torch.tensor([1.0] * len(bbox)),\n            )\n        ]\n    else:\n        y = [\n            dict(\n                boxes=torch.tensor([]),\n                labels=torch.tensor([]),\n                scores=torch.tensor([]),\n            )\n        ]\n    return y\n    \n    \nfor test_fname in test_fnames:\n    pred_row = test_preds_df[test_preds_df['file_name'] == test_fname]\n    y_pred = get_bbox_for_evaluation(pred_row)\n        \n    true_row = test_true_df[test_true_df['file_name'] == test_fname]\n    y_true = get_bbox_for_evaluation(true_row)\n    \n    metric.update(y_pred, y_true)\n    \npprint(metric.compute())","metadata":{"execution":{"iopub.status.busy":"2023-01-02T01:33:28.28601Z","iopub.execute_input":"2023-01-02T01:33:28.286452Z","iopub.status.idle":"2023-01-02T01:33:37.407617Z","shell.execute_reply.started":"2023-01-02T01:33:28.286414Z","shell.execute_reply":"2023-01-02T01:33:37.406374Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Cleanup","metadata":{"papermill":{"duration":3.660432,"end_time":"2021-07-18T07:01:16.804923","exception":false,"start_time":"2021-07-18T07:01:13.144491","status":"completed"},"tags":[]}},{"cell_type":"code","source":"if os.path.isdir(f'{BASE_DIR}') == True:\n    shutil.rmtree(f'{BASE_DIR}')","metadata":{"papermill":{"duration":4.440213,"end_time":"2021-07-18T07:01:25.213035","exception":false,"start_time":"2021-07-18T07:01:20.772822","status":"completed"},"tags":[],"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"os.chdir('/kaggle/working/')","metadata":{"trusted":true},"execution_count":null,"outputs":[]}]}