{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# detectron2 train\n\n\nObject detaction task to find a class and location of floor plans doors.\n\n`detectron2` is one of the famous pytorch object detection library\n\n - https://github.com/facebookresearch/detectron2","metadata":{}},{"cell_type":"markdown","source":"# Table of Contents\n\n** [Dataset preparation](#dataset)** <br/>\n** [Installation](#installation)** <br/>\n** [Training method implementations](#train_method)** <br/>\n** [Customizing detectron2 trainer](#custom_trainer) ** [Advanced topic, skip it first time] <br/>\n**   - [Mapper for augmentation](#mapper)** <br/>\n**   - [Evaluator](#evaluator)** <br/>\n**   - [Loss evaluation hook](#loss_hook)** <br/>\n** [Loading Data](#load_data)** <br/>\n** [Data Visualization](#data_vis)** <br/>\n** [Training](#training)** <br/>\n** [Visualize loss curve & competition metric AP40](#vis_loss)** <br/>\n** [Visualization of augmentation by Mapper](#vis_aug)** <br/>\n** [Next step](#next_step)** <br/>","metadata":{}},{"cell_type":"markdown","source":"<a id=\"dataset\"></a>\n# Dataset preparation\n\nPreprocessing x-ray image format (dicom) into normal png image format is already done by @xhlulu in the below discussion:\n - [Multiple preprocessed datasets: 256/512/1024px, PNG and JPG, modified and original ratio](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/207955).\n\nHere I will just use the dataset [VinBigData Chest X-ray Resized PNG (256x256)](https://www.kaggle.com/xhlulu/vinbigdata-chest-xray-resized-png-256x256) to skip the preprocessing and focus on modeling part.\n\nI also uploaded the original sized png images:\n - [vinbigdata-chest-xray-original-png](https://www.kaggle.com/corochann/vinbigdata-chest-xray-original-png) ([notebook](https://www.kaggle.com/corochann/preprocessing-image-original-size-lossless-png) on kaggle fails due to disk limit)\n\nPlease upvote the dataset as well!","metadata":{}},{"cell_type":"markdown","source":"<a id=\"installation\"></a>\n# Installation\n\ndetectron2 is not pre-installed in this kaggle docker, so let's install it. \n<!-- We can follow [installation instruction](https://github.com/facebookresearch/detectron2/blob/master/INSTALL.md), we need to know CUDA and pytorch version to install correct `detectron2`. -->","metadata":{}},{"cell_type":"code","source":"!nvidia-smi","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2021-07-31T16:58:50.176097Z","iopub.execute_input":"2021-07-31T16:58:50.176424Z","iopub.status.idle":"2021-07-31T16:58:50.857469Z","shell.execute_reply.started":"2021-07-31T16:58:50.176391Z","shell.execute_reply":"2021-07-31T16:58:50.856544Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!nvcc --version","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2021-07-31T16:58:53.840645Z","iopub.execute_input":"2021-07-31T16:58:53.841105Z","iopub.status.idle":"2021-07-31T16:58:54.568062Z","shell.execute_reply.started":"2021-07-31T16:58:53.841061Z","shell.execute_reply":"2021-07-31T16:58:54.567246Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import torch\n\ntorch.__version__","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2021-07-31T16:58:56.447775Z","iopub.execute_input":"2021-07-31T16:58:56.448136Z","iopub.status.idle":"2021-07-31T16:58:58.149441Z","shell.execute_reply.started":"2021-07-31T16:58:56.4481Z","shell.execute_reply":"2021-07-31T16:58:58.14875Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"It seems CUDA=10.2 and torch==1.7.0 is used in this kaggle docker image.","metadata":{}},{"cell_type":"code","source":"!pip install detectron2 -f \\\n  https://dl.fbaipublicfiles.com/detectron2/wheels/cu102/torch1.7/index.html","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2021-07-31T16:59:00.119374Z","iopub.execute_input":"2021-07-31T16:59:00.119713Z","iopub.status.idle":"2021-07-31T16:59:41.832133Z","shell.execute_reply.started":"2021-07-31T16:59:00.119672Z","shell.execute_reply":"2021-07-31T16:59:41.831336Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"train_method\"></a>\n# Training method implementations\n\nBasically we don't need to implement neural network part, `detectron2` already implements famous architectures and provides its pre-trained weights. We can finetune these pre-trained architectures.\nThese models are summarized in [MODEL_ZOO.md](https://github.com/facebookresearch/detectron2/blob/master/MODEL_ZOO.md).\n\nWe need object detection model, We choosen [R50-FPN](https://github.com/facebookresearch/detectron2/blob/master/configs/COCO-Detection/faster_rcnn_R_50_FPN_3x.yaml) for this kernel.","metadata":{}},{"cell_type":"markdown","source":"## Data preparation\n\n`detectron2` provides high-level API for training custom dataset.\n\nTo define custom dataset, we need to create **list of dict** (`dataset_dicts`) where each dict contains following:\n\n - file_name: file name of the image.\n - image_id: id of the image, index is used here.\n - height: height of the image.\n - width: width of the image.\n - annotation: This is the ground truth annotation data for object detection, which contains following\n     - bbox: bounding box pixel location with shape (n_boxes, 4)\n     - bbox_mode: `BoxMode.XYXY_ABS` is used here, meaning that absolute value of (xmin, ymin, xmax, ymax) annotation is used in the `bbox`.\n     - category_id: class label id for each bounding box, with shape (n_boxes,)\n\n`get_dicts` is for train dataset preparation and `get_dicts_test` is for test dataset preparation.\n\nThis `dataset_dicts` contains the metadata for actual data fed into the neural network.<br/>\nIt is loaded beforehand of the training **on memory**, so it should contain all the metadata (image filepath etc) to construct training dataset, but **should not contain heavy data**.<br/>\n\nIn practice, loading all the taining image arrays are too heavy to be loaded on memory, so these are loaded inside `DataLoader` on-demand (This is done by mapper class in `detectron2`, as I will expain later).","metadata":{}},{"cell_type":"code","source":"import os, glob, json, cv2\nfrom pathlib import Path\nfrom detectron2.structures import BoxMode\nfrom detectron2.data import MetadataCatalog, DatasetCatalog\n\ndef get_dicts(imgsdir: Path):\n    dataset_dicts = []\n    idx = 0;\n    for json_file in glob.glob(str(imgsdir) + \"**/*.json\"):\n        record = {}\n        with open(json_file) as f:\n            img_metadata = json.load(f)\n            \n        filename = os.path.join(imgsdir, img_metadata[\"imagePath\"]);\n        height, width = cv2.imread(filename).shape[:2]\n        \n        record[\"file_name\"] = filename      \n        record[\"image_id\"] = idx\n        record[\"height\"] = height\n        record[\"width\"] = width\n        \n        objs=[]\n        annotations = img_metadata[\"shapes\"]    \n        for annotation in annotations:\n            assert annotation[\"shape_type\"] == \"rectangle\"\n            rectengle_points = annotation[\"points\"]            \n            obj = {                \n                \"bbox\": [rectengle_points[0][0], rectengle_points[0][1], rectengle_points[1][0], rectengle_points[1][1]],\n                \"bbox_mode\": BoxMode.XYXY_ABS,\n                \"category_id\": annotation[\"label\"],\n            }\n            objs.append(obj)\n        record[\"annotations\"] = objs\n        \n        dataset_dicts.append(record)    \n        idx +=1        \n    return dataset_dicts","metadata":{"execution":{"iopub.status.busy":"2021-07-31T16:56:06.922098Z","iopub.execute_input":"2021-07-31T16:56:06.922472Z","iopub.status.idle":"2021-07-31T16:56:06.937612Z","shell.execute_reply.started":"2021-07-31T16:56:06.922438Z","shell.execute_reply":"2021-07-31T16:56:06.936233Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# inputdir = Path(\"/kaggle/input\")\n# datadir = inputdir / \"floor-plans-for-mta-image-processing-course/\"\n\nfor d in [\"Train\", \"Test\"]:    \n    DatasetCatalog.register(\"floor_plans_doors_\" + d, lambda d=d: get_dicts(\"/kaggle/input/floor-plans-for-mta-image-processing-course/\" + d))\n    MetadataCatalog.get(\"floor_plans_doors_\" + d).set(thing_classes=[\"Door\"])\ndoor_metadata = MetadataCatalog.get(\"floor_plans_doors_Train\")","metadata":{"execution":{"iopub.status.busy":"2021-07-31T16:57:45.554311Z","iopub.execute_input":"2021-07-31T16:57:45.554672Z","iopub.status.idle":"2021-07-31T16:57:45.576626Z","shell.execute_reply.started":"2021-07-31T16:57:45.554641Z","shell.execute_reply":"2021-07-31T16:57:45.574289Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# --- utils ---\nfrom pathlib import Path\nfrom typing import Any, Union\n\nimport yaml\n\n\ndef save_yaml(filepath: Union[str, Path], content: Any, width: int = 120):\n    with open(filepath, \"w\") as f:\n        yaml.dump(content, f, width=width)\n\n\ndef load_yaml(filepath: Union[str, Path]) -> Any:\n    with open(filepath, \"r\") as f:\n        content = yaml.full_load(f)\n    return content\n","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"execution":{"iopub.status.busy":"2021-07-24T15:07:27.934984Z","iopub.execute_input":"2021-07-24T15:07:27.93546Z","iopub.status.idle":"2021-07-24T15:07:27.944955Z","shell.execute_reply.started":"2021-07-24T15:07:27.93541Z","shell.execute_reply":"2021-07-24T15:07:27.944045Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# --- configs ---\nthing_classes = [\"Door\"]\ncategory_name_to_id = {class_name: index for index, class_name in enumerate(thing_classes)}\n","metadata":{"execution":{"iopub.status.busy":"2021-07-24T15:07:34.94947Z","iopub.execute_input":"2021-07-24T15:07:34.949796Z","iopub.status.idle":"2021-07-24T15:07:34.956095Z","shell.execute_reply.started":"2021-07-24T15:07:34.949766Z","shell.execute_reply":"2021-07-24T15:07:34.955275Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now all the preparation has done!\n\n`MyTrainer` overwraps `build_evaluator` method of `DefaultTrainer` provided by `detectron2` to support validation dataset evaluation.\n\n\n1. `build_train_loader` & `build_test_loader`: \nThese class methods deine how to construct DataLoader for training data & validation data respectively.\nHere `AlbumentationMapper` is passed to construct DataLoader to insert customized augmentation process.\n\n2. `build_evaluator`:\nThis class method defines how to construct Evaluator. \nHere implemented `VinbigdataEvaluator` is constructed (we can also use `COCOEvaluator` here).\n\n3. `build_hooks`:\nThis method defines how to construct hooks. I insert `LossEvalHook` before evalutor to work well.","metadata":{}},{"cell_type":"markdown","source":"Now the methods are ready. main scripts starts from here.","metadata":{}},{"cell_type":"code","source":"dataset_dicts = get_vinbigdata_dicts(imgdir, train, debug=debug)","metadata":{"execution":{"iopub.status.busy":"2021-07-31T16:31:22.708741Z","iopub.execute_input":"2021-07-31T16:31:22.709198Z","iopub.status.idle":"2021-07-31T16:31:22.745516Z","shell.execute_reply.started":"2021-07-31T16:31:22.709167Z","shell.execute_reply":"2021-07-31T16:31:22.743629Z"},"jupyter":{"outputs_hidden":true},"collapsed":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"data_vis\"></a>\n# Data Visualization\n\nIt's also very easy to visualize prepared training dataset with `detectron2`.<br/>\nIt provides `Visualizer` class, we can use it to draw an image with bounding box as following.","metadata":{"jupyter":{"source_hidden":true}}},{"cell_type":"code","source":"# Visualize data...\nanomaly_image_ids = train.query(\"class_id != 14\")[\"image_id\"].unique()\ntrain_meta = pd.read_csv(imgdir/\"train_meta.csv\")\nanomaly_inds = np.argwhere(train_meta[\"image_id\"].isin(anomaly_image_ids).values)[:, 0]\n\nvinbigdata_metadata = MetadataCatalog.get(\"vinbigdata_train\")\n\ncols = 3\nrows = 3\nfig, axes = plt.subplots(rows, cols, figsize=(18, 18))\naxes = axes.flatten()\n\nfor index, anom_ind in enumerate(anomaly_inds[:cols * rows]):\n    ax = axes[index]\n    # print(anom_ind)\n    d = dataset_dicts[anom_ind]\n    img = cv2.imread(d[\"file_name\"])\n    visualizer = Visualizer(img[:, :, ::-1], metadata=vinbigdata_metadata, scale=0.5)\n    out = visualizer.draw_dataset_dict(d)\n    # cv2_imshow(out.get_image()[:, :, ::-1])\n    #cv2.imwrite(str(outdir / f\"vinbigdata{index}.jpg\"), out.get_image()[:, :, ::-1])\n    ax.imshow(out.get_image()[:, :, ::-1])\n    ax.set_title(f\"{anom_ind}: image_id {anomaly_image_ids[index]}\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-07-24T15:12:54.560854Z","iopub.execute_input":"2021-07-24T15:12:54.561246Z","iopub.status.idle":"2021-07-24T15:12:56.840036Z","shell.execute_reply.started":"2021-07-24T15:12:54.561196Z","shell.execute_reply":"2021-07-24T15:12:56.839148Z"},"jupyter":{"source_hidden":true},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"<a id=\"training\"></a>\n# Training","metadata":{}},{"cell_type":"code","source":"from detectron2.engine import DefaultPredictor\nfrom detectron2.engine import DefaultTrainer\nfrom detectron2.config import get_cfg\nfrom detectron2 import model_zoo\n\nconfig_name = \"COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml\"\ncfg = get_cfg()\ncfg.merge_from_file(model_zoo.get_config_file(config_name))\ncfg.DATASETS.TRAIN = (\"floor_plans_doors_Train\",)\ncfg.DATASETS.TEST = ()\ncfg.DATALOADER.NUM_WORKERS = 2\ncfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url(config_name)  # Let training initialize from model zoo\ncfg.SOLVER.IMS_PER_BATCH = 2\ncfg.SOLVER.BASE_LR = 0.00025  # pick a good LR\ncfg.SOLVER.MAX_ITER = 300    # 300 iterations seems good enough for this toy dataset; you will need to train longer for a practical dataset\ncfg.SOLVER.STEPS = []        # do not decay learning rate\ncfg.MODEL.ROI_HEADS.BATCH_SIZE_PER_IMAGE = 512 #128 faster, and good enough for this toy dataset (default: 512)\ncfg.MODEL.ROI_HEADS.NUM_CLASSES = 1  # only has one class (Door).\n\nos.makedirs(cfg.OUTPUT_DIR, exist_ok=True)\ntrainer = DefaultTrainer(cfg) \ntrainer.resume_or_load(resume=False)\ntrainer.train()","metadata":{"execution":{"iopub.status.busy":"2021-07-31T16:57:56.2339Z","iopub.execute_input":"2021-07-31T16:57:56.234331Z","iopub.status.idle":"2021-07-31T16:57:57.191948Z","shell.execute_reply.started":"2021-07-31T16:57:56.234247Z","shell.execute_reply":"2021-07-31T16:57:57.188325Z"}},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# from detectron2.config.config import CfgNode as CN\n\n# cfg = get_cfg()\n# cfg.aug_kwargs = CN(flags.aug_kwargs)  # pass aug_kwargs to cfg\n\n# original_output_dir = cfg.OUTPUT_DIR\n# cfg.OUTPUT_DIR = str(outdir)\n# print(f\"cfg.OUTPUT_DIR {original_output_dir} -> {cfg.OUTPUT_DIR}\")\n\n# config_name = \"COCO-Detection/faster_rcnn_R_50_FPN_3x.yaml\"\n# cfg.merge_from_file(model_zoo.get_config_file(config_name))\n# cfg.DATASETS.TRAIN = (\"vinbigdata_train\",)\n# if split_mode == \"all_train\":\n#     cfg.DATASETS.TEST = ()\n# else:\n#     cfg.DATASETS.TEST = (\"vinbigdata_valid\",)\n#     cfg.TEST.EVAL_PERIOD = flags.eval_period\n\n# cfg.DATALOADER.NUM_WORKERS = flags.num_workers\n# # Let training initialize from model zoo\n# cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url(config_name)\n# cfg.SOLVER.IMS_PER_BATCH = flags.ims_per_batch\n# cfg.SOLVER.LR_SCHEDULER_NAME = flags.lr_scheduler_name\n# cfg.SOLVER.BASE_LR = flags.base_lr  # pick a good LR\n# cfg.SOLVER.MAX_ITER = flags.iter\n# cfg.SOLVER.CHECKPOINT_PERIOD = 100000  # Small value=Frequent save need a lot of storage.\n# cfg.MODEL.ROI_HEADS.BATCH_SIZE_PER_IMAGE = flags.roi_batch_size_per_image\n# cfg.MODEL.ROI_HEADS.NUM_CLASSES = len(thing_classes)\n# # NOTE: this config means the number of classes,\n# # but a few popular unofficial tutorials incorrect uses num_classes+1 here.\n\n# os.makedirs(cfg.OUTPUT_DIR, exist_ok=True)\n","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2021-07-24T15:13:02.346185Z","iopub.execute_input":"2021-07-24T15:13:02.346586Z","iopub.status.idle":"2021-07-24T15:13:02.383401Z","shell.execute_reply.started":"2021-07-24T15:13:02.346551Z","shell.execute_reply":"2021-07-24T15:13:02.382292Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# trainer = MyTrainer(cfg)\n# trainer.resume_or_load(resume=False)\n# trainer.train()","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2021-07-24T15:13:06.159992Z","iopub.execute_input":"2021-07-24T15:13:06.160386Z","iopub.status.idle":"2021-07-24T16:05:57.08732Z","shell.execute_reply.started":"2021-07-24T15:13:06.16035Z","shell.execute_reply":"2021-07-24T16:05:57.086358Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"It's actually very easy to use multiple gpus for training.\n\nYou just need to wrap above training scripts by `main` method and use `launch` method provided by `detectron2`.\n\nPlease refer official example [train_net.py](https://github.com/facebookresearch/detectron2/blob/master/tools/train_net.py#L161) for details.","metadata":{}},{"cell_type":"markdown","source":"<a id=\"vis_loss\"></a>\n# Visualize loss curve & competition metric AP40\n\nAs I explained, the calculated metrics are saved in `metrics.json`. We can analyze/plot them to check how the training proceeded.","metadata":{}},{"cell_type":"code","source":"metrics_df = pd.read_json(outdir / \"metrics.json\", orient=\"records\", lines=True)\nmdf = metrics_df.sort_values(\"iteration\")\nmdf","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-07-24T16:14:06.973203Z","iopub.execute_input":"2021-07-24T16:14:06.973611Z","iopub.status.idle":"2021-07-24T16:14:07.183736Z","shell.execute_reply.started":"2021-07-24T16:14:06.973577Z","shell.execute_reply":"2021-07-24T16:14:07.182945Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# 1. Loss curve\nfig, ax = plt.subplots()\n\nmdf1 = mdf[~mdf[\"total_loss\"].isna()]\nax.plot(mdf1[\"iteration\"], mdf1[\"total_loss\"], c=\"C0\", label=\"train\")\nif \"validation_loss\" in mdf.columns:\n    mdf2 = mdf[~mdf[\"validation_loss\"].isna()]\n    ax.plot(mdf2[\"iteration\"], mdf2[\"validation_loss\"], c=\"C1\", label=\"validation\")\n\n# ax.set_ylim([0, 0.5])\nax.legend()\nax.set_title(\"Loss curve\")\nplt.show()\nplt.savefig(outdir/\"loss.png\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-07-24T16:14:11.37569Z","iopub.execute_input":"2021-07-24T16:14:11.376017Z","iopub.status.idle":"2021-07-24T16:14:11.551569Z","shell.execute_reply.started":"2021-07-24T16:14:11.375987Z","shell.execute_reply":"2021-07-24T16:14:11.550545Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, ax = plt.subplots()\nmdf3 = mdf[~mdf[\"bbox/AP75\"].isna()]\nax.plot(mdf3[\"iteration\"], mdf3[\"bbox/AP75\"] / 100., c=\"C2\", label=\"validation\")\n\nax.legend()\nax.set_title(\"AP40\")\nplt.show()\nplt.savefig(outdir / \"AP40.png\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-07-24T16:14:18.756894Z","iopub.execute_input":"2021-07-24T16:14:18.75722Z","iopub.status.idle":"2021-07-24T16:14:18.900491Z","shell.execute_reply.started":"2021-07-24T16:14:18.757189Z","shell.execute_reply":"2021-07-24T16:14:18.89922Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig, ax = plt.subplots()\nmdf_bbox_class = mdf3.iloc[-1][[f\"bbox/AP-{col}\" for col in thing_classes]]\nmdf_bbox_class.plot(kind=\"bar\", ax=ax)\n_ = ax.set_title(\"AP by class\")","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-07-24T16:14:21.951803Z","iopub.execute_input":"2021-07-24T16:14:21.95231Z","iopub.status.idle":"2021-07-24T16:14:22.160862Z","shell.execute_reply.started":"2021-07-24T16:14:21.952266Z","shell.execute_reply":"2021-07-24T16:14:22.160192Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Our Evaluator calculaes AP by class, and it is easy to check which class is diffucult to train.\n\nIn my experiment, **\"Calcification\" seems to be the most difficult class to predict**.","metadata":{}},{"cell_type":"markdown","source":"<a id=\"vis_aug\"></a>\n# Visualization of augmentation by Mapper\n\nLet's check the behavior of Mapper method. Since mapper is used inside DataLoader, we can check its behavior by constucting DataLoader and visualize the data processed by the DataLoader.\n\nThe defined Trainer class has **class method** `build_train_loader`. We can construct train_loader purely from `cfg`, without instantiating `trainer` since it's class method.\n\nBelow code is to visualize the same data 4 times. You can check that augmentation is applied and every time the image looks different.\n\nNote that both `detectron2.data.transforms` & `albumentations` augmentations properly handles bounding box. Thus bounding box is adjusted when the image is scaled, rotated etc!\n\nAt first I was using `detectron2.data.transforms` with `MyMapper` class, it provides basic augmentations.<br/>\nThen I noticed that we can use many augmentations in `albumentations`, so I implemented `AlbumentationsMapper` to support it.<br/>\nHow many augmentations can be used in albumentations?<br/>\nYou can see official github page, all [Pixel-level transforms](https://github.com/albumentations-team/albumentations#pixel-level-transforms) and [Spatial-level transforms](https://github.com/albumentations-team/albumentations#spatial-level-transforms) with \"BBoxes\" checked can be used. There are really many!!!","metadata":{}},{"cell_type":"code","source":"# Visualize data...\n# import matplotlib.pyplot as plt\nfrom detectron2.data.samplers import TrainingSampler\n\nn_images = 2\nn_aug = 4\n\nfig, axes = plt.subplots(n_images, n_aug, figsize=(16, 8))\n\n# Ref https://github.com/facebookresearch/detectron2/blob/22b70a8078eb09da38d0fefa130d0f537562bebc/tools/visualize_data.py#L79-L88\nfor i in range(n_aug):\n    sampler = TrainingSampler(len(dataset_dicts), shuffle=False)\n    train_vis_loader = MyTrainer.build_train_loader(\n        cfg, sampler=sampler\n    )  # For visualization...\n    for batch in train_vis_loader:\n        for j, per_image in enumerate(batch):\n            ax = axes[j, i]\n\n            img_arr = per_image[\"image\"].cpu().numpy().transpose((1, 2, 0))\n            visualizer = Visualizer(\n                img_arr[:, :, ::-1], metadata=vinbigdata_metadata, scale=1.0\n            )\n            target_fields = per_image[\"instances\"].get_fields()\n            labels = [\n                vinbigdata_metadata.thing_classes[i] for i in target_fields[\"gt_classes\"]\n            ]\n            out = visualizer.overlay_instances(\n                labels=labels,\n                boxes=target_fields.get(\"gt_boxes\", None),\n                masks=target_fields.get(\"gt_masks\", None),\n                keypoints=target_fields.get(\"gt_keypoints\", None),\n            )\n            # out = visualizer.draw_dataset_dict(per_image)\n\n            img = out.get_image()[:, :, ::-1]\n            filepath = str(outdir / f\"vinbigdata_{j}_aug{i}.jpg\")\n            cv2.imwrite(filepath, img)\n            print(f\"Visualization img {img_arr.shape} saved in {filepath}\")\n            ax.imshow(img)\n            ax.set_title(f\"image{j}, {i}-th aug\")\n        break","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2021-07-24T16:14:28.333415Z","iopub.execute_input":"2021-07-24T16:14:28.333803Z","iopub.status.idle":"2021-07-24T16:14:34.098516Z","shell.execute_reply.started":"2021-07-24T16:14:28.333769Z","shell.execute_reply":"2021-07-24T16:14:34.097445Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"That's all! \n\nI found that the competition data is not so many (15000 for all images, 4000 images after filtering \"No finding\" images).<br/>\nIt does not take long time to train (less than a day), so this competition may be a good choice for beginners who want to learn object detection!\n\n<h3 style=\"color:red\">If this kernel helps you, please upvote to keep me motivated 😁<br>Thanks!</h3>","metadata":{}},{"cell_type":"markdown","source":"<a id=\"next_step\"></a>\n# Next step\n\n[📸VinBigData detectron2 prediction](https://www.kaggle.com/corochann/vinbigdata-detectron2-prediction) kernel explains how to use trained model for the prediction and submisssion for this competition.\n\n[📸VinBigData 2-class classifier complete pipeline](https://www.kaggle.com/corochann/vinbigdata-2-class-classifier-complete-pipeline) kernel explains how to train 2 class classifier model for the prediction and submisssion for this competition.\n\n## Discussions\nThese discussions are useful to further utilize this training notebook to conduct deeper experiment.\n\n - [1-step training & prediction](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/219672): The 1-step pipeline which does not use any 2-class classifier approach is proposed.\n - [What anchor size & aspect ratio should be used?](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/220295): Suggests how to predict more smaller sized, high aspect ratio bonding boxes. It affects to the score a lot!!!\n - [Preferable radiologist's id in the test dataset?](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/219221): Investigation of test dataset annotation distribution.\n","metadata":{}},{"cell_type":"code","source":"","metadata":{"trusted":true},"execution_count":null,"outputs":[]}]}