{"cells":[{"metadata":{},"cell_type":"markdown","source":"MONAI - Medical Open Network for AI - is a library for for deep learning in healthcare imaging, originally started by NVIDIA & King’s College London: [https://monai.io](https://monai.io)\n\n\n# Install required packages\n\nSince it is not installed by default with kaggle jupyter Notebooks, we first need to install [monai](https://github.com/Project-MONAI/MONAI)"},{"metadata":{"trusted":true},"cell_type":"code","source":"!pip3 install monai","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Import libraries\n\nLoad python libraries used later."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import pandas as pd\nimport os\nimport SimpleITK as sitk\nimport numpy as np\nimport torch\nfrom monai import transforms\nimport random\nimport matplotlib.pyplot as plt\nimport matplotlib.patches as patches\nfrom PIL import Image\nimport numpy as np","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Define some utilities\n\n## Method for metadata reading\n\nFirst of all, we need to create a metadata table and join filepaths to train.csv\n\nTherefore, we crawl train/test subdirectories for dicom files and extract their image IDs from filenames.\n\nThen we join filenames to the metadata via \"image_id\".\n\nNote that multiple annotation can occur in one image."},{"metadata":{"trusted":true},"cell_type":"code","source":"def _read_metadata():\n    \n    base_dir = '/kaggle/input/vinbigdata-chest-xray-abnormalities-detection'\n    \n    dataframes = {}\n    \n    for _type in [\"train\", \"test\"]:\n        dcm_files = os.listdir(os.path.join(base_dir, _type))\n        _df = pd.DataFrame(\n            {\n                \"filepath\": dcm_files\n            }\n        )\n            \n        _df[\"image_id\"] = _df.filepath.str.split(\".\", 1).str[0]\n        _df[\"filepath\"] = [os.path.join(base_dir, _type, _f) for _f in _df[\"filepath\"]]\n        \n        dataframes[_type] = _df\n        \n    train_df = dataframes[\"train\"].merge(\n        right=pd.read_csv(\n            filepath_or_buffer=os.path.join(\n                base_dir, \"train.csv\"\n            )\n        ),\n        how=\"left\",\n        on=\"image_id\",\n        suffixes=(\"\", \"\")\n    )\n    \n    test_df = dataframes[\"test\"]\n    \n    return train_df, test_df","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_info, test_info = _read_metadata()\ntrain_info.head(20)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Define dataset class  \n\nFor deep learning using pytorch, we first need to prepare a dataset.\n\nLater, we want to train a Faster-RCNN model + resnet for object detection and classification: https://pytorch.org/docs/stable/torchvision/models.html#faster-r-cnn\n\nTherefore, our dataset needs to return images, bounding boxes and labels.\n\nPlease note these caveats:\n* Faster RCNN requires the background to be encoded as 0. Our background in the dataset is 14. We therefore replace 14 with -1 and shift classes by +1.\n\n* Furthermore, we want to load an image only once. Therefore, from our metadata table, the unique combinations of filepath and image ID are extracted and stored inside `self.data_unique`.\n  + For each image ID, we check if only class 14/background was annotated by the expert readers. If this is the case, we replace all annotations with one (!) label + box of dimensions 0, 0, 1, 1.\n  + If there are other classes than 14, we only keep those and extract class labels and boxes from the metadata. \n  \nSince the dicom images are very big, we further reduce them by `self.reducefct` in x- and y dimension.  Bounding box coordinates are reduced accordingly."},{"metadata":{"trusted":true},"cell_type":"code","source":"class XrayDataset(torch.utils.data.Dataset):\n\n    def __init__(\n        self,\n        dataframe: pd.DataFrame,\n        datatype: str\n    ):\n        \n        # dataframe contains all data\n        self.dataframe = dataframe\n        self.datatype = datatype # \"train\" or \"test\"\n        \n        # data_unique contains only unique combinations of filepath and image_id\n        self.data_unique = self.dataframe[[\"filepath\", \"image_id\"]].drop_duplicates()\n        \n        # image size reduce factor\n        self.reducefct = 5\n    \n        \n    @staticmethod\n    def load_image(img_path):\n        img_sitk = sitk.ReadImage(img_path)\n        np_img = sitk.GetArrayFromImage(img_sitk)\n        # c, y, x\n        return np_img\n        \n    # image transforms\n    @staticmethod\n    def img_trf(spatial_size):\n        imgtf = transforms.Compose(\n            [\n                transforms.CastToType(),\n                transforms.NormalizeIntensity(),\n                transforms.ScaleIntensity(),\n                transforms.Resize(spatial_size)\n            ]\n        )\n        return imgtf\n    \n    def get_label_boxes(self, img_id):\n        \n        targets = self.dataframe[[\"class_id\", \"x_min\", \"y_min\", \"x_max\", \"y_max\"]].loc[self.dataframe.image_id == img_id]\n        \n        if targets.class_id.max() == 14:\n            \n            # test if only background class (=14) available\n            if len(targets[targets.class_id == 14]) == len(targets):\n                # create new dataframe\n                targets = pd.DataFrame(\n                    {\n                        \"class_id\": -1,\n                        \"x_min\": 0,\n                        \"y_min\": 0,\n                        \"x_max\": 1,\n                        \"y_max\": 1\n                    },\n                    index=[0]\n                )\n            else:\n                # if other classes than background are available:\n                # exclude all cases with 14\n                targets = targets[targets.class_id != 14]\n        \n        # extract labels; now 0 = background class\n        lbls = list(targets.class_id.astype(int) + 1)\n        \n        # extract boxes\n        boxes = targets[[\"x_min\", \"y_min\", \"x_max\", \"y_max\"]].to_numpy()\n        \n        return lbls, boxes\n    \n    def __len__(self):\n        return len(self.data_unique)\n        \n    def __getitem__(self, idx):\n        \n        if torch.is_tensor(idx):\n            idx = idx.tolist()\n\n        img_path = self.data_unique.iloc[idx][\"filepath\"]\n        img_id = self.data_unique.iloc[idx][\"image_id\"]\n        \n        img = self.load_image(img_path=img_path)\n        \n        # resize, normalize intensity, to tensor\n        c, y, x = img.shape\n        ynew = int(y/self.reducefct)\n        xnew = int(x/self.reducefct)\n        img = self.img_trf(\n            spatial_size=(ynew, xnew)\n        )(img) \n        \n        out_dict = {}\n\n        # https://pytorch.org/docs/stable/torchvision/models.html#faster-r-cnn\n        if self.datatype == \"train\":\n\n            # get labels and boxes\n            labels, boxes = self.get_label_boxes(img_id=img_id)\n            out_dict[\"labels\"] = labels\n            \n            bb = np.array(boxes / self.reducefct, dtype=\"uint16\")\n            out_dict[\"boxes\"] = bb\n        \n        \n        out_dict[\"image\"] = img\n        out_dict[\"image_id\"] = img_id\n        \n        return out_dict ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Define method for plotting"},{"metadata":{"trusted":true},"cell_type":"code","source":"def plot_xray(batch):\n    \n    blen = len(batch) if len(batch) < 20 else 20\n    \n    # Create figure and axes\n    fig, axs = plt.subplots(4, int(blen / 4), figsize=(15,15))\n    \n    _b = 0\n        \n    for row in axs:\n        for col in row:\n\n            im=batch[_b][\"image\"]\n            bx=batch[_b][\"boxes\"]\n            lbl=batch[_b][\"labels\"] \n\n            # Display the image\n            col.imshow(im[0, :, :], cmap = \"gray\")\n\n            # Create a Rectangle patch\n            for i in range(len(bx)):\n                bbox = bx[i]\n                rect = patches.Rectangle(\n                    (bbox[0], bbox[1]), \n                    bbox[2] - bbox[0],\n                    bbox[3] - bbox[1],\n                    linewidth=1,\n                    edgecolor='r',\n                    facecolor='none'\n                )\n\n                col.text(\n                    bbox[2] + 2,\n                    bbox[1] - 2,\n                    str(lbl[i])\n                )\n\n                # Add the patch to the Axes\n                col.add_patch(rect)\n            _b += 1\n\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Finally, have a look at the data"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_ds = XrayDataset(train_info, \"train\")\nbatch = [train_ds[_i] for _i in range(20)]\nbatch[0][\"image\"].shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_xray(batch)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}