{"cells":[{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"markdown","source":"# VinBigData Chest X-ray Abnormalities Detectionm\n\n## Automatically localize and classify thoracic abnormalities from chest radiographsm\n\nPage:\nhttps://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection"},{"metadata":{},"cell_type":"markdown","source":"# 1. Import Packages"},{"metadata":{"trusted":true},"cell_type":"code","source":"import numpy as np\nimport matplotlib.pyplot as plt\nimport pandas as pd\nimport seaborn as sns\nimport os\nimport pathlib\nfrom pprint import pprint\n\n#img\nimport cv2\n\n#pytorch\nimport torch\nfrom torch import nn\nfrom torch import functional as F\nfrom torch import optim\nfrom torch.utils.data import Dataset,DataLoader,random_split\nfrom torchvision import transforms\nfrom torch.nn import Module\nfrom torchvision import models\nfrom PIL import Image\nfrom albumentations.pytorch.transforms import ToTensorV2\nfrom tqdm.notebook import tqdm\nimport albumentations as A\n\n#FasterRCNN\n#from torchvision.models.detection import FasterRCNN\n#from torchvision.models.detection.faster_rcnn import FastRCNNPredictor\n#from torchvision.models.detection.rpn import AnchorGenerator\n\n#dicom\nimport pydicom\n\n#set Device\ndevice = 'cuda' if torch.cuda.is_available() else 'cpu'\nprint(device)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Path"},{"metadata":{"trusted":true},"cell_type":"code","source":"#-----path-----\n#train csv\ntrain_csv_path=pathlib.\\\n        Path(\"../input/vinbigdata-chest-xray-abnormalities-detection/train.csv\")\nsample_sub_path=pathlib.\\\n        Path(\"../input/vinbigdata-chest-xray-abnormalities-detection/sample_submission.csv\")\n#dicom data\ntrain_data_path=pathlib.\\\n        Path(\"../input/vinbigdata-chest-xray-abnormalities-detection/train\")\ntest_data_path=pathlib.\\\n        Path(\"../input/vinbigdata-chest-xray-abnormalities-detection/test\")\n\n#pathの確認\nprint(pathlib.Path.exists(train_csv_path),\n      pathlib.Path.exists(train_data_path),\n      pathlib.Path.exists(test_data_path)\n     )","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#get train image ids\nimage_ids=[x for x in train_data_path.iterdir() if x.is_file()]\nlen(image_ids)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#train csv\ndf=pd.read_csv(train_csv_path)\ndf.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#image id\nsample_=pathlib.\\\n    Path('../input/vinbigdata-chest-xray-abnormalities-detection/train/000434271f63a053c4128a0ba6352c7f.dicom')\n\nprint(\"suffix\",image_ids[0].suffix)\nprint(\"non suffix:\",image_ids[0].stem)\n\n#test sample\ndf[df[\"image_id\"]==sample_.stem]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#fill NaN\nprint(df.isnull().sum())\ndf.fillna(0,inplace=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#classid\nprint(df[\"class_id\"].unique())\nprint(\"uniques:\",len(df[\"class_id\"].unique()))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 5.2 Normalize Dicom Image\n\n### idea is below\n\n[reference] https://www.kaggle.com/raddar/popular-x-ray-image-normalization-techniques\n\n### [1] No-Normalization\n\n### [2] Histogram normalization\nThe general idea is to make pixel distribution uniform. This makes X-rays appear a little darker. This generates view, which radiologist would not see in his standard workplace.\\ Such normalization is used in popular open-source X-ray datasets, such as CheXpert.\n\nimg = read_xray(str(sample_ids))\\\nimg = exposure.equalize_hist(img)\n\n\n### [3] CLAHE normalization\nThis method produces sharper images and is quite often used in chest X-ray research. This generates view, which radiologist would not see in his standard workplace. However, it closely resembles the \"bone-enhanced\" view in some X-rays done (usually due to broken ribs).\n\nimg = read_xray(str(sample_ids))\\\nimg = exposure.equalize_adapthist(img/np.max(img))"},{"metadata":{},"cell_type":"markdown","source":"## My issue 1:\n\n## is this error?\n\n## Bits Stored' value (14-bit) doesn't match the JPEG 2000 data (16-bit). It's recommended that you change the 'Bits Stored' valuef\"The (0028,0101) 'Bits Stored' value ({ds.BitsStored}-bit) \""},{"metadata":{"trusted":true},"cell_type":"code","source":"def read_xray_normalized(image_ids):\n    #read dicom data\n    ds=pydicom.dcmread(image_ids)\n    #->to ndarray\n    dcm_arr=ds.pixel_array\n    \n    #Bits Stored' value (14-bit) doesn't match the JPEG 2000 data (16-bit). \n    #It's recommended that you change the 'Bits Stored' value\n    #f\"The (0028,0101) 'Bits Stored' value ({ds.BitsStored}-bit) \"\n    \n    #is this need...?\n    dcm_arr = np.right_shift(dcm_arr, ds.BitsAllocated - ds.BitsStored)\n    \n    #normalize arr\n    amin=np.amin(dcm_arr)\n    amax=np.amax(dcm_arr)\n    scale = 255.0/(amax-amin) # set scale\n    arr_rescaled = dcm_arr*scale # >rescale 0-255\n    arr_normalized = np.uint8(arr_rescaled) #->uint8\n    return arr_normalized","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"temp_img=read_xray_normalized(image_ids[5])\nprint(temp_img.shape)\nplt.figure(figsize=(8,8))\nplt.imshow(temp_img,cmap=\"bone\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#Histogram normalization(type:ndarray)\nfrom pydicom.pixel_data_handlers.util import apply_voi_lut\nfrom skimage import exposure\n\nimage = exposure.equalize_hist(temp_img)\nplt.figure(figsize=(8,8))\nplt.imshow(image,cmap=\"bone\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## simple Data Augmentation"},{"metadata":{"trusted":true},"cell_type":"code","source":"#Set Image Augumentation\nfrom torchvision import transforms\nimport albumentations\n\n#transforms.Grayscale(3) like RGB channels\n\ntransform=transforms.Compose([\n        transforms.ToPILImage(),\n        transforms.Grayscale(3),\n        transforms.ToTensor(),\n        ]) ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Resize Image and BoundingBox"},{"metadata":{"trusted":true},"cell_type":"code","source":"def resize(image, boxes, width, height):\n    # 現在の高さと幅を取得しておく\n    c_height, c_width = image.shape[:2]\n    img = cv2.resize(image, (width, height))\n    \n    # 圧縮する比率(rate)を計算\n    r_width = width / c_width\n    r_height = width / c_height\n    \n    # 比率を使ってBoundingBoxの座標を修正\n    new_boxes = []\n    for box in boxes:\n        x,y,w,h=box\n        x = int(x * r_width)\n        y = int(y * r_height)\n        w = int(w * r_width)\n        h = int(h * r_height)\n        new_box =[x, y, w, h]\n        new_boxes.append(new_box)\n    return img, new_boxes","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Dataset and DataLoader"},{"metadata":{"trusted":true},"cell_type":"code","source":"class My_Dataset(Dataset):\n    def __init__(self,df,):\n        \n        #dataframeを格納する\n        self.df = df\n        self.image_ids=df[\"image_id\"].unique()\n        self.image_dir=pathlib.\\\n                    Path(\"../input/vinbigdata-chest-xray-abnormalities-detection/train\")\n        #columnsを設定する\n        self.box_col=[\"y_min\",\"y_min\",\"x_max\",\"y_max\"]\n        #transform\n        self.transform=transforms.Compose(\n            [\n            transforms.ToPILImage(),\n            transforms.Grayscale(3),\n            transforms.ToTensor(),\n            ]) \n\n    def __len__(self):\n        return len(self.image_ids)\n    \n    def __getitem__(self,index,transform=False):\n        \n        #get dicom_arr\n        image_id=self.image_ids[index]\n        image=read_xray_normalized(str(self.image_dir/image_id)+\".dicom\")\n        \n        \n        #target\n        records = self.df[(self.df['image_id'] == image_id)]\n        records = records.reset_index(drop=True)\n        \n        if records.loc[0, \"class_id\"] == 0:\n            records = records.loc[[0], :]\n        \n        #-----bounding box-----\n        boxes = records[self.box_col].values.astype(np.float32)\n        #----area-----\n        #bbox:[x,y,w,h] and area=(w-x)*(h-y)\n        area = (boxes[:,2] - boxes[:,0]) * (boxes[:,3] - boxes[:,1])\n        area = area.astype(np.float32)\n        \n        #----labels-----\n        labels = torch.tensor(records[\"class_id\"].values, dtype=torch.int64)\n        \n        # suppose all instances are not crowd\n        #iscrowd = torch.zeros((records.shape[0],), dtype=torch.int64)\n                \n        #-----[target]:dict-----\n        target = {}\n        target['boxes'] = torch.tensor(boxes)\n        target['labels'] = labels\n        target['image_id'] = torch.tensor([index])\n        #target['area'] = torch.tensor(area)\n        #target['iscrowd'] = iscrowd\n        target[\"image_row_shape\"]=torch.tensor(image.shape)\n        target[\"dicom_id\"]=image_id\n        \n        #Transoformed Image\n        #transform\n        #image_transformed=self.transform(image.astype(np.float32))\n        \n        \n        #resize image and bbox\n        #resize scale;width,height=[512,512]\n        width=512\n        height=512\n        image_resized,boxes_resized=resize(image,boxes,width, height)\n        #print(\"boxes_resized:\",boxes_resized)\n        target[\"boxes_resized\"]=torch.tensor(boxes_resized)\n        image_transformed=self.transform(image_resized.astype(np.float32))\n        \n        return image_transformed, target","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#change collate_fn\ndef collate_fn(batch):\n    imgs, targets= list(zip(*batch))\n    \n    imgs = torch.stack(imgs)\n    targets = list(targets)\n    \n    return imgs,targets","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_dataset=My_Dataset(df=df)\ntrain_dataloader=DataLoader(train_dataset,\n                            batch_size=2,shuffle=True, \n                            collate_fn= collate_fn)\n\n# Sample Output Test\nimage,target =next(iter(train_dataloader))\nprint(\"------image-----\")\nprint(\"image_tensor:\",image.shape)\nprint(\"-----target-----\")\nprint(target[0])\nprint(target[1])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## EfficientDet\n\n[reference]\\\n1.github : https://github.com/rwightman/efficientdet-pytorch\n\n2.kaggle : https://www.kaggle.com/shonenkov/training-efficientdet\n\n### Import Packages for EfficientDet-pytorch"},{"metadata":{"trusted":true},"cell_type":"code","source":"!pip install timm\n!pip install omegaconf\n!pip install pycocotools\n!pip install effdet\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Using EfficientDet d1 sample"},{"metadata":{"trusted":true},"cell_type":"code","source":"from effdet import get_efficientdet_config, EfficientDet, DetBenchTrain,DetBenchPredict\nfrom effdet.efficientdet import HeadNet\n\n\nconfig = get_efficientdet_config('tf_efficientdet_d1')\nconfig.image_size = [512,512]\nconfig.norm_kwargs=dict(eps=.001, momentum=.01)\n\n#BackBone=True\nnet = EfficientDet(config, pretrained_backbone=True)\n\n#default 90 -> 19\nnet.reset_head(num_classes=15)\nnet.class_net = HeadNet(config, num_outputs=config.num_classes)\n\n\n#[MODE]:Train\nnet=DetBenchTrain(net, config)\nnet.train()\nprint(\"Loaded pretrained weights\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#configを確認\nprint(net.config)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"class AverageMeter(object):\n    \"\"\"Computes and stores the average and current value\"\"\"\n    def __init__(self):\n        self.reset()\n\n    def reset(self):\n        self.val = 0\n        self.avg = 0\n        self.sum = 0\n        self.count = 0\n\n    def update(self, val, n=1):\n        self.val = val\n        self.sum += val * n\n        self.count += n\n        self.avg = self.sum / self.count","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# sample output\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"summary_loss = AverageMeter()\n\nnet.train()\nfor i in range(1):\n    images,targets=next(iter(train_dataloader))\n    #print(images.shape)\n    #print(targets)\n    \n    batch_size = images.shape[0]\n    \n    boxes = [target['boxes_resized'].to(device).float() for target in targets]\n    labels = [target['labels'].to(device).float() for target in targets]\n    print(\"images:\",images.shape)\n    print(\"boxes:\",boxes)\n    print(\"labels:\",labels)\n    \n    \n    #train\n    target_dict={}\n    target_dict[\"bbox\"]=boxes\n    target_dict[\"cls\"]=labels\n    output=net(images,target_dict)\n    loss=output[\"loss\"]\n    print(\"loss:\",loss)\n    \n    #loss.backward()\n    summary_loss.update(loss.detach().item(), batch_size)\n\nprint(summary_loss.avg)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# set train models "},{"metadata":{"trusted":true},"cell_type":"code","source":"from torch import Tensor as tensor\n\ndef train_loss(images,targets):\n   \n    batch_size = images.shape[0]\n    \n    boxes = [target['boxes_resized'].to(device).float() for target in targets]\n    labels = [target['labels'].to(device).float() for target in targets]\n    \n    img_scale = torch.Tensor([1.0] * batch_size)\n    img_size = torch.Tensor([images[0].shape[-2:]] * batch_size,)\n    \n    target_dict={}\n    target_dict[\"bbox\"]=boxes\n    target_dict[\"cls\"]=labels\n    #target_dict[\"img_scale\"]=img_scale\n    #target_dict[\"img_size\"]=img_size\n    \n    \n    output=net(images,target_dict)\n    train_loss=output[\"loss\"]\n    \n    return train_loss","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def validation_loss(images,targets):\n   \n    batch_size = images.shape[0]\n    \n    boxes = [target['boxes_resized'].to(device).float() for target in targets]\n    labels = [target['labels'].to(device).float() for target in targets]\n\n    img_scale = torch.Tensor([1.0] * batch_size)\n    img_size = torch.Tensor([images[0].shape[-2:]] * batch_size,)\n\n    target_dict={}\n    target_dict[\"bbox\"]=boxes\n    target_dict[\"cls\"]=labels\n    target_dict[\"img_scale\"]=img_scale\n    target_dict[\"img_size\"]=img_size\n    \n    with torch.no_grad():\n        output=net(images,target_dict)\n        val_loss=output[\"loss\"]\n    \n    return val_loss","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def Train_ObjDetect(net, dataloaders_dict, optimizer, num_epochs):\n\n    device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n    print(\"device：\", device)\n\n    net.to(device)\n\n    torch.backends.cudnn.benchmark = True\n        \n    best_summary_loss = 10**5\n    \n    #dict\n    #hist={\"loss\":[],\"val_loss\":[],}\n    \n    # loop epoch\n    for epoch in range(num_epochs):\n        print('---Epoch {}/{}---'.format(epoch+1, num_epochs))\n\n        for phase in ['train', 'val']:\n            \n            if phase == 'train':\n                net.train()  \n            else:\n                net.eval()\n            \n            for images, targets in dataloaders_dict[phase]:\n                \n                optimizer.zero_grad()\n                \n                summary_loss = AverageMeter()\n                \n                if phase==\"train\":\n                    loss=train_loss(images,targets)\n                    print(\"train loss:\",loss)\n                    \n                    batch_size=images.shape[0]\n                    summary_loss.update(loss.detach().item(), batch_size)\n                \n                    loss.backward()\n                    optimizer.step()\n                    \n                else:\n                    loss=validation_loss(images,targets)\n                    print(\"val loss:\",loss)\n                    batch_size=images.shape[0]\n                    summary_loss.update(loss.detach().item(), batch_size)\n                    \n            if summary_loss.avg < best_summary_loss:\n                best_summary_loss = summary_loss.avg\n            #print(summary_loss.avg)\n           \n    return ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from effdet import get_efficientdet_config, EfficientDet, DetBenchTrain,DetBenchPredict\nfrom effdet.efficientdet import HeadNet\n\nconfig = get_efficientdet_config('tf_efficientdet_d1')\nconfig.image_size = [512,512]\nconfig.norm_kwargs=dict(eps=.001, momentum=.01)\n\n#BackBone=True\nnet = EfficientDet(config, pretrained_backbone=True)\n\n#default 90 -> 19\nnet.reset_head(num_classes=15)\nnet.class_net = HeadNet(config, num_outputs=config.num_classes)\n\n\n#[MODE]:Train\nnet=DetBenchTrain(net, config)\nnet.train()\nprint(\"Loaded pretrained weights\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.model_selection import train_test_split\ntest_size=0.2\ndf_train,df_val=train_test_split(df,test_size=test_size,\n                                random_state=64)\n\n#分割を確認\nprint(df_train.shape)\nprint(df_val.shape)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# TEST CODE　"},{"metadata":{"trusted":true},"cell_type":"code","source":"# test sample with reducing data\ndf_train=df_train[:30]\ndf_val=df_val[:10]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#DataSet\ntrain_dataset=My_Dataset(df_train)\nval_dataset=My_Dataset(df_val)\n\n#Dataloader\nbatch_size=5\ntrain_dataloader=DataLoader(train_dataset,batch_size=batch_size,shuffle=True,collate_fn=collate_fn)\nval_dataloader=DataLoader(val_dataset,batch_size=batch_size,shuffle=False,collate_fn=collate_fn)\n\n#dict\ndataloaders_dict={\"train\":train_dataloader,\"val\":val_dataloader}\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learning_rate=1e-3\n\nnum_epochs=1\n#-----optimizer-----\noptimizer=torch.optim.AdamW(net.parameters(), lr=learning_rate)\n#-----train model-----\nfrom tqdm import tqdm\n\nTrain_ObjDetect(net=net,\n            dataloaders_dict=dataloaders_dict,\n            optimizer=optimizer,\n            num_epochs=num_epochs)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## My issue 1:\n\n## is this error? \n\n> ## Bits Stored' value (14-bit) doesn't match the JPEG 2000 data (16-bit). It's recommended that you change the 'Bits Stored' valuef\"The (0028,0101) 'Bits Stored' value ({ds.BitsStored}-bit) \"\n\n### add  below code to solve\ndcm_arr = np.right_shift(dcm_arr, ds.BitsAllocated - ds.BitsStored)\n\n## but its causion continues..."},{"metadata":{},"cell_type":"markdown","source":"# My Big Current issue 2\n\n## get Loss output NaN... immediately\n\n### to solve this problem\n\n---Epoch 1/1---\ntrain loss: tensor(37132.6797, grad_fn=<AddBackward0>)\\\ntrain loss: tensor(117652.2500, grad_fn=<AddBackward0>)\\\ntrain loss: tensor(nan, grad_fn=<AddBackward0>)\n    \n\n## To solve:\n    \n1. change leaning rate\n    \n   lr=1e-5 -> output Nan\\\n   lr1e-7 -> output nan...\n\n\n"},{"metadata":{},"cell_type":"markdown","source":"## I'm beginner of Objective Detection...\n\n\nI would be grateful if you could give me some adbice"},{"metadata":{},"cell_type":"markdown","source":"# Next Inference test Data..."}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}