{"cells":[{"metadata":{},"cell_type":"markdown","source":"# References\n\n1. https://www.kaggle.com/dschettler8845/visual-in-depth-eda-vinbigdata-competition-data\n\n2. https://www.kaggle.com/trungthanhnguyen0502/eda-vinbigdata-chest-x-ray-abnormalities\n\n3. https://www.kaggle.com/bhallaakshit/dicom-wrangling-and-enhancement\n\n4. https://www.kaggle.com/awsaf49/vinbigdata-cxr-ad-yolov5-14-class-train\n\nThanks for above great works!"},{"metadata":{},"cell_type":"markdown","source":"# Goal of this experiment\n(Written in Korean, but I will edit in English asap)\n\n* Because X-ray imaging devices cannot focus like optical lenses, The resulting image generally tends to be slightly blurred.\n* Radiologists diagnose X-ray images with their own eyes, so it is determined that the black/white ratio and shape (blood vessels, lungs) are the criteria, which are determined by the edge.\n* Therefore, I think it would be helpful to emphasize this edge information on the image through pre-processing.\n* Image enhancement was experimented using several basic methods\n* And train yolo v5 model with enhanced images to see if this affects the actual performance.\n* Also, preprocessed image datasets (abnormal classes) are also provided as a result.\n\n"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np\nimport pandas as pd \nimport os\nimport shutil\nimport pydicom\nfrom pydicom.pixel_data_handlers.util import apply_voi_lut\nimport cv2\nimport matplotlib.pyplot as plt\nfrom skimage import exposure\nfrom glob import glob\n\nfrom scipy.io import wavfile\nimport warnings\nwarnings.filterwarnings('ignore')\nfrom sklearn.model_selection import GroupKFold\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Enhancement trial"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"dataset_dir = '../input/vinbigdata-chest-xray-abnormalities-detection'","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def dicom2array(path, voi_lut=True, fix_monochrome=True):\n    dicom = pydicom.read_file(path)\n    # VOI LUT (if available by DICOM device) is used to\n    # transform raw DICOM data to \"human-friendly\" view\n    if voi_lut:\n        data = apply_voi_lut(dicom.pixel_array, dicom)\n    else:\n        data = dicom.pixel_array\n    # depending on this value, X-ray may look inverted - fix that:\n    if fix_monochrome and dicom.PhotometricInterpretation == \"MONOCHROME1\":\n        data = np.amax(data) - data\n    data = data - np.min(data)\n    data = data / np.max(data)\n    data = (data * 255).astype(np.uint8)\n    return data\n        \n    \ndef plot_img(img, size=(7, 7), is_rgb=True, title=\"\", cmap='gray'):\n    plt.figure(figsize=size)\n    plt.imshow(img, cmap=cmap)\n    plt.suptitle(title)\n    plt.show()\n\n\ndef plot_imgs(imgs, cols=4, size=7, is_rgb=True, title=\"\", cmap='gray', img_size=(500,500)):\n    rows = len(imgs)//cols + 1\n    fig = plt.figure(figsize=(cols*size, rows*size))\n    for i, img in enumerate(imgs):\n        if img_size is not None:\n            img = cv2.resize(img, img_size)\n        fig.add_subplot(rows, cols, i+1)\n        plt.imshow(img, cmap=cmap)\n    plt.suptitle(title)\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Origninal images"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"dicom_paths = glob(f'{dataset_dir}/train/*.dicom')\nimgs = [dicom2array(path) for path in dicom_paths[:8]]\nplot_imgs(imgs)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Invert"},{"metadata":{"trusted":true},"cell_type":"code","source":"invert = 256 - np.array(imgs)\nplot_imgs(invert)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# filters\n1) 3x3 High pass filter"},{"metadata":{"trusted":true},"cell_type":"code","source":"hf1 = np.array([[-1, -1, -1], [-1, 9, -1], [-1, -1, -1]])\n\nafter_hf1 = [cv2.filter2D(img, -1, hf1) for img in imgs]\nplot_imgs(after_hf1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"It seems worse than originals..\n* noise\n* appear wave patterns"},{"metadata":{},"cell_type":"markdown","source":"# Histogram Equalization"},{"metadata":{"trusted":true},"cell_type":"code","source":"hist_eq = [cv2.equalizeHist(img) for img in imgs]\nplot_imgs(hist_eq)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# CLAHE (Contrast Limited Adaptive Histogram Equalization)\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"def clahe(image):\n    clahe = cv2.createCLAHE(\n        clipLimit = 2., \n        tileGridSize = (10, 10)\n    )\n    \n    image = clahe.apply(image) \n    #image = tf.expand_dims(image, axis = 2)\n    \n    return image","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"clahe_ = [clahe(img) for img in imgs]\nplot_imgs(clahe_)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"It looks best."},{"metadata":{},"cell_type":"markdown","source":"# Data Settings"},{"metadata":{"trusted":true},"cell_type":"code","source":"dim = 512 #1024, 256, 'original'\ntest_dir = f'/kaggle/input/vinbigdata-{dim}-image-dataset/vinbigdata/test'\nweights_dir = '/kaggle/input/vinbigdata-cxr-ad-yolov5-14-class-train/yolov5/runs/train/exp/weights/best.pt'","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df = pd.read_csv(f'../input/vinbigdata-{dim}-image-dataset/vinbigdata/train.csv')\ntrain_df.head()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"train_df['image_path'] = f'/kaggle/input/vinbigdata-{dim}-image-dataset/vinbigdata/train/'+train_df.image_id+('.png' if dim!='original' else '.jpg')\ntrain_df.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df = train_df[train_df.class_id!=14].reset_index(drop = True)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Data split"},{"metadata":{"trusted":true},"cell_type":"code","source":"fold = 4\ngkf  = GroupKFold(n_splits = 5)\ntrain_df['fold'] = -1\nfor fold, (train_idx, val_idx) in enumerate(gkf.split(train_df, groups = train_df.image_id.tolist())):\n    train_df.loc[val_idx, 'fold'] = fold\nval_df = train_df[train_df['fold']==4]\nval_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Train : 3515, val : 879"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_files = []\nval_files   = []\nval_files += list(train_df[train_df.fold==fold].image_path.unique())\ntrain_files += list(train_df[train_df.fold!=fold].image_path.unique())\nlen(train_files), len(val_files)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"os.makedirs('/kaggle/working/vinbigdata/labels/train', exist_ok = True)\nos.makedirs('/kaggle/working/vinbigdata/labels/val', exist_ok = True)\nos.makedirs('/kaggle/working/vinbigdata/images/train', exist_ok = True)\nos.makedirs('/kaggle/working/vinbigdata/images/val', exist_ok = True)\nlabel_dir = '/kaggle/input/vinbigdata-yolo-labels-dataset/labels'\nfor file in train_files:\n    shutil.copy(file, '/kaggle/working/vinbigdata/images/train')\n    filename = file.split('/')[-1].split('.')[0]\n    shutil.copy(os.path.join(label_dir, filename+'.txt'), '/kaggle/working/vinbigdata/labels/train')\n    \nfor file in val_files:\n    shutil.copy(file, '/kaggle/working/vinbigdata/images/val')\n    filename = file.split('/')[-1].split('.')[0]\n    shutil.copy(os.path.join(label_dir, filename+'.txt'), '/kaggle/working/vinbigdata/labels/val')\n    \nval_dir = f'/kaggle/working/vinbigdata/images/val'","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Setting for enhanced images"},{"metadata":{},"cell_type":"markdown","source":"* Enhance only train images (not for val images)"},{"metadata":{"trusted":true},"cell_type":"code","source":"os.makedirs('/kaggle/working/vinbigdata_processed/images/train', exist_ok = True)\nshutil.copytree('/kaggle/working/vinbigdata/labels/train','/kaggle/working/vinbigdata_processed/labels/train')\nshutil.copytree('/kaggle/working/vinbigdata/labels/val','/kaggle/working/vinbigdata_processed/labels/val')\nshutil.copytree('/kaggle/working/vinbigdata/images/val','/kaggle/working/vinbigdata_processed/images/val')","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"for file in train_files:\n    shutil.copy(file, '/kaggle/working/vinbigdata/images/train')\n    filename = file.split('/')[-1].split('.')[0]\n    shutil.copy(os.path.join(label_dir, filename+'.txt'), '/kaggle/working/vinbigdata_processed/labels/train')\n    ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Save Enhancemented Images with .png format"},{"metadata":{"trusted":true},"cell_type":"code","source":"path = '/kaggle/working/vinbigdata/images/train'\nsave_path = '/kaggle/working/vinbigdata_processed/images/train'\nimages = os.listdir(path)\n\nfor image in images:\n    img_path = os.path.join(path, image)\n    ori_img = cv2.imread(img_path, cv2.IMREAD_GRAYSCALE)\n    dst_img = clahe(ori_img)\n\n    cv2.imwrite(os.path.join(save_path, image), dst_img)\n\n    \n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Save train images after clahe"},{"metadata":{"trusted":true},"cell_type":"code","source":"print(len(os.listdir(save_path)))\nsample = cv2.imread(os.path.join(save_path, images[0]))\nplt.imshow(sample)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Setups for training yolo v5"},{"metadata":{"trusted":true},"cell_type":"code","source":"class_ids, class_names = list(zip(*set(zip(train_df.class_id, train_df.class_name))))\nclasses = list(np.array(class_names)[np.argsort(class_ids)])\nclasses = list(map(lambda x: str(x), classes))\nclasses","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from os import listdir\nfrom os.path import isfile, join\nimport yaml\n\ncwd = '/kaggle/working/'\n\nwith open(join( cwd , 'train.txt'), 'w') as f:\n    for path in glob('/kaggle/working/vinbigdata_processed/images/train/*'):\n        f.write(path+'\\n')\n            \nwith open(join( cwd , 'val.txt'), 'w') as f:\n    for path in glob('/kaggle/working/vinbigdata_processed/images/val/*'):\n        f.write(path+'\\n')\n\ndata = dict(\n    train =  join( cwd , 'train.txt') ,\n    val   =  join( cwd , 'val.txt' ),\n    nc    = 14,\n    names = classes\n    )\n\nwith open(join( cwd , 'vinbigdata.yaml'), 'w') as outfile:\n    yaml.dump(data, outfile, default_flow_style=False)\n\nf = open(join( cwd , 'vinbigdata.yaml'), 'r')\nprint('\\nyaml:')\nprint(f.read())","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"shutil.copytree('/kaggle/input/yolov5-official-v31-dataset/yolov5', '/kaggle/working/yolov5')\nos.chdir('/kaggle/working/yolov5') # install dependencies\n\nimport torch\nfrom IPython.display import Image, clear_output  # to display images\n\nclear_output()\nprint('Setup complete. Using torch %s %s' % (torch.__version__, torch.cuda.get_device_properties(0) if torch.cuda.is_available() else 'CPU'))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"!WANDB_MODE=\"dryrun\" python train.py --img 640 --batch 16 --epochs 30 --data /kaggle/working/vinbigdata.yaml --weights yolov5x.pt --cache\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Inference"},{"metadata":{"trusted":true},"cell_type":"code","source":"!python detect.py --weights '/kaggle/working/yolov5/runs/train/exp/weights/best.pt'\\\n--img 640\\\n--conf 0.15\\\n--iou 0.5\\\n--source /kaggle/working/vinbigdata_processed/images/val\\\n--exist-ok","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import matplotlib.pyplot as plt\nfrom mpl_toolkits.axes_grid1 import ImageGrid\nimport numpy as np\nimport random\nimport cv2\nfrom glob import glob\nfrom tqdm import tqdm\n\nfiles = glob('runs/detect/exp/*')\n\nfor _ in range(1):\n    row = 4\n    col = 4\n    grid_files = files[:16]\n    images     = []\n    for image_path in tqdm(grid_files):\n        img          = cv2.cvtColor(cv2.imread(image_path), cv2.COLOR_BGR2RGB)\n        images.append(img)\n\n    fig = plt.figure(figsize=(col*5, row*5))\n    grid = ImageGrid(fig, 111,  # similar to subplot(111)\n                     nrows_ncols=(col, row),  # creates 2x2 grid of axes\n                     axes_pad=0.05,  # pad between axes in inch.\n                     )\n\n    for ax, im in zip(grid, images):\n        # Iterating over the grid returns the Axes.\n        ax.imshow(im)\n        ax.set_xticks([])\n        ax.set_yticks([])\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Compare with non-clahe image train models"},{"metadata":{"trusted":true},"cell_type":"code","source":"weights_dir = '/kaggle/input/vinbigdata-cxr-ad-yolov5-14-class-train/yolov5/runs/train/exp/weights/best.pt'\n# train 30 epochs\n# val mAP_0.5 : 0.31919\n# val mAP_0.5:0.95 : 0.14161","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"!python detect.py --weights $weights_dir\\\n--img 640\\\n--conf 0.15\\\n--iou 0.5\\\n--source /kaggle/working/vinbigdata_processed/images/val\\\n--save-txt --save-conf --exist-ok\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import matplotlib.pyplot as plt\nfrom mpl_toolkits.axes_grid1 import ImageGrid\nimport numpy as np\nimport random\nimport cv2\nfrom glob import glob\nfrom tqdm import tqdm\n\nfiles = glob('runs/detect/exp/*')\n\nfor _ in range(1):\n    row = 4\n    col = 4\n    grid_files = files[:16]\n    images     = []\n    for image_path in tqdm(grid_files):\n        img          = cv2.cvtColor(cv2.imread(image_path), cv2.COLOR_BGR2RGB)\n        images.append(img)\n\n    fig = plt.figure(figsize=(col*5, row*5))\n    grid = ImageGrid(fig, 111,  # similar to subplot(111)\n                     nrows_ncols=(col, row),  # creates 2x2 grid of axes\n                     axes_pad=0.05,  # pad between axes in inch.\n                     )\n\n    for ax, im in zip(grid, images):\n        # Iterating over the grid returns the Axes.\n        ax.imshow(im)\n        ax.set_xticks([])\n        ax.set_yticks([])\n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Compare with GT labels"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}