{"cells":[{"metadata":{},"cell_type":"markdown","source":"<center><h1 style=\"color:blue\">VinBigData Chest X-ray Abnormalities Detection</h1></center>\n<center><h1 style=\"color:red\">Automatically localize and classify thoracic abnormalities from chest radiographs</h1></center>\n<img src=\"https://storage.googleapis.com/kaggle-competitions/kaggle/24800/logos/header.png?t=2020-12-17-19-26-15\">"},{"metadata":{},"cell_type":"markdown","source":"# **Competetion**\n\n- **Task**: Automatically localize and classify <span style=\"color:red\">14 types of thoracic abnormalities</span> from chest radiographs. \n- **Dataset**: Consisting of <span style=\"color:red\">18,000 scans</span>: <span style=\"color:blue\">15,000 train images</span> and will be evaluated on a <span style=\"color:blue\">test set of 3,000 images</span>. \n\n- These annotations were collected via VinBigData's web-based platform, VinLab. Details on building the dataset can be found in the organizer's recent paper [“VinDr-CXR: An open dataset of chest X-rays with radiologist's annotations”](https://storage.googleapis.com/kaggle-media/competitions/VinBigData/VinDr_CXR_data_paper.pdf).\n\nWe are classifying common thoracic lung diseases and localizing critical findings. This is **<span style=\"color:green\"> an object detection and classification </span>** problem."},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"import os\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport matplotlib.patches as ptc\nimport seaborn as sns\nimport pydicom\nfrom pydicom.pixel_data_handlers.util import apply_voi_lut\nfrom skimage import exposure\nimport warnings\n\nwarnings.filterwarnings('ignore')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Load training data (Train.csv)\n\n> For each test image, you will be predicting a bounding box and class for all findings. If you predict that there are no findings, you should create a prediction of \"14 1 0 0 1 1\" (14 is the class ID for no finding, and this provides a one-pixel bounding box with a confidence of 1.0).\n\nThe images are in <span style=\"color:red\"> DICOM format</span>, which means they contain additional data that might be useful for visualizing and classifying."},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":false,"trusted":true},"cell_type":"code","source":"df_train = pd.read_csv(\"../input/vinbigdata-chest-xray-abnormalities-detection/train.csv\")\nprint (df_train.shape)\ndf_train.head()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"df_train.isna().sum().to_frame().rename(columns={0:\"NA Counts\"}).style.background_gradient(cmap=\"summer\")","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"df_train.nunique().to_frame().rename(columns={0:\"Unique Values\"}).style.background_gradient(cmap=\"plasma\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# **EDA**"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(20, 8))\nsns.set_style('darkgrid')\nsns.countplot(y=\"class_name\", data=df_train, palette='Set2')\nplt.title(\"Class Name Distribution\", weight='bold', fontsize=22)\nplt.ylabel(\"Class Name\", weight='bold')\nplt.xlabel(\"Count\", weight='bold')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(20, 8))\nsns.set_style('darkgrid')\nsns.countplot(y=\"rad_id\", data=df_train)\nplt.title(\"Rad ID Distribution\", weight='bold', fontsize=22)\nplt.ylabel(\"Rad ID\", weight='bold')\nplt.xlabel(\"Count\", weight='bold')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# **Load DICOM data**\n\n1. Reference: https://www.kaggle.com/raddar/convert-dicom-to-np-array-the-correct-way\n1. Reference: https://www.kaggle.com/raddar/popular-x-ray-image-normalization-techniques"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"train_dir = \"../input/vinbigdata-chest-xray-abnormalities-detection/train\"\ntest_dir = \"../input/vinbigdata-chest-xray-abnormalities-detection/test\"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"def read_xray(path, voi_lut=True, fix_monochrome=True):\n    dcm_data = pydicom.read_file(path)\n    \n    def show_dcm_info(data):\n        print(\"Gender :\", data.PatientSex)\n        if 'PixelData' in data:\n            rows = int(data.Rows)\n            cols = int(data.Columns)\n            print(\"Image size : {rows:d} x {cols:d}, {size:d} bytes\".format(rows=rows, cols=cols, size=len(data.PixelData)))\n            if 'PixelSpacing' in data:\n                print(\"Pixel spacing :\", data.PixelSpacing)\n    \n    show_dcm_info(dcm_data)\n    \n    # VOI LUT (if available by DICOM device) is used to transform raw DICOM data to \"human-friendly\" view\n    if voi_lut:\n        data = apply_voi_lut(dcm_data.pixel_array, dcm_data)\n    else:\n        data = dcm_data.pixel_array\n               \n    # depending on this value, X-ray may look inverted - fix that:\n    if fix_monochrome and dcm_data.PhotometricInterpretation == \"MONOCHROME1\":\n        data = np.amax(data) - data\n        \n    data = data - np.min(data)\n    data = data / np.max(data)\n    data = (data * 255).astype(np.uint8)\n        \n    return data","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## **Visualizations of DICOM Images**\n\n1. Raw Image\n1. Histogram Normalized Image\n1. CLAHE Normalized Image\n1. Annotated Image"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"flag = 1\nwhile(flag):    \n    idx = np.random.randint(0, df_train.shape[0])\n    if df_train.loc[idx, 'class_name'] != 'No finding':\n        flag = 0\n        \n        img_id = df_train.loc[idx, 'image_id']\n        img = read_xray(os.path.join(train_dir, img_id+\".dicom\"))\n\n        fig, ax = plt.subplots(2,2, figsize=(20,20))\n        ax[0][0].imshow(img, 'gray')\n        ax[0][0].set_title(\"Raw Image\",fontsize=15)\n\n        ax[0][1].imshow(exposure.equalize_hist(img), 'gray')\n        ax[0][1].set_title(\"Histogram Normalized Image\",fontsize=15)\n\n        ax[1][1].imshow(exposure.equalize_adapthist(img), 'gray')\n        ax[1][1].set_title(\"CLAHE Normalized Image\",fontsize=15)\n\n        bbox = [df_train.loc[idx, 'x_min'], df_train.loc[idx, 'y_min'], df_train.loc[idx, 'x_max'], df_train.loc[idx, 'y_max']]\n        patch = ptc.Rectangle((bbox[0], bbox[1]), bbox[2]-bbox[0], bbox[3]-bbox[1], ec='r', fc='none', lw=2.)\n        ax[1][0].imshow(img, 'gray')\n        ax[1][0].add_patch(patch)\n        ax[1][0].set_title(\"Annotated Image\",fontsize=15)\n\n        plt.suptitle('DICOM Image',fontsize=25)\n        plt.show()\n    \n    else:\n        continue","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# **This work is in progress. Feel free to <span style=\"color:red\"> Upvote </span> and give <span style=\"color:blue\"> Feedback </span>.**"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}