{"cells":[{"metadata":{},"cell_type":"markdown","source":"In this competition, we are classifying common thoracic lung diseases and localizing critical findings. This is an object detection and classification problem.\n\nFor each test image, you will be predicting a bounding box and class for all findings. If you predict that there are no findings, you should create a prediction of \"14 1 0 0 1 1\" (14 is the class ID for no finding, and this provides a one-pixel bounding box with a confidence of 1.0).\n\nThe images are in DICOM format, which means they contain additional data that might be useful for visualizing and classifying.\n\nDataset information\nThe dataset comprises 18,000 postero-anterior (PA) CXR scans in DICOM format, which were de-identified to protect patient privacy. All images were labeled by a panel of experienced radiologists for the presence of 14 critical radiographic findings as listed below:\n\n0 - Aortic enlargement\n1 - Atelectasis\n2 - Calcification\n3 - Cardiomegaly\n4 - Consolidation\n5 - ILD\n6 - Infiltration\n7 - Lung Opacity\n8 - Nodule/Mass\n9 - Other lesion\n10 - Pleural effusion\n11 - Pleural thickening\n12 - Pneumothorax\n13 - Pulmonary fibrosis\nThe \"No finding\" observation (14) was intended to capture the absence of all findings above.\n\nNote that a key part of this competition is working with ground truth from multiple radiologists.\n\nFiles\ntrain.csv - the train set metadata, with one row for each object, including a class and a bounding box. Some images in both test and train have multiple objects.\nsample_submission.csv - a sample submission file in the correct format\nColumns\nimage_id - unique image identifier\nclass_name - the name of the class of detected object (or \"No finding\")\nclass_id - the ID of the class of detected object\nrad_id - the ID of the radiologist that made the observation\nx_min - minimum X coordinate of the object's bounding box\ny_min - minimum Y coordinate of the object's bounding box\nx_max - maximum X coordinate of the object's bounding box\ny_max - maximum Y coordinate of the object's bounding box\n\n**\nEvaluation**\n\n\nThe challenge uses the standard PASCAL VOC 2010 mean Average Precision (mAP) at IoU > 0.4.\n\nSubmission File\nImages in the test set may contain more than one object. For each object in a given test image, you must predict a class ID, confidence score, and bounding box in format xmin ymin xmax ymax. If you predict that there are NO objects in a given image, you should predict 14 1.0 0 0 1 1, where 14 is the class ID for \"No finding\", 1.0 is the confidence, and 0 0 1 1 is a one-pixel bounding box.\n\nThe submission file should contain a header and have the following format:\n\nID,TARGET\n004f33259ee4aef671c2b95d54e4be68,14 1 0 0 1 1\n004f33259ee4aef671c2b95d54e4be69,11 0.5 100 100 200 200 13 0.7 10 10 20 20\netc.\n\n"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np\nimport pydicom\nimport pandas as pd\nfrom pydicom.pixel_data_handlers.util import apply_voi_lut\nimport cv2\nimport matplotlib.pyplot as plt\n%matplotlib inline\nimport random\nfrom random import randint\n\n\ntrain_df = pd.read_csv(f'{dataset_dir}/train.csv')\n\n# convert rad_id to int type\nle = preprocessing.LabelEncoder()\ntrain_df['rad_label'] = le.fit_transform(train_df['rad_id'])\ntrain_df.head()\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def read_xray(path, voi_lut = True, fix_monochrome = True):\n    dicom = pydicom.read_file(path)\n    \n    # VOI LUT (if available by DICOM device) is used to transform raw DICOM data to \"human-friendly\" view\n    if voi_lut:\n        data = apply_voi_lut(dicom.pixel_array, dicom)\n    else:\n        data = dicom.pixel_array\n               \n    # depending on this value, X-ray may look inverted - fix that:\n    if fix_monochrome and dicom.PhotometricInterpretation == \"MONOCHROME1\":\n        data = np.amax(data) - data\n        \n    data = data - np.min(data)\n    data = data / np.max(data)\n    data = (data * 255).astype(np.uint8)\n        \n    return data\n        \n    \ndef plot_img(img, size=(7, 7), is_rgb=True, title=\"\", cmap='gray'):\n    plt.figure(figsize=size)\n    plt.imshow(img, cmap=cmap)\n    plt.suptitle(title)\n    plt.show()\n\n\ndef plot_imgs(imgs, cols=4, size=7, is_rgb=True, title=\"\", cmap='gray', img_size=(500,500)):\n    rows = len(imgs)//cols + 1\n    fig = plt.figure(figsize=(cols*size, rows*size))\n    for i, img in enumerate(imgs):\n        if img_size is not None:\n            img = cv2.resize(img, img_size)\n        fig.add_subplot(rows, cols, i+1)\n        plt.imshow(img, cmap=cmap)\n    plt.suptitle(title)\n    plt.show()\n    \ndef draw_bboxes(img, boxes, thickness=10, color=(255, 0, 0), img_size=(500,500)):\n    img_copy = img.copy()\n    if len(img_copy.shape) == 2:\n        img_copy = np.stack([img_copy, img_copy, img_copy], axis=-1)\n    for box in boxes:\n        img_copy = cv2.rectangle(\n            img_copy,\n            (int(box[0]), int(box[1])),\n            (int(box[2]), int(box[3])),\n            color, thickness)\n    if img_size is not None:\n        img_copy = cv2.resize(img_copy, img_size)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"\n\nimgs = []\nimg_ids = finding_df['image_id'].values\nclass_ids = finding_df['class_id'].unique()\n\n# map label_id to specify color\nlabel2color = {class_id:[randint(0,255) for i in range(3)] for class_id in class_ids}\nthickness = 3\nscale = 5\n\n\nfor i in range(8):\n    img_id = random.choice(img_ids)\n    img_path = f'{dataset_dir}/train/{img_id}.dicom'\n    img = dicom2array(path=img_path)\n    img = cv2.resize(img, None, fx=1/scale, fy=1/scale)\n    img = np.stack([img, img, img], axis=-1)\n    \n    boxes = finding_df.loc[finding_df['image_id'] == img_id, ['x_min', 'y_min', 'x_max', 'y_max']].values/scale\n    labels = finding_df.loc[finding_df['image_id'] == img_id, ['class_id']].values.squeeze()\n    \n    for label_id, box in zip(labels, boxes):\n        color = label2color[label_id]\n        img = cv2.rectangle(\n            img,\n            (int(box[0]), int(box[1])),\n            (int(box[2]), int(box[3])),\n            color, thickness\n    )\n    img = cv2.resize(img, (500,500))\n    imgs.append(img)\n    \nplot_imgs(imgs, cmap=None)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}