{"cells":[{"metadata":{},"cell_type":"markdown","source":"# About:\n\nWhen you have a broken arm, radiologists help save the day—and the bone. These doctors diagnose and treat medical conditions using imaging techniques like CT and PET scans, MRIs, and, of course, X-rays. Yet, as it happens when working with such a wide variety of medical tools, radiologists face many daily challenges, perhaps the most difficult being the chest radiograph. The interpretation of chest X-rays can lead to medical misdiagnosis, even for the best practicing doctor. Computer-aided detection and diagnosis systems (CADe/CADx) would help reduce the pressure on doctors at metropolitan hospitals and improve diagnostic quality in rural areas.\n\nExisting methods of interpreting chest X-ray images classify them into a list of findings. There is currently no specification of their locations on the image which sometimes leads to inexplicable results. A solution for localizing findings on chest X-ray images is needed for providing doctors with more meaningful diagnostic assistance."},{"metadata":{},"cell_type":"markdown","source":"# Data\n\nIn this competition, we are classifying common thoracic lung diseases and localizing critical findings. This is an object detection and classification problem.\n\nFor each test image, you will be predicting a bounding box and class for all findings. If you predict that there are no findings, you should create a prediction of \"14 1 0 0 1 1\" (14 is the class ID for no finding, and this provides a one-pixel bounding box with a confidence of 1.0).\n\nThe images are in DICOM format, which means they contain additional data that might be useful for visualizing and classifying.\n\n**Dataset information**\n\nThe dataset comprises 18,000 postero-anterior (PA) CXR scans in DICOM format, which were de-identified to protect patient privacy. All images were labeled by a panel of experienced radiologists for the presence of 14 critical radiographic findings as listed below:\n\n* 0 - Aortic enlargement\n* 1 - Atelectasis\n* 2 - Calcification\n* 3 - Cardiomegaly\n* 4 - Consolidation\n* 5 - ILD\n* 6 - Infiltration\n* 7 - Lung Opacity\n* 8 - Nodule/Mass\n* 9 - Other lesion\n* 10 - Pleural effusion\n* 11 - Pleural thickening\n* 12 - Pneumothorax\n* 13 - Pulmonary fibrosis\n\nThe \"No finding\" observation (14) was intended to capture the absence of all findings above.\n\n"},{"metadata":{},"cell_type":"markdown","source":"# Dicom to numpy ([From here](http://www.kaggle.com/raddar/convert-dicom-to-np-array-the-correct-way))"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"import pydicom\nfrom pydicom.pixel_data_handlers.util import apply_voi_lut\n\nimport numpy as np\nimport pandas as pd \n\nimport matplotlib.pyplot as plt\nimport  matplotlib.patches as patches\n%matplotlib inline\n\nimport seaborn as sns\n\nfrom random import randint","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def read_xray(path, voi_lut = True, fix_monochrome = True):\n    dicom = pydicom.read_file(path)\n    \n    # VOI LUT (if available by DICOM device) is used to transform raw DICOM data to \"human-friendly\" view\n    if voi_lut:\n        data = apply_voi_lut(dicom.pixel_array, dicom)\n    else:\n        data = dicom.pixel_array\n               \n    # depending on this value, X-ray may look inverted - fix that:\n    if fix_monochrome and dicom.PhotometricInterpretation == \"MONOCHROME1\":\n        data = np.amax(data) - data\n        \n    data = data - np.min(data)\n    data = data / np.max(data)\n    data = (data * 255).astype(np.uint8)\n        \n    return data","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Plot boxes"},{"metadata":{"trusted":true},"cell_type":"code","source":"t = [1,2,3,4,5,6]\nt[0:3][0:2]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def plot_dicom(path):\n    img = read_xray(path)\n    plt.figure(figsize = (12,12))\n    plt.imshow(img, 'gray')\nplot_dicom('/kaggle/input/vinbigdata-chest-xray-abnormalities-detection/test/f923cd5cb2daf790272dbef850f5647b.dicom')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"data = pd.read_csv('/kaggle/input/vinbigdata-chest-xray-abnormalities-detection/train.csv')\ndata.head(10)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"data[data['image_id']=='051132a778e61a86eb147c7c6f564dfe']","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We see that one image may appear many time in the excel file, since it contains many abnormalities."},{"metadata":{},"cell_type":"markdown","source":"Let's make a function to plot borders."},{"metadata":{"trusted":true},"cell_type":"code","source":"def plot_borders(imageid,data):\n    img = read_xray('/kaggle/input/vinbigdata-chest-xray-abnormalities-detection/train/{}.dicom'.format(imageid))\n    infos = data[data['image_id'] == imageid]\n    fig = plt.figure(figsize = (12,12)) \n    class_ids = infos['class_id'].unique()\n\n    label2color = {class_id:[randint(0,255)/255 for i in range(3)] for class_id in class_ids}\n  \n    ax = fig.add_subplot(111) \n    for index, row in infos.iterrows():\n        # Create a Rectangle patch.\n        x_min = row['x_min']\n        x_max = row['x_max']\n        y_min = row['y_min']\n        y_max = row['y_max']\n        color = label2color[row['class_id']]\n        # Draw the rectangle\n        rect = patches.Rectangle((x_min,y_min),x_max-x_min,y_max-y_min,linewidth=2,edgecolor= color,facecolor='none',label=row['class_name'])\n\n        # Add the patch to the Axes\n        ax.add_patch(rect)\n        ax.legend()\n\n    plt.imshow(img, 'gray')\nplot_borders('051132a778e61a86eb147c7c6f564dfe',data)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#Another example\nplot_borders('9a5094b2563a1ef3ff50dc5c7ff71345',data)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#The last one\nplot_borders('afb6230703512afc370f236e8fe98806',data)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(22,5))\nsns.countplot(data['class_name'])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The data is unbalanced, a lot of images have no findings of pneumonial anomalies. For the anomalies, the most occurent ones are Cardiomegaly,Aortic enlargement, Pleural thickening, Pulmonary fibrosis. Because of this, we would need some techniques to combat this unbalance problem"},{"metadata":{},"cell_type":"markdown","source":"1. I will try to make some insights on the coordinates containing anomalies, and try to see if there is any correlation between the abnormalities types and the coordinates."},{"metadata":{"trusted":true},"cell_type":"code","source":"mean_coordinates = data.groupby('class_name').mean()\nmean_coordinates['image_id'] = '50a418190bc3fb1ef1633bf9678929b3' # this is an image with no findings, we take it just to show the locations of the mean cordiantes\nmean_coordinates['class_name'] =  mean_coordinates.index\nmean_coordinates.reset_index(drop=True)\nplot_borders('50a418190bc3fb1ef1633bf9678929b3',mean_coordinates)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Crop images containig anomalies:"},{"metadata":{"trusted":true},"cell_type":"code","source":"def crop(imageid):\n    \n    w=12\n    h=12\n    fig=plt.figure(figsize=(9, 25))\n    rows = 1\n    \n    img = read_xray('/kaggle/input/vinbigdata-chest-xray-abnormalities-detection/train/{}.dicom'.format(imageid))\n    infos = data[data['image_id'] == imageid]\n    columns = infos.shape[0]\n\n    i=1\n    for index, row in infos.iterrows():\n        x_min = int(row['x_min'])\n        x_max = int(row['x_max'])\n        y_min = int(row['y_min'])\n        y_max = int(row['y_max'])\n        label = row['class_name']\n        \n        abnormality = img[x_min:x_max,y_min:y_max]\n        fig.add_subplot(rows, columns, i)\n        plt.imshow(abnormality, 'gray')\n        plt.title(label)\n        i=i+1\n    plt.tight_layout() \n    plt.show()\ncrop('9a5094b2563a1ef3ff50dc5c7ff71345')","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}