{"cells":[{"metadata":{},"cell_type":"markdown","source":""},{"metadata":{},"cell_type":"markdown","source":"# VinBigData Chest X-ray Abnormalities Detection\n##### **Automatically localize and classify thoracic abnormalities from chest radiographs**"},{"metadata":{},"cell_type":"markdown","source":"In this competition, the goal is to automatically localize and classify 14 types of thoracic abnormalities from chest radiographs. If successful, we will help build what could be a valuable second opinion for radiologists. An automated system that could accurately identify and localize findings on chest radiographs would relieve the stress of busy doctors while also providing patients with a more accurate diagnosis. \n\nBefore doing that, let's fisrt learn more about the competiton, the data type, the evaluation and so on.\n\n ----- WORK IN PROGRESS -----\n\n**My goal is to publish several notebooks, each one of them will represent a step in this competition. Please let me know if you have any feedback.** \n\n "},{"metadata":{},"cell_type":"markdown","source":"# Table of Contents\n1. [Introduction 📚](#introduction)\n 1. [What is the competition about and how did we get the data ?](#how)\n 2. [Chest X-rays](#chest)\n2. [The Data 💾](#data)\n 1. [Short presentation of the data](#presentation)\n 2. [Reading the data](#reading)\n 3. [The findings](#findings)\n"},{"metadata":{},"cell_type":"markdown","source":"# Introduction 📚 "},{"metadata":{},"cell_type":"markdown","source":"## What is the competition about and how did we get the data ? "},{"metadata":{},"cell_type":"markdown","source":"In an effort to provide a large dataset of chest x-ray (CXR) images with high-quality labels for the research community, VinBigData have built the VinDr-CXR dataset from more than 100,000 raw images in DICOM format that were retrospectively collected from the Hospital 108 and the Hanoi Medical University Hospital, two of the largest hospitals in Vietnam.\n\nOut of this raw data, they release 18 000 postero-anterior (PA) view CXR scans that come with both the localization of critical findings and the classification of common thoracic diseases. These images were annotated by a group of 17 radiologists with at least 8 years of experience for the presence of 22 critical findings (local labels) and 6 diagnoses (global labels); each finding is localized with a bounding box while the global labels reflect the diagnostic impression of the radiologist.\n\nThe released dataset is divided into a training set of 15,000 and a test set of 3,000. Each scan in the training set was independently labeled by 3 radiologists, while each scan in the test set was labeled by the consensus of 5 radiologis.\n\nWhy that is super cool ? because as the Vingroup said in their recent paper [“VinDr-CXR: An open dataset of chest X-rays with radiologist's annotations”](https://arxiv.org/pdf/2012.15029.pdf), most of the existing chest X-ray datasets include labels from a list of findings without specifying their locations on the radiographs. This limits the development of machine learning algorithms for the detection and localization of chest abnormalities. \n\nFor example, the MIMIC-CXR dataset release in 2019 contains more than 377 000 CXR images but these are image-level labeled (no bounding-box, just an image label) and like most of the other existing CXR datasets, its depend on automated rule-based labelers that either use keyword matching or an NLP model to extract disease labels from free-text radiology reports. These tools can produce labels on a large scale but, at the same time, introduce a high rate of inconsistency, uncertainty, and errors as mentionned the paper.\n\nSo with this competition, we benefit from the knowledge of several experienced radiologists to hopefully help building an automated system that could accurately identify and localize findings on chest radiographs.\n\nSo it is pretty clair by now that we are going to work with X-ray images and to be more precise, chest X-ray. Let's learn more about that part."},{"metadata":{},"cell_type":"markdown","source":"## Chest X-rays \n\n### **A short reminder of the definition of X-rays**\n\nX-rays are a type of radiation called electromagnetic waves. X-ray imaging creates pictures of the inside of your body. The images show the parts of your body in different shades of black and white. This is because different tissues absorb different amounts of radiation. Calcium in bones absorbs x-rays the most, so bones look white. Fat and other soft tissues absorb less and look gray. Air absorbs the least, so for example lungs which are full of air look black.\n\nThe most familiar use of x-rays is checking for fractures (broken bones), but x-rays are also used in other ways. For example, chest x-rays can spot pneumonia and mammograms use x-rays to look for breast cancer.\n\nA chest x-ray is an x-ray of the chest, lungs, heart, large arteries, ribs, and diaphragm. There are several CXR views. In this competition we will only have to deal with postero-anterior (PA) views. That mean that the patient stands in front of a radiographic plate, hands on hips, with the X-ray source 2 m behind.\n\n### **A normal CXR**\n\nIt is important to know the normal chest radiograph and common landmarks so that we can recognize what is abnormal. \n\n\n\nNote that the left side of the subject is on the right side of the image. In most CXR a little R or L will indicate the side.\n\nHere are some common landmarks for the normal CXR:\n\n- The trachea should sit midline and be in between the right and left clavicular heads.\n- The carina is the point or level at which the trachea divides into the right and left main bronchi. This is usually midline with the spinous process being behind it. \n- The aortic knob should be visualized in the normal chest radiograph around the level of T3 to T4 or just lateral to the carina. \n- Usually the heart is positioned with one-third of its diameter to the right, and two-thirds to the left of the thoracic vertebrae spinous processes.\n- The right hemidiaphragm normally sits slightly higher than the left due to the presence of the liver under the diaphragm.\n\n\nNote that the T4 (for example) vertebra is the fourth thoracic vertebra that makes up the middle segment of spinal column of the human body. The thoracic spinal vertebrae consist of 12 total vertebrae and are located between the cervical vertebrae (which begin at the base of the skull) and the lumbar spinal vertebrae.\n\n\nAfter this breif introduction about the construction of the dataset, the reminder about X-ray and the presentation of a normal CXR, we can move on to see some of the abnormal CXR and by extention talk about our data."},{"metadata":{},"cell_type":"markdown","source":"# The Data 💾 "},{"metadata":{},"cell_type":"markdown","source":"## Short presentation of the data "},{"metadata":{},"cell_type":"markdown","source":"As we said before, the dataset comprises 18,000 postero-anterior (PA) CXR scans in DICOM format. 15,000 for training and 3000 for test.\n\nIn addition, we have 2 files :\n\n\n - train.csv - the train set metadata, with one row for each object, including a class and a bounding box. Some images in both test and train have multiple objects.\n - sample_submission.csv - a sample submission file in the correct format\n\n\nfor the train.csv file, we will find these data :\n\n - image_id - unique image identifier\n - class_name - the name of the class of detected object (or \"No finding\")\n - class_id - the ID of the class of detected object\n - rad_id - the ID of the radiologist that made the observation\n - x_min - minimum X coordinate of the object's bounding box\n - y_min - minimum Y coordinate of the object's bounding box\n - x_max - maximum X coordinate of the object's bounding box\n - y_max - maximum Y coordinate of the object's bounding box\n\n\nFor each test image, we will be predicting a bounding box and class for all findings. If we predict that there are no findings, we should create a prediction of \"14 1 0 0 1 1\".\n\n- the first number is the class ID, can go from 0 to 14.\n- the second number is the confidence of our detection. This value is between 0 and 1, the closer to 1 the more confident we are.\n- the rest is x_min, y_min and x_max,y_max. The bounding box coordinate.\n\n\nHere is the distribution of findings and pathologies on the training set of the VinDr-CXR originel Dataset.\n\n\n\nAs you can see there are 28 classes in the original dataset. For the competition we have to localize and classify the presence of only 14 critical radiographic findings as listed below:\n\n0. Aortic enlargement\n1. Atelectasis\n2. Calcification\n3. Cardiomegaly\n4. Consolidation\n5. ILD\n6. Infiltration\n7. Lung Opacity\n8. Nodule/Mass\n9. Other lesion\n10. Pleural effusion\n11. Pleural thickening\n12. Pneumothorax\n13. Pulmonary fibrosis\n14. No findings\n\nWe will go through every one of them in one section below.\n\n\nThe images are in DICOM format, which means they contain additional data that might be useful for visualizing and classifying. Let's begin dealing with the DICOM files."},{"metadata":{},"cell_type":"markdown","source":"## Reading the data "},{"metadata":{},"cell_type":"markdown","source":"First let's import all what we need and define our paths. Then we can start to take a look at our data."},{"metadata":{"trusted":true},"cell_type":"code","source":"# All the import\n\nimport os\nimport re\nimport pydicom\nimport matplotlib\nimport numpy as np\nimport pandas as pd\nfrom tqdm import tqdm\nfrom math import ceil\nimport matplotlib.pyplot as plt\nimport matplotlib.gridspec as gridspec\nfrom pydicom.pixel_data_handlers.util import apply_voi_lut\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_csv = pd.read_csv(\"../input/vinbigdata-chest-xray-abnormalities-detection/train.csv\")\ntrain_path = \"../input/vinbigdata-chest-xray-abnormalities-detection/train\"\ntrain_dicom_path = [f\"{train_path}/{i}\" for i in os.listdir(train_path)] ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_csv.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(f\"The train shape is {train_csv.shape}\")\nprint(f\"There are {len(train_csv['image_id'].unique())} unique images in this CSV and {len(train_dicom_path)} DICOM images\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let see for one CXR what information we have:"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_csv[train_csv['image_id'] == '9a5094b2563a1ef3ff50dc5c7ff71345']","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"So one radologist can give several findings, if two or more radiologist have the same findings, the position of the bounding box may be differents (example with class_id = 3)."},{"metadata":{"trusted":true},"cell_type":"code","source":"# Let's read the dicom of the same image above\nimage_id = '9a5094b2563a1ef3ff50dc5c7ff71345'\ndata_file = pydicom.dcmread(f\"{train_path}/{image_id}.dicom\")\nprint(data_file)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"For now let's visualize some of the CXR we have."},{"metadata":{"trusted":true},"cell_type":"code","source":"# from here https://www.kaggle.com/raddar/convert-dicom-to-np-array-the-correct-way\n\ndef read_xray(path, voi_lut = True, fix_monochrome = True):\n dicom = pydicom.read_file(path)\n \n # VOI LUT (if available by DICOM device) is used to transform raw DICOM data to \"human-friendly\" view\n if voi_lut:\n data = apply_voi_lut(dicom.pixel_array, dicom)\n else:\n data = dicom.pixel_array\n \n # depending on this value, X-ray may look inverted - fix that:\n if fix_monochrome and dicom.PhotometricInterpretation == \"MONOCHROME1\":\n data = np.amax(data) - data\n \n data = data - np.min(data)\n data = data / np.max(data)\n data = (data * 255).astype(np.uint8)\n \n return data\n\ndef example(classe, number_ex, path=train_path):\n image_list = train_csv[train_csv['class_id']==classe][0:number_ex]['image_id'].values\n image_index = train_csv[train_csv['class_id']==classe][0:number_ex]['image_id'].index.values\n rows = int(ceil(number_ex / 4))\n gs = gridspec.GridSpec(rows, 4)\n fig = plt.figure(figsize=(30, 20))\n fig.suptitle(f\"The class {train_csv.loc[train_csv['class_id']==classe, 'class_name'].values[0]}\", fontsize=30)\n for i, (name, image_index) in enumerate(zip(image_list, image_index)):\n ax = fig.add_subplot(gs[i])\n ax.set_title(train_csv.loc[i, 'image_id'])\n ax.axis(\"off\")\n data = read_xray(f\"{path}/{name}.dicom\")\n ax.imshow(data, cmap='gray')\n if train_csv.loc[image_index, 'class_name'] != 'No finding':\n bbox = [train_csv.loc[image_index, 'x_min'],\n train_csv.loc[image_index, 'y_min'],\n train_csv.loc[image_index, 'x_max'],\n train_csv.loc[image_index, 'y_max']]\n p = matplotlib.patches.Rectangle((bbox[0], bbox[1]),\n bbox[2]-bbox[0],\n bbox[3]-bbox[1],\n ec='r', fc='none', lw=2.)\n ax.add_patch(p)\n fig.tight_layout()\n plt.subplots_adjust(hspace=0.1, wspace=0.1, top=0.95)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"example(1,8)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## The findings "},{"metadata":{},"cell_type":"markdown","source":"OK that cool, we now can visualize our data and the abnormalities in each CXR. It's now time to understand each of the abnormalities."},{"metadata":{},"cell_type":"markdown","source":"### Aortic enlargement\n\n\nThis is know as a sign of an aortic aneurysm. It's litterally an enlargement of the aorta (the body's largest artery), which can occur in the chest (thoracic aneurysm) or abdomen. \n\nHere is one figure that can help understand it.\n\n\n\nThe above illustration shows three types of aneurysm and the two aneurysm shapes. The three types of aneurysm are Ascending Thoracic aortic Aneurysm (TAA), Descending Thoracic Aortic Aneurysm (TAA), and Abdominal Aortic Aneurysm (AAA). The Fusiform Aneurysm and Saccular Aneurysm show the two types of aneurysm shapes.\n\n\nOf course, we are working on chest data so we won't see any Abdominal Aortic Aneurysm but it's still important to know that kind of stuff.\n\nRisk factors for it may include:\n\n Age\n Gender\n Smoking\n High blood pressure\n ...\n\nNote that we have the age and the gender information in our metadata! that can help us later.\n\n\nHere are some real examples."},{"metadata":{"trusted":true},"cell_type":"code","source":"example(0, 8)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Atelectasis\n\nAtelectasis is a medical term used to describe the complete or partial collapse of a lung. It is sometimes referred to as a “collapsed lung\". It occurs when tiny air sacs in the lungs known as alveoli deflate. \n\n\n\nAs we can see, this abnormality is recognizable as there is no air in part or all of the lungs. The air is represented by black and therefore I think that we can see Atelectasis whiter in the CXR:"},{"metadata":{"trusted":true},"cell_type":"code","source":"example(1, 8)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Calcification \n\n\nThe calcification is a deposits of calcium in the tissues causing the tissue to harden. We can detect this as the calcium absorbs x-rays the most, so calcification look white in the X-ray. As it's a deposits, the bounding box can be \"smaller\" sometimes so it may be more diffucult for us - non radiologist - to detect this abnormality. "},{"metadata":{"trusted":true},"cell_type":"code","source":"example(2, 8)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Cardiomegaly\n\nThe term \"cardiomegaly\" refers to an enlarged heart that we can see in the CXR. TAn enlarged heart may be the result of a short-term stress on the body, such as pregnancy, or a medical condition, such as the weakening of the heart muscle, coronary artery disease, heart valve problems or abnormal heart rhythms. \n\nOne way to notice it if cardiothoracic ratio is too large."},{"metadata":{"trusted":true},"cell_type":"code","source":"example(3, 8)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Consolidation\n\nThis abnormality occurs when the air that usually fills the small airways in the lungs is replaced with something else. Depending on the cause, the air may be replaced with:\n\n a fluid, such as pus, blood, or water\n a solid, such as stomach contents or cells\n\nOn X-ray, we will see that as a whiter zone instead of the normal black zone of the lungs."},{"metadata":{"trusted":true},"cell_type":"code","source":"example(4, 8)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### ILD\n\n\nInterstitial lung disease includes more than 200 different conditions that cause inflammation and scarring around the balloon-like air sacs in the lungs, called the alveoli. Oxygen travels through the alveoli into your bloodstream. When they’re scarred, these sacs can’t expand as much. As a result, less oxygen enters the blood.\n\nBecause it's always easier to understand something with a figure :\n\n\n\nOther parts of your lungs can be affected too, such as the airways, lung lining, and blood vessels. \n\n\nThe four patterns of interstitial lung disease (ILD): linear, reticular, reticulonodular, and nodular opacities."},{"metadata":{"trusted":true},"cell_type":"code","source":"example(5, 8)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"#### Infiltration\n\n\nThis when an abnormal substance that accumulates gradually within cells or body tissues or any substance or type of cell that occurs within or spreads as through the interstices (interstitium and/or alveoli, we saw them above - reffer to the figure) of the lung, that is foreign to the lung, or that accumulates in greater than normal quantity within it.\n\n\n\nJust like the consolidation, we can see it on a X-ray because of a whiter zone instead of the normal black zone of the lungs. The problem is it's so similar to the consolidation that it became hard to distinguish them apart."},{"metadata":{"trusted":true},"cell_type":"code","source":"example(6, 8)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Lung Opacity\n\n\nFrom [this](https://www.kaggle.com/zahaviguy/what-are-lung-opacities) notebook : it's any area in the chest radiograph that is more white than it should be. the notebook explained very well the whole thing."},{"metadata":{"trusted":true},"cell_type":"code","source":"example(7, 8)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Nodule/Mass\n\nPulmonary nodules are small, rounded opacities within the pulmonary interstitium. The differential diagnosis for a nodule can be refined by its size, location, and density.\n\nSize : \n\n miliary nodules: <2 mm\n pulmonary micronodule: 2-7 mm\n pulmonary nodule: 7-30 mm\n pulmonary mass: >30 mm\n\nA pulmonary mass is any area of pulmonary opacification that measures more than 30 mm. The commonest cause of a pulmonary mass is lung cancer. [radiopaedia](https://radiopaedia.org/articles/pulmonary-mass?lang=us).\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"example(8, 8)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Other lesion\n\n\nAs we saw, the original dataset has more than 14 classes, I think that in this class we have the original \"other lesion\" class (with about 363 records according to the paper above) + all the classes that are missing here (ex. Rib fracture, clavicle fracture, edma, etc.) \n\nNote : I have to check that :) "},{"metadata":{"trusted":true},"cell_type":"code","source":"example(9, 8)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Pleural effusion\n\nPleural effusion, also called water on the lung, is an excessive buildup of fluid in the space between your lungs and chest cavity.\n\nThin membranes, called pleura, cover the outside of the lungs and the inside of the chest cavity. There’s always a small amount of liquid within this lining to help lubricate the lungs as they expand within the chest during breathing.\n\nThe pleura creates too much fluid when it’s irritated, inflamed, or infected. This fluid accumulates in the chest cavity outside the lung, causing what’s known as a pleural effusion.\n\nCertain types of cancer can cause pleural effusions, lung cancer in men and breast cancer in women being the most common. other causes can be pulmonary embolism and pneumonia for example.\n\n\n\n\nImaging criteria are (found [here](http://www.stritch.luc.edu/lumen/MedEd/Radio/curriculum/Medicine/Pleural_effusion1.htm), pretty good details !):\n\n Homogenous density\n Density in dependent portion\n Upright: Costophrenic angle in PA view\n Lateral view: Anterior and posterior portions of gutter\n Lateral decubitus position: Along sides\n Supine position: Along posteriorly, giving diffuse haziness on the side of effusion\n Silhouette of upper limit of density\n Upper margin high in axilla in PA view (yellow arrows)\n Upper margin high anteriorly and posteriorly in lateral view\n This is just an illusion\n Loss of silhouette. In the images below note lack of identifiable left diaphragm before and visible diaphragm after clearance of fluid (Silhouette sign principle)\n Mediastinal shift\n\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"example(10, 8)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Pleural thickening\n\nPleural thickening occurs when scar tissue develops on the lining of the lungs, or the pleura. Pleural thickening can develop following asbestos exposure or other conditions, such as infection.\n\nDepending on the cause, pleural thickening may form in different parts of the pleura.\n\n\n Visceral pleura: The membrane directly covering the lung tissue\n Parietal pleura: The outer membrane of the lung attached to the chest wall\n Pleural space: The space between the visceral and parietal pleura\n\n\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"example(11, 8)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Pneumothorax\n\nPneumothorax is the medical term for a collapsed lung. Pneumothorax occurs when air enters the space around your lungs (the pleural space). Air can find its way into the pleural space when there’s an open injury in your chest wall or a tear or rupture in your lung tissue, disrupting the pressure that keeps your lungs inflated.\n\nA pneumothorax typically demonstrate:\n\n - visible visceral pleural edge is seen as a very thin, sharp white line\n - no lung markings are seen peripheral to this line\n - peripheral space is radiolucent compared to the adjacent lung"},{"metadata":{"trusted":true},"cell_type":"code","source":"example(12, 8)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### Pulmonary fibrosis\n\nPulmonary fibrosis is a lung disease that occurs when lung tissue becomes damaged and scarred. This thickened, stiff tissue makes it more difficult for your lungs to work properly.\n\n\n\nThe most common radiographic changes to pulmonary fibrosis are an interstitial shadowing of small (1 to 2 mm), irregular opacities, which are seen in about 75% of patients. Less common are small, round opacities, which are seen in 20% of patients. This finding is generally known as reticulonodular opacities. [emedicine.medscape](https://emedicine.medscape.com)"},{"metadata":{"trusted":true},"cell_type":"code","source":"example(13, 8)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The last class is \"No finding\", so a normal CXR. Even if we already saw one example, here are some more."},{"metadata":{"trusted":true},"cell_type":"code","source":"example(14,5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"markdown","source":"Sources :\n\n* [VinDR - CXR](https://vindr.ai/datasets/cxr)\n* [VinDr-CXR: An open dataset of chest X-rays withradiologist’s annotations](https://arxiv.org/pdf/2012.15029.pdf)\n* [EBM Consult](https://www.ebmconsult.com/articles/radiology-chest-xray-normal)\n* [Healthline](https://www.healthline.com/human-body-maps/t4-fourth-thoracic-vertebrae#1)\n* [UMCVC](https://www.umcvc.org/conditions-treatments/aortic-aneurysm)\n* [MSRBlog](http://www.msrblog.com/science/medical/atelectasis.html)\n* [Projects Harvard](https://projects.iq.harvard.edu)\n* [NursingStandard](https://dm3omg1n1n7zx.cloudfront.net/rcni/static/journals/ns/28/41/ns.28.41.51.e8849/graphic/ns_v28_n41_8849_0001.jpg)\n* [Medicaldialogues](https://medicaldialogues.in/pulmonology/news/pirfenidone-found-safe-efficacious-in-idiopathic-pulmonary-fibrosis-65865)"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}