5. Fusion of boxings","metadata":{}},{"cell_type":"code","source":"train_dataframe.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def fusing_boxes(dataframe, class_id):\n # filter on class_id\n filtered_dataframe = dataframe.loc[dataframe.class_id == class_id,\n ['image_id','x_min','y_min','x_max','y_max']]\n # aggregate on image_id to average radiologists's estimations\n return filtered_dataframe.groupby(['image_id']).mean()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Plot deduplicated class counts\n\nindexes = train_dataframe.class_id.unique()\ncounts = [fusing_boxes(train_dataframe, class_id).shape[0] for class_id in range(14)]\n\nsorted_dict = dict(zip(indexes, counts))\nsorted_dict = {k: v for k, v in sorted(sorted_dict.items(), key=lambda item: item[1], reverse = True)}\n\nx = list(sorted_dict.keys())\ny = list(sorted_dict.values())\n\nplot_distribution_classes(x, y, \n title=\"Deduplicated distribution of radiographic observations\")","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"--------\n\n**
6. Presentation of each radiographic observations","metadata":{}},{"cell_type":"code","source":"def get_rectangle_parameter(dataframe, index):\n \n \"Adapt coordinates of bounding box for patch.Rectangle function\"\n \n x_min = dataframe.loc[index, 'x_min']\n y_min = dataframe.loc[index, 'y_min']\n x_max = dataframe.loc[index, 'x_max']\n y_max = dataframe.loc[index, 'y_max']\n \n anchor_point = (x_min, y_min)\n height = y_max - y_min\n width = x_max - x_min\n \n return anchor_point, height, width","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def select_9_from_each(dataframe):\n \n \"For each class, returns 9 indexes and image paths\"\n \n # Initialize dictionaries\n class_id_index_examples, class_id_image_examples = {}, {}\n image_ids_train_dataframe = list(train_dataframe.image_id)\n \n # Loop over different classes\n for class_id in range(14):\n fusing_boxes_dataframe = fusing_boxes(dataframe, class_id).sample(n=9)\n # image_id\n fusing_box_indexes = fusing_boxes_dataframe.index\n # Infer indexes\n class_id_index_examples[str(class_id)] = [image_ids_train_dataframe.index(fusing_box_indexes[cid]) for cid in range(9)]\n # Infer image paths\n class_id_image_examples[str(class_id)] = fusing_box_indexes\n \n return class_id_index_examples, class_id_image_examples\n\nclass_id_index_examples, class_id_image_examples = select_9_from_each(train_dataframe)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def display_images(class_id, graph_indexes = np.arange(9)):\n \n # Get files\n files_index = class_id_index_examples[str(class_id)]\n files_list = class_id_image_examples[str(class_id)]\n \n # define subplot\n fig, axs = plt.subplots(3,3, figsize=(12,12))\n for graph_index in graph_indexes:\n \n full_filename = files_list[graph_index]+'.dicom'\n ds = dcmread(os.path.join(PATH, \n 'train',\n full_filename))\n \n\n# axs[graph_index%3, (graph_index)//3].set_title('Label: %s \\n'%class_id,\n# fontsize=18)\n axs[graph_index%3, (graph_index)//3].imshow(ds.pixel_array, cmap=plt.get_cmap('gray'))\n \n if str(class_id) != '14':\n \n # Add rectangle\n anchor_point, height, width = get_rectangle_parameter(train_dataframe, \n files_index[graph_index])\n rect = Rectangle(anchor_point, \n height, \n width, \n edgecolor='r', \n facecolor=\"none\")\n axs[graph_index%3, (graph_index)//3].add_patch(rect)\n \n # the bottom of the subplots of the figure\n plt.subplots_adjust(bottom = 0.001)\n plt.subplots_adjust(top = 0.99)\n \n # show the figure\n plt.show()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I am not a doctor and cannot talk about each of these class fluently, hence I copy-pasted some accurate definitions from trusted sources. The main goal of this section is to share clear explanations regarding each class with plotted images to illustrate.","metadata":{}},{"cell_type":"markdown","source":"## Aortic enlargement","metadata":{}},{"cell_type":"markdown","source":"An aortic aneurysm is an enlargement (dilatation) of the aorta to greater than 1.5 times normal size. They usually cause no symptoms except when ruptured. Occasionally, there may be abdominal, back, or leg pain.\n\nThey are most commonly located in the abdominal aorta, but can also be located in the thoracic aorta. Aortic aneurysms cause weakness in the wall of the aorta and increase the risk of aortic rupture. When rupture occurs, massive internal bleeding results and, unless treated immediately, shock and death can occur.","metadata":{}},{"cell_type":"code","source":"display_images('0')","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Atelectasis","metadata":{}},{"cell_type":"markdown","source":"Atelectasis (at-uh-LEK-tuh-sis) is a complete or partial collapse of the entire lung or area (lobe) of the lung. It occurs when the tiny air sacs (alveoli) within the lung become deflated or possibly filled with alveolar fluid.\n\nAtelectasis is one of the most common breathing (respiratory) complications after surgery. It's also a possible complication of other respiratory problems, including cystic fibrosis, lung tumors, chest injuries, fluid in the lung and respiratory weakness. You may develop atelectasis if you breathe in a foreign object.\n\nAtelectasis can make breathing difficult, particularly if you already have lung disease. Treatment depends on the cause and severity of the collapse.","metadata":{}},{"cell_type":"code","source":"display_images('1')","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Calcification","metadata":{}},{"cell_type":"markdown","source":"Calcification is the accumulation of calcium salts in a body tissue. It normally occurs in the formation of bone, but calcium can be deposited abnormally in soft tissue, causing it to harden. Calcifications may be classified on whether there is mineral balance or not, and the location of the calcification.","metadata":{}},{"cell_type":"code","source":"display_images('2')","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Cardiomegaly","metadata":{}},{"cell_type":"markdown","source":"Cardiomegaly is an enlarged heart. It is not a disease, but a sign of another condition. Less severe forms of cardiomegaly are referred to as mild cardiomegaly. As mild cardiomegaly does not always cause symptoms, many people with a slightly enlarged heart are unaware of the problem.","metadata":{}},{"cell_type":"code","source":"display_images('3')","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Consolidation","metadata":{}},{"cell_type":"markdown","source":"A pulmonary consolidation is a region of normally compressible lung tissue that has filled with liquid instead of air. The condition is marked by induration (swelling or hardening of normally soft tissue) of a normally aerated lung. It is considered a radiologic sign.","metadata":{}},{"cell_type":"code","source":"display_images('4')","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## ILD","metadata":{}},{"cell_type":"markdown","source":"Interstitial lung disease (ILD) is a group of many lung conditions. All interstitial lung diseases affect the interstitium, a part of your lungs. The interstitium is a lace-like network of tissue that goes throughout both lungs. It supports your lungs' tiny air sacs, called alveoli.","metadata":{}},{"cell_type":"code","source":"display_images('5')","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Infiltration","metadata":{}},{"cell_type":"markdown","source":"As part of a disease process, infiltration is sometimes used to define the invasion of cancer cells into the underlying matrix or the blood vessels. Similarly, the term may describe the deposition of amyloid protein. During leukocyte extravasation, white blood cells move in response to cytokines from within the blood, into the diseased or infected tissues, usually in the same direction as a chemical gradient,[1] in a process called chemotaxis. The presence of lymphocytes in tissue in greater than normal numbers is likewise called infiltration.\n\nAs part of medical intervention, local anaesthetics may be injected at more than one point so as to infiltrate an area prior to a surgical procedure. However, the term may also apply to unintended iatrogenic leakage of fluids from phlebotomy or intravenous drug delivery procedures, a process also known as extravasation or \"tissuing\".","metadata":{}},{"cell_type":"code","source":"display_images('6')","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Lung Opacity","metadata":{}},{"cell_type":"markdown","source":"Pulmonary opacification represents the result of a decrease in the ratio of gas to soft tissue (blood, lung parenchyma and stroma) in the lung. When reviewing an area of increased attenuation (opacification) on a chest radiograph or CT it is vital to determine where the opacification is. The patterns can broadly be divided into airspace opacification, lines and dots.","metadata":{}},{"cell_type":"code","source":"display_images('7')","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Nodule/Mass","metadata":{}},{"cell_type":"markdown","source":"A lung nodule or pulmonary nodule is a relatively small focal density in the lung. A solitary pulmonary nodule (SPN) or coin lesion, is a mass in the lung smaller than 3 centimeters in diameter. There may also be multiple nodules.\n\nOne or more lung nodules can be an incidental finding found in up to 0.2% of chest X-rays and around 1% of CT scans.\n\nThe nodule most commonly represents a benign tumor such as a granuloma or hamartoma, but in around 20% of cases it represents a malignant cancer, especially in older adults and smokers. Conversely, 10 to 20% of patients with lung cancer are diagnosed in this way. If the patient has a history of smoking or the nodule is growing, the possibility of cancer may need to be excluded through further radiological studies and interventions, possibly including surgical resection. The prognosis depends on the underlying condition.","metadata":{}},{"cell_type":"code","source":"display_images('8')","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Other lesion","metadata":{}},{"cell_type":"code","source":"display_images('9')","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Pleural effusion","metadata":{}},{"cell_type":"markdown","source":"Pleural effusion, sometimes referred to as “water on the lungs,” is the build-up of excess fluid between the layers of the pleura outside the lungs. The pleura are thin membranes that line the lungs and the inside of the chest cavity and act to lubricate and facilitate breathing. Normally, a small amount of fluid is present in the pleura.","metadata":{}},{"cell_type":"code","source":"display_images('10')","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Pleural thickening","metadata":{}},{"cell_type":"markdown","source":"What Is Pleural Thickening? Pleural thickening develops when scar tissue thickens the delicate membrane lining the lungs (the pleura). Pleural thickening can develop following asbestos exposure or other conditions, such as infection. It may be a symptom of a more severe diagnosis such as malignant pleural mesothelioma.","metadata":{}},{"cell_type":"code","source":"display_images('11')","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Pneumothorax","metadata":{}},{"cell_type":"markdown","source":"A pneumothorax (noo-moe-THOR-aks) is a collapsed lung. A pneumothorax occurs when air leaks into the space between your lung and chest wall. This air pushes on the outside of your lung and makes it collapse. Pneumothorax can be a complete lung collapse or a collapse of only a portion of the lung.","metadata":{}},{"cell_type":"code","source":"display_images('12')","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Pulmonary fibrosis","metadata":{}},{"cell_type":"markdown","source":"Pulmonary fibrosis is a lung disease that occurs when lung tissue becomes damaged and scarred. This thickened, stiff tissue makes it more difficult for your lungs to work properly. As pulmonary fibrosis worsens, you become progressively more short of breath.\n\nThe scarring associated with pulmonary fibrosis can be caused by a multitude of factors. But in most cases, doctors can't pinpoint what's causing the problem. When a cause can't be found, the condition is termed idiopathic pulmonary fibrosis.\n\nThe lung damage caused by pulmonary fibrosis can't be repaired, but medications and therapies can sometimes help ease symptoms and improve quality of life. For some people, a lung transplant might be appropriate.","metadata":{}},{"cell_type":"code","source":"display_images('13')","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"--------\n\n**
Back to summary**","metadata":{}},{"cell_type":"markdown","source":"# References\n\n#### DICOM files\n*
Deal with .dicom files\n*
Matching DICOM metadata\n\n#### Abnormality classes\n*
Aortic aneurysm - Wikipedia\n*
Atelectasis - Mayoclinic\n*
Calcification - Wikipedia\n*
Cardiomegaly - Medicalnewstoday\n*
Pulmonary consolidation - Wikipedia\n*
ILD - Webmd\n*
Infiltration - Wikipedia\n*
Lung Opacity - Radiopaedia\n*
Nodule - Wikipedia\n*
Pleural effusion - Clevelandclinic\n*
Pleural thickening - Mesothelioma\n*
Pneumothorax - Mayoclinic\n*
Pulmonary Fibrosis - Mayoclinic","metadata":{}},{"cell_type":"markdown","source":"
\n
Thank you for taking the time to read this notebook. I hope that I was able to answer your questions or your curiosity and that it was quite understandable. any constructive comments are welcome. They help me progress and motivate me to share better quality content. I am above all a passionate person who tries to advance my knowledge but also that of others. If you liked it, feel free to upvote and share my work.
\n
\n
Thank you and may passion guide you.
","metadata":{}}]}