{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.7.9","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"none","dataSources":[{"sourceType":"competition","sourceId":24800,"datasetId":1042002,"databundleVersionId":1831594},{"sourceType":"datasetVersion","sourceId":2090095,"datasetId":1253200,"databundleVersionId":2130530},{"sourceType":"datasetVersion","sourceId":1832961,"datasetId":1089494,"databundleVersionId":1870583},{"sourceType":"datasetVersion","sourceId":1810939,"datasetId":1075804,"databundleVersionId":1848423},{"sourceType":"datasetVersion","sourceId":1836030,"datasetId":1091476,"databundleVersionId":1873670},{"sourceType":"datasetVersion","sourceId":1913589,"datasetId":1140992,"databundleVersionId":1952040}],"dockerImageVersionId":30086,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# VinBigData Chest X-ray Abnormalities Classifier","metadata":{"papermill":{"duration":0.036932,"end_time":"2021-02-17T21:45:57.432627","exception":false,"start_time":"2021-02-17T21:45:57.395695","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"# Table of Contents\n\n1. Introduction\n2. Objectives\n3. Data\n4. Dataset Preparation\n5. Installation\n6. Exploratory Data Analysis\n7. Methods\n8. Image Visualizaion and Augmentation with Albumentations\n9. Model Building\n10. Model Training\n11. Model Testing and Validation\n12. Results\n13. Discussion\n14. Furture Improvement\n15. References\n16. Appendices","metadata":{"papermill":{"duration":0.034918,"end_time":"2021-02-17T21:45:57.502879","exception":false,"start_time":"2021-02-17T21:45:57.467961","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"**1. Introduction**\n\nWhen you have a broken arm, radiologists help save the day—and the bone. These doctors diagnose and treat medical conditions using imaging techniques like CT and PET scans, MRIs, and, of course, X-rays. Yet, as it happens when working with such a wide variety of medical tools, radiologists face many daily challenges, perhaps the most difficult being the chest radiograph. **The interpretation of chest X-rays can lead to medical misdiagnosis**, even for the best practicing doctor. Computer-aided detection and diagnosis systems (CADe/CADx) would help reduce the pressure on doctors at metropolitan hospitals and improve diagnostic quality in rural areas.\n\nThe annotations were collected via VinBigData's web-based platform, VinLab. Details on building the dataset can be found in the organizer's recent paper “VinDr-CXR: An open dataset of chest X-rays with radiologist's annotations”.und in the organizer's recent paper “VinDr-CXR: An open dataset of chest X-rays with radiologist's annotations”.","metadata":{"papermill":{"duration":0.036099,"end_time":"2021-02-17T21:45:57.573899","exception":false,"start_time":"2021-02-17T21:45:57.5378","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"**2. Objective**\n\nExisting methods of interpreting chest X-ray images classify them into a list of findings. There is currently no specification of their locations on the image which sometimes leads to inexplicable results. A solution for localizing findings on chest X-ray images is needed for providing doctors with more meaningful diagnostic assistance\n\nIn this competition, we are classifying common thoracic lung diseases and localizing critical findings. This is an **object detection and classification** problem from chest x-ray image (radiographs)","metadata":{}},{"cell_type":"markdown","source":"**3. Data**\n**3.1 Intro**\n\nIn this competition:\n\n* Task: Automatically localize and classify **14 types of thoracic abnormalities** from chest radiographs.\n* Dataset: Consisting of **18,000 scans: 15,000 train images** and will be evaluated on a test set of **3,000 images**.\n\nFor each test image, you will be predicting a bounding box and class for all findings. If you predict that there are no findings, you should create a prediction of \"14 1 0 0 1 1\" (14 is the class ID for no finding, and this provides a one-pixel bounding box with a confidence of 1.0).\n\nThe images are in DICOM format, which means they contain additional data that might be useful for visualizing and classifying","metadata":{}},{"cell_type":"markdown","source":"**3.2 Dataset information**\n\nThe dataset comprises 18,000 postero-anterior (PA) CXR scans in DICOM format, which were de-identified to protect patient privacy. All images were labeled by a panel of experienced radiologists for the presence of 14 critical radiographic findings as listed below:\n\n\n\nThe \"No finding\" observation (14) was intended to capture the absence of all findings above.","metadata":{}},{"cell_type":"markdown","source":"**4. Data Preparation**\n\nPreprocessing x-ray image format (dicom) into normal png image format is already done by @xhlulu in the below discussion:\n - [Multiple preprocessed datasets: 256/512/1024px, PNG and JPG, modified and original ratio](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/207955).\n\nSo I will just use the dataset \n- [VinBigData Chest X-ray Resized PNG (256x256)](https://www.kaggle.com/xhlulu/vinbigdata-chest-xray-resized-png-256x256) to skip the preprocessing and focus on modeling part. \n\nPlease upvote the dataset as well!","metadata":{}},{"cell_type":"markdown","source":"**6. Methods (Implementation)**\n\nLoading Libraries","metadata":{}},{"cell_type":"code","source":"# Loading Basic Libraries \nimport os\nimport gc   # No Version\nfrom pathlib import Path\n\nimport random\nimport six\nimport sys\nimport warnings\n\nimport dataclasses\nfrom dataclasses import dataclass, field\nfrom typing import Dict, Any, Tuple, Union, List, Optional, Mapping\nfrom tqdm import tqdm\nfrom tqdm.notebook import tqdm\n\nimport yaml\nimport pickle","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:32:48.930243Z","iopub.execute_input":"2024-12-29T23:32:48.930557Z","iopub.status.idle":"2024-12-29T23:32:48.961077Z","shell.execute_reply.started":"2024-12-29T23:32:48.93048Z","shell.execute_reply":"2024-12-29T23:32:48.959833Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Loading Data Processing Libraries \nimport numpy as np\nimport pandas as pd\n\nfrom logging import getLogger","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:32:48.963485Z","iopub.execute_input":"2024-12-29T23:32:48.963793Z","iopub.status.idle":"2024-12-29T23:32:48.96807Z","shell.execute_reply.started":"2024-12-29T23:32:48.963761Z","shell.execute_reply":"2024-12-29T23:32:48.967097Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Loading Visualization Libraries \nfrom IPython.core.display import display, HTML\nimport matplotlib.pyplot as plt\nimport matplotlib.patches as ptc\nimport seaborn as sns\n\nfrom plotly import tools, subplots\nimport plotly.offline as py\n\nimport plotly.graph_objs as go\nimport plotly.express as px\nimport plotly.figure_factory as ff\nimport plotly.io as pio","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:32:48.970935Z","iopub.execute_input":"2024-12-29T23:32:48.971397Z","iopub.status.idle":"2024-12-29T23:32:52.836464Z","shell.execute_reply.started":"2024-12-29T23:32:48.971365Z","shell.execute_reply":"2024-12-29T23:32:52.83539Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Loading Imaging Libraries \nimport pydicom as pdc\nfrom pydicom.pixel_data_handlers.util import apply_voi_lut\nimport cv2\nimport albumentations as A","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:32:52.837972Z","iopub.execute_input":"2024-12-29T23:32:52.838305Z","iopub.status.idle":"2024-12-29T23:32:53.849345Z","shell.execute_reply.started":"2024-12-29T23:32:52.838276Z","shell.execute_reply":"2024-12-29T23:32:53.847444Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Loading Statistics Libraries \nimport scipy as sp","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:32:53.851141Z","iopub.execute_input":"2024-12-29T23:32:53.851617Z","iopub.status.idle":"2024-12-29T23:32:53.855624Z","shell.execute_reply.started":"2024-12-29T23:32:53.851569Z","shell.execute_reply":"2024-12-29T23:32:53.85457Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Loading Models Libraries \nfrom sklearn import preprocessing\nfrom sklearn.model_selection import StratifiedKFold, KFold\nimport lightgbm as lgb\nimport xgboost as xgb\nimport catboost as cb\n\nimport torch\nimport torch.nn.functional as F\nfrom torch import nn\nfrom torch.nn import Linear\nfrom torch import nn, optim\nfrom torch.utils.data.dataloader import DataLoader\nfrom torch.utils.data.dataset import Dataset\n\nfrom ignite.engine import Events, Engine","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:32:53.857282Z","iopub.execute_input":"2024-12-29T23:32:53.858047Z","iopub.status.idle":"2024-12-29T23:32:55.850349Z","shell.execute_reply.started":"2024-12-29T23:32:53.857991Z","shell.execute_reply":"2024-12-29T23:32:55.848952Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"from pydicom import dcmread\n\nds = dcmread(\"/kaggle/input/vinbigdata-chest-xray-abnormalities-detection/test/002a34c58c5b758217ed1f584ccbcfe9.dicom\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:32:55.8516Z","iopub.execute_input":"2024-12-29T23:32:55.851961Z","iopub.status.idle":"2024-12-29T23:32:55.957172Z","shell.execute_reply.started":"2024-12-29T23:32:55.851922Z","shell.execute_reply":"2024-12-29T23:32:55.956022Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"arr = ds.pixel_array\n\nplt.imshow(arr, cmap=\"gray\")\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:32:55.958646Z","iopub.execute_input":"2024-12-29T23:32:55.958983Z","iopub.status.idle":"2024-12-29T23:32:59.422383Z","shell.execute_reply.started":"2024-12-29T23:32:55.958951Z","shell.execute_reply":"2024-12-29T23:32:59.420931Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"#print(np.__version__)\n#print(sys.version_info)\n#print(Path.__version__)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:32:59.423916Z","iopub.execute_input":"2024-12-29T23:32:59.424209Z","iopub.status.idle":"2024-12-29T23:32:59.428281Z","shell.execute_reply.started":"2024-12-29T23:32:59.424181Z","shell.execute_reply":"2024-12-29T23:32:59.42703Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"#import types\n#def imports():\n#    for name, val in globals().items():\n#        if isinstance(val, types.ModuleType):\n#            yield val.__name__\n#list(imports())","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:32:59.429783Z","iopub.execute_input":"2024-12-29T23:32:59.430117Z","iopub.status.idle":"2024-12-29T23:32:59.446011Z","shell.execute_reply.started":"2024-12-29T23:32:59.430088Z","shell.execute_reply":"2024-12-29T23:32:59.444932Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**5. Installation**\n\nDetectron2 is not pre-installed in this kaggle docker, so we need to install it in here. \n\nWe need to know CUDA and pytorch version to install correct `detectron2`.","metadata":{"papermill":{"duration":0.03611,"end_time":"2021-02-17T21:46:06.850146","exception":false,"start_time":"2021-02-17T21:46:06.814036","status":"completed"},"tags":[]}},{"cell_type":"code","source":"!pip install detectron2 -f \\\n  https://dl.fbaipublicfiles.com/detectron2/wheels/cu102/torch1.7/index.html\n!pip install pytorch-pfn-extras timm","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"papermill":{"duration":29.671583,"end_time":"2021-02-17T21:46:36.558082","exception":false,"start_time":"2021-02-17T21:46:06.886499","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:32:59.447389Z","iopub.execute_input":"2024-12-29T23:32:59.44769Z","iopub.status.idle":"2024-12-29T23:33:41.903049Z","shell.execute_reply.started":"2024-12-29T23:32:59.447656Z","shell.execute_reply":"2024-12-29T23:33:41.900767Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import timm\nfrom detectron2.structures import BoxMode","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Loading Loibraries after Installation\nimport pytorch_pfn_extras as ppe\nimport pytorch_pfn_extras.training.extensions as E\nfrom pytorch_pfn_extras.training import IgniteExtensionsManager\nfrom pytorch_pfn_extras.training.extension import Extension, PRIORITY_READER\nfrom pytorch_pfn_extras.training.manager import ExtensionsManager\n\nimport timm\nfrom detectron2.structures import BoxMode","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# To get python package version within jupyter notebook\nimport pkg_resources\nimport types\n\ndef get_imports():\n    for name, val in globals().items():\n        if isinstance(val, types.ModuleType):\n            # Split ensures you get root package, \n            # not just imported function\n            name = val.__name__.split(\".\")[0]\n\n        elif isinstance(val, type):\n            name = val.__module__.split(\".\")[0]\n\n        # Some packages are weird and have different\n        # imported names vs. system/pip names. Unfortunately,\n        # there is no systematic way to get pip names from\n        # a package's imported name. You'll have to add\n        # exceptions to this list manually!\n        poorly_named_packages = {\n            \"PIL\": \"Pillow\",\n            \"sklearn\": \"scikit-learn\"\n        }\n        if name in poorly_named_packages.keys():\n            name = poorly_named_packages[name]\n\n        yield name\nimports = list(set(get_imports()))\n\n# The only way I found to get the version of the root package\n# from only the name of the package is to cross-check the names \n# of installed packages vs. imported packages\nrequirements = []\nfor m in pkg_resources.working_set:\n    if m.project_name in imports and m.project_name!=\"pip\":\n        requirements.append((m.project_name, m.version))\n\nfor r in requirements:\n    print(\"{}=={}\".format(*r))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:34:36.94775Z","iopub.execute_input":"2024-12-29T23:34:36.948112Z","iopub.status.idle":"2024-12-29T23:34:36.960718Z","shell.execute_reply.started":"2024-12-29T23:34:36.948083Z","shell.execute_reply":"2024-12-29T23:34:36.959451Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Initialization of Configurations \nwarnings.filterwarnings(\"ignore\")\npd.set_option('max_columns', 50)\npy.init_notebook_mode(connected=True)\npio.templates.default = \"plotly_dark\"\n%matplotlib inline","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:34:41.407916Z","iopub.execute_input":"2024-12-29T23:34:41.408307Z","iopub.status.idle":"2024-12-29T23:34:42.088249Z","shell.execute_reply.started":"2024-12-29T23:34:41.408269Z","shell.execute_reply":"2024-12-29T23:34:42.087179Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"train_dir = \"../input/vinbigdata-chest-xray-abnormalities-detection/train\"\ntest_dir = \"../input/vinbigdata-chest-xray-abnormalities-detection/test\"\n\ntrain_files = os.listdir(train_dir)\ntest_files = os.listdir(test_dir)\n\ntrain_df = pd.read_csv(\"../input/vinbigdata-chest-xray-abnormalities-detection/train.csv\")\nsample_submission = pd.read_csv(\"../input/vinbigdata-chest-xray-abnormalities-detection/sample_submission.csv\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:34:45.247014Z","iopub.execute_input":"2024-12-29T23:34:45.247538Z","iopub.status.idle":"2024-12-29T23:34:47.480856Z","shell.execute_reply.started":"2024-12-29T23:34:45.247491Z","shell.execute_reply":"2024-12-29T23:34:47.479414Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**7. Exploratory Data Analysis**\n\n**7.1 Exploration of Train and Test Data**\n\nThe train set metadata, with one row for each object, including a class and a bounding box. \nSome images in both test and train have multiple objects.\n\nColumns\nimage_id - unique image identifier\nclass_name - the name of the class of detected object (or \"No finding\")\nclass_id - the ID of the class of detected object\nrad_id - the ID of the radiologist that made the observation\nx_min - minimum X coordinate of the object's bounding box\ny_min - minimum Y coordinate of the object's bounding box\nx_max - maximum X coordinate of the object's bounding box\ny_max - maximum Y coordinate of the object's bounding box","metadata":{"papermill":{"duration":0.052405,"end_time":"2021-02-17T21:46:36.664525","exception":false,"start_time":"2021-02-17T21:46:36.61212","status":"completed"},"tags":[]}},{"cell_type":"code","source":"train_df","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:34:56.729527Z","iopub.execute_input":"2024-12-29T23:34:56.729865Z","iopub.status.idle":"2024-12-29T23:34:56.770043Z","shell.execute_reply.started":"2024-12-29T23:34:56.729835Z","shell.execute_reply":"2024-12-29T23:34:56.769019Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"train_df.isna().sum().to_frame().rename(columns={0:\"Nan_counts\"}).style.background_gradient(cmap=\"cool\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:35:10.092833Z","iopub.execute_input":"2024-12-29T23:35:10.093335Z","iopub.status.idle":"2024-12-29T23:35:10.125354Z","shell.execute_reply.started":"2024-12-29T23:35:10.093213Z","shell.execute_reply":"2024-12-29T23:35:10.124274Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"train_df.nunique().to_frame().rename(columns={0:\"Unique Values\"}).style.background_gradient(cmap=\"plasma\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:35:10.127502Z","iopub.execute_input":"2024-12-29T23:35:10.127801Z","iopub.status.idle":"2024-12-29T23:35:10.190835Z","shell.execute_reply.started":"2024-12-29T23:35:10.127774Z","shell.execute_reply":"2024-12-29T23:35:10.189804Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"plt.figure(figsize=(26, 8))\nsns.countplot(x=\"class_name\", data=train_df)\nplt.title(\"Class Name Distribution\")\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:35:10.192471Z","iopub.execute_input":"2024-12-29T23:35:10.192953Z","iopub.status.idle":"2024-12-29T23:35:10.496849Z","shell.execute_reply.started":"2024-12-29T23:35:10.19288Z","shell.execute_reply":"2024-12-29T23:35:10.49561Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"plt.figure(figsize=(8, 8))\nsns.countplot(x=\"class_id\", data=train_df)\nplt.title(\"Class ID Distribution\")\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:35:10.49907Z","iopub.execute_input":"2024-12-29T23:35:10.499515Z","iopub.status.idle":"2024-12-29T23:35:10.703617Z","shell.execute_reply.started":"2024-12-29T23:35:10.499469Z","shell.execute_reply":"2024-12-29T23:35:10.702489Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"plt.figure(figsize=(8, 8))\nsns.countplot(x=\"rad_id\", data=train_df)\nplt.title(\"RAD ID Distribution\")\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:35:10.70785Z","iopub.execute_input":"2024-12-29T23:35:10.708375Z","iopub.status.idle":"2024-12-29T23:35:10.982398Z","shell.execute_reply.started":"2024-12-29T23:35:10.708323Z","shell.execute_reply":"2024-12-29T23:35:10.98102Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"plt.figure(figsize=(10, 10))\nsns.pairplot(train_df, hue='class_id')\nplt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:36:53.987093Z","iopub.execute_input":"2024-12-29T23:36:53.987447Z","iopub.status.idle":"2024-12-29T23:37:22.772633Z","shell.execute_reply.started":"2024-12-29T23:36:53.987415Z","shell.execute_reply":"2024-12-29T23:37:22.771463Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print(\"Number of train images: \", len(train_files))\nprint(\"Number of test images: \", len(test_files))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:37:34.374125Z","iopub.execute_input":"2024-12-29T23:37:34.374607Z","iopub.status.idle":"2024-12-29T23:37:34.381109Z","shell.execute_reply.started":"2024-12-29T23:37:34.374562Z","shell.execute_reply":"2024-12-29T23:37:34.379842Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"We have 15000 images for Train and 3000 images for Test. But, the length of train_df is 67914. So, are there any duplicacy or something? Let's have a sanity check on unique image_ids of train_df.","metadata":{}},{"cell_type":"code","source":"print(\"Unique images in train_df: \", train_df.image_id.nunique())","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:37:34.384075Z","iopub.execute_input":"2024-12-29T23:37:34.384593Z","iopub.status.idle":"2024-12-29T23:37:34.422185Z","shell.execute_reply.started":"2024-12-29T23:37:34.384548Z","shell.execute_reply":"2024-12-29T23:37:34.42102Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"It's confirmed that we have one or more than one ground truths in our train dataset. Now, we got a lot to explore, such as:\n\nWhether the boxes are overlapped on a single image? What about different classes? Are they over-lapped too for different classes? In a single image, what's the maximum number of class (or taregt present)? etc.\n\nWe will continue to explore these aspects and try to come with some insights, but let's worry about DICOM at the first place. Let's read out some image file names from both the train and test folders.","metadata":{}},{"cell_type":"code","source":"for _ in range(3):\n    print(train_files[random.randint(0, len(train_files))])","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:37:34.423834Z","iopub.execute_input":"2024-12-29T23:37:34.424269Z","iopub.status.idle":"2024-12-29T23:37:34.430057Z","shell.execute_reply.started":"2024-12-29T23:37:34.424224Z","shell.execute_reply":"2024-12-29T23:37:34.429122Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**7.2 Exploration of DICOM**\n\nDICOM files can be exchanged between two entities that are capable of receiving image and patient data in DICOM format.\n\nThe different devices come with DICOM Conformance Statements which state which DICOM classes they support. \nThe standard includes a file format definition and a network communications protocol that uses TCP/IP to communicate between systems.\n\nLet's read one sample and grind it.","metadata":{}},{"cell_type":"code","source":"sample_fn = train_df.image_id.to_list()[random.randint(0, len(train_df))]\nsample = pdc.read_file(os.path.join(train_dir, sample_fn + \".dicom\"))\nsample","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:37:34.43131Z","iopub.execute_input":"2024-12-29T23:37:34.431614Z","iopub.status.idle":"2024-12-29T23:37:34.550249Z","shell.execute_reply.started":"2024-12-29T23:37:34.431586Z","shell.execute_reply":"2024-12-29T23:37:34.549076Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"So, for a single patient(yes, there is one-one mapping in patients and image_ids, so there is no duplicates or more than one images per patient, as confirmed by the organizers in the discussion forum), we have following entities:\n\nPatient's Sex Samples per Pixel Photometric Interpretation Rows Columns Pixel Spacing Bits Allocated Bits Stored High Bit Pixel Representation Window Center Window Width Rescale Intercept Rescale Slope Lossy Image Compression Pixel Data","metadata":{}},{"cell_type":"code","source":"def read_dicom_df(fn):\n    _ = pdc.read_file(os.path.join(train_dir, fn))\n    pass","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:37:34.554376Z","iopub.execute_input":"2024-12-29T23:37:34.554761Z","iopub.status.idle":"2024-12-29T23:37:34.559263Z","shell.execute_reply.started":"2024-12-29T23:37:34.554726Z","shell.execute_reply":"2024-12-29T23:37:34.55792Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def plot_pixel_array(data, figsize=(10,10)):\n    plt.figure(figsize=figsize)\n    plt.imshow(data, cmap=plt.cm.bone)\n    plt.show()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:37:34.562812Z","iopub.execute_input":"2024-12-29T23:37:34.563153Z","iopub.status.idle":"2024-12-29T23:37:34.575149Z","shell.execute_reply.started":"2024-12-29T23:37:34.563125Z","shell.execute_reply":"2024-12-29T23:37:34.573937Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# ref kernel: https://www.kaggle.com/raddar/convert-dicom-to-np-array-the-correct-way\ndef read_xray(path, voi_lut=True, fix_monochrome=True):\n    dcm_data = pdc.read_file(path)\n    \n    def show_dcm_info(data):\n        print(\"Gender :\", data.PatientSex)\n        if 'PixelData' in data:\n            rows = int(data.Rows)\n            cols = int(data.Columns)\n            print(\"Image size : {rows:d} x {cols:d}, {size:d} bytes\".format(\n                rows=rows, cols=cols, size=len(data.PixelData)))\n            if 'PixelSpacing' in data:\n                print(\"Pixel spacing :\", data.PixelSpacing)\n    \n    show_dcm_info(dcm_data)\n    # VOI LUT (if available by DICOM device) is used to transform raw DICOM data to \"human-friendly\" view\n    if voi_lut:\n        data = apply_voi_lut(dcm_data.pixel_array, dcm_data)\n    else:\n        data = dcm_data.pixel_array\n               \n    # depending on this value, X-ray may look inverted - fix that:\n    if fix_monochrome and dcm_data.PhotometricInterpretation == \"MONOCHROME1\":\n        data = np.amax(data) - data\n        \n    data = data - np.min(data)\n    data = data / np.max(data)\n    data = (data * 255).astype(np.uint8)\n        \n    return data","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:37:34.577015Z","iopub.execute_input":"2024-12-29T23:37:34.577443Z","iopub.status.idle":"2024-12-29T23:37:34.589778Z","shell.execute_reply.started":"2024-12-29T23:37:34.577402Z","shell.execute_reply":"2024-12-29T23:37:34.588906Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"7.2.1 Visualizing Train and Test Images","metadata":{}},{"cell_type":"code","source":"print(\"Examining train images...\")\nfor _ in range(5):\n    fn = train_files[np.random.randint(0, len(train_files))]\n    file_path = os.path.join(train_dir, fn)\n    data = read_xray(file_path)\n    plot_pixel_array(data)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:37:34.591466Z","iopub.execute_input":"2024-12-29T23:37:34.591878Z","iopub.status.idle":"2024-12-29T23:37:48.462132Z","shell.execute_reply.started":"2024-12-29T23:37:34.591834Z","shell.execute_reply":"2024-12-29T23:37:48.460918Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print(\"Examining test images...\")\nfor _ in range(5):\n    fn = test_files[np.random.randint(0, len(test_files))]\n    file_path = os.path.join(test_dir, fn)\n    data = read_xray(file_path)\n    plot_pixel_array(data)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:37:48.463823Z","iopub.execute_input":"2024-12-29T23:37:48.46444Z","iopub.status.idle":"2024-12-29T23:38:00.78298Z","shell.execute_reply.started":"2024-12-29T23:37:48.464395Z","shell.execute_reply":"2024-12-29T23:38:00.781684Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"7.2.2 Checking the Localizations","metadata":{}},{"cell_type":"code","source":"for _ in range(10):\n    idx = np.random.randint(0, len(train_files))\n    img_id = train_df.loc[idx, 'image_id']\n    img = read_xray(os.path.join(train_dir, img_id+\".dicom\"))\n    plt.figure(figsize=(8, 14))\n    plt.imshow(img, cmap='gray')\n    plt.title(train_df.loc[idx, 'class_name'])\n    \n    if train_df.loc[idx, 'class_name'] != 'No finding':\n        bbox = [train_df.loc[idx, 'x_min'],\n                train_df.loc[idx, 'y_min'],\n                train_df.loc[idx, 'x_max'],\n                train_df.loc[idx, 'y_max']]\n        \n        patch = ptc.Rectangle((bbox[0], bbox[1]),\n                              bbox[2]-bbox[0],\n                              bbox[3]-bbox[1],\n                              ec='r', fc='none', lw=2.)\n        ax = plt.gca()\n        ax.add_patch(patch)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:38:00.784388Z","iopub.execute_input":"2024-12-29T23:38:00.78469Z","iopub.status.idle":"2024-12-29T23:38:18.458684Z","shell.execute_reply.started":"2024-12-29T23:38:00.78466Z","shell.execute_reply":"2024-12-29T23:38:18.457673Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def save_yaml(filepath: str, content: Any, width: int = 120):\n    with open(filepath, \"w\") as f:\n        yaml.dump(content, f, width=width)","metadata":{"_kg_hide-input":true,"papermill":{"duration":0.061176,"end_time":"2021-02-17T21:46:36.779472","exception":false,"start_time":"2021-02-17T21:46:36.718296","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:38:18.460127Z","iopub.execute_input":"2024-12-29T23:38:18.460426Z","iopub.status.idle":"2024-12-29T23:38:18.466146Z","shell.execute_reply.started":"2024-12-29T23:38:18.460398Z","shell.execute_reply":"2024-12-29T23:38:18.464956Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Flags Class summarizes all the configuratoin available during the training. \n\nThe users can change various hyperparameters to experiment improving your models","metadata":{}},{"cell_type":"code","source":"@dataclass\nclass Flags:\n    # General\n    debug: bool = True\n    outdir: str = \"results/det\"\n    device: str = \"cuda:0\"\n\n    # Data config\n    imgdir_name: str = \"vinbigdata-chest-xray-resized-png-256x256\"\n    # split_mode: str = \"all_train\"  # all_train or valid20\n    seed: int = 111\n    target_fold: int = 0  # 0~4\n    label_smoothing: float = 0.0\n    # Model config\n    model_name: str = \"resnet18\"\n    model_mode: str = \"normal\"  # normal, cnn_fixed supported\n    # Training config\n    epoch: int = 20\n    batchsize: int = 8\n    valid_batchsize: int = 16\n    num_workers: int = 4\n    snapshot_freq: int = 5\n    ema_decay: float = 0.999  # negative value is to inactivate ema.\n    scheduler_type: str = \"\"\n    scheduler_kwargs: Dict[str, Any] = field(default_factory=lambda: {})\n    scheduler_trigger: List[Union[int, str]] = field(default_factory=lambda: [1, \"iteration\"])\n    aug_kwargs: Dict[str, Dict[str, Any]] = field(default_factory=lambda: {})\n    mixup_prob: float = -1.0  # Apply mixup augmentation when positive value is set.\n\n    def update(self, param_dict: Dict) -> \"Flags\":\n        # Overwrite by `param_dict`\n        for key, value in param_dict.items():\n            if not hasattr(self, key):\n                raise ValueError(f\"[ERROR] Unexpected key for flag = {key}\")\n            setattr(self, key, value)\n        return self","metadata":{"_kg_hide-input":true,"papermill":{"duration":0.069686,"end_time":"2021-02-17T21:46:36.902284","exception":false,"start_time":"2021-02-17T21:46:36.832598","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:38:18.468283Z","iopub.execute_input":"2024-12-29T23:38:18.468768Z","iopub.status.idle":"2024-12-29T23:38:18.486558Z","shell.execute_reply.started":"2024-12-29T23:38:18.468719Z","shell.execute_reply":"2024-12-29T23:38:18.485346Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"flags_dict = {\n    \"debug\": False,  # Change to True for fast debug run!\n    \"outdir\": \"results/tmp_debug\",\n    # Data\n    \"imgdir_name\": \"vinbigdata-chest-xray-resized-png-256x256\",\n    # Model\n    \"model_name\": \"resnet18\",\n    # Training\n    \"num_workers\": 4,\n    \"epoch\": 15,\n    \"batchsize\": 8,\n    \"scheduler_type\": \"CosineAnnealingWarmRestarts\",\n    \"scheduler_kwargs\": {\"T_0\": 28125},  # 15000 * 15 epoch // (batchsize=8)\n    \"scheduler_trigger\": [1, \"iteration\"],\n    \"aug_kwargs\": {\n        \"HorizontalFlip\": {\"p\": 0.5},\n        \"ShiftScaleRotate\": {\"scale_limit\": 0.15, \"rotate_limit\": 10, \"p\": 0.5},\n        \"RandomBrightnessContrast\": {\"p\": 0.5},\n        \"CoarseDropout\": {\"max_holes\": 8, \"max_height\": 25, \"max_width\": 25, \"p\": 0.5},\n        \"Blur\": {\"blur_limit\": [3, 7], \"p\": 0.5},\n        \"Downscale\": {\"scale_min\": 0.25, \"scale_max\": 0.9, \"p\": 0.3},\n        \"RandomGamma\": {\"gamma_limit\": [80, 120], \"p\": 0.6},\n    }\n}","metadata":{"_kg_hide-input":true,"papermill":{"duration":0.063494,"end_time":"2021-02-17T21:46:37.018809","exception":false,"start_time":"2021-02-17T21:46:36.955315","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:38:18.488225Z","iopub.execute_input":"2024-12-29T23:38:18.488576Z","iopub.status.idle":"2024-12-29T23:38:18.507746Z","shell.execute_reply.started":"2024-12-29T23:38:18.488544Z","shell.execute_reply":"2024-12-29T23:38:18.506578Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# args = parse()\nprint(\"torch\", torch.__version__)\nflags = Flags().update(flags_dict)\nprint(\"flags\", flags)\ndebug = flags.debug\noutdir = Path(flags.outdir)\nos.makedirs(str(outdir), exist_ok=True)\nflags_dict = dataclasses.asdict(flags)\nsave_yaml(str(outdir / \"flags.yaml\"), flags_dict)\n\n# --- Read data ---\ninputdir = Path(\"/kaggle/input\")\ndatadir = inputdir / \"vinbigdata-chest-xray-abnormalities-detection\"\nimgdir = inputdir / flags.imgdir_name\n\n# Read in the data CSV files\ntrain = pd.read_csv(datadir / \"train.csv\")\n# sample_submission = pd.read_csv(datadir / 'sample_submission.csv')","metadata":{"_kg_hide-input":true,"papermill":{"duration":0.234439,"end_time":"2021-02-17T21:46:37.305923","exception":false,"start_time":"2021-02-17T21:46:37.071484","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:38:18.509265Z","iopub.execute_input":"2024-12-29T23:38:18.509549Z","iopub.status.idle":"2024-12-29T23:38:18.621209Z","shell.execute_reply.started":"2024-12-29T23:38:18.509522Z","shell.execute_reply":"2024-12-29T23:38:18.620029Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"7.2 Exploration of Normal and Abnromal Classes\n\nWe need to check how many normal class exist in the training data.\nIt is classified as \"class_name = No finding\" and \"class_id = 14\".\n\nHowever you need to be careful that 3 radiologists annotated for each image, \nso you can find 3 annotations as you can see below.","metadata":{"papermill":{"duration":0.053475,"end_time":"2021-02-17T21:46:37.414096","exception":false,"start_time":"2021-02-17T21:46:37.360621","status":"completed"},"tags":[]}},{"cell_type":"code","source":"train.query(\"image_id == '50a418190bc3fb1ef1633bf9678929b3'\")","metadata":{"papermill":{"duration":0.087359,"end_time":"2021-02-17T21:46:37.555345","exception":false,"start_time":"2021-02-17T21:46:37.467986","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:38:18.622605Z","iopub.execute_input":"2024-12-29T23:38:18.622878Z","iopub.status.idle":"2024-12-29T23:38:18.647563Z","shell.execute_reply.started":"2024-12-29T23:38:18.622853Z","shell.execute_reply":"2024-12-29T23:38:18.646596Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Is there an image that the 3 radiologists' opinions differ?\n\nLet's check number of \"No finding\" annotations for each image.\nIf the opinions are in complete agreement the number of \"No finding\" annotations, then it should be \n**0 -> Abnormal(all radiologists does not think this is normal)\" or \n\"1 -> Normal(all radiologists think this is normal)\"**.","metadata":{"papermill":{"duration":0.053793,"end_time":"2021-02-17T21:46:37.663683","exception":false,"start_time":"2021-02-17T21:46:37.60989","status":"completed"},"tags":[]}},{"cell_type":"code","source":"is_normal_df = train.groupby(\"image_id\")[\"class_id\"].agg(lambda s: (s == 14).sum()).reset_index().rename({\"class_id\": \"num_normal_annotations\"}, axis=1)\nis_normal_df.head()","metadata":{"papermill":{"duration":2.656924,"end_time":"2021-02-17T21:46:40.374779","exception":false,"start_time":"2021-02-17T21:46:37.717855","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:38:18.648878Z","iopub.execute_input":"2024-12-29T23:38:18.64923Z","iopub.status.idle":"2024-12-29T23:38:22.274841Z","shell.execute_reply.started":"2024-12-29T23:38:18.649199Z","shell.execute_reply":"2024-12-29T23:38:22.273849Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"It could confirm that **always 3 radiologists opinions match** for normal - abnormal diagnosis.\n\nNoticed that it does not apply for the other classes. \ni.e., 3 radiologists opinions sometimes do not match for the other class of thoracic abnormalities.","metadata":{"papermill":{"duration":0.054046,"end_time":"2021-02-17T21:46:40.483831","exception":false,"start_time":"2021-02-17T21:46:40.429785","status":"completed"},"tags":[]}},{"cell_type":"code","source":"num_normal_anno_counts = is_normal_df[\"num_normal_annotations\"].value_counts()\nnum_normal_anno_counts.plot(kind=\"bar\")\nplt.title(\"The number of 'No finding' annotations in each image\")","metadata":{"papermill":{"duration":0.205628,"end_time":"2021-02-17T21:46:40.743922","exception":false,"start_time":"2021-02-17T21:46:40.538294","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:38:22.276398Z","iopub.execute_input":"2024-12-29T23:38:22.27678Z","iopub.status.idle":"2024-12-29T23:38:22.406314Z","shell.execute_reply.started":"2024-12-29T23:38:22.276749Z","shell.execute_reply":"2024-12-29T23:38:22.405289Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"num_normal_anno_counts_df = num_normal_anno_counts.reset_index()\nnum_normal_anno_counts_df[\"name\"] = num_normal_anno_counts_df[\"index\"].map({0: \"Abnormal\", 3: \"Normal\"})\nnum_normal_anno_counts_df","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"papermill":{"duration":0.070477,"end_time":"2021-02-17T21:46:40.870129","exception":false,"start_time":"2021-02-17T21:46:40.799652","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:38:22.408139Z","iopub.execute_input":"2024-12-29T23:38:22.408536Z","iopub.status.idle":"2024-12-29T23:38:22.42264Z","shell.execute_reply.started":"2024-12-29T23:38:22.408494Z","shell.execute_reply":"2024-12-29T23:38:22.42155Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"normal_count = num_normal_anno_counts_df.iloc[0]['num_normal_annotations']\nabnormal_count = num_normal_anno_counts_df.iloc[1]['num_normal_annotations']\n\nprint(\"% of Normal X-Ray Images = \", round((normal_count/(normal_count+abnormal_count))*100),3)\nprint(\"% of AbNormal X-Ray Images = \", round((abnormal_count/(normal_count+abnormal_count))*100),3)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:38:22.424485Z","iopub.execute_input":"2024-12-29T23:38:22.424797Z","iopub.status.idle":"2024-12-29T23:38:22.445494Z","shell.execute_reply.started":"2024-12-29T23:38:22.424768Z","shell.execute_reply":"2024-12-29T23:38:22.444236Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"So around 70% of the data is actually \"Normal\" X-ray images.\n\nBut about 30% of the images need thoracic abnormality location detection.","metadata":{"papermill":{"duration":0.056173,"end_time":"2021-02-17T21:46:40.982761","exception":false,"start_time":"2021-02-17T21:46:40.926588","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"**8. Image Visualizaion and Augmentation with Albumentations**\n\nWhen you train CNN models, Image Augmentation is important to avoid model to overfit.\nThe below examples to use Albumentations to run image augmentation very easily\nAt first, I will define pytorch Dataset class for this competition, which can be also used later in the training.","metadata":{"papermill":{"duration":0.062022,"end_time":"2021-02-17T21:46:41.37108","exception":false,"start_time":"2021-02-17T21:46:41.309058","status":"completed"},"tags":[]}},{"cell_type":"code","source":"def get_vinbigdata_dicts(\n    imgdir: Path,\n    train_df: pd.DataFrame,\n    train_data_type: str = \"original\",\n    use_cache: bool = True,\n    debug: bool = True,\n    target_indices: Optional[np.ndarray] = None,\n):\n    debug_str = f\"_debug{int(debug)}\"\n    train_data_type_str = f\"_{train_data_type}\"\n    cache_path = Path(\".\") / f\"dataset_dicts_cache{train_data_type_str}{debug_str}.pkl\"\n    if not use_cache or not cache_path.exists():\n        print(\"Creating data...\")\n        train_meta = pd.read_csv(imgdir / \"train_meta.csv\")\n        if debug:\n            train_meta = train_meta.iloc[:500]  # For debug....\n\n        # Load 1 image to get image size.\n        image_id = train_meta.loc[0, \"image_id\"]\n        image_path = str(imgdir / \"train\" / f\"{image_id}.png\")\n        image = cv2.imread(image_path)\n        resized_height, resized_width, ch = image.shape\n        print(f\"image shape: {image.shape}\")\n\n        dataset_dicts = []\n        for index, train_meta_row in tqdm(train_meta.iterrows(), total=len(train_meta)):\n            record = {}\n\n            image_id, height, width = train_meta_row.values\n            filename = str(imgdir / \"train\" / f\"{image_id}.png\")\n            record[\"file_name\"] = filename\n            record[\"image_id\"] = image_id\n            record[\"height\"] = resized_height\n            record[\"width\"] = resized_width\n            objs = []\n            for index2, row in train_df.query(\"image_id == @image_id\").iterrows():\n                # print(row)\n                # print(row[\"class_name\"])\n                # class_name = row[\"class_name\"]\n                class_id = row[\"class_id\"]\n                if class_id == 14:\n                    # It is \"No finding\"\n                    # This annotator does not find anything, skip.\n                    pass\n                else:\n                    # bbox_original = [int(row[\"x_min\"]), int(row[\"y_min\"]), int(row[\"x_max\"]), int(row[\"y_max\"])]\n                    h_ratio = resized_height / height\n                    w_ratio = resized_width / width\n                    bbox_resized = [\n                        int(row[\"x_min\"]) * w_ratio,\n                        int(row[\"y_min\"]) * h_ratio,\n                        int(row[\"x_max\"]) * w_ratio,\n                        int(row[\"y_max\"]) * h_ratio,\n                    ]\n                    obj = {\n                        \"bbox\": bbox_resized,\n                        \"bbox_mode\": BoxMode.XYXY_ABS,\n                        \"category_id\": class_id,\n                    }\n                    objs.append(obj)\n            record[\"annotations\"] = objs\n            dataset_dicts.append(record)\n        with open(cache_path, mode=\"wb\") as f:\n            pickle.dump(dataset_dicts, f)\n\n    print(f\"Load from cache {cache_path}\")\n    with open(cache_path, mode=\"rb\") as f:\n        dataset_dicts = pickle.load(f)\n    if target_indices is not None:\n        dataset_dicts = [dataset_dicts[i] for i in target_indices]\n    return dataset_dicts\n\ndef get_vinbigdata_dicts_test(\n    imgdir: Path, test_meta: pd.DataFrame, use_cache: bool = True, debug: bool = True,\n):\n    debug_str = f\"_debug{int(debug)}\"\n    cache_path = Path(\".\") / f\"dataset_dicts_cache_test{debug_str}.pkl\"\n    if not use_cache or not cache_path.exists():\n        print(\"Creating data...\")\n        # test_meta = pd.read_csv(imgdir / \"test_meta.csv\")\n        if debug:\n            test_meta = test_meta.iloc[:500]  # For debug....\n\n        # Load 1 image to get image size.\n        image_id = test_meta.loc[0, \"image_id\"]\n        image_path = str(imgdir / \"test\" / f\"{image_id}.png\")\n        image = cv2.imread(image_path)\n        resized_height, resized_width, ch = image.shape\n        print(f\"image shape: {image.shape}\")\n\n        dataset_dicts = []\n        for index, test_meta_row in tqdm(test_meta.iterrows(), total=len(test_meta)):\n            record = {}\n\n            image_id, height, width = test_meta_row.values\n            filename = str(imgdir / \"test\" / f\"{image_id}.png\")\n            record[\"file_name\"] = filename\n            # record[\"image_id\"] = index\n            record[\"image_id\"] = image_id\n            record[\"height\"] = resized_height\n            record[\"width\"] = resized_width\n            # objs = []\n            # record[\"annotations\"] = objs\n            dataset_dicts.append(record)\n        with open(cache_path, mode=\"wb\") as f:\n            pickle.dump(dataset_dicts, f)\n\n    print(f\"Load from cache {cache_path}\")\n    with open(cache_path, mode=\"rb\") as f:\n        dataset_dicts = pickle.load(f)\n    return dataset_dicts\n","metadata":{"_kg_hide-input":true,"papermill":{"duration":0.594728,"end_time":"2021-02-17T21:46:42.028422","exception":false,"start_time":"2021-02-17T21:46:41.433694","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:38:22.446995Z","iopub.execute_input":"2024-12-29T23:38:22.44727Z","iopub.status.idle":"2024-12-29T23:38:22.466222Z","shell.execute_reply.started":"2024-12-29T23:38:22.447243Z","shell.execute_reply":"2024-12-29T23:38:22.465239Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"\"\"\"\nReferenced `chainer.dataset.DatasetMixin` to work with pytorch Dataset.\n\"\"\"\nimport numpy\n\nclass DatasetMixin(Dataset):\n\n    def __init__(self, transform=None):\n        self.transform = transform\n\n    def __getitem__(self, index):\n        \"\"\"Returns an example or a sequence of examples.\"\"\"\n        if torch.is_tensor(index):\n            index = index.tolist()\n        if isinstance(index, slice):\n            current, stop, step = index.indices(len(self))\n            return [self.get_example_wrapper(i) for i in\n                    six.moves.range(current, stop, step)]\n        elif isinstance(index, list) or isinstance(index, numpy.ndarray):\n            return [self.get_example_wrapper(i) for i in index]\n        else:\n            return self.get_example_wrapper(index)\n\n    def __len__(self):\n        \"\"\"Returns the number of data points.\"\"\"\n        raise NotImplementedError\n\n    def get_example_wrapper(self, i):\n        \"\"\"Wrapper of `get_example`, to apply `transform` if necessary\"\"\"\n        example = self.get_example(i)\n        if self.transform:\n            example = self.transform(example)\n        return example\n\n    def get_example(self, i):\n        \"\"\"Returns the i-th example.\n\n        Implementations should override it. It should raise :class:`IndexError`\n        if the index is invalid.\n\n        Args:\n            i (int): The index of the example.\n\n        Returns:\n            The i-th example.\n\n        \"\"\"\n        raise NotImplementedError\n","metadata":{"_kg_hide-input":true,"papermill":{"duration":0.079143,"end_time":"2021-02-17T21:46:42.17118","exception":false,"start_time":"2021-02-17T21:46:42.092037","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:38:22.468203Z","iopub.execute_input":"2024-12-29T23:38:22.468605Z","iopub.status.idle":"2024-12-29T23:38:22.486015Z","shell.execute_reply.started":"2024-12-29T23:38:22.468566Z","shell.execute_reply":"2024-12-29T23:38:22.485048Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Here mixup augmentation in the dataset. It makes interpolation of 2 images, with the label is also modified according to the mix ratio. \nmixup augmentation is especially useful when the number of data is limited.\nBecause it can make combination of any 2 images, instead of just using 1 image.\n\nI also added label smoothing feature. \nSometimes it is difficult to learn label is 0, 1. \nLabel smoothing changes its label 0 -> 0.01 & 1 -> 0.99. \nBy smoothing the label the loss surface becomes more \"soft\", and sometimes model can learn well.","metadata":{"papermill":{"duration":0.061858,"end_time":"2021-02-17T21:46:42.29527","exception":false,"start_time":"2021-02-17T21:46:42.233412","status":"completed"},"tags":[]}},{"cell_type":"code","source":"class VinbigdataTwoClassDataset(DatasetMixin):\n    def __init__(self, dataset_dicts, image_transform=None, transform=None, train: bool = True,\n                 mixup_prob: float = -1.0, label_smoothing: float = 0.0):\n        super(VinbigdataTwoClassDataset, self).__init__(transform=transform)\n        self.dataset_dicts = dataset_dicts\n        self.image_transform = image_transform\n        self.train = train\n        self.mixup_prob = mixup_prob\n        self.label_smoothing = label_smoothing\n\n    def _get_single_example(self, i):\n        d = self.dataset_dicts[i]\n        filename = d[\"file_name\"]\n\n        img = cv2.imread(filename)\n        if self.image_transform:\n            img = self.image_transform(img)\n        img = torch.tensor(np.transpose(img, (2, 0, 1)).astype(np.float32))\n\n        if self.train:\n            label = int(len(d[\"annotations\"]) > 0)  # 0 normal, 1 abnormal\n            if self.label_smoothing > 0:\n                if label == 0:\n                    return img, float(label) + self.label_smoothing\n                else:\n                    return img, float(label) - self.label_smoothing\n            else:\n                return img, float(label)\n        else:\n            # Only return img\n            return img, None\n\n    def get_example(self, i):\n        img, label = self._get_single_example(i)\n        if self.mixup_prob > 0. and np.random.uniform() < self.mixup_prob:\n            j = np.random.randint(0, len(self.dataset_dicts))\n            p = np.random.uniform()\n            img2, label2 = self._get_single_example(j)\n            img = img * p + img2 * (1 - p)\n            if self.train:\n                label = label * p + label2 * (1 - p)\n\n        if self.train:\n            label_logit = torch.tensor([1 - label, label], dtype=torch.float32)\n            return img, label_logit\n        else:\n            # Only return img\n            return img\n\n    def __len__(self):\n        return len(self.dataset_dicts)","metadata":{"papermill":{"duration":0.084332,"end_time":"2021-02-17T21:46:42.442066","exception":false,"start_time":"2021-02-17T21:46:42.357734","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:38:22.487398Z","iopub.execute_input":"2024-12-29T23:38:22.487721Z","iopub.status.idle":"2024-12-29T23:38:22.50686Z","shell.execute_reply.started":"2024-12-29T23:38:22.487678Z","shell.execute_reply":"2024-12-29T23:38:22.5055Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Now creating the dataset is just easy as following:","metadata":{"papermill":{"duration":0.0619,"end_time":"2021-02-17T21:46:42.570849","exception":false,"start_time":"2021-02-17T21:46:42.508949","status":"completed"},"tags":[]}},{"cell_type":"code","source":"dataset_dicts = get_vinbigdata_dicts(imgdir, train, debug=debug)\ndataset = VinbigdataTwoClassDataset(dataset_dicts)","metadata":{"_kg_hide-output":true,"papermill":{"duration":75.178981,"end_time":"2021-02-17T21:47:57.812584","exception":false,"start_time":"2021-02-17T21:46:42.633603","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:38:22.508461Z","iopub.execute_input":"2024-12-29T23:38:22.508752Z","iopub.status.idle":"2024-12-29T23:38:22.640219Z","shell.execute_reply.started":"2024-12-29T23:38:22.508719Z","shell.execute_reply":"2024-12-29T23:38:22.635617Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"You can access each image and its label (0=Normal, 1=Abnormal) by just access `dataset` with index.","metadata":{"papermill":{"duration":0.255738,"end_time":"2021-02-17T21:47:58.326815","exception":false,"start_time":"2021-02-17T21:47:58.071077","status":"completed"},"tags":[]}},{"cell_type":"code","source":"index = 0\nimg, label = dataset[index]\nplt.imshow(img.cpu().numpy().transpose((1, 2, 0)) / 255.)\nplt.title(f\"{index}-th image: label {label}\")","metadata":{"papermill":{"duration":0.489165,"end_time":"2021-02-17T21:47:59.077261","exception":false,"start_time":"2021-02-17T21:47:58.588096","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:42:37.97267Z","iopub.execute_input":"2024-12-29T23:42:37.973472Z","iopub.status.idle":"2024-12-29T23:42:38.022993Z","shell.execute_reply.started":"2024-12-29T23:42:37.973403Z","shell.execute_reply":"2024-12-29T23:42:38.018815Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"To run augmentation on this image, I will define `Transform` class which is applied each time the data is accessed.","metadata":{"papermill":{"duration":0.258913,"end_time":"2021-02-17T21:47:59.598184","exception":false,"start_time":"2021-02-17T21:47:59.339271","status":"completed"},"tags":[]}},{"cell_type":"code","source":"class Transform:\n    def __init__(\n        self, hflip_prob: float = 0.5, ssr_prob: float = 0.5, random_bc_prob: float = 0.5\n    ):\n        self.transform = A.Compose(\n            [\n                A.HorizontalFlip(p=hflip_prob),\n                A.ShiftScaleRotate(\n                    shift_limit=0.0625, scale_limit=0.1, rotate_limit=10, p=ssr_prob\n                ),\n                A.RandomBrightnessContrast(p=random_bc_prob),\n            ]\n        )\n\n    def __call__(self, image):\n        image = self.transform(image=image)[\"image\"]\n        return image\n","metadata":{"papermill":{"duration":0.641247,"end_time":"2021-02-17T21:48:00.497145","exception":false,"start_time":"2021-02-17T21:47:59.855898","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:42:52.569267Z","iopub.execute_input":"2024-12-29T23:42:52.569583Z","iopub.status.idle":"2024-12-29T23:42:52.578095Z","shell.execute_reply.started":"2024-12-29T23:42:52.569556Z","shell.execute_reply":"2024-12-29T23:42:52.576732Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"To use augmentation, you can just define dataset with the `Transform` function.","metadata":{"papermill":{"duration":0.261496,"end_time":"2021-02-17T21:48:01.020729","exception":false,"start_time":"2021-02-17T21:48:00.759233","status":"completed"},"tags":[]}},{"cell_type":"code","source":"aug_dataset = VinbigdataTwoClassDataset(dataset_dicts, image_transform=Transform())","metadata":{"papermill":{"duration":0.269682,"end_time":"2021-02-17T21:48:01.5532","exception":false,"start_time":"2021-02-17T21:48:01.283518","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:42:52.580578Z","iopub.execute_input":"2024-12-29T23:42:52.581254Z","iopub.status.idle":"2024-12-29T23:42:52.620287Z","shell.execute_reply.started":"2024-12-29T23:42:52.581207Z","shell.execute_reply":"2024-12-29T23:42:52.617782Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"Let's visualize, looks good. <br/>\nYou can see each image looks different (rotated, brightness is different etc...) even if it is generated from the same image :)","metadata":{"papermill":{"duration":0.264415,"end_time":"2021-02-17T21:48:02.080813","exception":false,"start_time":"2021-02-17T21:48:01.816398","status":"completed"},"tags":[]}},{"cell_type":"code","source":"index = 0\n\nn_images = 4\n\nfig, axes = plt.subplots(1, n_images, figsize=(16, 5))\nfor i in range(n_images):\n    # Each time the data is accessed, the result is different due to random augmentation!\n    img, label = aug_dataset[index]\n    ax = axes[i]\n    ax.imshow(img.cpu().numpy().transpose((1, 2, 0)) / 255.)\n    ax.set_title(f\"{index}-th image: label {label}\")\nplt.show()","metadata":{"_kg_hide-input":true,"papermill":{"duration":0.788107,"end_time":"2021-02-17T21:48:03.148359","exception":false,"start_time":"2021-02-17T21:48:02.360252","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:43:06.472043Z","iopub.execute_input":"2024-12-29T23:43:06.472399Z","iopub.status.idle":"2024-12-29T23:43:06.898134Z","shell.execute_reply.started":"2024-12-29T23:43:06.472369Z","shell.execute_reply":"2024-12-29T23:43:06.895059Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"class Transform:\n    def __init__(self, aug_kwargs: Dict):\n        self.transform = A.Compose(\n            [getattr(A, name)(**kwargs) for name, kwargs in aug_kwargs.items()]\n        )\n\n    def __call__(self, image):\n        image = self.transform(image=image)[\"image\"]\n        return image","metadata":{"papermill":{"duration":0.272641,"end_time":"2021-02-17T21:48:04.50634","exception":false,"start_time":"2021-02-17T21:48:04.233699","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:43:06.899551Z","iopub.status.idle":"2024-12-29T23:43:06.900065Z","shell.execute_reply":"2024-12-29T23:43:06.899794Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**9. Model Building(CNN Models)**\n\nI will use `timm` this time. You can just re-use latest research results without hustle.","metadata":{"papermill":{"duration":0.262749,"end_time":"2021-02-17T21:48:05.030201","exception":false,"start_time":"2021-02-17T21:48:04.767452","status":"completed"},"tags":[]}},{"cell_type":"code","source":"class CNNFixedPredictor(nn.Module):\n    def __init__(self, cnn: nn.Module, num_classes: int = 2):\n        super(CNNFixedPredictor, self).__init__()\n        self.cnn = cnn\n        self.lin = Linear(cnn.num_features, num_classes)\n        print(\"cnn.num_features\", cnn.num_features)\n\n        # We do not learn CNN parameters.\n        # https://pytorch.org/tutorials/beginner/finetuning_torchvision_models_tutorial.html\n        for param in self.cnn.parameters():\n            param.requires_grad = False\n\n    def forward(self, x):\n        feat = self.cnn(x)\n        return self.lin(feat)\n","metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"papermill":{"duration":0.273325,"end_time":"2021-02-17T21:48:05.579219","exception":false,"start_time":"2021-02-17T21:48:05.305894","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:43:06.901258Z","iopub.status.idle":"2024-12-29T23:43:06.901701Z","shell.execute_reply":"2024-12-29T23:43:06.901476Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**11. Model Validation**","metadata":{}},{"cell_type":"code","source":"def build_predictor(model_name: str, model_mode: str = \"normal\"):\n    if model_mode == \"normal\":\n        # normal configuration. train all parameters.\n        return timm.create_model(model_name, pretrained=True, num_classes=2, in_chans=3)\n    elif model_mode == \"cnn_fixed\":\n        # normal configuration. train all parameters.\n        # https://rwightman.github.io/pytorch-image-models/feature_extraction/\n        timm_model = timm.create_model(model_name, pretrained=True, num_classes=0, in_chans=3)\n        return CNNFixedPredictor(timm_model, num_classes=2)\n    else:\n        raise ValueError(f\"[ERROR] Unexpected value model_mode={model_mode}\")\n","metadata":{"papermill":{"duration":0.362691,"end_time":"2021-02-17T21:48:06.203144","exception":false,"start_time":"2021-02-17T21:48:05.840453","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:43:06.902693Z","iopub.status.idle":"2024-12-29T23:43:06.903151Z","shell.execute_reply":"2024-12-29T23:43:06.902929Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def accuracy(y: torch.Tensor, t: torch.Tensor) -> torch.Tensor:\n    \"\"\"Computes multi-class classification accuracy\"\"\"\n    assert y.shape[:-1] == t.shape, f\"y {y.shape}, t {t.shape} is inconsistent.\"\n    pred_label = torch.max(y.detach(), dim=-1)[1]\n    count = t.nelement()\n    correct = (pred_label == t).sum().float()\n    acc = correct / count\n    return acc\n\ndef accuracy_with_logits(y: torch.Tensor, t: torch.Tensor) -> torch.Tensor:\n    \"\"\"Computes multi-class classification accuracy\"\"\"\n    assert y.shape == t.shape\n    gt_label = torch.max(t.detach(), dim=-1)[1]\n    return accuracy(y, gt_label)","metadata":{"_kg_hide-input":true,"papermill":{"duration":0.274867,"end_time":"2021-02-17T21:48:06.739132","exception":false,"start_time":"2021-02-17T21:48:06.464265","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:43:06.904118Z","iopub.status.idle":"2024-12-29T23:43:06.904548Z","shell.execute_reply":"2024-12-29T23:43:06.904334Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def cross_entropy_with_logits(input, target, dim=-1):\n    loss = torch.sum(- target * F.log_softmax(input, dim), dim)\n    return loss.mean()\n","metadata":{"papermill":{"duration":0.270145,"end_time":"2021-02-17T21:48:07.270432","exception":false,"start_time":"2021-02-17T21:48:07.000287","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:43:06.905617Z","iopub.status.idle":"2024-12-29T23:43:06.906089Z","shell.execute_reply":"2024-12-29T23:43:06.905831Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"class Classifier(nn.Module):\n    \"\"\"two class classfication\"\"\"\n\n    def __init__(self, predictor, lossfun=cross_entropy_with_logits):\n        super().__init__()\n        self.predictor = predictor\n        self.lossfun = lossfun\n        self.prefix = \"\"\n\n    def forward(self, image, targets):\n        outputs = self.predictor(image)\n        loss = self.lossfun(outputs, targets)\n        metrics = {\n            f\"{self.prefix}loss\": loss.item(),\n            f\"{self.prefix}acc\": accuracy_with_logits(outputs, targets).item()\n        }\n        ppe.reporting.report(metrics, self)\n        return loss, metrics\n\n    def predict(self, data_loader):\n        pred = self.predict_proba(data_loader)\n        label = torch.argmax(pred, dim=1)\n        return label\n\n    def predict_proba(self, data_loader):\n        device: torch.device = next(self.parameters()).device\n        y_list = []\n        self.eval()\n        with torch.no_grad():\n            for batch in data_loader:\n                if isinstance(batch, (tuple, list)):\n                    # Assumes first argument is \"image\"\n                    batch = batch[0].to(device)\n                else:\n                    batch = batch.to(device)\n                y = self.predictor(batch)\n                y = torch.softmax(y, dim=-1)\n                y_list.append(y)\n        pred = torch.cat(y_list)\n        return pred\n","metadata":{"_kg_hide-input":true,"papermill":{"duration":0.297132,"end_time":"2021-02-17T21:48:07.828847","exception":false,"start_time":"2021-02-17T21:48:07.531715","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:43:06.907191Z","iopub.status.idle":"2024-12-29T23:43:06.907611Z","shell.execute_reply":"2024-12-29T23:43:06.907398Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"What kind of models are supported in the `timm` library?","metadata":{"papermill":{"duration":0.430302,"end_time":"2021-02-17T21:48:08.682483","exception":false,"start_time":"2021-02-17T21:48:08.252181","status":"completed"},"tags":[]}},{"cell_type":"code","source":"supported_models = timm.list_models()\nprint(f\"{len(supported_models)} models are supported in timm.\")\nprint(supported_models)","metadata":{"papermill":{"duration":0.273098,"end_time":"2021-02-17T21:48:09.347884","exception":false,"start_time":"2021-02-17T21:48:09.074786","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:43:06.908521Z","iopub.status.idle":"2024-12-29T23:43:06.908973Z","shell.execute_reply":"2024-12-29T23:43:06.908728Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"434 models are supported including **resnet** related models, **efficientnet**\nYou may wonder which model should be used?\nI will go with `resnet18` as a baseline at first, and try using more deeper/latest models in the experiment.","metadata":{"papermill":{"duration":0.261865,"end_time":"2021-02-17T21:48:09.872562","exception":false,"start_time":"2021-02-17T21:48:09.610697","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"**10. Model Training**","metadata":{"papermill":{"duration":0.275512,"end_time":"2021-02-17T21:48:10.412027","exception":false,"start_time":"2021-02-17T21:48:10.136515","status":"completed"},"tags":[]}},{"cell_type":"code","source":"\"\"\"\nFrom https://github.com/pfnet-research/kaggle-lyft-motion-prediction-4th-place-solution\n\"\"\"\nclass EMA(object):\n    \"\"\"Exponential moving average of model parameters.\n\n    Ref\n     - https://github.com/tensorflow/addons/blob/v0.10.0/tensorflow_addons/optimizers/moving_average.py#L26-L103\n     - https://anmoljoshi.com/Pytorch-Dicussions/\n\n    Args:\n        model (nn.Module): Model with parameters whose EMA will be kept.\n        decay (float): Decay rate for exponential moving average.\n        strict (bool): Apply strict check for `assign` & `resume`.\n        use_dynamic_decay (bool): Dynamically change decay rate. If `True`, small decay rate is\n            used at the beginning of training to move moving average faster.\n    \"\"\"  # NOQA\n\n    def __init__(\n        self,\n        model: nn.Module,\n        decay: float,\n        strict: bool = True,\n        use_dynamic_decay: bool = True,\n    ):\n        self.decay = decay\n        self.model = model\n        self.strict = strict\n        self.use_dynamic_decay = use_dynamic_decay\n        self.logger = getLogger(__name__)\n        self.n_step = 0\n\n        self.shadow = {}\n        self.original = {}\n\n        # Flag to manage which parameter is assigned.\n        # When `False`, original model's parameter is used.\n        # When `True` (`assign` method is called), `shadow` parameter (ema param) is used.\n        self._assigned = False\n\n        # Register model parameters\n        for name, param in model.named_parameters():\n            if param.requires_grad:\n                self.shadow[name] = param.data.clone()\n\n    def step(self):\n        self.n_step += 1\n        if self.use_dynamic_decay:\n            _n_step = float(self.n_step)\n            decay = min(self.decay, (1.0 + _n_step) / (10.0 + _n_step))\n        else:\n            decay = self.decay\n\n        for name, param in self.model.named_parameters():\n            if param.requires_grad:\n                assert name in self.shadow\n                new_average = (1.0 - decay) * param.data + decay * self.shadow[name]\n                self.shadow[name] = new_average.clone()\n\n    # alias\n    __call__ = step\n\n    def assign(self):\n        \"\"\"Assign exponential moving average of parameter values to the respective parameters.\"\"\"\n        if self._assigned:\n            if self.strict:\n                raise ValueError(\"[ERROR] `assign` is called again before `resume`.\")\n            else:\n                self.logger.warning(\n                    \"`assign` is called again before `resume`.\"\n                    \"shadow parameter is already assigned, skip.\"\n                )\n                return\n\n        for name, param in self.model.named_parameters():\n            if param.requires_grad:\n                assert name in self.shadow\n                self.original[name] = param.data.clone()\n                param.data = self.shadow[name]\n        self._assigned = True\n\n    def resume(self):\n        \"\"\"Restore original parameters to a model.\n\n        That is, put back the values that were in each parameter at the last call to `assign`.\n        \"\"\"\n        if not self._assigned:\n            if self.strict:\n                raise ValueError(\"[ERROR] `resume` is called before `assign`.\")\n            else:\n                self.logger.warning(\"`resume` is called before `assign`, skip.\")\n                return\n\n        for name, param in self.model.named_parameters():\n            if param.requires_grad:\n                assert name in self.shadow\n                param.data = self.original[name]\n        self._assigned = False\n","metadata":{"_kg_hide-input":true,"papermill":{"duration":0.287512,"end_time":"2021-02-17T21:48:10.962713","exception":false,"start_time":"2021-02-17T21:48:10.675201","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:43:06.909991Z","iopub.status.idle":"2024-12-29T23:43:06.910412Z","shell.execute_reply":"2024-12-29T23:43:06.910195Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"\"\"\"\nFrom https://github.com/pfnet-research/kaggle-lyft-motion-prediction-4th-place-solution\n\"\"\"\nclass LRScheduler(Extension):\n    \"\"\"A thin wrapper to resume the lr_scheduler\"\"\"\n\n    trigger = 1, 'iteration'\n    priority = PRIORITY_READER\n    name = None\n\n    def __init__(self, optimizer: optim.Optimizer, scheduler_type: str, scheduler_kwargs: Mapping[str, Any]) -> None:\n        super().__init__()\n        self.scheduler = getattr(optim.lr_scheduler, scheduler_type)(optimizer, **scheduler_kwargs)\n\n    def __call__(self, manager: ExtensionsManager) -> None:\n        self.scheduler.step()\n\n    def state_dict(self) -> None:\n        return self.scheduler.state_dict()\n\n    def load_state_dict(self, to_load) -> None:\n        self.scheduler.load_state_dict(to_load)\n","metadata":{"_kg_hide-input":true,"papermill":{"duration":0.273877,"end_time":"2021-02-17T21:48:11.499394","exception":false,"start_time":"2021-02-17T21:48:11.225517","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:43:06.911527Z","iopub.status.idle":"2024-12-29T23:43:06.911979Z","shell.execute_reply":"2024-12-29T23:43:06.911735Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"\ndef create_trainer(model, optimizer, device) -> Engine:\n    model.to(device)\n\n    def update_fn(engine, batch):\n        model.train()\n        optimizer.zero_grad()\n        loss, metrics = model(*[elem.to(device) for elem in batch])\n        loss.backward()\n        optimizer.step()\n        return metrics\n    trainer = Engine(update_fn)\n    return trainer\n","metadata":{"_kg_hide-input":true,"papermill":{"duration":0.324862,"end_time":"2021-02-17T21:48:12.088299","exception":false,"start_time":"2021-02-17T21:48:11.763437","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:43:06.912938Z","iopub.status.idle":"2024-12-29T23:43:06.913356Z","shell.execute_reply":"2024-12-29T23:43:06.913141Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**11. Model Testing and Validation**\n\nWhen we have few data, running stable evaluation is very important. \nWe can use cross validation to reduce validation error standard deviation.\n\nI will use **[`StratifiedKFold`]** to keep the balance between normal/abnormal ratio same for the train & validation dataset.\n\nAccording to [this discussion](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/208837#1139712), using multi label stratified kfold https://github.com/trent-b/iterative-stratification may be more stable.","metadata":{"papermill":{"duration":0.262059,"end_time":"2021-02-17T21:48:13.705455","exception":false,"start_time":"2021-02-17T21:48:13.443396","status":"completed"},"tags":[]}},{"cell_type":"code","source":"skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=flags.seed)\n# skf.get_n_splits(None, None)\ny = np.array([int(len(d[\"annotations\"]) > 0) for d in dataset_dicts])\nsplit_inds = list(skf.split(dataset_dicts, y))\ntrain_inds, valid_inds = split_inds[flags.target_fold]  # 0th fold\ntrain_dataset = VinbigdataTwoClassDataset(\n    [dataset_dicts[i] for i in train_inds],\n    image_transform=Transform(flags.aug_kwargs),\n    mixup_prob=flags.mixup_prob,\n    label_smoothing=flags.label_smoothing,\n)\nvalid_dataset = VinbigdataTwoClassDataset([dataset_dicts[i] for i in valid_inds])\n","metadata":{"papermill":{"duration":0.365045,"end_time":"2021-02-17T21:48:14.334805","exception":false,"start_time":"2021-02-17T21:48:13.96976","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:43:06.914309Z","iopub.status.idle":"2024-12-29T23:43:06.914736Z","shell.execute_reply":"2024-12-29T23:43:06.914505Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**10. Model Training**","metadata":{"papermill":{"duration":0.268025,"end_time":"2021-02-17T21:48:15.041167","exception":false,"start_time":"2021-02-17T21:48:14.773142","status":"completed"},"tags":[]}},{"cell_type":"code","source":"train_loader = DataLoader(\n    train_dataset,\n    batch_size=flags.batchsize,\n    num_workers=flags.num_workers,\n    shuffle=True,\n    pin_memory=True,\n)\nvalid_loader = DataLoader(\n    valid_dataset,\n    batch_size=flags.valid_batchsize,\n    num_workers=flags.num_workers,\n    shuffle=False,\n    pin_memory=True,\n)\n\ndevice = torch.device(flags.device)\n\npredictor = build_predictor(model_name=flags.model_name, model_mode=flags.model_mode)\nclassifier = Classifier(predictor)\nmodel = classifier\n# optimizer = optim.Adam(model.parameters(), lr=1e-3)\noptimizer = optim.Adam([param for param in model.parameters() if param.requires_grad], lr=1e-3)\n\n# Train setup\ntrainer = create_trainer(model, optimizer, device)\n\nema = EMA(predictor, decay=flags.ema_decay)\n\ndef eval_func(*batch):\n    loss, metrics = model(*[elem.to(device) for elem in batch])\n    # HACKING: report ema value with prefix.\n    if flags.ema_decay > 0:\n        classifier.prefix = \"ema_\"\n        ema.assign()\n        loss, metrics = model(*[elem.to(device) for elem in batch])\n        ema.resume()\n        classifier.prefix = \"\"\n\nvalid_evaluator = E.Evaluator(\n    valid_loader, model, progress_bar=False, eval_func=eval_func, device=device\n)\n\n# log_trigger = (10 if debug else 1000, \"iteration\")\nlog_trigger = (1, \"epoch\")\nlog_report = E.LogReport(trigger=log_trigger)\nextensions = [\n    log_report,\n    E.ProgressBarNotebook(update_interval=10 if debug else 100),  # Show progress bar during training\n    E.PrintReportNotebook(),  # Show \"log\" on jupyter notebook  \n    # E.ProgressBar(update_interval=10 if debug else 100),  # Show progress bar during training\n    # E.PrintReport(),  # Print \"log\" to terminal\n    E.FailOnNonNumber(),  # Stop training when nan is detected.\n]\nepoch = flags.epoch\nmodels = {\"main\": model}\noptimizers = {\"main\": optimizer}\nmanager = IgniteExtensionsManager(\n    trainer, models, optimizers, epoch, extensions=extensions, out_dir=str(outdir),\n)\n# Run evaluation for valid dataset in each epoch.\nmanager.extend(valid_evaluator)\n\n# Save predictor.pt every epoch\nmanager.extend(\n    E.snapshot_object(predictor, \"predictor.pt\"), trigger=(flags.snapshot_freq, \"epoch\")\n)\n# Check & Save best validation predictor.pt every epoch\n# manager.extend(E.snapshot_object(predictor, \"best_predictor.pt\"),\n#                trigger=MinValueTrigger(\"validation/module/nll\",\n#                trigger=(flags.snapshot_freq, \"iteration\")))\n\n# --- lr scheduler ---\nif flags.scheduler_type != \"\":\n    scheduler_type = flags.scheduler_type\n    print(f\"using {scheduler_type} scheduler with kwargs {flags.scheduler_kwargs}\")\n    manager.extend(\n        LRScheduler(optimizer, scheduler_type, flags.scheduler_kwargs),\n        trigger=flags.scheduler_trigger,\n    )\n\nmanager.extend(E.observe_lr(optimizer=optimizer), trigger=log_trigger)\n\nif flags.ema_decay > 0:\n    # Exponential moving average\n    manager.extend(lambda manager: ema(), trigger=(1, \"iteration\"))\n\n    def save_ema_model(manager):\n        ema.assign()\n        torch.save(predictor.state_dict(), outdir / \"predictor_ema.pt\")\n        ema.resume()\n\n    manager.extend(save_ema_model, trigger=(flags.snapshot_freq, \"epoch\"))\n\n_ = trainer.run(train_loader, max_epochs=epoch)","metadata":{"papermill":{"duration":1673.867537,"end_time":"2021-02-17T22:16:09.177329","exception":false,"start_time":"2021-02-17T21:48:15.309792","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:43:06.915714Z","iopub.status.idle":"2024-12-29T23:43:06.916162Z","shell.execute_reply":"2024-12-29T23:43:06.915943Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**13. Discussion**\n\n**Extensions** - Each role:\n - **`ProgressBar` (`ProgressBarNotebook`)**: Shows training progress in formatted style.\n - **`LogReport`**: Logging metrics reported by `ppe.reporter.report` (see `LyftMultiRegressor` for reporting point) method and save to **log** file. It automatically collects reported value in each iteration and saves the \"mean\" of reported value for regular frequency (for example every 1 epoch).\n - **`PrintReport` (`PrintReportNotebook`)**: Prints the value which `LogReport` collected in formatted style.\n - **`Evaluator`**: Evaluate on validation dataset.\n - **`snapshot_object`**: Saves the object. Here the `model` is saved in regular interval `flags.snapshot_freq`. Even you quit training using Ctrl+C without finishing all the epoch, the intermediate trained model is saved and you can use it for inference.\n - **`LRScheduler`**: You can insert learning rate scheduling with this extension, together with the regular interval call specified by `trigger`. Here cosine annealing is applied (configured by Flags) by calling `scheduler.step()` every iteration.\n - **`observe_lr`**: `LogReport` will check optimizer's learning rate using this extension. So you can follow how the learning rate changed through the training","metadata":{"papermill":{"duration":0.266428,"end_time":"2021-02-17T22:16:09.710208","exception":false,"start_time":"2021-02-17T22:16:09.44378","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"**11. Model Testing and Validation**","metadata":{"papermill":{"duration":0.273231,"end_time":"2021-02-17T22:16:10.250914","exception":false,"start_time":"2021-02-17T22:16:09.977683","status":"completed"},"tags":[]}},{"cell_type":"code","source":"torch.save(predictor.state_dict(), outdir / \"predictor_last.pt\")\ndf = log_report.to_dataframe()\ndf.to_csv(outdir / \"log.csv\", index=False)\ndf","metadata":{"papermill":{"duration":0.618642,"end_time":"2021-02-17T22:16:11.698253","exception":false,"start_time":"2021-02-17T22:16:11.079611","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:43:06.917075Z","iopub.status.idle":"2024-12-29T23:43:06.917487Z","shell.execute_reply":"2024-12-29T23:43:06.917275Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**12. Results**\n\nWe can submit **14 prob 1 1 0 0** where the `prob` is the normal probability in the discussion [[Scoring bug] Improve your LB score by 0.053, just adding \"14 1 0 0 1 1\"](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/211971)!\n\nPropose new post processing (similar to [this](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/211971#1157809) by @pestipeti):\n\nHere `p` is the **normal probability**.\n\n1. `p < low_threshold`                   -> Do nothing, Keep det prediction.\n2. `low_threshold <= p < high_threshold` -> Just \"Add\" Normal prediction, **keep** detection prediction.\n3. `high_threshold <= p`                 -> Replace with Normal prediction with normal score 1.0, **remove** all detection predictoin.","metadata":{"papermill":{"duration":0.265518,"end_time":"2021-02-17T22:16:12.230515","exception":false,"start_time":"2021-02-17T22:16:11.964997","status":"completed"},"tags":[]}},{"cell_type":"code","source":"# --- Prediction ---\nprint(\"Training done! Start prediction...\")\n# valid data\nvalid_pred = classifier.predict_proba(valid_loader).cpu().numpy()\nvalid_pred_df = pd.DataFrame({\n    \"image_id\": [dataset_dicts[i][\"image_id\"] for i in valid_inds],\n    \"class0\": valid_pred[:, 0],\n    \"class1\": valid_pred[:, 1]\n})\nvalid_pred_df.to_csv(outdir/\"valid_pred.csv\", index=False)\n\n# test data\ntest_meta = pd.read_csv(inputdir / \"vinbigdata-testmeta\" / \"test_meta.csv\")\ndataset_dicts_test = get_vinbigdata_dicts_test(imgdir, test_meta, debug=debug)\ntest_dataset = VinbigdataTwoClassDataset(dataset_dicts_test, train=False)\ntest_loader = DataLoader(\n    test_dataset,\n    batch_size=flags.valid_batchsize,\n    num_workers=flags.num_workers,\n    shuffle=False,\n    pin_memory=True,\n)\ntest_pred = classifier.predict_proba(test_loader).cpu().numpy()\ntest_pred_df = pd.DataFrame({\n    \"image_id\": [d[\"image_id\"] for d in dataset_dicts_test],\n    \"class0\": test_pred[:, 0],\n    \"class1\": test_pred[:, 1]\n})\ntest_pred_df.to_csv(outdir/\"test_pred.csv\", index=False)","metadata":{"papermill":{"duration":21.336725,"end_time":"2021-02-17T22:16:33.832623","exception":false,"start_time":"2021-02-17T22:16:12.495898","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:43:06.918468Z","iopub.status.idle":"2024-12-29T23:43:06.918916Z","shell.execute_reply":"2024-12-29T23:43:06.918674Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# --- Test dataset prediction result ---\ntest_pred_df","metadata":{"papermill":{"duration":0.289321,"end_time":"2021-02-17T22:16:34.394933","exception":false,"start_time":"2021-02-17T22:16:34.105612","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:43:06.919748Z","iopub.status.idle":"2024-12-29T23:43:06.920197Z","shell.execute_reply":"2024-12-29T23:43:06.919976Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"sns.distplot(valid_pred_df[\"class0\"].values, color='green', label='valid pred')\nsns.distplot(test_pred_df[\"class0\"].values, color='orange', label='test pred')\nplt.title(\"Prediction results histogram\")\nplt.xlim([0., 1.])\nplt.legend()","metadata":{"papermill":{"duration":0.54479,"end_time":"2021-02-17T22:16:35.219098","exception":false,"start_time":"2021-02-17T22:16:34.674308","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:43:06.921234Z","iopub.status.idle":"2024-12-29T23:43:06.921661Z","shell.execute_reply":"2024-12-29T23:43:06.921444Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# pred_2class = pd.read_csv(inputdir/\"vinbigdata-2class-prediction/2-cls test pred.csv\")  # LB 0.230\n# low_threshold = 0.0\n# high_threshold = 0.95\n\npred_2class = pd.read_csv(inputdir/\"vinbigdata2classpred/test_pred.csv\")\nlow_threshold = 0.0\nhigh_threshold = 0.976\npred_2class","metadata":{"papermill":{"duration":0.303469,"end_time":"2021-02-17T22:16:36.359934","exception":false,"start_time":"2021-02-17T22:16:36.056465","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:43:06.922615Z","iopub.status.idle":"2024-12-29T23:43:06.923055Z","shell.execute_reply":"2024-12-29T23:43:06.92281Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"NORMAL = \"14 1 0 0 1 1\"\n\npred_det_df = pd.read_csv(inputdir/\"public-kernel-vinbigdata-detectron2-prediction-v9/results/20210110_train_all_500k_512/submission.csv\")  # You can load from another submission.csv here too.\nn_normal_before = len(pred_det_df.query(\"PredictionString == @NORMAL\"))\nmerged_df = pd.merge(pred_det_df, pred_2class, on=\"image_id\", how=\"left\")\n\n# 1. p < low_threshold                   -> \"Keep\": Do nothing, Keep det prediction.\n# 2. low_threshold <= p < high_threshold -> \"Add\": Just \"Add\" Normal prediction\n# 3. high_threshold <= p                 -> \"Replace\": Replace with Normal prediction\n\nif \"target\" in merged_df.columns:\n    merged_df[\"class0\"] = 1 - merged_df[\"target\"]\n\nc0, c1, c2 = 0, 0, 0\nfor i in range(len(merged_df)):\n    p0 = merged_df.loc[i, \"class0\"]\n    if p0 < low_threshold:\n        # Keep, do nothing.\n        c0 += 1\n    elif low_threshold <= p0 and p0 < high_threshold:\n        # Add, keep \"det\" preds and add normal pred.\n        merged_df.loc[i, \"PredictionString\"] += f\" 14 {p0} 0 0 1 1\"\n        c1 += 1\n    else:\n        # Replace, remove all \"det\" preds.\n        merged_df.loc[i, \"PredictionString\"] = NORMAL\n        c2 += 1\n\nn_normal_after = len(merged_df.query(\"PredictionString == @NORMAL\"))\nprint(\n    f\"n_normal: {n_normal_before} -> {n_normal_after} with threshold {low_threshold} & {high_threshold}\"\n)\nprint(f\"Keep {c0} Add {c1} Replace {c2}\")\nsubmission_filepath = str(outdir / \"submission.csv\")\nsubmission_df = merged_df[[\"image_id\", \"PredictionString\"]]\nsubmission_df.to_csv(submission_filepath, index=False)\nprint(f\"Saved to {submission_filepath}\")","metadata":{"_kg_hide-input":true,"papermill":{"duration":2.051364,"end_time":"2021-02-17T22:16:38.683617","exception":false,"start_time":"2021-02-17T22:16:36.632253","status":"completed"},"tags":[],"trusted":true,"execution":{"iopub.status.busy":"2024-12-29T23:43:06.924Z","iopub.status.idle":"2024-12-29T23:43:06.924381Z","shell.execute_reply":"2024-12-29T23:43:06.924192Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**13. Discussion**\n\n(1) General\n - The baseline submission of score is 0.141\n - Augmentation is very important hyperparameter to improve model's performance, and \n   you want to experiment with various configurations.\n   updated `Transform` function is written to **support all the augmentations implemented in \n   albumentations**.\n   You can specify `aug_kwargs` from external configuration inside `Configuration`\n - **Exponential Moving Average of model weights** is calculated by `EMA` class during \n   training, together with showing its validation loss. We can usually obtrain more stable \n   models with EMA\n - Obtrained training history results really easily by just accessing `LogReport` class, which \n   is useful for managing a lot of experiments during kaggle competitions.\n - Now, the score improved by about **0.8, which is significant**!\n\n(2) Details\n - Tried to change several `low_threshold` value and lower `low_threshold` achieved better \n   results. So it may be okay to set `low_threshold=0.0` which means **always add \"No finding\"    prediction with the predicted probability.**<br/>\n - Noticed that setting `high_threshold` value less than 1 is important, which means we have a \n   benefit to **remove abnormality predictions** for the images which is highly likely to be  \n   normal.<br/>\n - Curently used [training kernel](https://www.kaggle.com/corochann/vinbigdata-detectron2-\n   train) with abnormal image during training and model tend to produce more abnormal boxes. \n - Better to include normal images for training to learn where there is **no** abnormality. \n   <br/>\n - Try to **include \"No finding\" class during detection training** (by adding virtual \"No \n   finding\" boxes, or by adding global classifier together with the detection).","metadata":{"papermill":{"duration":0.330672,"end_time":"2021-02-17T22:16:39.288624","exception":false,"start_time":"2021-02-17T22:16:38.957952","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"That's all!\n\n<h3 style=\"color:red\">If this kernel helps you, please upvote to keep me motivated 😁<br>Thanks!</h3>","metadata":{"papermill":{"duration":0.304096,"end_time":"2021-02-17T22:16:40.047354","exception":false,"start_time":"2021-02-17T22:16:39.743258","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"14.Future Improvement\n\n(1) Changing training configurations by just changing Flags (flags_dict) configuration.\n\nData\nimgdir_name: You can use different preprocessed image introduced in [Multiple preprocessed datasets: 256/512/1024px, PNG and JPG, modified and original ratio]\nModel\nmodel_name: You can try various kinds of models timm library support, by just changing model_name.\nTraining\nepoch, batch_size, scheduler_type etc: Try changing these hyperparamters, to see the difference!\nAugmentation: Please modify Transform class to add your augmentation, it's easy to support more augmentations with albumentations library.\n(2) Checking training loss/training accuracy: If it is almost same with validation loss/accuracy and it is not accurate enough, model's representation power may be not enough, or data augmentation is too strong. You can try more deeper models, decrease data augmentation or using more rich data (high-resolution image).\n\n(3) Checking training loss/validation loss difference: If validation loss is very high compared to training loss, it is a sign of overfitting. Try using smaller models, increase data augmentation or apply regularization (dropout etc).","metadata":{"papermill":{"duration":0.269808,"end_time":"2021-02-17T22:16:40.595416","exception":false,"start_time":"2021-02-17T22:16:40.325608","status":"completed"},"tags":[]}},{"cell_type":"markdown","source":"15. Reference\n\n(1) Classifier Pipeline from the kernel of VinBigData 2-class classifier complete pipeline\n\n(2) Trained Detection Model from the Kernel of VinBigData detectron2 train\n\n(3) Training 2 Class Classifier from the Kernel of VinBigData 🌟2 Class Filter🌟\n\n(4) VBD Chest X-ray Abnormalities Detection | EDA📊🔴\n\n(5) Deep learning Pytorch in Computer Vision!\n\n(6) Image Augmentation Library developed by famous Kagglers\n\n(7) Pytorch-Image-Models with Petrained Weights\n\n(8) Pytorch-Ignite Traning/Evaluation Abstraction Framework\n\n(9) Pytorch Feature-Rich Functionality on Ignite","metadata":{}},{"cell_type":"markdown","source":"**16. Appendices**\n\n- There are several other X-ray images in this research field.<br/>\n- The paper of this competition dataset [\"VinDr-CXR: An open dataset of chest X-rays with radiologist's annotations”.](https://arxiv.org/pdf/2012.15029.pdf)\nsummarizes the existing public datasets.\n![image.png](attachment:image.png)","metadata":{"papermill":{"duration":0.270941,"end_time":"2021-02-17T22:16:41.14019","exception":false,"start_time":"2021-02-17T22:16:40.869249","status":"completed"},"tags":[]},"attachments":{"image.png":{"image/png":"iVBORw0KGgoAAAANSUhEUgAAAp4AAADGCAYAAAB7NIAgAAAgAElEQVR4AeydBVhUWRvH3fDbUtd2jTXWtSUMSkBAEBXsRrFr7dZV19a1u9fuQBEDLEJQsejGxI5VUAxi4vc9d4YYmAERFYf18jzzzJ0bJ/7ve8753feceymA+CcqICogKiAqICogKiAqICogKpAHChTIgzzELEQFRAVEBUQFRAVEBUQFRAVEBRDBU3QCUQFRAVEBUQFRAVEBUQFRgTxRQATPPJFZzERUQFRAVEBUQFRAVEBUQFRABE/RB0QFRAVEBUQFRAVEBUQFRAXyRAERPPNEZjETUQFRAVEBUQFRAVEBUQFRARE8v2AfSIjZzZDWdtjZafq0ZMAC3yzVkcqCWdm3FW3bzMU3Qa52nkz+gN1j29K65XhcH0rVjud0x5voo6xe4kxkYk6vSD9PLn/InnFtadNqMqef5b4M6Sl+nq3Ee/sZ1X6ORp0/dYnk8ud4rRpMx5bjPsiO6eWUcv3AGPpM90rflYOtz6mBpuK9uryEPoN3EKPBreTyR+yb0IlFJ14jk91h5+guLHdP0JRMjve9jdzIH92XE5iLdpDjTPLgxOS4Cyz9oz0Oow7xVPbhGb64doTZg9pgVq8Odeoa0bLHLFzD4xUJy3nNuRW96dRzNWEZ5E/g4pq+dOq1RrE/6fEhRrexV+kH7WnVthuDxy3jzPW3Ggupbf6osZAfceeXVt+PKJ2YlAYFRPDUIMqXsutN2Ezq/VCQb7/9lm+/+ZqvvirAV199rfz9bUEMHVyylEIq8WTQb99TvEBHXF5rAs9oZpoU5ocCZmy4qWF0zjLl9AOJT7yYbFGaogXs2Rennkf6mZq3Eu6uwuKnQvz008/0WfeQjzDOac7oE+9NuLubIXbTOacB8D9x1iTELMH0x/qM3XaOO68/Rm5SovcMo/tk93ckloT3zCa0HHuWJOBzaqCpoC+O9qZmjamEStSPyuQ3mWtekn7r4pDJY9g6tC2LT2UgH/WL3rHnle9YdIr3w+OdPpBRt3ckm8vDuc1DSvhSKyrWHcxen5toRrqcFyk+ZCmWpYtRv9UYlmzaz97tyxjbriZlirdm9zWlYRJiNtGiTFHaTg9U+JGQ+svA2RgU/pWhO+8r+oSE24sx/u5X2k9cwrJlyxSfxfMm0KlBCX4p1YeTz9T7npz7Y261yrkOOT/zfcqS8dyc1zfnpRHP/HIVEMHzy7V9hpq/Oj+aKt98RVXduVxX4UTZ2/v4nz2Oi4sbF0Ifp3XequDp/OIxQV5u+AQ9TBtMZHJN4Ckj/lE4PqdPciHkUVpaGQoCJD7eQx/DGpT68Wu+KlCAIrkCTymhSy2p9PsY/h5QhTqW63ikgTzlybHcuX6Lp69UKp25QCm/pYlPuHXjPi81wEYWlwASYu/e4mGc5ovk0uc8ePgq68uzOCKXxnH31vuWJYvEFLtlvHl6i5v3XqBa0jeBU2lQpBcn36oPvumpZV/H9POy2koi7uFNbj+IV8k7idOj6mDW61SWfiKklhh3hxs3n/BGzXwSYh885IVqZbLKHpBLY3l4P04l/2xOBtLBU0Lc/Zvcey7gsfJPFTxT96l+y+WveXTnIa/Vyqx6lrCdxLOYmzx6KUUTeErfPuVW1HXuPUtQuanSrJvmc5X5yZNf8ODmLR5rdGxNttWcR+LLB9y69ZBXWWou4cLUhhh1dUE1cPuudpUce58nyiBmmkAy+TUWNCmFcZeDPFFp1zJZFLPNS2A12DvFb6RE7+hIhR/NWB2YiDQ5kGkmJTHpejAt4iqAp8n/6rM4MDktfWFDCaRlGHso54is7o+atcpNv5Ac94BHsVmJK+H5/Ucq5c+53TT7RlblVslClsDjWzey7tveo29VSVXc/AIUEMHzCzByTqqoCTxfBCzBusJ3CvgrUECIhv6AUbst3JJCKnh+940eVha/8M1XBSjw1bdUazCRs4+lZAZPYfrRebwpxb/9itS09JrO42qsOtAkx51jzZQ/mTixD8alv80VeMrkEcwwLk7LScE8OzOEKt9as/W2cqQXpuBXtqpG7wXLaFurIhUrFqdwobqM3XNHZQBPV02WHMnGfgaU+KEQxYr+wM8lGzF53y3e/LudNmX1mOudPow+PdKfmhX6czpeSszJKVhW/pkixUtQ5IcyNO27k+sJIJPfZWmLGvRbsogWv/6EgYMzd3d2omb18VxJSUrOC/b1q45Zz6P8e3MJTYo5cOS1HKHsLn81pWKRwhQvUZiiJQ0Zuy2St/Io5lqWo9vCmwi1THy4jqbFi2I76rxiAJa8PYxjpd+Z6KIetnwVvYOeDcvwY6FiFP3he6oZDOfYTQkJd9ZhV/5nCn7zI+V/acN2wfAZ/mSa65go4/bB3uhU7c/JJ8I1Mq7vdkC3zih8YiWELm2KQft9ipReX9uOo34pChX7hVKFf6KW0WTOPkskYElzyhQqSMEfS2LQbjv3b6RrkPhkAy3LdWf+ik7U+LUSFYr/QMXq/TmaMu8dF7GFbrol+Onn4pQq0YCBU3thrDeVoExjtiT5GH0qWzF4ajuqlShKkZ/+R7mqndly+aXihsFvngWGXZzSICn5xVZaF2/G1hipAjwrV3NgZOcaFCnyMz8VLETDlou5+lzwfdWI5zXmNanIiJ2C7gmE7htCg9KFKVqqOMV+rknvxZdRt4iMJ1eW0bp6Eb776Wd+LvQb9j1tqZES8RR84+yi1lQp8hMly5Wl6A8/Y9h6NeEJEjXdnmdxboSCpZII3tGXuiULUeyX0hT5sSytJ5wmVgFyWdlWPY9/JVH806ceJX4qRpnShSleypx5J55laktSota3o2yRgnz7Qwnqt1hLTGKExnal8N8Ha2j+a08WLm1Nhe/LMPZgRvh7e20eDb+tz+KAjLAo+NqTYFecz0TyJsVXhfa2qVNlahjPZddsMypW6MdphV8qT8gKPBP/XU+THyoxRUO0OuHm0rQ2mZU/HolR9+PnZKGrhn7BsKsLL1wHUafxYCa2r0mJoj/z0/+KYNx+NUGCiwrtal1LGnZfzLzWlSn7fS/FPo39TqK63bLyDU1+pNr+IJHgnf3RL/UjhYoV5cfvfqHFYCfuSlH0T+/Tt6aYSPz6whQQwfMLM3hW1c0MnnLi2elQjm+/Kk/XOYc4fWoH/fR/4rsC+iz2T04DzwIFvqZy/UGs2raOkdbl+OargtiOvkhCpojnE9f+lPvma2pZzsLtkjcbhzTgp68KYj3MJ21gz1w2qeQSo+v8mCvwVCwj+J8Jq8IkSJJP069yUbrMV0KZsP50iU0xylXpy7HbyQgD+Zmx+lQ3WMxdleiJsjxSQlbYULZMB/aEv0XQ5dJyeyr+2Jo99+6zvs0vNBmmhDs5sex0+BXzPqeJ+3cP7cpWoPeKEAVYvLqxh67VitNpXiTJ8jssaFKcMpXastrzFi+TZCQ+3Ypd0RrMSoFYycsDdCj9G5NPvObtjQU0+qYdzq+l3N3XlUolW7M1WAgBJRC6tyfVf7BgfcQbzk2ph07TzTwDnjg5ULJQIarpKSPY8T6jqPF9R5wzgb5wgzDfqhQNW23m2luQvg5kfosK6DXboogQZxfxTMymjhL5PbZ2rYqZowv37+2kbcU6TD0ZC0gJWdgYPbtdCGvwdnevgEH7PTwSbmbivRmuW5yuC24ron2qEc90DeQkPl6DxQ8/Yu7oxD2JcJ0Pw+oWofPft0iWBTK5YUma9D/OYym8uXMAh9+/p2KViQSogacLDqW+o2KtEXjcT0aWdJOdfetQufJ4riZJuDLTmHqt96VF8ZNjN2L7P0v+ua0Ez2Jf/4jtsBM8lsDr2wdwrFkYy0FevM0AntHMalSKP7a+5k34PAwL12a88z2kSIlxG0TN/zVhc6alKNJETwZV+xmLPs7cTYI3947SX6cQZYoop9oT//2HpoWqM+HoEwXcvb63lda/VGaSmzCdnzFSld25koRDdCr1K0N33FfcrDw9Pw6d/1mx8ZaU7GwrzZTH88OOVCzRlcP3JIq2dHpcPWoZL+OeWltKj3gmCH6QRbvad09K4v0VmP9UiAYtF3H+1guSMqUV6+zIr9/3ekckPr1HSXy0lw6/fs9339Zg0vHn6QdSIpsm31Vj4GoXjh07pvg471vHxFY10LdcRnj6fWXadTn1R1kmrbLTVVO/IETWi3/zIzaD3VT8rAg2Iy+QhIzolTYU/v53es734k5c0nvZLTvfyOxHqvVNuLEc858r0v+fCEXf/Tx4GU1/KUW/DfeRvlffmianuPGFKSCC5xdm8Kyqmxk8Bai5H36ZSxf9CPA/y6Gt02lR+TsKflWNGZ6JaeD5vwK6LLisjDok3F2O2fffUKH8GC4mRKWt8Vx/8zXOA6rwzVfF6TLXFXd3d04dnYjBd1/za4WxaVG+zGXLPXgKA1wDahotTXn4I4HjQ6pRvcFCRbRWCZ4lsJsYkDa1GuvkQLXa04jIBCdCtGShdQla/hmUdq5U4s3gaiUYsPEZ93Z3ouqvyjokx+2hXcnazDmXyMNdHfitygSuJkiQSIRPIoHzzalttoZH0hgWWBXHfmJgWppy+VM2dSiH9XAlxD7Y14Vqlcfhl0gaeB5885yN7X6h6ShfEhRpSkhKjmCGSQkclz0k/txoahXugdurBNyG18Zs6Agaf9+CnY8S8Ztlgo7NRv7NNIAnPliJxXcZI0fx50dT49s2HIiVkx14ZldHYfoz6fEBOleujJ7+rzQf6ZMS2VMFzzcc7FeFqg3H4uL3INMNSEaAUh34BPC0/E6fxX5Kv5Pzhh1dytHyz2BehsxA/xtbtj9Ijc5K8J9rQqWswLNkcQZs/DfN9RIfr8Xqhzr87fv2neBZoWB7nJ+nRuxl3FzfnMoVBN+/obLGUwmeg7a+wG+2CTWNV/AwxQYy2V08d+7m0r3UsiqLEe8zkmoFW7JXoHHFn4yYTS2pkAKecmk8jx88UwKxLIGHoetoVa5sSlQwo27ZnStJPE6P8iVoMeoAwQ8yrkHN3rYZ84h160+FQo2Yus+fxxkDkynlT/1KB8+32bSrgRtjleD5v3qKm9zUq1W/n+3uSIXig/B+55pX5VVJ/7rQ47fv+e4bXeZ4Zpy3V06p/8gvVWtSs6byU61KSQp/V5Nh26I0LvXIqT9KMoFndrpq6hcE8Cz/TRsOPM3oZ1WqTMRfIoCnNb/VmEpYSr+VXfpPZBntlp1vZA2eUq6takrVmtMIT+srEzk9qja6zbcRJ7/PEpuc9a2q9hS3vywFRPD8suydZW3VwVPGzWPjaFT+R77++ntKV9anVgV18Py5QCsOxCs7RUmyG73K/4+SXzng8iYdPNfdesb61sUVU+zCNLvqp+TX3XDNYv1gbsFTKvVlVK0f+brgD/z000/Kz3ff8kMBI1aGSVCCZ0kcl6Y/cBR7sJtG8JTKrjBOpxgD/kkfrBQw2qQEHWZdI/nFXiVs+iRwd1cnataZQYRESuB8U/73XVHKlitHOZWPfpOlREsE8CxJz5VPMtjj4f6uVKs0jstJT9jYoQKtJgcrwDR1kDvwKpwZRoX58eeyGdIsV64ijgsikSS60fvX6vzpfpUphrX40/0ykxtWYMi+KJbalqHbYvWlBG/8p6CrYkOhQAKMNi7YiDXRkmzAM/s6KtcJS/Cd3pCiX6Uvc1CNeCryuu/GX12MqFCkIIVL1aFl78Wcuy+MaBkHyVQNnF8rI55W39qwNQUuBfDc2VUJns9PDOD3ov3wVAGSZ7s7IAzUGiOepWsw0ys9pCWRnKZfxbKMPhivATz/yRDxTL0xSDXiS/chVPuhJ65vNIHnc44MqoqRw9FMgJ16dfq34IuVy43gYnqxiPcaRo2iyoinsGzl1JK+WOj9Rvnylalj3IiaJX7RDJ7Znivl5smZtDesSOGCP1Chlg3DFnjwWPou22a0jUx2D5eZXWhQsQj/+74M9Zr0Zc2ZB4ooanqthK108HzzjnYlRDwbf9OMnY9TgStjSvGeQ/nt65bs0/Dgzyv/LUz6c0caGAl67er5O783mMLKEbpUqjKScyoPK2qeak8icmNrfv1fV46l9G+qJcipP6qCZyLZ66qpXxDA8/fyo7ik4gupfnbyrVQBnnUar1fMcghtK7t+57o0o93ex49S63vodXLKzMo2lSUiMq6tsqF6vfnckAjgmbO+VVVPcfvLUkAEzy/L3lnWNjN4CtA3ps6P/PStBSsDXyIlln/altQY8Vx4VRl5Sry3ksY/CBHP0fhminju61WBggV+Y+qZ9MiKVCpBKs0UglMpYW7BM/78GKp904hFvjHcvXtX8bl9+yC9Khei1ZQQkhXTQSXpsexR2jq0rMBTJr/BHLPiCshMjT9JJVcYq1OcXqueICeOXd0rYTnqKKva/EqXeTeQIuP2hhaKjlh1WWTiiwfcefwaGXcU4Nl7lTApnv4niXei0y91mHR8PfYlLNlwXRlSSO30nV49ZHmzUhnKIgBa7IMY/n2FYppzV/eKmAwdiXVxB47GJ3BiRG30+o6kTUlTBUim56bcEgbdRimQmXrslf9kdL5pzq7H2UU8s6+jkFbCrQ00r6CHlXkFTLqmvj4nPeIplP3l0ye8SAR58lNCz25jqFkZGrZ3ygF4NmVbymu6VMEzzncstb9ug1PakgIpEcusqJwVeJYqxfDd6TcVyS+307JINaZ5vFYDT8EOxt+kT7VX/NERt1fpYPRwZweqZBnxjMX7Tz10bLYQlyK0XP4Ynz2b8Ex5AjtVfwEyf/9fJ1xU4Ojxns78mhLx/PdIb34tZssK7/sKiJXKrjJe71eN4JndufKkeJ4+iVNE9BL+jeT09tEYlyjD8L1x2fpv5puCxPinPI0VyCiJR5FebBxjzq8/OHD8Zbo2yrqlg+fbd7QrJXg2Z5cGsBTSkrx0olOZkvRZcy+tDSvzSODUqNpUrTuLaEWDlXFjnwMVfzBhhV8CklceDK71M437uqWsZVU+RKTp4aK3NxZi/K0Z62+khfZSTZQ2C5F+I6TZHzOCZ/ZtRlO/IIBnxW8zwq/gZ5XLjsA3URnxrGOxAWERi7DmM7t+J7PdsvONzOem9kGHXksIW2KZqW9L4tzkegjleKqIeOasb00TU9z44hQQwfOLM7nmCmcGTyHy07eiMDWlz+itrjj/MxDdQl/z7Ve/Z5hqF9Z4/mY4nPW7NzHO9le+/aogVkO8eaOyxnP9zWTu7etKia+/Qaf5fE5eucihhe0o/83X1G+9R236N7WEmsBTKgtimmkZShQxYJFv5gcLBPh6w/Gh1alhtJQ7qaSoSDCBE8NrUKX6VIKTlXflOQFPRZRmWkMqVhuBz79CglJuOPehasHGrEuZl396pDflf61OzaJ27EqZNn17bREmhWszweWBYmCUvvVjikkpWivAVzN4CmtN9/aqQqXqv6NnvSHtKXzVTj9osQUVqw7i9H1l5WIDZ2FUyIBlQUot7u3sQNFChalrvobHMnh2uAclCxfWGM0VZJHK/JioXwybP9x5LgMherW7d3VqGCzihrBGMpun2rOrY5IsksW2lbD/8xIvHuyhfYXfGLlfAP108BTWl85qVAz7if6KaWNhzaf7OF3qWm5IA89Gjq4KMErV4F0DfcJbN3pXKkGn2UGKNKXxvoxrWDjrNZ4lv6OmyUIiXgsPRsTiNc2UCkX7cvqlhJBFjfldZzbREsGv4vGZbkyxAhZpazyLf12MrvNDFPkkv7jABKPiNO57htdZrPGM8xpG1R/NWeGnfIvBy5DZGP5kwPLgjH4siT9Mt3LFaTf1CsKZ0jeBzGhcgl+K9MM9QUrMhhYq06synlyegWHh0ow+IMxzK6Nagm6JwoMs2ZwrgJVRwQYsuqp8vEn69hzDaxen9+rHZGfbVJhS5iFV6PRb7amEpDwlFes7jjrCjUvaUoHUFp0OnglC9DObdvUu8BQecPGda0bpEtYscX+QsmRFwh3PKRgWK4bD4muKiGvCve20LleUNlP906bMn3gOp/p3VRnrrFwjqzniCYn3lmP2Pz0WpSzpSK2F8J1Tf1TVSnjvQXa6KtZ4WpVE9YZUscbz66J0mhuc4me+TDQujrHjUV4p1nhaK4BPCZ7Zp69alnf5hqofKcqdts5cuBmdhu53Okw9/a+ib0t8cJBuVYvS6e8oJO+4qU9+eol9G/ZwJW0pjKqq4vaXooAInl+Kpd9Rz8zgKecVHjPN+Pkb4Sn0ryha1pp2VuX4tkAhOs4NT1/j+a0OjU1LpzzVXpA6jedw6Zn6U+0y+W12DqpHka+VT7UX+Oo7apn+iUc2HZBm8PRnol4hChaopVhLmblakgRhuvlnui2JyRQJQbEGsuo3dZnje1sxHZQz8ITkF95MtijHz6VqUr/B75QoVJEei5SwJOQveXuMHhW+V0B06gAgQNSF5a0oX6gYv9c3QqdCMeqazuZynDzLiKeQ1nPX/pT9piRDtqdHQ1UHOQFgF9hXpkjR36hvpEv5n8vTYfo5FA+5Kl5FtRarH79TPM0vxGmUDxB8h83Ii2kDb0bNZDw5NxPTMoUpW6MB9aqVoExZe/5JgZHswDOrOl6KS8J/aVNq1p6CvwJGhKfaO1P1F0eOPUpKe7hIiNDc2N+byt8XppKuCcb6v1K6lDnzTwsPf0jwn2dKkZJ6dJ54jEcqA5+wxtPq26wiTDJuu46jQamiVNJriG5VAxx7mFFVZR1cav0lyS44lK6IuV0DKpSvg17NMhQtYsDUQ8oo2tuIFViV/okKdS2xNKqFWa8/aFbGKg08K+t1wKFBeSrW1ad6mUJUN5yM96PMT7WnP1ykeLPDWCNKFC6PrmF9KhUthd3YM2kR0NRyCTc30Qf6U6PQT5SrWY9a5ctj2rcLhqWUU+0J93bTuUoxKulb0cRch3oWvWjXoDC/1RnKsfuJGXR7nM25R+7FsL1XdX4sVAF9ExN0fy1GLaMpnHsuVTz4lZX/ZrbNkwf7cKhaiGLl9DBupEeFn8vSZqIHcWqTGengKcRHs2tX7wZPkElj2DfGnNIFC1K0bHVqVSvNjwVL0WyQ8glrmfwWa9pU4HedGQSlT7QgPAToMrQOv5TuwfEHUsVrkzRFPCVvnelauhAtJlxON03KlmqbzN4fM/rxi/fsFwTwLFvOnGYNK1CxjuBnP/Gb3khOCo+QawDPrNqk0O9ktlt2vpHZj1Tbn3CDdnqmFaV/Kk2tBvWpWrww+k2XKW48UpcxZdW3vrr6JzW/UT4noCaquOOLUUAEzy/G1LmpaCL3Qzw5ccaPe/Fqo0hagrLEB1w5cYQzl2+T/eswJTyP8efMseN4B9xNe91JWkJavCGXxRF9wZUjx88R+TD1RS3vLvCrx+F4ux7D89JtRfTq3Ve8+wwhqvsg/Byux87gdyt9mvjdV2Z9RvKLG5w77sIJnzCeqL/fJ+sLgQ+tY/zdq5w4fIhjZ65yV6U6suSHXDlxCFffO1lAs3qx5LLHhPuGcPtBOD4nPQh98IrwZVbUNlub9t7G1KuU4KnLvIsvueN3kqOuF7j+LP19nMJ5Sf9G4HnsMCfO3+S1hiYge3sHX9cjnLl4M4fvDJXw/PZVzhw7wcXIZ2kPl6WWSfX77b/RnHc7wunL99TWhUpe3uS82zE8L9/ipRTePrzCcZfz3HkLmXXL7lwBVG5fPcXhQ0fwuhKjsm5PWZKsbJs5D9mbu1w6dRhnF3cCbuf8vbS5bVeqOsU/CMT9yEEOubgTeEdYyqJdf5m1EkqXla6ZSy6AZ42qk/B7dZ/LJ45w6nw0sRldNPMlit9ZpZ+5LNn5RuZzM2Yk5dmNCxx3ccU39JEiGpvxuPhLVCBrBUTwzFob8YiogKhAPlNAWH87Qb8sdqNdiHj4iGuXtuBYqzTdllxXe9glFTznX8w41Z3PqiwW9z+sQCp4BqovM/0P11qs2n9dARE8/+sWFusnKvCFKfAyfA8j25lQ67dK1NC1YuDME6Qsic2ghCT5JIN1mrDSXwTPDMKIP7RGgZenR2JiOjftdUlaUzCxIKICH6CACJ4fIJ54qaiAqICogKiAqICogKiAqEDOFRDBM+daiWeKCogKiAqICogKiAqICogKfIACInh+gHjipaICogKiAqICogKiAqICogI5VyADeAr/yrBbt27iR9RA9AHRB0QfEH1A9AHRB0QfEH3gg3zg+XPh9XgZ/zKA57Vr19i1a5f4ETUQfUD0AdEHRB8QfUD0AdEHRB/4IB94+1b4xxYZ/zKAZ8ZD4i9RAVEBUQFRAVEBUQFRAVEBUYGPp0AG8AwKCmLhwoXiR9RA9AHRB0QfEH1A9AHRB0QfEH3gg3zg1Sv1fyiRATyjo6MRP1+mBr6+vujo6Ij217I2MGnSJNEmWmYTHx8f9PT0RLtomV3Gjx8v2kTLbOLp6SnaRMtskteM9+LFC7VQabbgGRl5nD/t6qGjq4uuXgMsW/Rj6WG/LB0p6po/Hsfd8Yv8MHiLDPDA7cxVIr9wg+Wlg3h4eFCpUqUsbfsxyxJyZjZ2ejroCn6lq4eBaWvGrnYnQoO9BZ9a0deBFWcj8qRsH7OeHyOt1q1b51m9o6IusWvRak6EnWTdgh1cVGnHwrF/hjXFbtAWPFb1ZsiSS+8sV7DbdHr8sYOAK8vp47CUSyrpfQxtPlcaJ06coHr16u+s/8cqX9iVPSxd6UrQ6XUs23Y+T/rF8KC19O+4gPP5yGbNmzfPtU1CLy+nZ+tZeER82Nj1Pja/tK4f/f/2zrU9NfWN2dkt+OR0HP/YTqiGfla13JrSVT3+PttHjhzJtU3eJ5/cnhuwfzQd/9jxTk1ymr4m/bO2czgu07swcUvIOzXSlG5Oy/S5z1OjTiBb8IwId6JfTWuWXIkiKtKfU9vHYvabNYvdwzUKFRG+m14G3dkV8mGNN2BvP2w6bflozvC5hc8P+ecleAYdH4Fh/cn4KAa1CC4dG49JmXZsDIxS86uoa1eYbmPCjNOafS4/aPshZcxL8ED+DAoAACAASURBVIyI2IiD+UhcLszA3mY+fioDVET4NrrrK9t28AVXTp57tz0CDg3GsuUqgsPO43b8/H+mPec1ePpv7k6bwUdwn9WMbnOuqLWRD/GvrK4N959Ls/rj8cxDEMuqLDnd/0HgeWEaFrWH4Rb+YWNXTssqnHf+72bYjzqj8YY7J+lo6huzs1uQ82As7Ve9sx1qSjcn5dF0jraDp//mrhjavVsTTXXTtE+T/lnbOZw9g43osyz4nW1aU7qa8tfGfR8EnsoKCULp0XKsByHeK+lq0gAjk4bo67RkoWswLvNaU7V4ZZoN28rlTMcXuYZz1XkStg0aYGigg3nL+YpO7cqhqdgZNcCwfl0at5mHW5ArM1pVp0zFRkzaHPBOg2ij0PmxTJ8PPKOJCF5P+8otWecfRWZ/OBmaDp4RQQf5s6UxDYzqo1+/A8uOh6j5oeBnEcEHGde8IQ2MDNDXb8fy0+FoujbVTuH+q3GwHMvx8GgiwvczxKoPm32d1PKKijzLEsdGNDAyoaGOHg7T3QgJWE5H2644mNZj+Jp3dyCpeeb0O6/AM+TkIno216Vc2fo0bVydClUtmLTpakr7C+fQXDuqFP8NuxHbObO4A73m+HJ5lQPNuw6kXXNL9GsYMW7rVUI8l9PZsBa6BkZYWDfAyH4VAVcX08VuHj5BK+hs40D/Ds2xqFeTpgO24xflw5oBFug3NMG8eRvsrYfiFBrInkktaNDACAOd+vSYe0ar+oG8A88Iji7qS1PdslTWb4Zp9bLomo5k11XlDVrUNXWdNPloaMhKOpq3plsbGxpWr0PLPr1oZWNG3aqNmHYwhKDjE7Br1g57MyPq19aj818uBKaBZwB7p7bCqIER9es2pOdcV0I15JtTf/6U530M8HQND2RxF0taO7amiWEN6jXvTY82NpjWqYbdKGc06RsRHYnHpgGY6TWkUaPmtG5uy8TdIWp92alMAZlUIAnL1K8tPnaK6fbN+PuUcHMXzoGxTemz0J09anZI7xtTdU0FFI8wL7W+yt95MPp6VrQwNaJeTR06/OlMcHS0WjlT+9zpZ65+cDvMj+AZFeXLxuHW1KtvSENdMwav8CQ86gL/DLWhgYExBvVtmLzjqsaxJ1V/1Ru2VDsHZWIigZn2DG5AfUs7GhnVo06dZsw5FEjUNfU2l94e1dt8qu219fsjgGc0F+bZYtp9DwEHZjFltReR0REcG29Cu0nnCVOJeGY+3nayF9sH6NFh2jkios6xZeZCnAPcmWBhxvTjYQh3WSs769BrqT9ixDPv7rpTnTWvwbN2yWqYNG5M48am1K9ahdZ/HiMk0kPNH3os90qJeIZw4q8mtBzlSnh0NBf+6YJJi5WcOzBTzQ/9DgzA0GwWvpGReO2YxfIDfhqvDUyJ6EVFXWBGUyMmOocRsLsPjVut5OBfVmp5+YXtZ+aYdYpIbfCZcZgbT8Pn6lwsilmzxPPTLAXIK/AU/ODKus50GH2KE1Oa0WdJYAbYi4jYSreGfdgfGonnzCaK9n5xYTN0bJZyOTqai6vaYtFjG6u61aXTLGH6MJTD4wzQab6KgMuzsDX+i7OBC7AuYc/ai1GEB62kde2+bN0zADOr+VyIjMbfeQi6ZbuxM2gvvWs3YdG5CMIubGfOfKcMZUn12c/1nXfgKfQDQSzvaMu00+5MsGnHBpVZgYjwfWo6RWjy0cAFNK3gwK6AKAJ29ECv4V/4REZyYoo5rcd5IUTC6pbvxi6/KCL8N9OlVgsWuc9RRDzd3KdgbTiOU8JNmd96OtRow5or2mmfjwOe/sy2rcagjQEIUf6uVUz52zOCkLOTsTCYhCZ9z4fuo0+9Ziz3jiAy7CD9a//GgC2uan2ZMLap+qwAJHajTnI8U79m3GI5B/40w370acLCnehfvwXznSep2WF9QNbgeSZYva/yOPgHdct3Z4/CzpvoUsOGxedOq5Uztc+dcnK3mn+plj8n2/kRPIOOj8TUcJJi2UWo7yJa1OrN5n1/YNZ4jmLpSYDLJLoO2sJFDWNPaNoNW/oYngqeVzMxU9vJZ9k1uC5GnbcREB2N75buGJj/jZeHeptb5aNsj2dC1Nt8TuzwOc/5COAZwdHxRjQd4kqo90ZGdbbDtoUdTXTK0XqiTwbw1HTcz20BXS3qoVvfgu6jNuIZ/A/tKlXFpFlzhE7DupE+PWZ7i+CpMsWZVw6T1+DZUH8Ux86d5/x5H45u7IWB3mhcNfhD97nHFeA5/UwAq7v8TnWjZgpfadbUBFO7eXh4qftheJgrcxwt0ddpgE37MWzzCtR4bfr6tUi85jen2R8HWONoyMANlzSe7xPuxdpxnWlh24IWTXWo1eAvzl6dSzO90Zz+RFOSeQOeEXjt/pvBzWph3HEMDsY1adZ/Kcf80ztPzeDZnGbDXBVThf6bHbB0XMk4s0bMPKOE8IA9fTAVIp4q4NlcbzSnIqIRpvU76/dhzSwbmg9zU6QREb6WtjUc2RMagstCR8z1dTBs3JHJG7wyDNh51SayyievwDMq0odtfw/HpkY9uoxzxKhaY0YsOUzqDVN0tLpOQkROzUcDF9BCfzRnIqIJcBqEZes1iulW77lNaTnGQwGejWwWK6JfQhR1vn09hu6YqQDPo/90oUplY5o1F/poG0x127DMO1Ar7fPxwNOYqW7hitmPfvU7sik4ivArc7BtOFER8VTT98pcbNP6gBCWtdOl76bVamOb4xwP9k/vgE0TazqO2oWnAJ6jXVmpoV9z95iMlcEEju4bgFXLVZzdpG6Hpd4X1ZYhpUbchIhn5nKeOfgHjayXpNl5np0+w/euVSunap/7oe0wP4LnlZVtseq9X9FGIiPdGW3UhIlTm2L7x7EMyyI0MU524BmUiZla/enJrsH16b5AeZMfHrIEu5qD2LGhs1qbm39idsrSF/U2n1U/pS37Pxg8wy5uxqG2LhOcAtjYU4/Os72JiI7AdaIpbVXBMzRU7XibPz1xWTObbWfCCfN3Y4KVDgO37uUPQ3tWXRSmjkI5tmY6/7iFi+D5BYBn+hpPYXp7Pe2rdmJT0CE1f1h/Qtm5Tj8dzL7hjeg+76ICQoJOrmHWyiNs6KHuh0En1jBvgzvhkX64TG2CWdetatfOXnkiYydyaR7W9Uww0+vJ3tBwjedf3dkTI8t5nI+IJthzAo2NUsDzE66FyxvwjOTKmZ0MMjZg0Or5dKxnw19bXTgf+i7wbEGLYUodBfBs4riZhe10GLheWCITideCZuhninimAlAqeG5a1xnTthsIio4m5OwUGgmRtyBXVs/ciFdEBJdO/oVV9R5fJnhG+eG2exim+n1ZsbALJpYT2Onik7ZGLzLSTU2nAE0+mgPw1K+hXN8YGXmUwfUaM81VOdC5Og/DqukiRVQ7MvwEq6eu5mSodtonL8BTk77eoRvoWKMz2wKiiIzyZLzJ7/TbonlsU4WB1Ijnnkz9mtA3hUd5McmiEfb2xgzeGEiQy3A1O5yOyDrieXR7D7W+SgBP/ZrDORERjcLODSyZc/Zgln3uX6eOqvmXavlzsp0fwTNgXz+EG7Er0cIyMGEGoCPL1nfFrM16xU1fqM8Khgxfz8oeumoMlBV4CjcYmccqgYl2DdbBOgVohUirmcl0zhxWb3NuV5QRT3ctbXvZ+UKuwLNvjZJUqquLrk4datcyod/fxwmOjuDE3GbomXSlb6+O2JtWR998FsdDD9Jftw5dZu7jSKbjeuYzWD/bltp6Lenm2BJj3c5sOB/KySXt0dNvQcf2ZtSvN4h9V6MIOjwE/RotWbT/46+Zy06gL/lYXkc8M4LnXnpWa8CU4yFq/rDHL71zDTm7mFY69bHr1A7T2oaM2XZJox8ePj0H6+r1aNW9O3YG9ei37DyZrx27LXX9ohKuhHU9s6xL0qin8k5X0/khXnOwrmWKQ7+edGhtSs1K1sw+oIwMqa7p+Zh+lDfgKQxEh/jDshfb/VbQ3myC2kMlmiOeGcHTuvsuLh4ag1H1hrTqbIeZSR3lGk+ViGdm8NwT4MI4Cz3M2nWhjb0xOtX6sD/Ynem2tTCw745DK0Mat1v5RYKn4EdBh4fRymEbF1a1p93ojA+iREZ6qOmkyUfnHJzzzohnnQo1MbPuQFvzWph1Wc8FIZJffzzuYR783U4Pg+adaGNWF5t+O7gaqZ32yQvw1KTvwqMBOE22Qt+oHZ3b2mFUvQ4jdgWq9WXC2KbaN6ROwQZ6ZezXlH1TJKdnWlC+bF8Ohgjt01PNDn4aHrxMjXi6nZmt1ldNXzaQBtXrYW7dkbbmdbHssZmr0RFq5Uztc6eeOqXmX6rlz8m29oNnF8qV+E355h5dXQwMh7Ar2I3pzfUwbtmVNo10aDnmIMHhRxlvJfRT3WhpqIfjAndcMzGOwEBHLysBUXU8SL3BOJ7pfD3z6SwYZEAtfVOadmyPaZ16DN94WaOtL6dM4buHaWfby84X3hs8s0tMiFBeOHWIY2euEBbmi5vzGcVrlAK8j3LI9SIRGo9HcsXrCAed3LiostA64IIbzs5nuJL2RGEQXkecOOn7adbNZV+v9CjPl3ReXoLnu3RV94d0m0QFn8f10GE8LqU+Va3ZD8P9PTnidJCT59NfVaF+rUq6UT5MtzFkwqGwtMFB0/nBl07i7OLO1fBQLrgdxuNKZNr576pXbo7nFXhGRV3F/cQ5gsMucPL0pVy/4kWoY6ifO0dd3PFLa8/pOmfWIOz8ThavcuLkUVe8vGbQzHgyZyOjiYryw+OIE86uF9IifJmv/Vy/82qqXahfuJ87p31CCPY9gcdFdV/TpNP7+qjyaeeVXDx3nKOuvhr0DuLciUMcOX1Zsb5aKJemfD+XPVLz/RDwTE0jJ9+Z9XW/7M32Bas5ePIobj7uTLG2YI57ynITtbFNc1vQ1Neol0XdDurnpKefuZzKvioS/3PHOXbqUgY7Z9XnfqidtR08s9IvKjIQ7+OHOe7pnzYzJrzZx/PoIVy9U9e/ax57skozK2ZS5nUE94uhKmNJ1rb+UJtkXb503/mY53xk8Pw0hfyYFRbTyrmNtAk889pukVEnmd5WFxP7hZz9RGs1c1unvALP3JbvQ6+LDDnE+BZGNLK2wcKoMSPX+X4Q9H5oeXJyfV6CZ07K86HnKMHz471S5kPLk9vr8wo81csXxN7JdhgZW2NtYUSLPhv+M++sVa9rzscU4dr8Cp4fWm/x+nQ/eSd4Cv9lYN26deLnC9Tg77//pnLlyqLttcz2ZmZmX4ZN1q7NN/WcOXMmv//+e74p7zv79LWrWbVqTb6vj4mJyWeuw1rWaln/8U7bf+LyTps27TPbROSZz+0DCQkJauyZ4QXybm5utGzZMs8+bWxHY/D7+LT82jQdg2HZ2dja510Z8rK+2pyXra0tNWvWTLOFNpf1Sypb/fr1MTc3F+2Sh/3Su/xLaCu1atX6D9jEniamxlg2f1d/a09TcxMaN7XX6jrXq1ePxo0ba3UZ3+Vb/7XjNjY21K5dW7SJFvVfee1jz549yx481Y5+yh0ScB4op3NzWVou3hPkdDGWcTd9V9oxcePTKnD79m3Fv8z8tLmIqb+vAo6OjuzYseN9LxPP/4QKREVFKf5l5ifMIk+STopz44DzI1K7WzmvuREcpZK3hMREqeK3TH4Tpx3evFU5qm2bnTp1Yv/+/dpWrC+6PMHBwejo6HzRGoiVV1cgQ8RT/XDqniS853dhnlsCcvkTzizpTXOrJjSxsqLbiD1EJ4BMfo3Vjo0wNDLCyMgIE7NmDJgs/J/WjXQ3M8bIyBCDhgaK442MO7MpMBkSYIqDstt7clDO/K1yFnUVwTNV9bz8zq/gKecFAU7zGDN4GNOWuXFHMTJKuOW5hkmjxrFsfxhvUoRMuO/BysmjmTTXiahXyp2a9uWl7u/KKz+DZ1KsO9t2hb+rivnuuDaB57OwA8wZO5iRE1fifiMjFsb5b2Lprmsky0LZNKY/ffr0Sfn0Y96+W8Sf3siecIlC/7hoDw7sPcjuzS6K3/KkBN4mB+PiFEZSUhKQhPfGTQQlaq+5tAs8k/BZ2Aab9quJVrJ7joRLeu7E2MH7ePTKhYn9dvAw9a5A5WqZLIKV/aZzIUGetldOPE7jBuN09z0yS7v6023kZ/CUSi4zu5UpA5eH8z6q5taGkjdHsrT5p7PQ50k5h+CZyJHBugzf+ZpXF8Zi0WQFNxJALn+O2ygTei2/g0QWzBTDpmyKUZooKT6IOdbVGev0WlEzieQk/at14+jr9MaSBp5S2GQjY1x3Gd0KyFjqKk+7C/88snx5ueZX8HxyfABGZn9x1Pc8O0aZ0XKML3GBM7E0HMGBc27MsGvAWKfnSGV+TDEzY+Jeb47Ms8e6+2GeadgXp2Wmz5/gmUiUxwYmtqmBZT93LVP0w4ujLeApiT9KL50mzHG+wNkDI7HQ+xO/FDCUxHsyXL84TYb6kCh/ToSvN2fPnsXL+zhz2jVm6olYXjivZ889KdLXvmxesJ+IOAmRbm4KgWRvb3NqtSMN6nZjg/sNQIL/xg0ZYOfDlfy4KWgTeEolZxmqb0yj+kYs9k/OcUXlyY+ICHvE6/jNdDRdSMpwmuF6qewKE8164/Y2fSyVE8u6tjasu668kchwwWf8kZ/BM/7cGOoZNaKR3lTC3kPW3NowORubf0YTfpKs3xs8X5wYSP1G0/B9oFwwKvk3ksAbL9TAU84rDg3QY8SueEXBswXP1KpJECOeqVrk8Xf+BE8pYdtHseyUMtLzJmQGVjarcZ9rSe/lDxU3L7Fug2jaw5W4qLnYWq/liQwkicfppzuAI8Fz1PZ5qEQR8tgEGrPLr+AZfdaJrX/ZYjtABE+Nhv0IOxNubGP8dHcE1pTKApncsCU7HssVs1KHhrVh6Jj22A/zQYhXpv49vzCFHsNO8wJ4fXEre68qoUgue4Kf237WLtpDrM9m9gUkcM3DjQu+xzirgJkETm3axvX3Cf2kZppH39oEnnHuw7BqtwvfpU2xH3eF1/c24thqJbelSlvNbtWfQ49us2dYU8ysbGhsYMzgtZEkxu1mYNet3I9LBU8p152H08TUCmsLA2w6/kNU8hUmmDSilYMtlqb6WLVZQfCbdPCM9V9FFytzLE0bYtdjK9c+Y5Q6v4KnnDccHWrAwB1XmdfEkAUXk0nUYMO9QfsZZG2GlU1jjA26sStcQnImG95KvqNm5ySZug39Hm1Ku9nQJht+iub73uApk97BZU5PLHSrUbthCwZN2UlorAypLJjJBhXQa2KHnZ0dTRvXwdB6BVGpd+CaIp6fokZimrlSIH+CZ3pVX989wZQWhgzeGs3R4SaMd06HUZvmm7nrNZxmjkfTB2nDtqw7OlRt377Y9ChCeuqfbyt/gqdSrxcnBmEngucndx5Z0h2OzLLHuv0O7kll3DowgB4Tz/PQpW8G8JTJI5nf1pH9KdOx0uQgjjlHkjmYI30bRkhExiid9K0fx4/deK8px09e8UwZaAt4KoIuAw0YvT+OhDvLsa01nssJ0fxtbcP6axJe+U2heZsdPE64yIa/j3BfBm9vLqR5kzXcf7qBDk1WcjcFPG9LE/BZv5BTMVLFcrY5FvZsfHCRcXp1+Ov0S+TEcWSoAX03RikinquuRbPIrhlrw5IRoqCHBhoyep8y+JNJrjz5mV/BU/L2OH11euEqzAKssKXFSF8S5NfUbHjLdz0rDt9BhpTQhTb0XvaIxOcZbXjjrbqdH0muqNmw6+olCvC8mXxTq2z4KRwlW/CMfxDBrSdCt5Q+1R7/6Dr344SFJzJe3rvIxmHGmHU9RKxiqt2KZQEPePToEXeijzPU0JJVocpuTWPE81PUSEwzVwrkV/CUy2PxWdUDy8Y9WXkyhiSScB9nyph9SvB85TcZ61Y7eeQ7jmZdDynBU3KZ8fU7seXMGLV9h+JF8MyVA2m4SARPDaJ81F0ynlxZSVdzS/74+zT3kkDy0hlHXXtmbdvL1j+taWA3h5NBzxW5xp4eRts/PBRtILUYb26FcysTl0jjX/Aqw9pCGf9GhPNQi6OdQn20BTwlrw7jUL4MtRtZ0NiiAZUKV2eG1xsiV9jTbV4EHn/aMGZ/HDLZXY7OH0jnDp3p1M6AeuYruZsJPGOkUm6cmk+/zh3o3KUtBuWasv7eRSYadOVwyrK15/u6YTfBXQGeyyKP0Lt6LWw6dlLo0dbGhBEb76eaO8+/8yt4Pj3Sm/Ila9LIwgKzhpWpXH4Ul5KkajZMvHeC2QO60KFzF1obVMJx8QM18BQinpnt/EByRc2GFqOnKMDzeoKrVtnwUzhNNuAp4fIME7rMEe5yX7OnZ13GHXpF0HxrOs2ISLtLjr84HjPLNTyQZFzjKTw5dLB/Aya6pEzJixHPT2G/j5Zm/gRPGfcO9aGFwz7ups0nyri9uT0dJwcjQcadbR1pM/4qr59spL3JDCIkkPjvFjo0mMKlB/+o7QvKHP75aArnLiEx4pk73T7lVdqyxjPpmRM9jHvjcjN9LlX6JpC9SxexaNEiZvc3orb5SHadfYTwEN6eXubMPpt+7qfU6HOkrS3g+dCpOxZdnFE+vyglcqUtTfp78PL+auytutHVYhjer+XEe4/CquVWHkkhIWYxdpbq4Hkj0ZMh9Tqw/54E4c0Cc61aKMBzvL4Jy4KFzkrC1VlW9FgZrgDPFdGXmWLpgPO/wg10EqEua3GL/HydWn4ETznP2dFNl9EHXircWCa/xULruvx1OoFEFRuefZPIiZFGDNx4HylSIhbb0lMDeAZ7jFSzswCemW3YfvEiBXjeSPbXKht+iracDXjCS795WNa1okc/O4z1R+DxXE7S0+OMMNWhcYd+DOjXAYv6tizyjFVMtas+XCQ4vef4+rSfpnyqVYx4fgrzfbw08yN4Cutw9vasSKnKdahbt67iY91nP0/i3BndxAKHoX2xMXLE6YZE+SDcRBuadBlKb2tThm25iUR4OC7TPm0L6ojg+fF8/GOlpC3gGXukF2WKV6FOiu/X1xnI0efpEfsXR9Kn2iXJJ+lftzduqg93fixBtCQdbQBPufwpmzrqMf5wymszhPnCe6uxrTwQ99d3Wd6iNDYjLijW3SbEbKRNPVsGjx9JH0db9Kq34Z9zazJMtQvTrqvbG2A/aDwj+nfHum4tHNbuYULjepg1bUffvs1p1HAsXrHKNZ5rrycSvac3Jiad6denGaaNJnMhLt0n8tpU+RE8E5/vpEO1fpx5laqbjOg1zbHofQKZ/J6KDaVEbuqIkfUgxo3qh4OtDibNNxD8OONUe+RNdTtvCbqgZsNTd1LXeEq1yoafwmeyBU8hw4TYmwRejeCpyps6hPe93Qu7xCX/KJ4qH1r/FGUT08xDBfIjeGYnjzz5GdeDI3j8WnXOMImnN4KJevBG5a0JmvZll3LeHsvP4Jm3SuVdbtoCnnlX4/yRkzaA5/sqlfDvNfwDbxInSeJRZAC3Y1X7K2Vq8uSnRPkFcfu5hISnkQRdV757Q/rmAWFB13meNtuTnvurx5H4+98g9vMFOxWFyY/gma5iTraSeHzNj5AbsSQnPyHC/wYv1E1IVnbODzbMiQrve847wfN9ExTPz58K/NfAM39aQb3UIniqa/K594jg+bktoDn//Aiemmvy39n73wfP/46t8rImInjmpdpanJcIntppHBE8tc8uInhqn02EEongqX12EcFT+2yiDSXKAJ73gI/5uXZezkovuSLNUG85f/WS8ddGOdc/cj4fs8xfalq+MTFUqVLlo9r/S9XyY9a7Q8+eok20rL84Gx1NzZo1871d7iLF19cL75fKPjorv72LjIBgb04/lmp1nVt16aLV5ctK3//y/tMhIaJNtKz/ymt/0wS62YLnLYkHf1g1ZsyB52nOczv5PCNamDJk2wMUHdKVdTi0sqC+YUMaWvdlqW+c4txbwXIm1pQxdJdM8cTxtDYyvBPlHNki57zk4wJuXgv5X8zvQsr/as9vdbvLC1yd5jFg8DBGL3Pj0ls5d5FwznMNQ0eNY8b+MKJVGv6NWHeW7QpP82ehvnd5zeGN89gcLcmwXxu0aOfoqHVlyqkumrTO6bXafJ5Xyv9q15YyZtb55rPLLJg5it4jZ7Dq3GPuADGvwli9YBx9RkxnZcq+G3FurHd+pDgu1EVoB+eCo9L8TWhHNxKVsBkju8H6HWe5ptKWtKX+qeWw79Qpreyp+z7X912SOLCwDRbtV+MtyR7sVct4/fkBBg3eh98rF4b024G/TH2sjJFFMLPvdFwS0tO9Szxrxw3mn7vadXNwKjhYa2yiqnN223dJZPtUG7rMvpTm79eub6Dn6GNpdQk6tYbVgcnk1s7Z5f9fO/b+4Jl4EPtKpWjYejuhKR1O6NlR1C5dhg7zbnP96T5a6TZj7oXnCK9Q9bs8E3OdURxJaRDX9ssZs1vO3VgYUEDG+H4yhg2RcVVDY/qviZ3f6pNfwdPvWH8amP3FFt/zLB1pSrMxvoQEzMDEcATrfNwY1aI+Q5yeI3QmXh4bGNymBub93NM6EMFO/mfHUrNQDSZ6JWbYrw02zI/gmZ3W2qDph5ZBW8BTk8535E9Y7GCIw3J3DnuvpkXdjmx49JotwwxoNc0Nl/PCE7btWHtfStjpf1gTrrzZCo3yYN3eg6zc7KJoA3eTEohODmaTUxg3k5IUA6zTxk2cTkyHnQ/V8WNfr03geVtylp76xtSvb8R0/+Qc9yt3kh/hGfaIyJebaWm6kItSdfC8LbvCYLPe7Hybbos7xDKnrTULrmvXzXP+BM83LHesTOlfzJl5OUFhu6iQGVjab1e2DeJZP2Uah97Kya2dP7bva3N6uQLPNvUcaNesCxseS7lLAttH2dKmpxUd593EZ509TUf6cjMFSoXo06mTpzmbMnWTBp6vYFhLmSLytK+HnLX/pjcYbRbsSypbfgRPYarw9LZRzDr1Vtk5BE/HCly1qwAAIABJREFUzGY1u+da0G35Q0UkJ8R1IFY9XLlBImfPHmDxFFuaDEgHzxvPjtHTfhh92huJ4PmRolkCEGnS+r/SnrQJPDPrHCO7xable/BKlhMjv82f1qbMCH7A3HYNGefxmhjZbSY3NWVmYDKhh9ax5p6U2699WbRgP15xyXi4uSnaUszbW+xY5Yhu3W4scL+hmEU4/s/6DFE2bbOnNoFnyJmhmLfbhfNSG5qNu0LkvY10aLVSAZK3ZYGMbdWfjY9us3KYDQ2tbDA0MKbP2kiuxe3GsetWLselgqcUb+dhNDK1wrSxARYd/8Er+TKDjBth42CLiak+Zq1XcOpNOniG+K+ilZU5JqYNsemxFZ/PeLOQX8FzRQ9Dei0fh7nZPDwT5aiC560EV8ZOclVEQzPb+fZH6kO1rW19SHlyBZ7t641h+ZI29NnwmJvJnvSxHsuqZc3pMD+aPePr0XNdrKKj0lSwVPC8J4Ntg2T8OVHGwJYyziaL4KlJr8+5Lz+Cp6pekXdPMKyFAX22RrN5uDHDnNNh1KL5ZoS3yQrnh7oNVPz/cGH7jvw+S/u15S+ff1nV20QEz4/caapqrWqr/L6tLeCZqqMmnSN852FR71catNjMJSn4e47k91LVqaNTlvp2W7kig8iLW1h7VRmNuyN7wjG3fcxdtIcQn02sC0jA290VZ99jHLwuUQQddm3axrn3mDZOLV9efWsLeN7lFesGNOSP/XFExyzDotZ4jiZEM6GJDQuuSQi/OhnrNjvwT7jIvL+PKGYAo28soEmTNVx+ugG7Jiu5lAaeCTitX8juGCkx8mjGWtiz8MFFBurVZszpl9whjo1DG9J9Y5Qi4jnnWjST7ZoxPywZIQq6boABf+yLz3KM/tS2yc/gOc7rGTvHm9FpQQRhKhHP0DPTmXIoHk12dvuMkP+pbZnb9HMJnuNwur6cVq024e01irYjfPBcbUfHedc5PseMdrOvE5MyYAmRDtcDq9l9TUO4XwaBt+REaZg6yG2FxOvUp2Fyq0l+Bc878lgOrHLEpHFP5pyM4SZJ7B7XiMH7lOApdPKNW+1MW+epOkj7He6NQbNpLNu7k35WNekw8yiez7VrjVR+nGpP9UFVrVP3/Re+8wN4Cjrfeh3GlNYNGXziPENMmjDrajwxybeZ18WAoYfiuZ0cxFbnSDJHaWLehnImIuP08O23fmw7diOtr9dGO2oLeN565Uyr8qWp3sgCI4sGVChcnfFeb/BYIYybEez505oh++O4I7vDpvkDaNmhM/btDNAzX8klNfCU4nNyHl06d6Bll7bolWvKgnsXGWzQlS2vlQGcoL0O2E5wV4DnjMgjdKxWC7OOnRD0sLUxYcDG+yJ4vsdN9V3eIEQ8J3gmciv+FI4G9sw88pdiql2Y9d0xfQo7X8rRZOcJWrhc63O31VyD59HEW0xp1ZpOjs2Y7J2AtwI8bxPp9xcGuhNwTZlav/XKlU41WrBCyxY4f27h80P++RE8hYfbLh7sjbXDPi4lKiFc2HdhU3vaTA7mNjJ8t3bAbvzVtMFVFYZCgw4wddEipiyaRwfDytgO38TRhyJ4fix/VdX6Y6WpDeloM3jeSjxCd7OJHEsQHrJLZNcYIxx2ONG5bh92xSsfvHOZYYrjqicKGIm6GcaF+IwzULfjXxClsg5faFNBEeH4a3nQQFvA8+qBbph2cSZK8cCWFI8VTbHo70HYvdVYW3WjtcVQDr2WE3Z2JOYttxIgheiYRVhbqoPnuQRPetTrwPp7EoQHvMZbtVCA50B9E2YFC5FoCUdnWdJlZbgCPGdHX2aopQObngq2TuL04bXsiNAQCHoPEPuQNpefI54CeAq+f8V1IHV06tCwxXZuS7wZP+GAYgYtKzvfyCNtP8QueXltrsHTNVGK52pbKlceh7CdCp7Cms4Dy9pSV7cprXt2oZG+Hp0XXEl7EiwvKyfm9WHRz/wJnm9Y2bMCJSrXoXrduopP4z77CYhzp5+VBW2G9sXMyJF/bqR3vJpgSLiLFafaP8x/NLU/TVprOi+/7dNm8BSmV7dMMKdes7449rPDxHwKznGvcVrUkvqWPejW3x5D41Hs1fJXI+XGJ7QBPO/In7Kgoy5DD79KizJeu7uKxpUGsvfVXaY1L4XViAuK5yKuxWzEtp4tPcaPpJNjU2pXb82Cc6szTLVfSL7JrPYNsR40nj79umNatxbt1u7hj8b6GDRtR6e+zWjYcCwHYp8rwHP+9UQ89/SmgUlnuvSxxaDRZFziMt5Y5Ebb3F6T38FTqPcd+WNW9KxCXdvtRFycw6Qdz8jOzgdSItG51ey/dt17g2dOBbgWe4tTVwO58EQ5vZnT68TzPv5gn1tN8yN4ZlfXO8nP8AmOIOC1LG0AyO58bT2Wn6fatVXTDy2XtoFn5voIkS7/a1c5HhRDeEqUUojchD8IxS3wFuH/0dfZaQN4ZrbFu35f+zcat8CbhEmS8IsIwDdWfcblTvJTvPyCuPg8mWtPIzh1Xflcxe03D/AIuk5oymyPal6RjyNw879B2Ge2dX4ET1UdM2/ffHKXgNfaM25nLp82/v5k4KmNlRXL9H6N478Gnv8V+4vg+X5+nBd213bwzAsNtDGP/Aie2qjjxyzTfw08P6Y2X0paIniKay+yjP6J4Kl9gCN0TCJ4ap9dRPDUPpsIbUUET+2ziwie2meTvAZeETxF8MwSPC/GxNC2bdssj+e1s4r5KTusPyZMwCMiQrSLFrVd72vX6Nixo2gTLbKJ0F/0Hz0awTZi36E9sOMeGsqKnTtFm2hZW8nLNvJO8NR0Qq72JcC25XLFpbJnsH24nKnDZES9zVVq4kV5oMDtlH+ZmQdZiVm8hwKOjo7s2LHjPa4QT/3UCkSl/MvMT53Pp09fSsxFL6JevisnGU+CfQh5LH3XiZ/1eKdOndi/f/9nLYOYeUYFgoOD0dHRybhT/PXFK5Dhf7WrqyHj/rklOLawokkTK5q2GY1zWAKQhM+CLsxzFbaVf28Cl9BnrBtIwHmgnM7NZYoDfmPlrA2VE+sp50SEEkZTrxG/tUeB/Aqecl4Q4DSPMYOHMW2ZG3cUNzcSbnmuYdKocSzbH8YbFZmTYt3Ztkt4nXz6n5zX+G6ch0u0JH2nlmzlZ/DUpLWWyPpBxdAm8EyO82PngomMGb+SszHJinrJ5c+5uG8BE0ZOZMPJO0KXjDQ+lD3zxjJk6CQ2HL9JotCLx7lxwPkRyp4ahHZwI1h4CVDqn4TERCVsyuQ3cdrhjTbHDrQLPJPwWdgGm/ariX4PXk967sTYwft49MqFif128DDVOKkmAWSyCFb2m86FhPTxVE48TuMG43T3PTJTSfNTbeZP8EzkxFQbes2+lObvCdc38MfoY2kyPT+1ll2BQnvLnZ3TEvpCN7IFT6nUhxH1W7ApUgBMGY/OjsKqyRruyxI5PkSXYTtep8n26vwYGrfapfydAFMclC1mT2MZQ3rJmNRSxvGY9IaSdqG4oRUK5FfwfHJ8AEZmf3HU9zw7RpnRcowvcYEzsTQcwYFzbsywa8BYp+dAIlEeG5jYpgaW/dwzaP7Eexw6hWow00sYjrXrL3+CZ9Zaa5e6uSuNtoCnTB7N0lamDP8/e+8ZFVW27v2ece8Y5/1wz/lwz/vucHY4u/c5d7t3t9pGQHJGARWJiiKKKAZQDKDdhhZT26jbgGiroJgDtJhRFAEjBlAykpMKKkGShAq/O1aRCigRbIUqLMZgrKq15prh/8zwm8+cq1ZQNNePe2BitJ08sZjkPTaYuwZzM+Yos7TGsi/rDWGzRzLp+3PcvXeaBTpabL7TQPX1IE4Kj7gDlZk3CT31CycOnpd9lzbW864pifNhqTQ2NsoG2FtBwSQqXxNpM6IygadYFIvn8DHojNRiW0LzhKAto918kDaVkJ5aQm31QRx0t1CggCPFkkes0JtJxLv28VQqvKVokhk/ZyvX5FkVwVNKHSdc/srvf6/P9ofNzrW6lHWYWR9pbhtUc3bVWm69k/Kxdu6mCnwRl7oFT1HTFWb+zQj/6BcIXY9U+pr0x7lU9QI8z06Qcr5aSsMlKd/tUTB9+yJkVv5CqiZ4ikk9spgdkc1+mLpkP4zNAonaZMTMnS9lnpyKiLmYT79CAw1kxoYRstpC9srMVos0ll1mgbUXi+y0WK8Gz1ZZfuVRsda/MlKluV1ZwPNd7hYsjXbzorGWmrpaXheWUiPNYqORJYeKGqitqaPmTSEl1ZkEL9zAHRmoiHjgp4Pbrle8Dd/HyWIx4tr7HPQ/Q3qliIyICJnOknf5RAa6MGrIVPZHCT+JLSIhaH8HL5vSGKQlI8oEnpVRXhjbHuf+dnOsfR5RWxyEy4QA8sUgljxlw4TZnC3J56SXOXrGZhhojGH+3gwaKk/gMSWE55Wt4CkmO3whJrrGmBpqYOZwgGdNj1iurcMEZwuMdIdjbLOLpLp28KxI2M1kY32MdEdjNT2ErH6cLKgseE7XZMFOH0z1NpPeAPLgKaqPYN33V2Te0M52Vi7sV7YW2p6fbsFT6Gyyr25kqsm3/G2QBhNnrCEsoQIJgsfz7/zpHzro6urK/scM+xMjrLt6PF+dkrLMQ8IGQwmHktpnaO1ZUH9SBgVUEzzblastusoqS03mh2RycaE2vuHtMGo27iBvW4K+vToXqznNHk+J9AXHZtvif/sNp2bqqMGzXc5P8kle608SoZJEoizgWX17MUMHGzDeciKWOqOY/MNtKkUxzB86FMPxlkyw1kHPZD0Pqlr73QayI9dhOcKZsHwRtXEhnHrcsjwveUV8xBn2bj1Jxe2DnH5ST9bNCO7dv0SszItWT2TwYbIVeOCUxCwoC3hKqeGshwZLzlRSX7gTi699eVifyY+mZuzLElETv4pxNkcprY9j/48XeC4BYRIxTlhNfL0fe5MAilrAM19cz+19W4gsECMRJhWG1gS9iMNn2GBWX69CSiUXPDWYFfRM5vHcnZXJVqux7E1tQvCCnvXQZMnp6n4zkSqD59qYMq776jPNP50qOY9n5Q0/fjpbjSI7x/cj5PebkT8i4W7BU1L3kpzCtzLPkaiuiFvHPBnzX65cqazvfqm9U0bqSqDkgxvYO92k/tqnCqgqeEqlFdzePR0jA1cCrhXQSCNRProsPd0MnjXxKzGdcIzWTSHyMFR63g3tsT9w5NQxlpj8g6nrL5JWrlxeedVcam+uuvJa92ll/syJKRN4amhtafaiie6yeLgtR0ujmT/IkJ+zBN9LAzd9tfDYX0bT2wdsczFk3DR/YvObR0dxUyKXwjNke0DlJRO/SyU5vePysPhdPJcv5aDE3Kk04CmqOYfzH3/HNzqGGBiO4i//Pgi/mDoydlkzdXM6N78zY+mZSiSSIi7+5IGTvROOthqM0A+gqBN4FojF5ET+hLuTPU6TJ6HxB3P2FcexQmMK52qbJxTlp6ditTxKBp47Mi4wc9DXmDk4yvSYZKbNoqDn8ubt08+qDJ7rohsQVUcyT8OaXRdWtyy113Nt7SoiqqQosrMybtfqU4P3MLFuwfNd1k+Yj/kRWR8mbFAX3cX723EcflHXK/DsYV7UwfpRAdUETwnFZ92wdD5NkbAXRPYnIf+gHQ4rkxAhofCwAza+j9sGV3kYqkwMZfvWrWzdupkZmn9l/MJgHpQo19CqBs9WuyrPUVnAs6E4kEmmOymWCH3zI5Zr2HGyLJ9/WlkTLKzp0sidVbq473/G8RkGeJ/Ik22ZkleyLi+NvE4OMXH1W2o6zL8kvElP46VyNQ35Ysg+K4vH82XYNAwnh1Mjy5WYjAALTGbfpOp5INbGU5li6MWtWinVtxZjPD4EocupL9iGlVFX8MxpiGbBCHvOFIsQHvDaZGwpA0/f4drsSBImFyIerzdmekCaDDx3ZT5klZEz4W8EKG0k5fxeIjL6bwFY1cFTeLblZcRcRg4djJblEcSiW2xYESpbQXufnduGoi41VH2iVYFuwVMqLSHcR5/hYxxw95iFrdFonFbG8la21N7Nw0WtsauPKqOAKoKnsAn8lOt/8ZuvBjNkyBDZv6nbGV5VRrHExBBnz1mYabkQltPe8cqDZ7tx6jmtXmpvl+MTfVKs9SeKvB+jURbwFPbcX/AxwWDSbGZaj8FueTRVSHge6YPRGFvc3a0x0FtFTGk4U373G/57cHMbGTJkJCtOCA/cDaw/ZQBPwSbBDsPwPdeMnYLCwgTB4isPomqL2Gn5W8wW3ZNNAOoLgrAZYcF8X2/cXCwYNsiGA3f2dFhqz23KJdBOA+u5viyaPQ3TIV/jvPckyw1GoGduy6xZ49AZvYyYiuY9nnuzG8g8ORNtbSfc3caiq7OSe5WtWy363t6qD57Csy2lnHT9b4ZaHKEmbhObjpbJnnd5n52jWzzRfa+26qTYLXi2FqP6ZSoP4p6QVSL/wzStV9XHgaCAKoJnd7pLm8rITkqntLaD66a7W5Tymip7PJVS0E+QKWUBz+aiNPImJ5GU3Io2r75wvr4ih6TEPCrb51yfoOTKHYUygGdvFap/k0XC01wqRY2UZDwhv6JrfyVtes2z+ETyy0XUv84gMbtSloy47gWpidmUK3Cx1ZRmkJCQQ0U/218VwbM7Gza+LqK0dd9WdwHV17pVoEfg2W0M6osDQoGBBp4DwiiAGjyVz5LKBZ7Kp09/5UgVwbO/tOqrdAcaePaVbgM9HTV4DnQL97B8avDsoVB9HEwNnn0seA+SU4NnD0TqhyBq8OwH0T+QpBo8PyDQF3q5A3j25fs7e5JW1l0pATFS2Xtew1dLWWAjwXuZhIdi5XkXbU/KoQphHhYXM2LECPU7dZXsnbozFy5U20TJbHInJwctLa0Bb5dCSRHno9M+WM5CaRnXYp+QKOnffnna3LkfzKsq9MUDKY8309PVNlGy/quv65citu4WPAulr9jpNYvdWaIPVp5n6XuYtSqGtPJQ5s4/TdKv7ITykqSs+IcEz+MSisXwk5uEqHIpuV+4ET9XpbnX8q72zxX/54q3iLdcCdvMnPleLNkRwYN3UooQcSd6D56LffA7k0qmXJ3JqYhix/H2wTS3Mp4d/iuY47uLXwqaPljPP1c53hevrYuL0uXpfXntfL6z1lnPb7Ju5WK8NoVxq6Z5Qtn5HlX4HtPyrnZlyGty+i8sX+aFx5pgLpeIFdaVznZITA3FZ9l83Ffs4nTOu7Z7BIA8fuCU7HsREuIjTnLiZXucBXU5RKVXtIUvEjWQ0+IEyMoL5cDN2rZr/aGNtaNjv6YvX+YiGgndYoOhXSC3RD2v69kt42d8zXkWuB8lQcE4WiBJZ92stZyvb4+3iGr2+sznQFG7veTz01+fI5OSlMYmPdWgiAaOrDFj8oYHZLWMHVnZ+3FdcqmtLImRewh82sTH2rmneRkI4T4CPJ+zeqwO6xI/PCCnPV6JxdRwMptKiE4t+SSAmHVGytITUtlP5azRkbB6iYTZ1pJeNeSBYLi+KIOqgmf8pdmM0lvNoft32e6ty9il90l+4oe25iJ+vh3BYsuRLAgrR+hMYm7uZ77N39F3j5J1IAXSZ6wZr8OsA9EcO+6BvtF27iuZN10VwVOR1vmSeDz19PA6dYugzVYYTTtHmtyEoC/q+KdKQ1nAM6/pBlNHWLAq4iEnD05Hz/IgT+VARZEd8qov4DDUhOXh9zgT6o3OsO+IaJCSkRfF+jUT+frPC2Rto4h6juw/yB2RMJGr5sbNUAJ/Oc6us6my6zn1DWQkh3MkuYHcRhH5onts3/+4bWD+VFr3Jh5lAs98USyuw8cwcqQWaxM+PH62lrOwZfzMqDrIeN0txCnoj/Ilj5ivN5Nj79rBs5AKNk4yxT/7w06i1rT64qia4FnHTpev+O3v9Vn3sF5Wp58l+2FkfaSlbVSzb9UPnH0n5WPt3BfaK0savwo8k87OZ8K873B0HM+oYcasvlFJVkEozkYjGGFojImNPgbO4aRXHsdlSggJokICvMwYbWyGpsYY3PZmkE8tu+baM/17NywnaDNadzXnKkXcCvdCR9cYXQMNDB0OyDo7QbQ28BRB/IvmRhY2Q8reN+0NTlnEVfV8qCJ4FiHm+uHFrI9s9to8S1qLnlkgJzYZMnXnSwqB5CseGE+/Qg4NxMaGsm2VBSZzmsEzM8cfU6PdxDfW8qyulqcFpR28o8pgU1UFz85aP8vYiJHpXhkY5TVcYvK3czgj57FRBq17mgdlAc/sskOMG+LNhVopmUW7MBnpR3RTe98ogGdnO2TlhDB/bRTZQL7kKZ6jrQkolfIsN5bAEysx/MazZXCt4+e9B3kigaS4g2w4mUyKOJNjl5vBM70ghlUuI9B22sXZ7BoKJEls2xOtBs+WyVTyDU/0bY8Tvt2MsT6PyCgOwn5CgAwkBd2XTZhNUEl+lzEyq/KEbPx8WNkKnuIu42NM00PmjtHBzNkCbd3h6E3cRWRdO3gmJ+xmgrE+2rqjMZsewu2G9jrR0zr+qcKpKnjumq7JjJ0+6OttJrpBijx45tVfYdn3V2Te0M52zlfRyfSnsreieH4VeD45ao+G7WlSgKdnXRm76Cb7PEfjeqAY4X0xV34yQMMxnLSy/ViZBPCoPo7NP17gsQSEAd7EZA9PJNVstv1vZh9/I5tF75quw/Jbbwnbt4UTBWIKpJksM7RmV8vyTht4NsImEwlrfpDg5SRpA1NFhVSf+7h9VqoInvK2zii6ipelBm4hmRxcOAav8HYYNRx3sM27lhLhIXtXu3Bv6m1v/jHYAFPLiRjpjGLSD7d5pmQdhyqCZ6tdOmgdvRATl4vtwKM5iZ8r+m9AbM3jxxyVBTyFbSYh3kP53d9G8bf//DOuQQWyyVbnMsnbofVaQWMBQeusMLA7yqMWr1pe02Umj/RqAc9GfjkYwo0WkM15/YRDZ/aw/thDTgUd5kZjLqcv3yf80g0ZTOU1RbEzOEMNnkARNfw8ZzTzzlSSWbADw699uVifyXITM/yzRAirg6Y2R0lQMEY+fN08fj5oA8/6LuPjlhdxeAz7hqXXqyikkiDP0UwLeibzeG7MymSl1Vh+Sm1C8IL+PEeDeaer+80uqgyePjFlHPPVw9E/nVQ5j2fKjbWsOlut0M7C6kFrG1Mfm1nkV4KnI5PWplMgDNjX5mPh9QvfjTVjS27znpK0O0sxcW4Hz3hRIcE/zWG8vRPWthoM0w9oBk87IzalimRLnwfmGbE8uo7b1zYz2cme8ZMnMewP5ux43nWfSpHg9cyVkim3lKQ27MdBpiLdVBU8C6UVhO52QdvAlY3XCsilkRM+Osw/3QyeQidvMOFYmydTfhAWwHOEVvNyVn7TXdyH2/KzknnTBwp4pt3zwWTK2WbwFD3EY6QjwdWq2UkrC3g+e7IWXZ313KiWklF0mPHD3DmpQFP5Oi/s3XzyaBcT9Y1w/fE6jxra+xB58BT6iIykCxxP67pMnJ7ylDtynlUhbGr8RU738zKvsiy159WEM+GPv2WQjiFahqP4078Pwjemjpu7rHDYnM7J70xZcKaSQknXMfJBF/AUdxkf/YvjmK8xhUO1ze0n8ZQzFsujZODpl3EBh799jZ6DI4IeFmbazAl63m8wpMrguTy6gbzqSFw0rFl3YbVsqV3YgnJ07SqOVUlRZOflMQ39prWicV0Zzv1K8HTC1u9ZG3haekWyY4YG3pdqEDqz+0fsGePUDp6x0YvQHx/CEzFkFmzF1KgFPO2N+TG9HTwXRl1l+gh79hWLKJDk4GtsqRA8lUHAgZwHVQRPod7F/TITU+fTPGgZQIVz94LtsFmZRL5QL0PssfJ9TOsSiPwgnFW0m3GmO3kkgXzRI+aOtmNfuXLB0EABz+xXQVhpNy8FZ785hPWoVVzvBC+q0r6UBTxTrnlgOPWcbFKVL0nEU3MCgaVd6698nc8uC8VuzEwO5XYdIDuDpzDI3k3J7jCQCu0r/W11B89qobSc6NTnsrGhP22oLOD5OHQqupPDZasnwnagm7vMMZx9k9TiQEyNpzLR0JOztVJSY727jJGdwfNOfXSX8VEAT4/h2qxPEsZRERfXGzE5IE0GnhsyH+Jp5Ezwa2FvbiPXz+3laHr/7ftUdfAU6vujKx4MHjqY0ZZHyBfdwnd5qGwF7X12zlGyVbP+bJNC2or+PvBUe/vDRU+OdgRPK89bPH38I9rfGjBpzhQMTEdhKOfxvJd3AIsRFkz39cbRxZxvBk0kILmczZ3Ac8nNdNbbjcZ0ri9u7tPQFV4JtielQ2fX38J9CemrJnjWEeD6J/73V4MZNGSI7N/A7QxPKqNwNzbExnMWelouHMhp73jlB+FC6WuClhmjNWk2DtZjGL88mnQl6zQGCngKcHJ4uRl6kz1xMNXB/VBuv4PKx7ZrZQHP/PpH+FhpYOAyD4fxWoz1vEySpJHTKzWZtK79Z2zk63zyBVf+z3/8ta29DB3qweGWyVZn8PxYffrrPmUAT6FP8Xf4Fs9zNW1jmDDBNfiLB6dqivhh3G8wXnRP9vBtVkFQlzHS/06gbKta61L7vabcLuOj7d6TzDMYjoa5LY6zxjJ69DJCK8pl4PlTdgPRJ2cyStuJyW4WaOis5Hxl18lIX9lI1cFT0KlQWsou178yxOII6XEb+f5oGd3ZObTFE91XGit7Or0Gz54UKOdtHtcT80gTtS/ZtN6X9SaTiKe5pIoaiU9/wv2KrkvoQtjCptfExCcSV95E1ut0IrPbf7KjNS71sau+n1ITVQTP7spf2FTG7aR0ntRK2gYAReEFr0BiTiJRueVtXlFF4frrnCqDZ2fNBK2fZicR+6K2g8esczhl/64s4CnoVCSp4UHKY27I1d/s14fZuLe423qv7Bp/TP6UATx7m++ejJHvGx/z615wMzGbFLntEq3pZ5SmE5GQQ6qCcbk1TF8cVRE8u9Ml91URT2o/71jcXfqqeO2zgKcqCqHOc9eGM9AhrWOsAAAgAElEQVTAc6DYeCCB50CxiTKBZ2dNhaXBhORH3FPh30ntXKaefldF8Oxp2VQ13EADT1W1Q3/mWw2eSraM2p+VoXPaavDsCuOdNeqP72rwVD67KDN49kcdVZY01eCpfG1FDZ7KZ5O+bq9q8FSD53uX3+IKChg/fvx7r/d1ZVWn19xhefj4cC8vT20XJWq7t7KymDRpktomSmQTob+Y5e1NXGGh2i5KZJcbKSkcvnJFbRMlsklfj60fBE9FAQbaucc7pM1FksCNtVLWzpYQXdxybqAVthflyW95ZWYvblEH7QMFXFxcOHr0aB+kpE6ipwo8a3llZk/DD/xwYgriYnhWJV9SCa+SbpNcKpY/+Vk/Ozo6cubMmc+ahjry3imQlJTE0KFDe3eTOrRCBSSSYuKihfe9QUNeDHcyRUilZSTGPuGNROEtSnuyw1PtinLZ9PYhAQsmoK+tiYaGOYsCHiD0LxVXvmPFoWI+prySmgLScyoRS19yytueDeffyOKRSks4s8SBjRffKMpKr85JatM47e/DwkVrOX63VBZ/+mEps/+lOceN0eD7o4TGDPD1ktDYq9gHXmBVBU8pb3kStpml8734YUcEhe8E24jIi97D94t92HEmlTo5czVWRHH4eHPjfZd2hMXubri5Nf+7u63hUq5ILnT/f1RV8KxI/4UffbxYtiaYR30IH31hMWUDT/k6LZRfIi7m2t6VLF32ExdTamSSKOoP5bUS16ZyeosvS3z8uZTcfE/r9cqEYLYfz0JASCnVJJ71Z5mXN5sO3JMNeI2VEYSGl3QZCyTSXMKO3kLWJFsj+4xH5QLPRm5vscHMLpDMXrB3Y3kYy+afpqTmPCvcj/JSwQArkaQT4L6We/XtDhPBLmE+8wkr6kVin9EWrVGrMniKRQ/ZMEEXj51psrrfWqYPHT/WhqK6C++1OUgojThJ5Esxosoc4q8e5+SVJ+SVS2jIDyPsZu2HsqVU17sFTwEEj04bxZSNcVRKoKkyjlXGGvjdrOfNsck4b8zplUFaS15zzwfb2ddlsFeTugXTIR7cKBNReG4WBuZ7yfvVbaeey16aOPwQQdzdIKaOsOXU8+ZIr0xtbqwVP0tZc14K9bDSViKD6db8fYlHVQXPV5fnoKW3mov373J0sR7jl96n8uk6jDQXEXonAj+rUSwLKxfmiDy7uZ8VNn/HyD1KZmJRRSb3b8USGxvLzQh/7LQXcquivTNXhnqgiuApaopi7ggL/CMecvOgK8aWBylVMIAqg74fkwflAc+udVqYdN1bZ4TlgtPEXNuE1dBZRJS/e29/KJRf6OdDnLVw3XadezcDcNIS7mluB6LqaBYO/w9MPG/L+uuq6IXoGq3j2qPb7JmuyZx9L3h7PYiTaYL3pYQw7zH81x80WXOuFGjkVlAwiQ0fo3Lv71Em8BSLYvEcPgadkVpsS2jqcWGkTSWkp5ZQW30QB90tFCgYC8WSR6zQm0nEu/a+Siq8pWiSGT9nK9fEWZXBs/rOUkZo6aAzbI3s1wF6asSPtWFTNzYXQCVi/0GyxdD0Jpno8GOcCL9FWqkEsfgeIfsf9zR7ShGuW/BsKN2L9WBf4uU6jvK0SKKTq3hzzIlR49xwtjNj9DdG+EdVIaWGe4EuGOsbozvKAO/gdN69e4y/kyF6xoZoj5nBycxkfp72LX/6H3N2RAoz60YSto9Fx3kJDiNtOZnTteG8Cp2L0zwvLEcv5kL2L8w11cPYzIAxGlM5mvGaU/Pt2fNEaNyN3Fpnx5qLufxsq8Ham7VIpAVsNddj+9Pmxt8KnnXHpXx/QgrVsMJe0mezcqWwuoJMqCZ4ikk9spgdkc0+lbpkP4zNAonaZMTMnS9lHpiKiLmYT79CAw1kxoYRstpC9spMeQmEentzpQN+VyvkTyvFZ1UEz4byQ9gO8eZ+rZT64gAsR/mR3rVZK4W+H5MJZQLPznVaIkljrd54jrwQI6WOsx6a+FwofG9/KJRf1HQZ92ELuCXzoDVyY5k2S069Qyp9xVkvGzyX2mHtJYCnhKLDDoybH8s7xGTutsR2ZRJl4T9zsliMSHSNmX/+X/zLv/zf6LndkK08JATt7+CZ+xi9e3qPMoFnZZQXxrbHub/dHGufR9QWB+EyIYB8MYglT9kwYTZnS/I56WWOnrEZBhpjmL83g4bKE3hMCeF5ZSt4iskOX4iJrjGmhhqYORzgWdMjlmvrMMHZAiPd4Rjb7CKprh08KxJ2M9lYHyPd0VhNDyFLbvzuqZafKpyqgqfQdi56auBx9DGbTTTxj2uiQYENTyWe6cAjx9NENHWyYV5TYRc7N0q62jC+JLhtstHFho11hO09SEnLBL7q8grWhzePe2JJEgf3RH8qk/VJPN2CZ02cL/pjD/JWQVbeHHNA2/EXKoGyczOZtCSOqsJd2Bj5kyUCUdVlZg1z5WT0cowsDvBKIqEoZj+hD2qR93gKUQsektl//Tcslz9B0fhUEjKJkWODeS6B6of72XWuEAliUraYMXPHCwoOO+LwfSINolgWjZnF1Sopr2O8+fo3gxg59A+MtgqhuMVgreApKYX14yTsmSNhU5i0yzKRgiIP6FOqCZ7tJqktusoqS03mh2RycaE2vi2NUoBRs3Htdfjt1blYzWn2eLbeXZ+1nSmOIQqXtVrD9NdRFcFT2P5wxXsY//m3UQz+zz8zJ6hgQLUv5QHP5lopX6dFopvMH+nOdZk3TMQDPz08fq54b38oxCCRZrDJRIc1V4opK4rAc+T/ZvrOl+SFzmH6iru8PD+rBTxBVBPFvMH/yd9HDeGr39pyJl9MbVwIpx4LE3sRGefX4zlvA9dyhe/1RAYflnlp+qL9KAt4ChPZsx4aLDlTSX3hTiy+9uVhfSY/mpqxL0tETfwqxtkcpbQ+jv0/XpCNa+9ytzDOZA/PX+/H3iSAohbwzBfXc3vfFiILxEikWWw0tCboRRw+wwaz+rrg7KnkgqcGs4KeyTyeu7My2Wo1lr2pTQheUGHiseR0dV/IrzANVQVP0bvLzBo6gyuVIjJ2WWDpfZ96aVYXG+bd39eJR0poKO9ow5x3Xe1cInrUxYZTAv8pA8/cptwuNlx8uow7B0NIbgGk+qwbxGQ0fxH46UhwhkL9lfVkt+BZl7ERYy1/2SyttQC1KaHsP5ctW2qfvCFbttReFbkAe+97vLk2l78PMsHB0RFHR1vMNV05kZtMsJc12hp62M7cTHSBqBN4ingaYI224xwsh07mF9nagoi4vbOYNNEG9023KQyxx9EvQ5ZWQ/FVNsyZjL3TZCZq/AWXbS949yYER10/7kd6MWHWVWok8azUNmHH42okTQUcmKyJ79mujU9SC89LhN0T6j9VBU+ptILbu6djZOBKwLUCGmkkykeXpaebZ4M18SsxnXCM1h0w8oO0YHUptVxYYMra6321E613dU0VwbPuqR/GuutJrobaoiM4DXMnsrp9WbB3CihfaGUGT7H4Dt6jZ7QswzZye6U2s0JiP9gf1mT9wqqZdjhM/Y5V0zWYsf8YLt9as/7wKUK+M2WU1UauJr7inp8BDmsSqKWRzKPOmLtcoKopkUvhGV2cBuJ38Vy+9HHbsT7G6soCnqKaczj/8Xd8o2OIgeEo/vLvg/CLqSNjlzVTN6dz8zszlp6pRCIp4uJPHjjZO+Foq8EI/QCKOoFngVhMTuRPuDvZ4zR5Ehp/MGdfcRwrNKZwrra5TZWfnorV8igZeO7IuMDMQV9j5iCMwY5MMtNmUdDzj5Hzk9yjquD5+sJM/vh//oGOoSF6o7/iqz8u5kGjuIsNFfFIZ/AUPJ6d7fxC9KiLDQ2XrJKBZ3b9FYU2rE26wNX05pXbpleppL1o3ofxNuEiUUq2xeJDladb8JTtJRmtycbbzY8rCp6McI9vmbGruMMez1bwrEz0Y4LdcYQFS4noGeGB50hJPUfI5TxEkrc83W3DuHkxHcCzNm0bZt/OJaqsiWch9ugL3slO+1pKQhyYIttP2sBVby08gp4jRkz6Ngtct71ALC1ln70epqZ6+EXVI2qKYNYQN67KBjsRD/30mLX71Ye0+KKvqyZ4Sig+64al82mK2p4Ok5B/0A6HlUmIkFB42AEb38dtg2Jn8BTVnMVVYxkP+3E5qruKp4rg+fbaXEynnpPBvliSyErNiRwvVYNnd3b+Ndfk67Swz3LPJGPZ1iKptJSf7fRYd/9Ct/2hVFpO7IGDPKqUIpEWsdPagB/vPeTU9q1s3bqVDbO1+Ebfm2O38jnnMZplYc2T+LoUP0zN91MG1OWlkddhbi/hTXoaLzv15b+mnB+6V1nA82XYNAwnh9P8iJaYjAALTGbfpOp5INbGU5li6MWtWinVtxZjPD6EEjHUF2zDyqgreOY0RLNghD1nikUID2ttMraUgafvcG12JAkeLxGP1xszPSBNBp67Mh+yysiZ8DdCe2sk5fxeIlo8Yx/S73NcV0XwlFLO0anfsiS0mXsk0jy2mA5h9fV6GuRsGFunmEc6g2fSTe8udhbAs7MN7bZtlYFnTlPCe2xYT25qdgczCW03I1XgIdX66xY8haK8ebSd8cOHM3aKK47mI9Ad+0+e1qIQPBskBRyaoY2h4xxcx47G1jeGVwXB2AzRwcXLiymGOvieKaHu6Rq0hzlz+G4caw1GsPx8mczrKBg40GYIbntyOqjYDp5iMoId0DKdi89id5wthqI9bj8ZIglFR+34n698SWgQvFh13Nk6gTFG0/GYPR69MYu58UrVTNNBgs/+RRXBU7DzKdf/4jdfDWbIkCGyf1O3M7yqjGKJiSHOnrMw03IhTG7fsPwgLYj69vp8TJzDlXaPryqCp7j+ERutNbFwmYfbeC0meF6mbAAtKyizx1NYvymJXoGp3hTmzzZlrNNRCsSK+8OaRyvR09pMlljE0z02jNaZjKudDlazw3klZ6+3F9qX2t/Gb2HcKBNc57ljpanPDxdeKc2KkTKAp1T6mmCHYfiea/9lgIbiQCy+8iCqtoidlr/FbNE92YNa9QVB2IywYL6vN24uFgwbZMOBO3s6LLULy66BdhpYz/Vl0expmA75Gue9J1luMAI9c1tmzRqHzuhlxFQ07/Hcm91A5smZaGs74e42Fl2dldyr7L9JnyqCZ0P5Mez/5s6NmlbdJGTuGYfhzKtIpMVyNlTMI0mlHZfaM3K72vlQ4r0uNowsbN3jKVYqG34O+PggeAqJShvLyU16TFLWaz7kGBJg4EVGPInZ5W1eJlFVIYkPH5NZUi8rgxCmIPkhWR8Fg42UZsWTnFNBU9Mr0hNyeCuRkH/IHvsV8ntEJVS9SCX+aR6VijaOfg41VThOVQTP7uSWNpWRnZROaa3cCNrdDUp6TRXBU5BSKq2hKPUxybkVbf2Akkrc62wpG3gqKkB9eQ7JaS9511b9u/aHgnf0kH9wy95mMWV5iaTmV3/QeyKqfU7K40QKypWrY1UG8FRki+7O1b/JIuFpLpWiRkoynpBf0WawttukTa95Fp9IfrmI+tcZJGYLT1aAuO4FqYnZlLet9rTdQk1pBgkJOVT0s4lUETzbVezJJ0U80vW+99lZFWzYtTS//kyPwPPXJ/M5YxCTfHAGY0a58kteP7eyz1nMzxz3QAPPzyxXn0WvquDZZwL1Q0KqAJ49kaXpTQoJme2euZ7co8xhVBE8lVnPT5G3gQ+en0KlLy+OAQCeX57RPkeJ1eD5OVT99XGqwfPXa/ipYxgo4Pmpdenv+NTg2d8W6Jq+Gjy7aqI+Ax3As6/f4aks6WXdlRIQI5W9Tzb+gpQVrhJ+uiSl8At6v2r8y5fo6+ur36mrZDZfuHq12iZKZpP7BQWYmpqq7dJilyLE3L8fw62q5j5U6NeLkPAk6RbXS8V9ptO85cv7LC1lGbuUPR+3srLUNlGy/quv64wi0P7iwTMvScqKf0jwPC6RPRm9coKE+w1Sfpog4UpDe0fa18bq6/Tutbyrva/TVadHtx2zrYtLt9fV+nWv3+fQJ6blXe2fI25VjDOnMoJ94SVdJuoFkhz2HY0lq48GXmtHR3Vb6SOte1pPI5OS1DZRMpv01HafKtxHgWfu24f4LZiAprYmwzTMcQ94QDqQdGUFCw8Vd+lsepLZgpp8onMqyJe+IMDbjhXn38jiKZSWsHuJPd9dfPNJKmuhtIzjB051iSslPoi1x7MoaKkQWWekLD0hpagcFk6VIDxTf3KKhP2VavDsiT37M0yR8GPlYZuZM9+LJTsiePBOShEi7kTvwXOxD35nUsmUa/g5FVHsOJ4mqxOZqYdxd3fDya35f4rbGo7kirrUl/4snyqDp7zWrRoWUcu5oM0czFQunVvz15OjsoGnIp079335VckEbF6Gq+f3+F/ORfhRls5l7ck9itpW6vUD7EkTIfTfe721+OMfNFl6rpQiGgkLCuZ6H03glQk8hbKHbrHB0C6QW6KejyPZ5aHMnX+a+JrzLHA/SoKkq50KJOmsm7WW8/Xt8RZRzV6f+Rwo6jsPc+f6o+i7KoJnEQ0cWWPG5A0P2iZNWdn7cV1yqa3NJEbuIfBpk6yOf4ydFWk1UM/1GjyFjmT7tJHYbowjTQK5lXF4GY9m+c16nhx1wn5jThu89Ua0tLs+WM++LgO8tFR/9IZ4cKpMxL1wN3TM93Jf3LWx9SZ+IWxGXhTr10zk6z8vaKsswvm8qmhmDv9/MfS8TW5n8KwFb3uJDFQO2ko4UtvesHubvqqFV1WPZ/yl2YzSW82h+3fZ7q3L2KX3SX7ih7bmIn6+HcFiy5EsCCtH6Exibu5nvs3f0XePktWJvIpnnLsVS2hsLCcj/LHUXsjZCuWyuSqCpyKtW9tDQuwy/vFvf2dFTEOHdtl6XRWOygKe79O5c99XSCV73Edg+f05zt47xXQdLVbd6ah/T+95pqBtpZz9mT3FYvJEV3H487/KXpmp43ZDNgG8fGBfB0D6nPZVJvDMF8XiOnwMI0dqsTahqcd1vbCphOjUEjKqDjJedwtxCsbCfMkj5uvN5Ni79r6qkAo2TjLFP1u5JnSqCZ517HT5it/+Xp91D+tltnuW7IeR9RHZZwHy9636gbPvpHysnT9nO1C2uHsNntklezEd7EuE3Iw1KTWS0OQqnhx1ZNg4NybamfHt10asjqqiSHhVWOA0dPSNGT3KAI/gdDLfPeY7RwM0jA0ZNWYGezKT2TRtKH/4H3PWRdbIZgyXt1ug4bwEq5G27Mnp2nDiQz0YP88L49GLOZQdxjRTPXTMDBip4UxA+mt2z7Nj85Pm2ccZP1uWX6zhWW4sgSdWYviNZ1ujL5S+Yp+XDa5L7bDw6gqewms1j8yR8P1aCfPnSkhRMNtUNqN+qvyoIngKe8uuH17M+sh3zZ1D0lr0zAI5scmQqTtfyrzoyVc8MJ5+hRwaiI0NZdsqC0zmNINnq3ZCvT250h7fqxVtdaX1Wn8fVRU8FWmdU3YJV2sv3Oy01OCpwOPY27omgKcinTv3fQXSTPwXbuBcy2rAOT9tpu161aGu9+SeqQEvuLLJqEvbSow7xN7HQv8rIvr8OmbM28CJXOF7PceDD3OnFx6/3mogH16ZwDP5hif6tscJ327GWJ9HZBQHYT8hQAaS+ZKnLJswm6CSfAK8zBhtbIamxhjc9maQVXkClykhPKxsBU8xt8K90NE1RtdAA0OHA8Q0PWTuGB3MnC3Q1h2O3sRdRNa1g2dywm4mGOujrTsas+kh3JYbv+X16ovPqgqeu6ZrMmOnD/p6m4lukCIPnnn1V1j2/RWZN7SznfM/QbvuC7v0ZRq9Bs+0+75ojz1ImgIxnxy1R8vxF9m1xPAZWC+JI7VgF2MN/bndJCWv6hJO37oSEO2LrsUBnkgkxMXsZ/+DWuQ9noIAeU1RTPnr/4PF8icoMtzjQzYMHxvMYwmkPdjHhvBCChETucWUqTtecDfEAZvvE8kWxeI2ZhbHWza55zVdZvJIL1kHK2x2vxM6B6cVd3l0zo2xcuDZwQgSeFogJesLgk6h/KoInvJ2yyi6ipelBm4hmRxcOAav8HYYNRzXXodTIjyw6ASemZn/ZJJjiMJlLfk0+uOzKoJnq07yWhdKn7PdfRKrb79h90xtNXgq6FNbdevtUV7n1nvl+77WcwKo3opch/EIZw7kd53gf+ie/fm1hChoW8lNTwkJz+jSd+e/i+fwpY9bFWvNc2+OygKewkT25zmjmXemksyCHRh+7cvF+kyWm5jhnyUi7fFKTG2OklAfx+YfL8jGtcwcf0xM9vDw9X6sTAJ40Aae9YTt28KJAjHCBGKZoTVbXsThMewbll6vQvBmB3mOZlrQM5nHc2NWJiutxvJTahOCF/TnORrMO13dYZLRG01/bVhVBk+fmDKO+erh6J9OqpzHM+XGWladrZY52jrbWd5J92u1Gyj39xo8n2VsQE/Lv4O7Pz0llC3nsmVL7XYbsmVL7anX5jPB+x5PrnrwP4NMsHJ0xNrRFn1NV3bnJuPvZcUoDT3GzdxMaIGoA3gKs+SIACtGOc7BeOhkggrEspnzuT2zGDvRhmkbb3P/kD2T/DJkaWUXX2XZnMlYOk3GTOMvOG17QeabEMbr+hEe6cW4WVfb9mXId6R5VWeZ9K01PodPsXWFCcOtNnI8sbzfGqSyVSpVBc9CaQWhu13QNnBl47UCcmnkhI8O8083g6fQyRtMONa2z7PzIC3sOQxaYILP9ebwymaXgQKe8edmojH2B3acOoa78T+wX3eR6HLl2o/WU9sry1J7a34712nhvHzfJ3zPffuAVS4GGE/z55f8jsvsrfF86B5h3+L72taz3FTuVbcv/QoT/cT0NBIULBW3pvepj8oCnnk14Uz4428ZpGOIluEo/vTvg/CNqePmLiscNqdz8jtTFpyppFBSSPBPcxhv74S1rQbD9AN40AU8xdy+tpnJTvaMnzyJYX8wx784jvkaUzjUshUs8ZQzFsujZODpl3EBh799jZ6DMAY7YmGmzZyg5/02zqkyeC6PbiCvOhIXDWvWXVgtW2oXvPhH167iWJUURXZersJbiD51e2yNr9fgKewlmTdKk+9vV7V4Dd+y32MozruKO+zxbAXP5KdrGWt3nBSgQJTBgd3nuJF6ju2X88gXvyVi90TM5sV0AM/0tK3ofzuX02VNRB+yQ9sqhIedOqvHwluJNuaQL2z69dbENeg5BYiJ2mbOlG0vyJeWssleDx1TPXyjmvdkCIWW70jz656ye/tWVm3dyrLZWvxD35uA2JJ+a5CtRlGWoyqCpzC4xf0yE1Pn0zxoaN4XLJy7F2yHzcok8pFwP8QeK9/Hbd6YzoN0bs0v2Gss43I/Lkd1VwcGCnimJIayZqvQ/jZjr/kVFguDufhSDZ7d2b6n1zrXaeE++b5P2GK0c4Y+Hify2va1K4r7Q/d8qG0pirMvzykLeD4OnYru5HCeyX5WSszNXeYYzr5JanEgpsZTmWjoydlaKamx3uiPD+GJGDILtmJq1BU879RHM32EPfuKRQi/EuBrbCkDT4/h2qxPEsmcNBfXGzE5IE0GnhsyH+Jp5Ezwa+Ehy0aun9vL0fSu3u2+souqg6dQ5x9d8WDw0MGMtjxCvugWvstDZSu977Oz8HByX+mrCun0GjyFQj15tB3T4cMxnOKKtfkItMb+k8haqULwzJbks3XGGMY4zsFu7GjG+8aQUBCE+RAd7Ly8mGiog+fpEjKerGbUMGd23I1jscFwvM6XyfbjFUhyWW8zhKl7cjoYrhU8Bdi8GWzPSNO5eCx2Z4LFUDTH7SdWJCbuiC1//cqXq3IAId+Ryhso5Xw3S+1faKVRTfCsI8D1T/zvrwYzaMgQ2b+B2xmeVEbhbmyIjecs9LRcOCC3b7jzIJ0SOQ9D5/A2L7l8PVGGzwMFPFu1FDwG6qX2Tzswda7TgtbyfV9eQzjjf/d/+PPg5jby9yEjWXiinLSH36OltZnbLXswe3JPTjdtq9XG/XVUBvAslL7G3+FbPM/VtI1hWUW7MfiLB6dqivhh3G8wXnRPNgHIKgjCYoQF0329cXQx55tBE/G/E9hhqf1eUy7r7UZjOtcXN/dp6A75Gtu9J5lnMBwNc1scZ41l9OhlhFaUy8Dzp+wGok/OZJS2E5PdLNDQWcn5fvx1FlUHT6EuF0pL2eX6V4ZYHCE9biPfHy2jOzuHfkEPJfekrX8UeMqEbyjnbtJjIrNeKfwZDvnEi6jjUXo8kdllbV6mvKoCrj18zK2S5uVMIcz9pIfcetX7mZgwi0vIiudGTjm5Ta+ITsghTSLh7kE7JqxQvEdUPn/qz4oHPVUEz+5sWdhUxu2kdJ7UStoGgO7CK+s1VQZPZdX01+ZL2ZbaP7Y8wq+WbPUP7vXeZmVtW8oAnr21RdabTCKe5pIqaiQ+/Qn3K7quAhQ2vSYmPpG48iayXqcTmd38EGR+3QtuJmaT0rLaI592Rmk6EQk5pIoU9/fyYT/nZ1UEz+70yH1VxJPa/tW0u/wp47WPBk9lLExrnoQnm68ddGXkKFeC8noPsq3xfOnHgQaeA8WeavBUvk5+oIBn7ptkrma2e+ZUvc2oIniquuYfyv9AA88PlVd9vWt/PSDBU23orob+GE2E1wCam5urtHfwY8qt7Pe4L1lCfIl6L7Iy2Ul4DaCVlZW6rSjZtqQZnp48edXxZ6KUqd58iXm5kZLChQcP1G1FydpKX9bFD4KnogDKfK4uAX6aIWH9cgl5ImXOqfLnLb/llZnKn9MvK4cuLi4cPXr0yyq0kpf2WcsrM5U8m32YPTEFcTE8q5JPUsKrpNskl4rlT37Wz46Ojpw5c+azpqGOvHcKJCUlMXTo0N7dpA494BXo8K52RaVtevuQgAUT0NfWREPDnEUBDxD6l4or37HiUDESRTfJnWssPctCm+Vcf9PcATW+Poe3zXdEtXyXC9qjj1KqyU/N5a0Ebn0v5QdrTjYAACAASURBVEallCw/KdvuS3t0vzqQYgVUFTylvOVJ2GaWzvfihx0RFL4TyiciL3oP3y/2YceZVOrkitxYEcXh48Iv0zb/VaT9wmYfb1Zu/oWMmtazynNUZfDsrLWgqpRa7gdt5nym6s4UlQk8y1JD2bhsPt4rAojKkVV+xNUpnNy8jAWe37P/ci4NctW5MiGY7cez6IyDTWUPCV63hIXefpy4W9rWr0ulZVwPOiUXg4RXCaf4cYU3a3Zc57kIGisjCA0vabunNbBEmkvY0Vs056r17Oc7Khd4NnJ7iw1mdoFkdha7Gwkay8NYNv80JTXnWeF+lJcKBliJJJ0A97Xcq28f84RxMcxnPmFFvUism3x8qkuqCZ4NXF1jxowND9rqbn32fuYtudQmS3nkXo4/bQI+zs5tEX2hH7oFT6m0hKPTRjFlYxyVEmiqjGOVsQZ+N+t5c2wyzhtzunRgXXVs4PEWM8xmXqZcUsJJNw1mBHTt+Lrep/iMWHybxQae3GppdIW/SPH4NwnXy9sboeI71We7U0BVwfPV5Tlo6a3m4v27HF2sx/il96l8ug4jzUWE3onAz2oUy8LKgQae3dzPCpu/Y+QeJZOi4dVJnEbaEXg9jojAKVhOC6eyO5H64ZpqgmdXrVule3XLh6H/9nfWxcjjUOtV1TgqC3iKqi8yY6gJG8PvERvqjeGw73jcWEnY7JFM+v4cd++dZoGOFpvvNGstqo5m4fD/wMTzNo1yUkulrzjsrMWsnVHcvxWI/RAHwl6KqcuLInCNDUP/vKAtdH12IBM03TkcE02why7T/LOovB7EyTQRwngR5j2G//qDJmvOlcoG5VtBwST2kamVCTzFolg8h49BZ6QW2xIEQOnZn7SphPTUEmqrD+Kgu4UCBRwpljxihd5MIt61j3lS4cfiJ5nxc7ZyTehUETyl1HHC5a/8/vf6bH9YLzNcXco6zKyPyD4LkH921VpuvZPysXbuWW0YuKG6Bc+G0r1YD/YlXq7jKE+LJDq5ijfHnBg1zg1nOzNGf2OEf1QVUmq4F+iCsb4xuqMM8A5Ol3VwElEqm801mLbMEWOrYArFIJYk8aOzEzOmjcNIbySmdntJqwdx3WP+Oc0UfWNdxui6cTqtgVehc3Ga58W40YvYfWAa3/x2EI5ro3kSI+WNBCqPSPnuaHsjHLjm+nwlU03wFJN6ZDE7Ipt9KnXJfhibBRK1yYiZO1/KPDAVEXMxn36FBhrIjA0jZLWF7M1FgpJvI+Yyzv26rI4KE5pFI125IteZfz61ex6zqoJnZ62FEjeWXWaBtReL7LRYrwbPnleC94SszzmM79oomUdTLHnKytHjOVSSSfDCDdyR1WMRD/x0cNv1CgEuz3rZ4LnUDmuvjuApkeZzfudJMkQgoQB/Uz12Jouoy4vl1ImVjP3GsyUHYlK2mOP6z2Iaa2uorXtNUUktb8P3cbJYjEh0jZl//l+yd7Xrud2QrTwkBO3v4Jl7T1E+yWllAs/KKC+MbY9zf7s51j6PqC0OwmVCAPmyse8pGybM5mxJPie9zNEzNsNAYwzz92bQUHkCjykhPK9sBU8x2eELMdE1xtRQAzOHAzxresRybR0mOFtgpDscY5tdJNW1g2dFwm4mG+tjpDsaq+khZMmN359E6F5EorLgOV2TBTt9MNXbTHoDyIOnqD6Cdd9fkXlDO9tZubC/F4bq46DdgmdNnC/6Yw/yVkGm3hxzQNvxF5mHqOzcTCYtiaOqcBc2Rv5kiUBUdZlZw1y5Ut0MhFUxXnz1fw1ja3zz7E+YtfkM+4ZV1wRgreSipxbzQ0rJCJjA1I3pCAYsjZiL6eRfyAmZxMixwTyXgLzH89EKKctXSPAfKyEkRQ2eCszU41OqCZ7txastusoqS03mh2RycaE2vuHtMGo2rr0Ov706F6s5zR7Pd1nbMB/lzc3iSrKvLWXEv1pwpESBi6E9mT7/pJrg2SyTvNYS6QuOzbbF//YbTs3UUYPnJ6xJksZCLqy3xtTuKMVt1beB7Mh1WI5wJiy/kbzQOUxfcZeX52d1Ac/WrNTc/4lxI/6MhuVBmXNAOC9quoL7KK+WIA1c8x7BMP3xWE60QnfkWLbHVlEbF8Kpx0K/LiLj/Ho8523gWq7wvZ7I4MNkt+WpNaXPc1QW8BQcMGc9NFhyppL6wp1YfO3Lw/pMfjQ1Y1+WiJr4VYyzOUppfRz7f7wgG9fe5W5hnMkenr/ej71JAEUt4Jkvruf2vi1EFoiRSLPYaGhN0Is4fIYNZvX15rHzgqcGs4KeyTyeu7My2Wo1lr2pTQhe0LMemiw5Xf15BO9BrKoMnmtjyrjuq880/3Sq5DyelTf8+OlstczR1tnO8k66HsjzxQbpFjzrMjZirOUvm6W1KlSbEsr+c9mypfbJG7JlS+1VkQuw977Hm2tz+fsgExwcHXF0tMVc05UzxWKEPSlbLTVx9BiH0aRjzQApLBdoTOZcbTMwlp+ZxgTfu4R7DGaosQNCJ+JgZ4rFlBCeBtvj6JchS0sePIU8VRVBSYdN7a05VR97o4CqgqdUWsHt3dMxMnAl4FoBjTQS5aPL0tPN4FkTvxLTCceobRFDHoaEgTHlrB+utrbM+O4Hpo5wIrxGuSYwAwU8S8+7oT32B46cOsYSk38wdf1F0soVbGDrTaXtp7DKstQu+CdfPQpgir4R8368TnHL+nnT2wdsczFk3DR/YvMbEFWF4/KtNesPnyLkO1NGWW3kWqKw/aTrn6gujX9OHM2qiOYlxq7gORSnjc39fnWcL0aWhyhrSuRSeIbMWSAfo/hdPJcv9WQ7lvxdH/9ZWcBTVHMO5z/+jm90DDEwHMVf/n0QfjF1ZOyyZurmdG5+Z8bSM5VIJEVc/MkDJ3snHG01GKEfQFEn8CwQi8mJ/Al3J3ucJk9C4w/m7CuOY4XGlPax8/RUrJZHycBzR8YFZg76GjMHYQx2ZJKZNouCnn+8qL/yTlUGz3XRDYiqI5mnYc2uC6tbltrrubZ2FRFVUhTZWZW3EP1KU/fq9m7BU7aXZLQmG283k53wIEe4x7fM2FXcYY9nK3hWJvoxwe44FYBE9IzwwHPkiBuJ32aOhdsVysR57LUdzuygfBolj/AdPobtic0z5cfrDZn+z1zu+5njFfJGVoi6zHME/fKM5yEOTGnZT9oZPHtVWnXg9yqgmuApofisG5bOpylq27QmIf+gHQ4rkxAhofCwAza+j9sGRXnwbKq8w8EDd2UPyzW82MtEjTWyH1x+r0j9cGGggGdlYijbt25l69bNzND8K+MXBvNAybzLPTWvsoBnY1kY08fM5Hxu+1qqsKR+fIYB3ify2vZxiuuecmq7oP1WNszW4ht9b47HlrQVV9Rwkbl63/FAxpoN3Fg6hoXHmp+06wieEjIDJzBjS4FsG0tNwipMrUJkq151eWnkdXCsSXiTnsbLPvJ2CoVRFvB8GTYNw8nhNCsoJiPAApPZN6l6Hoi18VSmGHpxq1ZK9a3FGI8PQWgG9QXbsDLqCp45DdEsGGHPmWIRwsNam4wtZeDpO1ybHUnCuqCIx+uNmR6QJgPPXZkPWWXkTPgbYQLdSMr5vUQIeyj66U/VwVOY3L2MmMvIoYPRsjyCWHSLDStCZavA77Nz21DUT5qrQrLdgqdQgDePtjN++HDGTnHF0XwEumP/ydNaFIJng6SAQzO0MXScg+vY0dj6xlCctAmjb+Zwo6y5B6ovCGLCoIkcyrqPr8bfGGVmi9uscWgO95I9+V5fdIJpmrpMnjMT85FGbI6qpEQePCXxLNcaxdx9jzo8rakKYitzHlURPIVN4Kdc/4vffDWYIUOGyP5N3c7wqjKKJSaGOHvOwkzLhbCc9o5XHjwFT/z2CSMxdJrJJG1dvgtr3heqTHYaKODZrmk9p9VL7e1y/IpPFRdm8Lv/+CuDW+r+yKEehL0MZ8rvfsN/D25uD0OGjGTFiXbv5tsL7UvtNY9Woqe1mUxJBVeWG6A1dhbz3K0w0F/Fvcpmz39H8ITG8qssMtLFbvYsLLWM+emG8jyOpwzgKZW+JthhGL7n2n8io6E4EIuvPIiqLWKn5W8xW3RPNikQxkKbERbM9/XGzcWCYYNsOHBnT4el9tymXALtNLCe68ui2dMwHfI1zntPstxgBHrmtsyaNQ6d0cuIqWje47k3u4HMkzPR1nbC3W0sujor22z5K6raR9+q+uAJUmkpJ13/m6EWR6iJ28Smo2V0Z+follXcjxbtC7jxg+ApaCBtLCc36TFJWa8/CHsCDLzIiCcxu7zNy6RIx9Yn8y6WvSA1MYM3cr+5IX1XQlr8E/LK2oFBPo6a4mQep7/uwRP18nepP3engCqCZ3flkTaVkZ2UTmlt98u5UkkFOYnJFFX1oWumu4x3uqbK4NmpKAPmq7J4PH+toMJT6If8g1t+sqeR0qzHPE4s4O0HmoK0qZysxCQKyhX3z782Xx97vzKAZ2/zXv8mi4SnuVSKGinJeEJ+Rdf+Str0mmfxieSXi6h/nUFidjPsi+uEsTObcgUutprSDBIScqjoZxOpInh2Z8PG10WUtu7b6i6g+lq3CvQIPLuN4SMvtoKn/E9CfGRU6ts+gQIDDTw/gSRKEYUaPJXCDB0yMVDAs+lNCgmZ7Z65DoVUwS+qCJ4qKHOvsjzQwLNXhVcHfq8C/Qaewt4JUUOj2mv5XtP07QU1ePat3j1NTQ2ePVWq78INFPDsO8X6JiU1ePaNzr1JRQ2evVHrywnbATz78v2dnyKtK9ulbe+ALXoHO3e2f/8U8X9JcSSUlmJhYdGm55dUdmUu68otW9Q2UbL3HD8sLmb8+PFqu7TYpQgx9+/HcKuqd/1vbnki4fGlFH4i+/pu2qS2ySfS8lP1iffy8r5Im+QUxxGeUoXQNm5FR3NfjEyH7LwYwjNFFErLuBb7hERJ8/lPpbcyxqMIp1UWPGMOS3H9F4nMmEUiCPKQYD+u+bsyiq/sebrX8q52Zc/nl5Y/WxeXL7LjVmY7x7S8q12Z89iXecupjGBfeEmPATKvoYECkA3KMWeOc6G2d8D6vrJZOzqq24qSgWdkUtIXZxMBKkOOX+SpREJSzlNCjh3jxJNsEsuzuXz1GHuuPOF+uZisvFAO3Kwd8Pr0GjwLpBn8MFWH4VpajNAag47FTPyuFH+wgymQpLJulh/n67t2KHn1l/CcfZSEjyD9Iqq5l5JLWsu9R5zbQVPweC6R+/6+zkl9XvEMS1XBs4i3XAnbzJz5XizZEcGDd1KKEHEneg+ei33wO5NKplxnnFMRxY7jaW2NPTntF1b4eLPwx1+IrelaX/u7vqgyePZE69yyh/ivW8xMbz923/l03q/PaTdlA8/OOhdKyzl/2p95i1aw5Voh+UChuIhje1YyZ9lPHE6paav/rToV1KQS6O+D26K1BLTYQVE8Qh989aw/Hl7efHfgnsxjk3r9AHvSBC9OCXu9tfjjHzRZeq60SxpFNJLd0MCVs+HcamwkVwQZ94MJfNjQJWxrvnpzVCbwFMoausUGQ7tAbol63q9kl4cyd/5p4mvOs8Bd8ThZIEln3ay1HcZXwS57feZzoEj8SbTsje7dhVVF8CyigSNrzJi84QFZLWNHVvZ+XJdcatM2MXIPgU+bUGTnvKZI/rk/A+FHx+KTY9l37BiHYpN4/CqJ0+HH2B0eS3SpmHzRPbbvf9wWZ3c6qvK1XoNnviSBBSPHsfOFWDaYP360Ad2/zOL0u+4bUr7oIfP03DimIFxu3REm6m0hrsX13BtB88W3cTfw5GwL0KrBUzFE9kbT1rCqCp7xl2YzSm81h+7fZbu3LmOX3if5iR/amov4+XYEiy1HsiCsHKEzibm5n/k2f0ffPUrW2LNfnWD8SDs2Xo/jcOBkzKaFkyYHqa3a9OdRFcGzp1qnSF+xzVkT551RnLsViOUQBw68VK6BU5HtlQU8FeksLO1d2zMRI9dgTsUcwUlrLFsz6zm7zhCzBac5fW0TJkPdOF7e3ocXUc8hLw0m/BDB+bvCT/zYsvd5o4J4RKTc9ELTaB3HHt1mw3QNXPe9IOnsz+wpFpMnuorDn/9V9spMHbcbXQbUIuqIjQ1gwrDhuOy4SVyNlMyUIHbdqO8SVpHuHzqnTOCZL4rFdfgYRo7UYm1CU4/LV9hUQnRqCRlVBxmvq3iczJc8Yr7ezA7jayEVbJxkin+2qMdpfUjPT3FdNcGzjp0uX/Hb3+uz7mFz3XyW7IeR9RGZtgLk71v1A2ffSVFk57yGs2z7+bnMQSe0rf0+PpxqYaGUy8tZEf5OFk+BJIlte6KVyl6fwuad4/hV4ClElvfuEg5/sWH3mwICvMwYbWyGpsYY3PZmkCfJY+cCY4ZoGKI7fiLaX8/g2DsxcTd/wEzfEG0DfZz8bpNYd4SxQ6xwmmqLzqjBjF8RQ4bwirHAaejoGzN6lAEewenkUMvZnU5o6hmjpaHLnEPPuBriwv/320FM9IuSebHU4Pllg6cwyF4/vJj1kc0N+VnSWvTMAjmxyZCpO1/KGn7yFQ+Mp18hhwZiY0P5/9l7D+CosjRtc2I39t+I3ZnY3X/+6Zme7p42013dVQVFYQQICYQMwhthhBPeIwrvCu+9NwIh4b0EwgsvQIC8994hj7xNKc2zcTOVPiWErRR1FQF589xzj3k/957vnMzcs9oRuxkq4hl7byb20x6RBgiLmimdJnLOxGLJ0JC+5Pu2Sjxbg/Wp2gw8D1ziWaOCLEUmK+2t2BxjXoHTlKzNiXga4pylSGZJ7/7sz5GQXF1LYnE2YZVxLLQeyEFlAqGWYzMs+OmWluxlK4rY6tSFpU9ryJJnsqqPFWsi4ozaiayWE3h6BA5znpOCDL9D/Ri0KprYwFO4hQrZHyl+NzcyafZmLqbrk63M2hfsOxlFYuZTzga85tzDNOV2e/zTkxyL/zQyNyfiGfPYlZ5OF/DZ50DfpSEkvvFgxOBDyoRLpjySJYOn41GQaRRHU8ov4jLmNMHlauIp44XPPHpY2WLVywKbkSd41hjMrO49cBjriKXVj1gPOcjDWi3xjAk/zGDbnlhadcFhwmn8JdpFhil9/pxlbZV4HpzQlUkHltLTejt+EgW6xDOj/h5Lfr6nzIYaylnYXciUBrDfM1y50yDYxNNHj3glVcXqlJTHeCeq9D2j8QkHPBNF4mmogELGc06n/6LjkFEMHOWEVce/0nvGDULqA9m+7RahckhO24md3VFePJyL9YBTyi30hMTtdP3/XDhVHcRsKyeO5sjIbAxn9YR1XCo9g+NvXThXqiCj7hojuq7AO+UAfW124t+oIKPyDs4/TMSz/DXTOg3haJ6MtEI/dp8JUZID3Yyn4XjF9x9ORNtqxlMt88Sc+8zrb8GU08mc/Kk785pWlQIZtel3UpPJjPWdiWMT8UxO3k2vzgu48qaM5w8W0e5/OHKwwLwybm2ReKpl0lqsEwK2Y9PxD3Tuf5KgD9gJUff3pV7NhXiq56uLc6b0GePbt6f7oP44DOxBN9tNXC99zISO05RZFyEQ3thgxaRjZXoBL9xvAX/9t2/4vv1v6TTgNK8lxu3cqVSQUfWE8d/9B3/p3I4//MaJ45mCb4/itE+iMtCqx2T4mkMNjyOS9OoIZbdv+X7Q7pdh+8J7cyGeOVRzbEYXZl8tJzlrPzbfLuN2fTLL7RzYmSIlPnQV9kPPEW4ijgYXuzPA7hBBGuJZj/fxXVzMkqFcVNgMZFdeIDM7fMfiR5VkU46HaxfGeyQpM55bUpJZNaAvO+IaEbKgwiJj9pUqPVmbwu5zlbVl4rn0WQnnl1kzamcCcToZz9jH61l9vQpTcvaVCEe9JNy7eYuXjcK1nLDYGCKbjgdmFMXil6uKMXFht7liZhnqz6EHH5TxnNPZljUBiTxPTSWosFaZRcqWZ+O5YwaDRjgz0MmCH3oe4Mah/ozclqG8nyl7wZTOk/EoPs0gi4081znjotxq77FT6WwypL6M77mMC7dn8N/f2DFglIrg9uw6kWO5Nfien4eNpQWW/aex6V62SDw/4zZwWyWe2YoyvA67YNlrIlseZJFOAxeX9mDOFVUWVHDyvQaf15zz1A3SwjbIo+vrGeHkxMgV6xj6ozOeZnbO82shnu/COqMmjtVDurDIV5uJ+xxO8FO0ae7E0+UbGyXBEQLgpWVdGeV+h6ldhB0oIRA24LWqO1NPaclIpjyMuZZ2bAqtIqsxk+2jLZhy7S6G7Ux0L+T6hp4MXRtOEg34nRuDrcstpW0lpcfxuqr5zFqOvIr4Su2ZfEEOGWUJPM/5NNlOoT1zIZ4Z1T4M/t1v+KaHDd1sOvP7f/mGZc9qeXpwACO3J3BppT1zr5ZjGEc79DxEkBHxlOH/YDujnUcwaPQwOvxnH3a+CWSOxRhONX0oK+ryWByXP1ESzw2Jtxj5t2+xHinE0lE4OlgywyNXJJ7vETuFIyFCxnO5n4SMqoe4WAxk4601yq12wY+dW7+a88IizISclz9TnVfOkqTzLFlrY4Z+Rzg/7ReXq8z6G9772t5/EPFUn/HUBSPu+QJ6DjpNhAySs3Zj3/sQT69PxnbKfaUTSn17hr6/m8DJKl/GdpzOlSoF2Yocjvy8hQuFZzRnPNXE80rIOvoOv0AskCVN5MThGzxviOfEsfsEyWREJh2iT/v5IvF8D+PRlVdrrtsi8RRWk4HXJmM/9gpBElW2Vyh77TmcoauiyUROwOkRDFgWqsm06BLP9PKX7DnxigQgJdeNvhZredrYfPBsDY6fus7XQjxNYX2/5hbjrVdwp16VJbiwuBszzht/8OVTY/qx7Zkz8RSOLKzpP5C9mcK5/Aa8V/dg/Il4Ng3tzcbIRrIVhWwdbs3aIO1WeEbjPZzbTeFClSAHKTc3WOF8JNSonYkn8nCf2Zm53qqAmhS7nl593In+jH7pfWRlLsQz1GscVqN9SGr65P7Tg32wmf6UuDdHsLcdxxAbV67XKDAVRw2J58t6PyZ0HMHxN1Ky5Gkss+2vJJ4zf7RkU7RUKa/bm3oz+lC8knhuTg7GtfdYPItVi4xHN9w4l/DpyP37yEOo25YzngLxFOJJyL2ZfN/+e7r0P0um9AXLlnspd9Cak7NwdOt9cfqa638y4pmS5YFjR0cmLFvAKJc+fPfNEPaGv2Kh3Y9YjZvOoH52dPxeWGFLuHVoKF3sJuMyoTe2oy/yutqYeN6uz2T3pO50HzWD4X27MGjZM2Ll6Wwe3oHuY11xGd8b+8nXEFbms7t1ZtLxEFJF4X5S5W6bxLOWQxN/z7/+6Xu+addO+a/XlKtElD9hmq0NQ12nYt3NhRNpWserSzyFT4euGdwJS+fJOFpaMd9bdS7UnJzA10I8TWGdRRmnlvekY9+puEwbgGXP1dwoNy/ib0oXzJl4CoEy+OESLLs74SxgaiVgKiPMbznW1mNwmWaPrfM55ZGG+OCf6dZtOy9ktXjvHkSn3hMYN30gXbsv5HJRo4l2FMSG7aJ3ZztGzJ6GbdeeLLlVpNzlMoXTly4zB+KZrShm58gfcL2hXUCl5Bym1x9ncrk6h3X9/g3b+a9JFxa7JuLozpdH9LbaXzems2l4F+xnLWPKtPFYtfsWJ7dLzO71IxZ9nBg1tS9duizBq6xUSTx3pErwuzSZzpbOjJ7iiEWPVdz8BW2qrRNPQYeFxdrBiX+mneNZEgK38PO5ElqSs9cn+nqwL20/n6u/9yaeLQ0k5W0yvpHpxEkbCEuIIKBMRra0nFfR0QSVaVfTwgdA4nJiuB+Tq9nuNNWukN4OSQjjYWqJJjuVrajgdVQw9xMKNEQz8U009xKKfhUpalM4fa6ytkg8W8Iiu7EE/+gEImr0t/cMn8mWl/EyKobgSvM626keZ1smnuo5qF9NYS1k5cJTQrkblUV8GzjfKczF3IinGl/d15SyVB5GZRDf9KEG4V5KaSqP4/NJaTpvJnz90e6dnspz+QJhjc+LxTfS4BkT7WTW5PI4NIqgUq2f1+37l7o2B+L5vnM3FUcN28huLOZZWBSBpY2kFCfwMFV1PjezNo+nUanENu326D6XWJiAb3gacTry173/pa7bIvFsCZv0ohwiasSMZksYGd77pMTTsHHxfdtWxq+NeH4t+vg1Ec+vRSZtgXi2Buv0tzHcT9Zm5lrzjDnXaYvE05zx/BRj+9qI56fA5NfWhkg8xe35ZrfnA7OysLa2bvb+r81YzGW+k1xdiS0vF+ViRrb7PDkZW1tbUSZmJBPBXsfNmEFcRYUoFzOSy+PYWPxTU0WZmJFMvnRseyfxNFXhayoL3a9QTUcCt1YoWDdWzsXQprKvaaIfMJfMpp/M/IBHxUc+IwIuLi6cO3fuM/YgNv2+CCQ1/WTm+z739daXkRX4jKRK3RnKKYr2J6ZQplv4Wa9HjRrF1atXP2sfYuPvh0B0dDTt27d/v4fE2l89Anq/1W44W5n0KStse7HWq1RzSyZ9xdr+Viw9k0dDiRcLJriTIZNwf609g+b6UK6pKcFvy0D6Tb6AQlHEhXlTOZ8iBeQUhR5n5mAbLLt2oYf9VE4FaJ/SPN5MPYUin8sLRrD55lvkgEJRwNVFI9lyu5BYjwl079qNbt260a17TwaNW8sD4XfZgIQzCqb/k/AESG4rWOkmR1YGy4bI0fOXyhq/vv/aKvFUUEGE93YWz5nHuv2+ZNcJspOS4XeUnxcuZf/VOGp1xNlQ9oQzF4TfJ1L9lcVfY/vSBazafo3EanWp+by2ZeJpiLWAqoIaAjy2czNZZZfmg3TrR2I+xFNOfsgZ1i90Zem6k4QUypDLY/FcPJ0pU6Y0/ZvGpiu+RmXbr2QYTVihKOGRx2W9cuMyY9tqKPfFy6dA6Y91H5Yr0vE+9wKlSere+EzX5kU8G/DfNRSH4UdIfg/u3VDqzZI5VyiovsmKaefIV4UsPcTk8gQOTVvP63pt0kRBFd5LgR4ypgAAIABJREFU5+Cd8x6d6bX6ed60TeIp8BkHJm0O0uhufao7sxfd0YBU+tCNC5GNwIfJWdPQr/SiReIplVzD+Y+/oeuQs5Q1AVT5YhEdf/PvjNmeiaTwKAM7biBBWstFl7/yh9+NxPutyhhkjX7M6fJb2nXbi1yRx76+VuyLaqSh+CpjfujL4delyJFTGLwJh/YLCdAxIqGrlupVx+3Cvt1MHpdIyb4xlV593MiQSQnZaMX4XdkqQiqvIHh3H6zG3NCI9t44raEihVcbFSzdJRcuf/V/bZV4Ft2dQTfrNdwOeMW5hdYMWhxAeeRGenedj9dLXzYM6MwSb2HhJCHpqTsrhv6d3tOeKOUtKbqEc6fhHHkUiO+RMfQfr7twMg+VaJvE0xhrNZpFL5bS/p//zsZnEnVRm3s1F+LZUHKZ0Z2cOf4kmEduztgMOkuRrJSEgBc8f/6cZy/ussWpF6t8043K1t5Xe3QV/LUZTziydijt/zBXIw+TZSZsq+qRB5fipcokgPeC7vzXf3Zl7Y1CZVB+4eFJ1BcStTkRT5n0Oa4/dqdHp27sCRcISuv+FI0FJMQVUFN1kpFWu8gywSNl8hBWWE/Gt04bzxTCl8UPc+BYqnlFs7ZIPBUIfObP/Md/9GRfcL1ScLWxG3EYeFZ5LZD866vX86JOwYfKuXXa8PXWeifxHN9xLC59R+NVJFhAPQ8W9WX8RDvGGRLPCd0YM7Efcz2KlMRPIKiOLhPob61LPCWkHx9E/wUBNDRhKmSsoh88IlEv7Sh/R70Gwvf1pcfYRYzs5MSlNMHY9Imn0Hyp70x6DtduvaiJp7wUjoyU4/lIoRnH1yvi1s2sbRJPGXFnF7L/oSqnUhuzAVuHIzzZ2pvJB/KVeljmO4s+E+4hQULyc29Or3FU/nKRgEqF7yz6TXuk1AGZzJ/5nSZyT8eZtw65z1urrRJPQ6wFlBpK7jJ34DzmD+/GJpF4frTiSLJu43Y+Rrlwlrw5QL9Om/Sya6WvVzNh3iMqdHoyVSbcrs14zuWLq+j7naumtnGZlIittka2VeRzjEtvZEilD5j8h/9T+Vvt1lMeK31yuIe7XmZO0/hnuDAn4ln+ZB62ThcI2NeHgUtDqHnjgcvgQ2TKQCaPZPPg6VwvyOTSvD5Y2zrQy6I7c9wSkZRfZOaY0+SWq4mnjFSfn7CzssXexgKHkSdIagxhuWUPBo91pLfVj9gOPUh0rZZ4loUfZrRtT3pbdWHAhNOkfCHib0qkbZZ4TujK3ANLsbfeToIEdImntN6XjT/fU2ZDDeVsXrTflETMo+ydxNOl42Iu7B3KHPdCGhv9mG+/hMv7+5sgnj1YfnUXo4aeolhez4OF/Vl+bQ+De2qJ596oGp4u68yMY/qrbWMoGt5ZT9r4hOl//mf6L49oylgKxNOS//rOngEDBtDfsSft/zoAz2it1amJZ/FhBSP/U87yoXJWbZdr0unG4/j1lLRN4qmVT03OfVb378qc08nc/smSZT5aMurQ76Qm+Fbcn8WAGaqMZ13KHvp0XsDTN+WkPlhMx//hyNkCEykGbTdf/KptEk8VTLpYC7se56c7sdP/LZcn9xCJ5yfTpAb8dgymw+//zDT3dNTaK1cksmOYC1d1tl5NlekOQ9p4j2md5+kWoV8m4Z4J28oLPMXlUCGrJyXx5iZcZ2/mQbrwvp6HnmdIVQ9Kr+VP/8ZciKeCaq7PtGDR1XLqsw/g+O0yguuT2WbvwPEUKdVhq+k39ByF9YG4b7tFrhzq0nfRz+4oucXujLA7RE4T8cyU1eN/fBcPs2TIFSlssRmIR14gSzt8z5pHlSgo55arBVM9kpQZz8Mpyewe0Be3uEaELOj1mV1ZdKXq04PdyhbbMvFc/6yER8t6Mn5nApU6Gc/yxxvYcb0KU3IO09KNViL066z2TuI5oeNS/FMPMmqwJ2nPFuE835+kowNNEs8N/onsGTyKMzlPmO+wjBfZh/WI576oOsK39MR5c6rGQQpboOFeR7ibHMz+KU4MHTqaI89qmq33QHlOVErkoYFYjppB//ajuabcj1BlPEeujyS/oID8vER8FvVg1Abteb5fp4hbN+u2SjwVijL8D0+gd6+JHHqQRQMNPFlqxeIrKuJZHbYK+8HnqWmCQZcMCYEx9voGJjo5MWnlOsZ1dManWrt91TrkPm+tr4V4Ft6cgmXfdZy9fJ5Fdv9g3KbbxJeaOMD2eeH8JK2by1a7djJyyhIPM6T9HF40HVkqezSPYbOfohsHTZVp28CAZKru6BNP07ZV2RjFHZ9EoyNLsrow7t5J0/H1ur19+mtzIZ7S6huM/d2/810PG3rZdOaP//ING57VknhQiJsJPF3pwOKr5cjlOdzeMRPnEc6McrKgY89D5BgQzyyZjLSHO5jmPALn0cOw+M8+HH8TyAqLMdyoUfmq0ivjGLD8iZJ47k+8xeRvvsVh5CgEPIY5WDLfI/fTg93KFtsy8dzoJ0Fa9ZDZFgM5eGtN01Z7PQ/Wr8a3UoEpObflI0StFOknqdYq4hkkyWDPkCFMcenH1hf1pDZDPLf415F02IkhU8czeuErqvMNiWcjteFr6fHDckIqVUYjrfZlyt/7c1ZnZS7MrKV6NfF7cPhhFk9KGkk6PYKeA06TY3DGU2ij4t4M7Cfe/SRAfe2NtE3iKefN9Sn0H3uFHPXZDeRknhzOyFXRSJGTfWYkQ5eFaoKiLvFsLH/JyROvlB8uk+S5McRirfILl81J1l8L8SyP8mLf7t3s3r2dSV3/zKCfPAkys+xya+VuLsSz/O5chi/2Vx0VkfrzU8ex3KpWIBxfujSpJ5ufa2mnqTLD+eqTTNVd/bLmbas2I54MvcSanLcJ8eR/oWynMFpzIZ753uOxGe2D6rOKMhIPOWI3/SmVuUcYaDuOMTbzeFGjoOrFQmwHnUYwg/qsPQzobUw80yR+zO04gqtvpAgf1tpq219JPJf9aMn+aNURs9BNtkw4FK8kngeTg1ndeyw+ys9aNBB70w3fxF9uA7itE0/hw9D5vrPo1P57uvU/i0z6gs0rvJQ7aM3JWROKDA1MfK9BoFXEM1QiJ/loX/78p6UI180TTwmSN0fo/c/fscVfgiT/iEHGU0j/V+C/34lOHfowZuJo7H/swISdIUbb3c3Vq26IZH2vjiy/WaI8wydXZHBkaDsmH03S+3CRMMPqoGV067xFM1nxonkE2iLxFA6BX574X/zbn76nXbt2yn/2U65SVP6ERXY2jHWdikM3F7yVZ4BVc9clnsKnQ/cN7oSN82SGWVqx0lt1LrR5lL78na+FeGqRq+eKuNWuheMjrqSVj1lo1ZlBU+cyeUB3hi/3Uy6ipI0PmN5uMr5NGTGhC1Nl1SGrsO62nZQmcqhPMlUDMyxrqGjetj5iKp/kUXMgngpFMZ4jO7DshvYrMoSY6PinmTypyeFA/9/gMP+1crFQn+XB0I6OzFm2gCkujnT4ZignXh7V22pPb0znyHALBs5axvzp47Fv9y1j3S6xvFdHrPs4MXVqP3p0WcKzMtUZT7dUCcmXJmNp6cy0KX2x6rGK1+W/3C5O2yeewjfnFHJp4l9o73iW6sCtbD1XQkty9tOxu0+i2F9hIy0Sz88537qyDKJDI0kvUn1qrLm+WluvuefF8tYh0BaJZ0szUzSWkBqdQGFNy9u5CnkZaVEx5FR+wdRMSwM3uNeWiafBVL6at+aS8RQAVTQWkxQaTGxm+XtvaQtfRXdqp6fJr+xpSVitta2W2vgc98yBeL7vvOrfphAemU65tIGCxAgyy4z9lVLGYVFklkqpL04kKlX19YOy2jziolIpNZFiqy5MJDw8jbJfLtmphKItEs+WZNhQnEOh+txWSxXFey0i8IsRzxZHJd784gh8bcTziwP4mToUiednAvYjmjUn4vkR06DxbSzhydrM3Me0ZQ7PtkXiaQ64fc4xfG3E83Ni9WtqWySevyZptzBXkXi2AM4veEsknr8g+M10/bUQz2am12aLReJpfqITiaf5ycQcRqRHPL/0b3iK/WE2v2Eb9fYtgwcPNpvxiLqh0o3tx4+LMjGz3zkOKyjAycnpVymXHGQEBDzjRaVCM/9sRQkPnkcQJf9l/dnmI0c0YxL9xy8rCzX+QTk5okzMzH+pZfOlXk0RXZF4/sqVQq18r5t+q139Xnw1D8ft5OIiOm4zs9FnTb/V/mu0kbRyX477FJAN5EglpMlUdpKS4cWJpzW/qK4OHDXqF+3/16gP75rzw+hoUSZm5r/eJbNPff+9iWd80DZGLriF8E2YwmCyFcUcmj+GbaF5HJo/n9OFMpNKlZRwhP7du9KxWzc6W/dj9OLTPNVZIZuaWLaiDO/jc7HtZcmPXbrSd9JR7pcrSC28xuQhy7n8VtVXWrEPU4eu5MrbBh56uNCxazdlPx2796TP2LVcTJcajSm14BqbT8SR+QkUQFjxP79yAI9U435MzautlLVV4plDBfe8tzNjzjwW7fclqE5BDlJe+h3FdeFSNlyNI7lJ7lFxXixdModpKw5yJa1OqScpuU/ZuGoh87Z686Jam8UxF7m1VeJpCuuYhGssXzKPmWs9uVtg2neYC+4tjcPciGda2RP2X4jX+L2s6jiO7FzKlPnrOfSyUEkSdedjSuezZTmcP7qKGUt2cCa2WtmWqbK4Ryc4Gi8lrV5CYowPZ2MkpDdIyZS+Zp97qGYMuv19qWtzIp45NOC1ayg2w4/wQtp6v5Ja6sWsOVcIq77J3GnnCDeRRc6SJ7Bx6npu1mvbzaEKt6VzOJFjXnbVlolnpjSYJYN7MPFAPFnvwR0+VIYZtc3L/EvZ0Ofo572JZ0bdXZy/dcbzrUrBU0vOMuDbufjUSQiNjydSYjorFB+6Cqu+nkrCml6dydkdA+jW14PgptWx4eRykPPay4WutjvwLZWRrSjlyiZ7HOY+JwcJt3c5YDv5LjHyAg5N7sKYQylkIeXWxh6M2pWtdKzZsgpu7nag+5gbes4vhxrOLJnCnpRPRxRTi88wZaaPhpAbzqctvm+rxDPsznQ6W6/hVMAr9i2wou/iAGIiNmDZdT7H/H1Z2L8Tc71Lyai6xcj2diz3ec1VrwX06LCSW/WhuFpbM+/yCzy2D6D3+BtmJ9O2SDxNYX2z+jHjOjqy2jeYSycnYN3/JJEmgmpbsB1zIZ6Cb3z21J05Q/9Oz2lPlH4vh3pOzbNg8Dpfbr4Svq7HCbdcLRnJlIeZ0Hkp1zfa4DD3ClcebMWu/RQulDaaKFMQe/0YR9/ISMh6xmqXjlg6H+R6ajVZ8mj2HPXT871fWpbmRDwzpc+Z+GN3OnXqxvrwxlbjkt1YgF9cAYmVJxlktYtAEzEzUx7CHOvJnK/TEs9sytgyzJ6dZpYQacvEM85/Ed9364FFh7U8bdRi/S69/lAZplc1L/N39WnO99+beOZQyynXTkzyKFKSu6ALo3CY/YxUStgzdRLHCio5OGsEE36eQv/BlnSxWsONcgW6xFMAJFMeyuzO9uzMLGH/9FGMm+9IH9c7GmPMVuSyrn8Xfn4p0ZRlVcZy8UGs8n2WNI4VDl0YtmQUPQd4EiRDmdXSJZ5CP1H3ZtBj+FVNG0JZRu01xg7eT7Ac0t4+xXWIFV1696S7wwLOZ0pJfXuOcePPESvUFX4ybsguHpZdYfSI2bg49mC+lz8rR/XCwtaGzt0n4Z4qJZtidowbz7FmMr7mrATNja0tEk8h+/zozEI2PVRlL5Oi12PtcISLW20YdyBfqbMx92ZiO+EesWmnmbP+CalKfYzEtctANr/aTG97NyUBypDcYfQPM7iqk0VoDqsvWd4WiWeKCay3JXrSr90CbtUoSM45iF2nDfi9hzP/kpi/qy9zIp7Pn3uxZ7UjdjNUxDNbUcRWpy4sfVpDljyTVX2s2BipJT5JiVuMdP5CbRwLrQdyME+G4POPzbBg5q1wo7KfbtWTGHgKt9BGsuTpXLkbgM+dx0pylNH4hAOeiXq+9104fur75kQ8Yx670tPpAj77HOi7NITENx6MGHxIiVWmPJIlg6fjUZDJoXkOdLF1oKtFd6a4JZJSfhGXMacJLleTEBkvfObRw8oWq14W2Iw8wbPGYGZ174HDWEcsrX7EeshBHtZqiWdM+GEG2/bE0qoLDhNO4y9pPWn61DJpq8RTsANP1y5MPBfKCjsL1gY2kmpChkeirjLe3poeDr3oZDGWI/FS0g1k+Lox20jOafIQIxneKfDULDbMSYYfqxPvTTyFDuNeLsaqnycR8hJ2OXdnxXMJ2RSxcfAA9uaVs93pL0y/8BYh1X9wQg/lfUPimU05W4f9yDL/ArYM/iMux1VZSvWEMqWvmPr9UI6VNW8gsc9c+f3/1oF1YSonKmyn3trYjd99Z4/dgAHYOvbkH38dwN5oLXkV2o9/uZShrs9JF+pvssV5VypZQob1wmj6zn1OfNFR+vdxI0ognpJrjOi+hrulntj921COZsmID1qBleMJIuRyAp+54x5UoyK9mwcx/4aK8Kjn0ZZf2yLx1MU7Mec+8/pbMOV0Mid/6s48Hy0Ztel3UpPJzGrIwmPjAHoNP8fjx/Owc7mtJaNdh7Wog7r9fanrtkg81djoYh0sq+D0gvb8+98687ff/oGJHllGW8Dq58z91VyIpxqnWN+ZODYRT6Es3G8Bf/23b/i+/W/pNOA0ITqZ5Ti/n4x0fn/xYyZ0nMblpmMqNzZYMeKYj1HZpGNlZDZGcdon0ejYUlzYba78wtk2cyGeOVRzbEYXZl8tJzlrPzbfLuN2fTLL7RzYmSJVJmbsh54jvD6Q7dtuESqH5LSd2NkdJbjYnQF2hwjSEM96vI/v4mKWjCxFMktsBrIrL5CZHb5j8aNKhNjq4dqF8R5JyoznlpRkVg3oy464RoQsqLCImH2l6hdbELRV4plRdwfn9pM4Vy7l6YE+9FkQQIrCWIavAo6z2UfgMzIe7rJn3P4CUkv0ZfiyLsBIzmHSECMZDj2yV0k8Xzemm5UM1X7mQ18/iHhmSgOZ0XkwOxPPM6TLcnwlCn3iObw3W+Okyi3xE7N7s8JPYpTxzFIksaSXNRvjitkyrDdb46VkysNZN8UJxyGj2egXzvzuvdmhsx2e1RiD28HbqoynPIGf+3Vl4Mx+WA87rzRUFfHswdD1kQifMg3NS8R9kSXDNmjPOglARXtPZNiaWDKp5YBLD1a/UhHT5KzdOPQ5TkgzxHNg9228lCrIaohn57wBdLawpt/k7XhlCXOV8/LoYCa5lf1iBv2hStDcc22VeApng70Ou2DZayJbHmSRTgMXl/ZgzhUV8RQWQb0GnycJOREhBxnSszcTtz0iRALxr5diN+a6inhKg5nZaRSeVc0vfprD7nOWt0XiKdiHIdZJEeux6rGJx1UKEnPOMKjDNC6ZGdatlaM5E09hK32upR2bQqvIasxk+2gLXK9riYcpnT9a7s/ULpOUW7fKs4mrujPm1H2jsqmnVO0kpcfxWkd2wtEov7jc9zoH11qs36eeuRDPjGofBv/uN3zTw4ZuNp35/b98w7JntTw9OICR2xO4tNKeuVfLyZZn47ljBoNGODPQyYIOPQ8RZEQ8Zfg/2M5o5xEMGj2MDv/Zh51vApljMYZTNSpfFXV5LI7LnyiJ54bEW4z827dYjxyFgIejgyUzPHJ/sTjVVoln+M1J/Pu//oPONjZYdPkTf/rdQu5IpEYyTH1znyUzRtPfeTQOFn/EeU+eEfEUMp6Gcg6VhhjJsMeiVUri6V9/z6xk+D42aKruBxFPgeD5rLOia19bBi4JVq509TKeI2zZltA88RSCUODjn7D4YS0PpdrtAN0BCn14r+2G48JXJDUd4g2/PxMLWzcER3hnTx/sptwjRpbBFqcOTPDINDrjKbQXe3c6vSfe1TOyuMc/MWxRgIaQTDtTorwf/dgVW5dbxArE0+4QEcKqM3s/vdutVmY81edrkhNvsO9uBpmyCnwPD1EeNRC2eH13DGTuZTHjqSvHL32t1K1rk7Efe4WgpvPGQtlrz+EMXRVNJnICTo9gwLJQkkq8GN59MqfStRnx1CIPBliqtnxT355iYOfVPDKz7d+2SDxTTWAd+2AmNuNuKD/olSmPwrXrYI4UmhfJb63+mjPxFI4LObebwoUq1Yfsbm6wwuVwkcYnmtL5B435bBraW7kln60oZOtwa1YFZRuVrQ3Sbtm3FqsvWc9ciGeo1zisRvsoY5kQK54e7IPN9KfEvTmCve04hti4cr1GQdzzBfQcdJoIGQiJEPvexsTzZb0fEzqO4PgbKVnyNJbZ9lcSz5k/WrIpWoi7Um5v6s3oQ/FK4rk5ORjX3mPxLBbk38CjG26cS/h0n294X3m2ReKZTSn7xrZnjlel0m6EYyUr7b9n8aN6UnRkeK2mnrMLujLRQ1h0yXiypw9jTBDPB0/nG8lZIJ6GMhywZ7eSeL5qDDMrGb6vzA3rfxDxFBpJit9Mu/+jA+tCVY6nNcTzm//1V37o1o0fOnekU+/ZHI+tUab+mzsAnVkVxIphnejgMAanMQ60bz+YnUE1xEVvxfLbGVwuUR2QT8nywP6bIRxOrdf7cJEwzvigZXTuvEXjZIWy1NLTjBh2QvkdcykpnvS1sMFp9mSsOvdnR3A9mY2vmN6lA/3mLcLZpT/drNbqEc+ULE/6tOvB8HnzGGLTA9crBQhbKQenjmBfpvbQviHYbe19W8x4CudwDk38Pf/6p+/5pl075b9eU64SUf6EabY2DHWdinU3F06kSYm5NZH/9T//rKnXvv1MTpWUcGa5A9ajXRlp34Npp9J/8ayNod60ReJpCusTecEsHWBBL5fZjBzUjb6ud4nW2QI2nLc5vzdn4inYhPfuQXTqPYFx0wfStftCLhfVcWVVV4ZtTFB+cNNY5+WE+S3H2noMLtPssXU+R5DMVJnpD5Oai6zMgXgK3/yyc+QPuN5QfTOAgE1KzmF6/XEml6tzWNfv37Cd/5p0oTzLA8eOjkxYtoBRLn347psh7Hx5RG+rXdh23TS8C/azljFl2nis2n2Lk9slZvf6EYs+Toya2pcuXZbgVVaqJJ47UiX4XZpMZ0tnRk9xxKLHKm6W/3ILvLZIPFNLzzPgb1O50vQtJ0Iyw+9IX3pOuk+2QleGMp56jqCT/SxmLpzGYMf2dO3nzsPC43oyfJp+wkjOe6NeG8nwQrb6jKfMrGT4sfb9wcTzYztu7fPCpzSjMqO5F5FKdN2ncXKCI9g3fTruTV/fklmbx5OIGEIqtaQxqyaHh+EJRDbTZ0ZlFg+CQ3lRoMpwZtTeYtqEE8ot/9bOzdzrtUXi2RKm2Y0l+EcnEFEj11uIGD4jZAUiU6N5nicsjD6Nzhn28THv2yLxbG6+OfJqgmJDeZxeanRGsLlnzLHc3IinIUZCoIzPi8U3MoN4qUqnhW/i2OL2RmkLzel8Smkqj+PzSdFZEJgqM+zPXN6bA/F8XyxS3ibjG5lOnLSBsIQIAsq0cUndVnZjMc/CoggsbSSlOIGHqaojXkIsexqVSqyJb5dJLEzANzyNuCb5q9v60q9tkXi+D0aCLYWnhPE4rZT0xiL8wtOI17EfdVvNybktyFA9hw99NXvi+aETe9dzCXEHWbRPOOf58cRC2Dp5duZntr36Zb8s+V1zft/7XxvxfN/5m2v9r4l4mivG7zsucyeehvMRiGh4TAivzfB7ag3H+jHv2yLx/Jj5toVnv3bi2RZk8EuP8Z3E83VGBu7Xr4v/foUY7HB35y9/+YsoezOTfe9+/USZmJlMthw9yjfffCPKxczkYm1vL8rEzGSy/sABUSZmJpMvzfHq6+uNuKfeT2Y+ePBA+RvEwu8Qi/9EDEQdEHVA1AFRB0QdEHVA1AFRBz5UB0pKSlomnkZ3xQIRAREBEQERAREBEQERAREBEYFPhIBexrOqqor09HTxn4iBqAOiDog6IOqAqAOiDog6IOrAR+mATCYzoqt6xPPChQv8+c9/Fv+JGIg6IOqAqAOiDog6IOqAqAOiDnyUDhQUFLRMPI3uigUiAiICIgIiAiICIgIiAiICIgKfCAG9jOcnalNsRkRAREBEQERAREBEQERAREBEwAgBkXgaQSIWiAiICIgIiAiICIgIiAiICHwOBETi+TlQFdsUERAREBEQERAREBEQERARMEJAJJ5GkIgFIgIiAiICIgIiAiICIgIiAp8DAZF4fg5UxTZFBEQERAREBEQERAREBEQEjBAQiacRJGKBiICIgIiAiICIgIiAiICIwOdAQCSenwNVsU0RAREBEQERAREBEQERAREBIwRE4mkEiVggIiAiICIgIiAiICIgIiAi8DkQEInn50BVbFNEQERAREBEQERAREBEQETACAGReBpBIhaICIgIiAiICIgIiAiICIgIfA4EROL5OVAV2xQREBEQERAREBEQERAREBEwQsCIeDbkBvIytlKvYmNZNK/CCpHrlYpvvj4EZGQFPiNJT/xyiqL9iSmUfX3TbXMzEuXT5kQmDlhEQESg1QjI5W8I9ItX1pdkPONlshSFooSo5xG8FQlIq3E094p6xFMQ8L0Ltyk2ErCMZK+LBNQoNPNRUE1uYgzR0dHEpZUg1dwxvKgnNyqCzGrDcu17hbyC9KAIcppvBJBRmh5CfJ6WANUWRBOdUqNt6BNdqecWl1pMg7JNOWU58cREx5CQnEJiTDTR0TFkvtUOuLogiZjoaKJj4sksrEU7StODaqzMJCQ8qwXcTD8nlEorEgiJKtFbCNQVxhKZVNH8Q62401Dui5dPQVO7UiQS1SzkinS8z72gTqcNeU2eSRx0qigvFZJ8osPTqDW8ofO+NfJXyMtJDY5Bl/9+Dvk3VqQTGp2PVFFOfEAMpUa2IAxcRllmMNEZjTqzaP5S0WJbzT9neEdfPqCghrToJEzKp1Yln/iMMj09kdfmkxQTjVAUaj5zAAAgAElEQVTe/J+xrTVf9+PvmNLn1rZqSi9a++ynqKdoKCAmNBm9tdqnaNgM2tC1y/eRkbFdttZe5JSkv+D2jUDyVY73E6EgpzQhiHiTzKWle63r/lPp4MfEBN2Rfip/o9vmu67VftMw7unq0LvaADmFvpd4mC9DWp5G2P0LXLoXQUapHEmmN95Pm4v1csoLCluMMe/u27jGp8ZRWpHGs1s3CX0jMersQ+L3p9I7o8E0U6CoLqG0DuTVWYS8CCStVCvtqpxwnj0N4U2TiAQOlR7ix8vYYpMcR494Shsfcco9ERlysu8soPPvf0/fubeVK42agJNcDNYFTEZRwHI6/s+ReOXWNzNUaCwLYKmlHcdStSTNsHJl9g2mfj+Ey2VaYmtYRy7JxHNsJ5Z6qeiPQJKfrerJmC1p7yR5hm29+72M4vCNWPzzt/x8u0gZuCXlT1jktIyXxSU8Wd6ZzkPOUqAz7caaIJZ1+SOT9t3nuvtPDOw1n/u5WsHo9qmgluRbs+jp6MmHUEVJxg2Ono9TCrSysJBqRTnBm+wYvibuo7CoeuTBpXgpioZ66hqjuekdR0ODEAEaeOHhSZSO+AWlN4WD7jwFR1IQvAo7y11kmYZCWb018q/OPs/o76dwv06lI59H/vUkXRiP3bgb1CnSubbvEkkm1FZel8XVWR2Z7tYSedMiIRDD5tpSVBdSXKWt29KVWj5CnfLkp3hdvsbFkzdNy0dRQ/Rue377/07ivsaupETuteO3/9dwrphwfuq+DW1NXd6aV0EfmwsPzT2vq8/N1Wmu3FAvdOspqKKooIUVr25lg+t3zUPVdiXFsZvo03kTyS3ot9D0u9oz6P6j3n7MvHU71rXL1srIlF221l7qkg7x885Awi8f50GGCcPTHVzTdevsR0bSjQP4xJhaKLZ0z0SHJopa0kET1U0WfWxM0G20JX+jW+/TXWv9pk6IUDavq0Pv7q8eX/eTpMqg8W0Mfj7nuejzgvhCOTLZa067h5psQq5IZItdP9yTW6czJhvRKVTr1KfEUS5PYP+CvQTEXOakT6YmTn9M/G5J7z7GB5jyVQ0FTzh7LpS35Q9YPesoCeVxHJw2h2sZUqqCt7FiTywVJQ/5edpBkmXVvL6wi6OnT7DIvhvrH+sQpSac9YmnxAfP47nIkXBz1n/zv//TP/Hbf52Nf72CulhPzho0IMk9iMN/zSNAok8Y5ZI8Ip49IzyrBgVlHBvmwMHIJIL8Y3jbtJIV2P/rZ4FkVaLM2Gy3dWoinnJK0gJ4EZCOOmTIa7IJDYjh9gpbDfEUxl9weiTOmyNJCnxJfGED8pocooKDCU8qobEul9jgBIp1Vs664xKeF7JxaRmFZEX6E2kQiKWSa2yeNhGLPw7lQnIjCkUBHvvOU46c5EOO2M541pQNVSEpV+Sx1/F71j6RoKAc96G/ZYa7Ng9i2Hd91l4G9lMRT3lNFkHPXpKYrxVQZU44L55FkNsEQn1xJtkFqUTEFaNQVJCZXkhFyinG9lvE3fhC3l4ey/C1IaSGPiciW9VOfXEqmcXFxL0OJrNSRnlmMK8jC0yuQIRZVPgc59IbGfK6TB4ecaFzu3G4P0kTcqyEe7jzul5XzqZxENqpzYviuV8Ib6qhseokI612EJUSSEB0kSaDrCtjwcDV8hdWmcmvnxGSpmVjldlhBMTeZ7H1VA3xFPp5X/nrYmgoD2GFlhEaSPij5QwafwMJcsozMjTZf315yEjY68jMJuIp6F3o8+dEZ2spV0VWMM/94yhSrpO0bSnnF/SKsLh8qhqSOD5mEKuvxfNWJqc0LZjXwfEUaqeuUq6m/9XykdUEcHLnVRLKpST6+jYjHyi/vpn+Nn9m3G7V4kwm8WfXyrHY/GEuL5tkqT8vMLY1Q3tsoDAzm9y0MJKK5HqyrtLRR2HxqitjvYmgP1e1PsupISc2mKCgIIKCQkjOb8CUPqjbMtQLfZnWE316HM7zfEgulOmNU3he8D9Br0JIzFMZmK4/0p2HkPlObpKXVrratmNzzjGm2yZCM0LwD8lBFXjryYl6zovgbGUWRrc9LT/VryMs7nRx1R2Per7Ca21BKpmFOUT4h6F2F/r6px2bMG/tn35/QqY2Iyuf9BB/IrNV+xF1helk5WcS/CKM3Bp9v6yVkSrTnhn2DP/IfKU96+NubJfCDoHWXkzrhbwul5db+jJw4V1iMqsQ7DEz/AUvw3OaMln6+AjzkuvYT7FM/76uDxLqSiszySqQm/T56ntQT15qJvlvInkVkqvx7/q2rEVUuDLUQUN9NXpvAnd1i7oxwVD+hhjr+jJjndDxNyZinDIbFfyMgNiCJn1VjwDeFTMM/YWx31TZlqnYruv7tLak7Vsg395uJylo2mWqvLuCTT6qRJNMHs3Jo37ayjpX1cFHWDbBhv4/vdLITHtbQn56FrlZIbwKzdHs2hnOQ41nWMRrPZ+sjgHG+mhaV9T96tqkMK/MsM307bqUBxFZGl6j6xdMxW9D3VG3Lbwa6p2+fuj7AP17rfd9au+hUORzdsHPPKlSUOI1nkGLAmlATpb7EIatiiLnzCiGLgmksjGMDRM3Ed1QRUW58LSMmP1T2PJEy2vUc9AjnjJZAGc9w5XEpD7jBuvnzGXPjQzl+6qnp7gar7+iMEU8BYJ2YswEzuWnsWvkZG6Vl+E29AfGrD/Izp96Y+t8lTdFd9m/6z6xwZsY0HUD4Q1pSuJxqUxG7r19HPFN4PnmfjitDkFS+5xtC92ILohl/7D2LGnKeAoTKDg9nK6D13Ng/zx6f+/CzfxCLk7qzMIrlcikYRzdfovyppkajut2WRURe/vRZcBmLlxewwCrdcTqTE8quc6xQ0lEeTjxY4c1BFXk43ngAhUtEs9vWeqTTfLrAzh1GsnlpiyvYd93yhSonUyZPIZdU1bxOOMu077rwqKrsaTfX83yg9Hkp11get9F+KYHstL6H4zesIfN+3xJvDSBHoPOU5x/mMEdV/Iqt5qSy2OwHL6Hy1fXM7Dbz4SWJ7F/6Hc4r/fG69AYejmu5cLta6zsMxD3FJ2JqjUBqAk8zeVQISsgI+WpL68D7vBcOYd6HnqeUa5EtdVNE09J0RkmjvYgJ3M/E8Z68bbqJIP/4cTGQ3uZbfMj8y/mkWMg44Ym4nmhtIhbe/byMCmAtX16sS+wkVL/7aw+FElOghtD/zbZgHi2Xv6N5SEaDDftvaqno7fLKnm8bQknI/IIPzYcmzE3eJt1kbH/PZLrNTJyDeTxpKRBE0gbSu+xes5RYt8mc2rGINbfLqEmbi+uyx6R6DuHH3+YzqVHJ5vakpJwdAkHQwp5deQE/lU57B3Ygy1Pc3mbdpRVm0LJC3HjnMECT425Vj6gkBcR5nsVt92XEAKmsXyg4vphDnovxeK/FxNcLyfPZx/H/PbjqCSexvO6n/NU39a8awzsMYjcsPVY/8WJjXvXccD7iJ6sczX6WGkkY12Nq8/QzvXsk3KNPlfL47l5/jnJiRcZ36EfHvF5RvqgxsJQL+7V5hvItJ7Uo0MYtOQZmRmn9MZZpkjj0MLthL0N5JibHw0l9/T80ctstV1VEKcjL+3CS6JpO6fkPCP+MZr9Vy+zul8vdgTUkXXWhZ/c8kg4MJpFl8qo1uBS3XTsQW5Qp4TCcC2uB69d1htPZBN48tp4tjl+x9iNFzm7ri+jVkZQa6B/627nkXJksHLeBVXqcyL6/S24mMGrDTZ0HrwRD/d1DGg3FM+oQH62/gdjN3hyZP0gevZzJ7NR7ZfrNDKqpYxH+zbhHfuGG/NsWX4nXQ93wbcZ+uVbBSp7meFWYqBPIZpFsKKxlLAdjgxfFUj22zfcXPUTp6KKiL04E2fXOySGrNPo3YmHqp0GecObJvvJIU3nvqFelknSOTXhe2adLjTy+VE1qntzT9ZQlrgHh78OZfuli6x0tGNPWCN1cfv0bPl2nPbAkbEO6vuv3YG5evq7J7DICPfz6VrLUMeE4hL9+BjRqK/b1zKDNb5sw86TRjpR3uS7rtWainFSwvZOZ8vDVK7P6YDDJG+Sm6Ykr0tuIWY0GPnBx6UVRn6zymDs2tiu7/u0tqS2aOG1gZcnTxPTBEl9ymOeJareSBufcNYzUbey8lpI8tzaf4yI4suM/vt47pToJ0eKIjYo9WbLyeP8POhHph5NI8fAn+vG1427L2l8ckGGKgZ4Vxfr6eNo13ukxxvrinpwhjFh3e188uO3089yPaFZpRpyrOsXDON3WL2+LgmxUP1nrHf6+nG7TOv78qpy9ezzZmlqK32f2leBJP8oo4aeUR4nKrs7DWuny0peVXp1LL0nPURWH83+YT/SuddUzsWqSKawUI30Xk4fy8ncSNWOXT0HPeIJEsJv3sJwV1w4SxZyy9dou9QU8RS2VyvyYnj55B6rHOxxSy9WZjyFrXZBeWZ9N47jbmMYtfIS3t4X8DjkRVSVysFdKC3k6OhBrL/sjdfFExy/HEG210Qmbc9ALmz3rrIzyniqttrruevaiQUXa6iN2kD/ASdIDT/F+WdaJ2E4ruNpUiofzmWY6wvqpEEsM8imqYhnOlJFETfmdcJy7DF27HsH8ezzF0atPc+1Wy/JKFc7fQFqfUyEvtVOpkRyDZcOiwhsqOXClFGczS/lpLMNeyIEYUm4P78zc88VcnGiLbuVpBDq0nYyaOh5KuvO4my5jTQZlF0ex8g1cTTKo1jTczy3aiQ8XmrHylv1NJadYHjPveTIJdxfaMea+8YrEGGUssYo7vgkaoKBWklkdWHcvWN4pME08VQoqsiJec3DB+sY0GM3qWVCxlO11V7pNw/rkSfYbyDj6ibi6Zl2grF9V3HZ25vzJw5zIzyfs+OH4J4uQ2ZCRkLGs7XyF1adWgz15bE/7gITHQV8oDp0lTLjWatIZnNvZ7xrK4zk4Xq2XEM8iy6NYfjKKNXi7MUCbIZe4o3PJAa4+lMr8WLGkOPky1RtCSQ25+ZMeljO4PRLIVtUxYmRdhxOkNLw9gZTLKxZePQVRTpZerUM3l8+KuJ5Ki2NvY5/Y/qZQE7tuUp27mEl8fSXGM9rxE+j9GxtkVemkT1WSK4xsfsaZXAwknWVSh9TpMZ2XJb+APcjh3E7eo/4Ah+9uar1WbBWBRU8WNqLCXsSqS3x0NOHm2Gq7KSwjWSsFzI9vyPY2Nvzo3HekITUQCczpYVcmdmdvlOPEZxfR47nOD1/FF58psmu5Aby0kpD3bak4aIy45kkkxK80Y4F52uQVucS9foJNzc4MmZrOvU6dqpuwbCORIOr3Gg8seqtH2EnarYtG/0kGj3NMaF/WeedlfNWZyyEPg37e+szmeHLhSRDA/6rujPLs0BjHzJpMMs6juBKhcovC0eg1DIqLzmJy6gLqiykTIrUhG8ztMv5FyuV9jLVLclIn7Rnv+VknxjGhF3ZNNRcYHS3dQh5DpnUjznfj+ZKibdG79QYCnqgth9hh6o5vcySyYjZ6YjryRoTPl+quSeV3mdmj8UENmhjTZmBLRc1uXVTOujzRt9/XfY7bKS/ZQa4z/bU7oopY0L/E8R6jtXTx9hqfX/lllapkZUQIwx1Qu27rtcojObrW1/LxQnd2fhMQtm16cw6nK9zBryh2Zix6mGhkR+cdOqUgd+8ToqBLalju5BU0vd9ainqv9ZE3+J+goqoNBbFaT7TURF+myeGxESIkHmnmDN+PUePHWCG5W8Zs0s/TunqRXXYKnoPPsbRFuKrrk7Jm2LAxaLzRvp4teyeka6oZ2IqJpSWHMep9wFydWhBo45fMIzfl7P1delDfZ+w86TLx46m5rbS96lnA9WByxkw/qYyOy6XJnNuyXSW7jzBtnE/MHZnBpKca2zZfJZTaxzoYXdIczxN4I2RJ0bhMP6OtrGmKwPiCXJJBgkpGk+nrCYtSyQpR9eNqZ6WvNHfapcrMnjt/4ido+dxt6SaU6MdOZqmJZ4yaSArB6/gpvtwhv8cpWT+0rdJJOanKDOeF8uLcXPqxd4IYcNKTlFiItGnRtB//msaWiSegqMYwdYXEuSKNxwb0ZOxS09qVk7CaOXyWHbrjEsgwi0Tz2scPdC0RdkQxVb7P9Jh8EllxjPJYKu9ISOQoKxszVZ7E7aaF1N9q4lnBRKCjy5l87nbPHyZTj01XJncgSXXBBk08HxVX1beKdVxNPrEc1T3rcpMpKHi6hNPD0b0UhNPW9b4miaewoBrM+LJ0NvqlfM2IZ58I/HLMYXDsyd7mDzrOkWSi4ztuYsUHeJZHbKa8YvvcMhAxoVSVYA7leHBCIv1xEtAoSgmMTGR404WbPGXvIN4vlv+usTTUB57Iz0Y/v1SQiXGxPNabbWRPFbdrtEQz9JbU+g99qYyEFeHrMJpqi91jZEcXLyOi3d8CU6tQ+3ABOJZlJtDVvBxxndx4HBMKSdG2HIoXoq0KJfsvCAOTerOmM2xGt0xvGi9fKDi2kFOp0opujudv/7Rln2PqpHkH6bPH+biLzHWs8mLB+vZ2qJr2Ub2mFenDfB18fv1ZV11FkEfU+TGdpz3NonA168IeJ1IYYH+XNWkRiCeJa+W08f+IKkyOcXph/T0ISmhUAmHglIjvbhTE2Nk32/POzNqfSLVBuPMkBaRm5OLv/tkbHru4M2pkXr+KC7zlHIewhh05XU0WrtyV7dtSDznn68g7MB4Vl0tofTyOMZtSVMST7WdquQpNaqjJZ4YjSdFvfdogmTkm9C/3POjlPPWmqxxf1riKSVooxNrH2h9jEweydpB83lWZ0w8KytOMbzDSiLrBRstIiM9yAh3LfFU22WtingeTzHSJ+0HWXWIZ/0NJnwzhQfVCoSYsdx2DvcrmyGeavvRIZ6GevkxxFNmYMtqezSlgzfz9P1XbNA+I/3VEk8V7ut0kgDqmBB/coSePibmRelh7Jb64cRTOCNfHX2U5avPcffea7J0czPoEk/9mLHqfomRH5zvfdzIb6adMhh7U2wXiGdztqTGVPVaT3pcql6RQlFKYlyu5lyk9qaUCLfd3H2rynJWR6yh29+XEaZz0FSXeNbGbGTs7NucbyG+Kolnk06p/falEh8jfXxQ7dss8TQVEyqbIZ5qv2AYv6++0delD/V9jQa850hKXit9nxbluqRtDBl5RXNMQbjTUPGY+X2m41vYQMzu4Sy/VoeCEk6PGcRRnbO2jRUnmTOtFcRT213LV0IqNfryFP7+//Ri6WEP3I/vZekoB1befMTiLp1wWXeEFQM7MulgADfm92Pu4VvcvrQP99u51L+9zcwu7bAfPZ1lW24Tn3WTKd91Z4t/Phl35tHlu96Mm7aA/TfTqSu9i2tPO2ZsOMjPw9szemMoFU2rhrIHCxg88Qi37l3kwIG7mrMhZX6Lmb4+Vi9zJ5O9MhpX7KEBdB10kqSM0zj9wYFj8U0ZFUUFsT6uDB56mNimT0I25Hkxa9YZSiqTOD3lO/7bcgFHPU5wdN9KnG0Wcj3ei8n/+A3OGwMoMMhYGfcdROrtWXT+cQWv856zsn8fxk+byU+Lt3Envo7quONMm7AR74eX2bH+Iolvw1ht8w2z3BOolNeS4jODzp1XE1zwhLmdbVlx+gm+66ywHHWZ1KwLjPovK3YFRnNk+DdM2JNA6stldPvLdO5lJHFw6DdM2pdC1atVjFnop0n7tyxt47vSZnDw8VtB1x+c2eC2iv7th3EsyBtX+6kcv32XC3sP8fhNIzkGMq7IVcl/o38MV+Z148deY5j103YepjWQ67uI3jZT2HhoLUO/7cPBYO2ntN9H/o0V4RoMyxr0dWHCgaf4LOpNv0kb2LfeiW4223gZ68Xkv/dgR8BbKg3kkVKTjdesDgxa9IrCuhgOT53MNq/7XNy+ieuJjVQHrqO//Timz3Rl5eZrhKVcU7a1PTAP//3LcfML4uLatdx4U8v9hd0Y5HoO/8d7WLPPj1de69l24c0nkE8yl+b2Zfr+CN42RrJlxi4SGsqJvjSVv//f9uwLKDDSs/hCA1vbFEzcLV17TCU/chM2fxjNxfhyKgNX6sn6eOhdpT6uPBNIjN5z6XpBoyZ0v2auWy+kaPX5jT9LOn3HlD1eeJ3fydYTfnjr6YN6W1JupBd7gnxZpON3Jh+IpOrFIiyt5+Pp+RMWOjrpHurH3iUH8Qu5yoafL9Ng4I/SJX6qeZx9xb09Wnnd1Fl8q9r+CbdzS7H8/XiuZ6RzatzfGLYmlDsru9FjxHoOrhmI9YAjRJY1tXcmmDKl72rg6cruOnUO8ez1Rg2uhv4xu4lByutT2NXvb0x3TyH+rDMde+4msSzaSP/U874apP7mC8P+juB/cQLdB27l6s2TbFx9gXSZkAn7kXGbr3Lt9Hq2n01BbZebX2aQ0ORzworfcntZD37oPoZFa47xPOeFnl8VcDe0y7w6lb0MXOxPxE1dfdLqhXAO7dbCLvSeco2MijrC3Kczc50X965uZ+fZBPJ09E6bNBJ2cAT7OcPD5xs0+BnqpUdAHJdmtGfw8uf4H9D3+YfD45X3hi4LIiPlIH3/MIILqam4j/obY7YlUGRgyykaomasg/uDk7iqo6/3U/OM9LfMZ6Ie7hma1UG9Jib4x/voxcd0ib6/GrvzosaXldcY60RYktrfZBNtEOPc4kvxWzeQvmOnMct1CXu9EjTnPOV1GS3GDMO4lNpYbOQ3g1Nv6I1dHds3v8zhuYHve7xqMFuaOVZkHHEMS+QURx5kuMNSAopUIFZmn2X47/+dsdsjNBxBIJ5jvx3GLq8beKxbx/VkqZHf04+vap06x4sYVQzYHphLiJ4+JlGSZqwr6uy9rFE/JlxLrCLj0UIsvpnJnVThUyKqPyGb31z83h2RZqQ7qqeM9a4l33cx4J6eXxx74FzrfJ/GV4FMHsZ6l02qXS5qyAi6xI7l6/CKVp3UrY7Yx+x5njwPvsXWxYeJqkvCY/FPHLj6gNtnPHmsc5xELUWjjKf6xse8yuqrqWkARUMtdU2xorGmijqNkQkpyHoqKuo1QtDtT1ZfQWW9WjzCHQnV1Q1IperAo62tkNZQXavXMG9uHueGiU9FmhqXtqV3XzVIDBjlux/R1Giu75rEsxy+lEBeViqxUY9xP/2iiQyq5qxpoJkLaU01El2omqlnXFyPpPnEp3H1VpfIqa+uVs6hoba2ifxLqanS/4opYxmrO5BTX1GhNyeFRNAnKSbEz/vIX92D8GpKHhJh3FJh+9DUX8vyUD6rfKyBqDP78EnIIzM1nrAnx7jqpwO0XE5jdTnVTaqkkNVQo9R1OTJpNRVVah37XPIxnJvhvIxtrUVZGchaVx9bek5/roZj0n1vrA/qu4Z6YSxTGTXVgo8x1km5XEp1eZV24WXgjzTzMJCXum/hHLSqbW2J5kper/RXwo5Fba1KmzTtqSuZqKO+pXw1GI/ePRNvtPon3DQxNoP+hMyb04pAyqtUuqneEdj5upRqHXU10ZWySLBttdc1xh0TdqltqXm90NYRrlTy1S8zfKe1H907xvLWvdv663fYsmaMur7JUF/13xvi3uxYDORvCuNmn23hhlwaz+ndV0nKzyIlPpIHbmcJ1PvQaAsPK28Z+gsw8psGY9e0aGBLivp6DenV1PnEF+qMZ1hZlUFfxvNQd21ap1qnj+o2hFd9m9S9o7028gvaW8pdX8NYqL7det9nHOta7fvUnQFFL9w46VeGok74UGK5UYxUNJRRkFeu8aeNlfm8Kao1ye+EZj8L8dQZ7xe9lEoes7CXHbP3vnrvr3T5ogPV6aw8YAvD+ruw4OctHDrpTUgrvxtSpwnxsgkBc5C/EMD9tgxn8PifWLP5AGevvn7H99OK4hMR+LIIKBRlBG22o/vIk6SoUrA0VkSyxubvzDmRoMkWfdlRmV9vn9qWTeH+pWctq3/FumEDmThvFdv2n+LGq0wjEvGlx/T5+pNTFLMZmz+O5WK8dqfs8/X3NbcsJScsiCxTX0fwAdP+qojnB8zfTB6RI/+grKWZDF8chh4Cyl//iha2VGSk+Pmh3k4Tf4lDDybxzS+IgEz4YJBUpslaCllSmZDtl6nzmL/g4Myta7mQM/80f8a4f5p237cV+a8l4Mhkyp1SUa3fV0M+b32ReH5efMXWf2UIaH/9S/i+wkjunT/Po4g03pamvscvcfzKQBOnKyIgIiAi8P+3d+5vUV3nHv9b+kN/aPs0z6nmJDWNSYwtMaZqSJSktW1MGm1QKx5vTag2ajxHE6NRQY0pNwsqAYkGLwiCCCLCcB9GLoIMcpvhDjNcBmbge541M2v2Xmvt2cwgIc7j4gfmutd63+9+37XeWXtmfaQCT40CTOFJNiz1jwmcO02mp/rR2yd+sp6f/rUQaTMjAoPDU2n1AfjHYjnQaSxHM910NGipte33j2ALLYwpxbHpYyz1RJvZX72jg3lNTf+yGAtwMTkZmbeN6OquCYjEQTarLiioh4dGRlCtJg+KtaYGtfUWnymTgx2werfsGmuvhznA2CE51lCQiat3233fx6GNzg7/qB17tM2n7Zbm3GRAGOAnRx2aY+KoLNqoNU7PLnbEtgN5JpCxOBh/Aukz2PcEgxilMaP93fJAetaebwI5MpD3kM3s5wr1OD1pgbHgKjJzyA9l9XtXzvMP65++FfLVH0IBpvCcS0SUnrHkuzOmzCTkchuLzk//IiItEESgHp5K9JXtYyYs1uRwNXa//CKO0Z1zxQZ1n9Gyn2jsD8sZWhhTBcemh7HUEygQf/WOD+Y1p4/+RY4aR8bOncjxYj71SRxTsObH4p9792NHxEIsjkhFp6MEhz54H5u3bsWWv6/C0hUJ7i22quL/hk0HM5B+7CjiryZj14YvYFJtIeLfXieqT2xHzL0qXDj5Pdq9VYYHETcVMP5R3b5W7NHXydYks0VWUmwdbSsUbtU51xYABngufdLTWu81jw1KjgUSRupx+nFiZ7b+zzwWB3E1IpQAAA/ySURBVOePnh2zjcOAEaM647SeXexr7HzDvvZ4j+YS9Tg13YrEjw+ieLALeSdO4bbG4pPaWuU8O3WQp+oj5P1QUYApPMnemT5E1GQ3mh/2wFpXjPJmGybtZpQUVKPb+7GMR5KJ2EcWjUY+JavRc4Qs9PmWWIGGQ/snn7LUSDsqKI9141FzPKqqzVQKg6EJA64RtNYa0NDl8OHTSJszIwJFPBU5jkwy7aZSlFW3orO1GqWlZXjY50TPg3I0dCp9zITFom0lvPMKvjQ0w5Bfhg7vlh28xiIaTct+qhR8m9QTHrwamzWfGFOPfyz+bqi1xo2MHBtqQVVpNdqGtfGlIo5NFZ9BoOD8+auOFbJzQme9AYYSE1oe3UdZaSPIuMij6/zFJVVdTf8iqNG6nBz3Bv/kdX0SxwTuG6rcODXn5HWsX7gR+UP9GPBuqWsr2I3oMxY4h1Pxx+c/QaXThQc3snDvdgzO5Ggvd/KYN1uXAXt++yL+daXSrTmxSY0d7Br1bIbO4h/ZPKZ+uo8VULYKlnEEs8e2qW3q9RbH9i4TDKU1bqQjweJWPujHRP8DVJi64IDSr2dLExbJSW2eDmpMY5F4DtjQUm1AQ+c4+s3VKKvyoO/UeUX6oXsxDnihCOcGbDAbyRjUiH7nKNpMBjQy+63xtrP9kj0OtNC+bOzqac2+xsezmGMetabGOmEsLUVtaxdaqj1jaL+rD41l92F1ePLQMqYgX7Vjhyov5pHmWDbaBsOtfNR2espf1kdPWzwq0D2vqBC7/vwhR/PnSssG9ZjLxyFvj148BYMYpTFDxmn6RxGOBJFM5mUeP8tjPBXsJ8DjIEU/tfOD9K32kcxxc4d6dKK96gCWL96NvJIS3CsqRqN7q0Ix3okd/HlW/NN+P6uHf/+ovvL2x1WAKTztPtSWHcbT4VgYtg9pl49hzX8vd29wfuFfy/CXQ00Y5bCIWtjHZgaLeA9GAT03jsuRy3G4VNmUmfb/rZ3FOlGJJofKGKzbi8tOo6xUQakdz0gV8IVd2ZFYvCweBFR15/ghZHUqiDQXj+MUEIEGdPtFNk6hOfFtvL7xNuzWZET8agvuOqbQnvIFEu43uhFtBMOmh8Vy75PvLWIT1vwX1h1ORfK+1xG+vQR2XmMNXJ4eTpRoRge0wWkWqTXfGFM1/o7gxizWk1j56/9Dk8uG1HUvYE9en4CyMzptAo6NYiyDRcGRwlPAtvaIqEubORFvL/grkjO+wslsC8YE/Jt2XNL49Nyq6V9TsNQaYfVeUgqExEHacFjjsXnDJR/ulRTFOZ9sxvkOFyb7U/BeeAzum27gcuFtxH2ehqaWQqQlp8NAlzBJTE2LmLeuPiP2LX0FXxS3YsC7tKVgBzswOH5OwD/yCFt6OVArd9SoyK3nLT6sZLDYtopRBSU64tXO0RmP8Ge2osjRibiIRfg01wFHbwq+inmAhxyiUo3kpPhRQtEIZkwzC/hEB8ynV2DN3gY4Ri/hvQVRKHKwecWgcL2FpxsD/N06LHnnIsgmY/diDyO7n2L9WJQlwWvy2MbDFQNCbpRb2dhVYzJ5rdX4vNbWTAbFWamRY3TFk4A4ToU/j715YzCfexfLP8qHY9qKswe/Ru3DFDcCNm1AOU9C7NylLUFAkpYPmQTMo2PoFo5+eg7GrmuI/E0ksjpYHwmKtpcbi6+NsljBL0v6hTGDWsFjiy91iDaM8GPuhOLfQC9rz54rj1DtZ4780yETTF6scSCIUTpO08JTjff97GiOgBm1cUjey5V17vkmMsEmYC2z24yC1nYVspbmBxl3fljUowvdVXuxdPEBlNVXISbiOezOHhfinWBKeSTk1SFlzubzQwtrmpr91Yz4YXbMlo/mWwGm8KQ79RPUlv3uDqxYdw3j6EHsijCcaXG5N2Retf4mHByCrnk0ncE+JnV2smi01HLUM6hAd5ijIHoxolKU3+fT/tPtaqQdHTo8q4wUe0ixbik9CtFCC1U1PF2HA797CwnmEqQkGDBBwPVefFovh+PkEYGnU+/gm7Ur/CIbnfZM/PU321E4cBO7nluMfYUtuBifid4ppQ89LNb3I54JiHyyJCue5FL7aO0+vLE6FaOcxq0uEY3Wwdm/I92O+uxvEBsTi/grDbC3HsEby+LctCU1Nms+MaYfJP1bwI2ldlMkoQOZm5Zgb65DoEhd7r/A4dguYbYoOLri6cO2/mwt4hMiBNTlCPkVeuJqLFp5Fn1TIrqwxqYdl3zSatG/1O/xT+IgBWMfbp06iQKr8m07lzMfn6w/4wUkTKHblIe88g40ZxzBmetnsPaN/aiy1yHryn1fN5Ma2MGMYStO/H4Z4lqVtt2kDi+208nhHzclP2Tz+EKlB5UIgM+dHWljApbR6kVWBottM44oKETqEEFppn/4Kvbm9+La7hewKuoOmjISkd3tEvod84MfDWpMc4hkkl6vPw5nNj56Yav7g6Y6r9QoXLriSXCTrqkKRL8UjqTWMiR9XcjsJ8ijLB0a2EaesHY2UYxdBZPJIvIUdGgdzBzOsKJHzDFltAV6rm5A+OYCdOXtxovP78SdRxlI+a4bTi9K8OLosIKs5GKHoEM9f2IeGUf4sSwDTQnvIirRs7xPfl3Pj+UvvRkvjMU8ojKtOFEYMxR/WF1ON48IqMlxYcxV4pC3h6BxLTpz5JAXaxwIYpQvPMl8QOe56WkRP9shYDw9881HiZ0C1nLjf3oFP4f95IeWj3OFeiSxQBDbK8O+Ri8cuL5lCfZkj4PHlG672K2Bww0Oa9rWzSJ56Rgib58cBQIqPE+ufM1XeK78MAeDHILO7GKxj2PgkXn3YWpTUIEe9NwEcrYvRfQVZbdipfBkkXZULnVCUqxbrgqlpoWqcmAK5sQILHtrP74j8F9V4dl1drUuItDaUIIv3nzWL7KRfIcve9ureDfyOFKvfYzlKyKReIPwJtnC0x8WS6/w7NPQmGfyms9yONH0EfQ0luBu0V2U1vVg1Ft4DnDYrPnEmO7OSxVwY1kDSViziKx4OvD9xle0C09rgoBj0ys89VBw6sKTYltvpK8TUJcODCI/7jOsD1uCg4V2AV3Y0GVhcGM0LufqlqzK1Vy9gCLzJMglxN5ez/qirTAaO0+0M1u6uEYKEHs0F8Wfv47IuD5MDt7C9dwBnylODexg7pifwpNiB7niYdM5s1/EIZ87O9JtAgbS6kVWBotta7QM+VCiPofI5bc727D0zc04+W0mdr62Cv8TexMjEHGQE37wo+rCc6YxrUmz8CQM9Ho4nDfwt0VbUTgmYjppEaEuPMml0ubENViychfOlylXeMhXMXi8ZiCF57m094TYpZhMXmvyYYviPVs5nKHpQZyQY0qhBrgmb+PvL72Fbce+RUb0ckSsP4VC27QPAesuPP3EjlJ4igjQRsuYUAy1JL2N328rdf/gzdljRs0FNj9XffgtYjh8roCoNBzz6w+PyT3VJBaeA8KYa/PFodbc0qcqPPl4ooVnIIhRGjN0xVM9z00Lc2k9LA4WyUvnm8iEbgFr+cmVIUHrET/5oeXjXKEeSR4HVnh2CDjcrDH9wpPHmgaKH1aPLfL+/CrAFJ7D7RletN8j3I9bjYW/PY66tgJELfwFtl96COOZcDz7uxhU57FYxDPlORz2cYTBIh69XI8CFS6LoOfIpY/TH0Ti+gC97ATQ/vcXZeOwCmlHJXEnJIN1a2AQfk4OVUXwheSPrExG/eVrtE8BU2OPfIg0S+/MiMBHOshG0vao8QDe33IL9qlaHFgb7eZ9q/vomfCPxTpa6YGiTw6X4eNFP8XmlIeoP78Wzyzch5JcVuNviisFXF7tI85+FU6UFMUKljOfwdrNL8bUweHvGuBwGhD9yiL8cedBbH9zAd6PrUI5h3aLNbUIODaKsTx4LzgUHCnoeGwrHysX6wdhPL8Nn/3Hgv7CrVjwsw1IN6Qx+LeHjlIGN+ZyFmDXa7twNyjyB41m/nYCNQnv4tkFryIsLAxhL/8JSSZSeDqQ8/EGJJqVVUpSsFTHH8GVDiesWdHY9Gkm8rPuoEVdNUDUfajrBjb+6pfYkdak+kUpRcQlIefWHrygwj+G/7MSDRzelFpBLsupUbYR+4vx/T9YVGRxVpQbWRksto2s7lOUaKXqBwjkw+be1TtQNOJE+cF1OFRIHBZxkCW3j/qQnAQ/Sv7I2FEfxJj21ZXPRXxi+T4seS4Cuw7twspnwhFnyuXySkHh5lVleDHAVvcG3c7xm9ga/r9ooN9VcFsl2l5Uc4zp952DFSjlcuNEzT2/mExeazU69OyNZKxXoYrNThF5WO3dUN4TnU6UHPgD9t8cwWjdIXy04667MKTjNEEJXvMiK8XYqfZBPHgkaYu9URjLTB03EPX8L/DyO1twIPYWuhwsdjCj3iFgUkVEpUUYM6g/PLZ47ZeXBRsquHktvrLXF4dllipG84v1Q37jaUHYEeSe/9CNNZ4JMfrecSVmSq2e+UqN9x0mV164HBziMJ4NA62+Oe2RkcUtN9pErSsKxPwg5/uHRT260J6zEQt+HoXbrfX48o2f4M9HHsAioCdNeMDNuUfvVCNl/S8RKNa0ooj173Hx0J5ckP/nUgGm8Ay8YRZJNuQH+8ig0Thcls2YgJgLLaoNjNneBayTd/IglyBmwrppoarIZsnafzMjAnk8FduOC3RzWucEM6v43qaPxfK9jbvDaqzdMjlEtJ9ryP1QC7k2fxhTETc27bS7UacuKp6W0cQ7XYylclAgKDjBX9q+0ox4j8O/8XE5Pq6s2IsHz8UzUxi1eRCkSmtODA7YmRVQ5TX23uyxg0o7TB4rT4uxx2EZyUoMxUry8cfryCN0/WHrfPHinFCIK0K/PH6UMdrPg8DybdJO0L9KzvN++WkcE72XEZ9sFsc7wXZ/LYjPs+Ocf63V54HXmbSqm2PEV3fXTmgNb/7Ok2Atl0fC6+4nCGKUHadZH+k4ooeo9O/PzOdKjAHeP94ebT/EZ2dCjIpHsM8oOTgzxpPOCWwL6kf6+RGIj/pzGosJVfcc6H39OVfdiqhH6q3RHwk/rLZL3tdTYJaFJ9tksNjHKVcnyoub4f3xNtuYziOJddMR5yl/6elCwT3lJzuE3Ccrrdd2LcHyPxyDwaZc3QkhF6SpT5ACJJ4kklc5IVIPRYtQujcnhafH4fnAPkqsWygF149h61ODgvsxxJV9SgWkAk+GAnOI8XwyHHpMK6Qejyng/B4+h4Xn/Boue5MKSAWkAlIBqYBUQCogFQgtBWThGVrnS1orFZAKSAWkAlIBqYBUIGQVkIVnyJ46abhUQCogFZAKSAWkAlKB0FJAFp6hdb6ktVIBqYBUQCogFZAKSAVCVgFZeIbsqZOGSwWkAlIBqYBUQCogFQgtBWThGVrnS1orFZAKSAWkAlIBqYBUIGQVkIVnyJ46abhUQCogFZAKSAWkAlKB0FLg/wHtYrZofXP7rwAAAABJRU5ErkJggg=="}}},{"cell_type":"markdown","source":"# Import Libraries","metadata":{}},{"cell_type":"code","source":"import torch\nimport torch.nn as nn\nimport torch.optim as optim\nfrom torch.utils.data import DataLoader, Dataset\nimport pandas as pd\nimport pydicom\nimport cv2\nimport os\nimport numpy as np\nfrom torchvision import transforms","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:50:25.932913Z","iopub.execute_input":"2024-12-30T00:50:25.933263Z","iopub.status.idle":"2024-12-30T00:50:25.938591Z","shell.execute_reply.started":"2024-12-30T00:50:25.933235Z","shell.execute_reply":"2024-12-30T00:50:25.937186Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Define the VGG16 Architecture","metadata":{}},{"cell_type":"code","source":"# Define the VGG16 architecture\nclass VGG16(nn.Module):\n    def __init__(self, num_classes=15):  # 14 classes + \"No finding\"\n        super(VGG16, self).__init__()\n        self.features = nn.Sequential(\n            nn.Conv2d(3, 64, kernel_size=3, padding=1),\n            nn.ReLU(inplace=True),\n            nn.Conv2d(64, 64, kernel_size=3, padding=1),\n            nn.ReLU(inplace=True),\n            nn.MaxPool2d(kernel_size=2, stride=2),\n            nn.Conv2d(64, 128, kernel_size=3, padding=1),\n            nn.ReLU(inplace=True),\n            nn.Conv2d(128, 128, kernel_size=3, padding=1),\n            nn.ReLU(inplace=True),\n            nn.MaxPool2d(kernel_size=2, stride=2),\n            nn.Conv2d(128, 256, kernel_size=3, padding=1),\n            nn.ReLU(inplace=True),\n            nn.Conv2d(256, 256, kernel_size=3, padding=1),\n            nn.ReLU(inplace=True),\n            nn.Conv2d(256, 256, kernel_size=3, padding=1),\n            nn.ReLU(inplace=True),\n            nn.MaxPool2d(kernel_size=2, stride=2),\n            nn.Conv2d(256, 512, kernel_size=3, padding=1),\n            nn.ReLU(inplace=True),\n            nn.Conv2d(512, 512, kernel_size=3, padding=1),\n            nn.ReLU(inplace=True),\n            nn.Conv2d(512, 512, kernel_size=3, padding=1),\n            nn.ReLU(inplace=True),\n            nn.MaxPool2d(kernel_size=2, stride=2),\n            nn.Conv2d(512, 512, kernel_size=3, padding=1),\n            nn.ReLU(inplace=True),\n            nn.Conv2d(512, 512, kernel_size=3, padding=1),\n            nn.ReLU(inplace=True),\n            nn.Conv2d(512, 512, kernel_size=3, padding=1),\n            nn.ReLU(inplace=True),\n            nn.MaxPool2d(kernel_size=2, stride=2),\n        )\n\n        self.classifier = nn.Sequential(\n            nn.Linear(512 * 7 * 7, 4096),\n            nn.ReLU(inplace=True),\n            nn.Dropout(),\n            nn.Linear(4096, 4096),\n            nn.ReLU(inplace=True),\n            nn.Dropout(),\n            nn.Linear(4096, num_classes),\n        )\n\n    def forward(self, x):\n        x = self.features(x)\n        x = torch.flatten(x, 1)\n        x = self.classifier(x)\n        return x","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:50:25.943999Z","iopub.execute_input":"2024-12-30T00:50:25.94443Z","iopub.status.idle":"2024-12-30T00:50:25.960204Z","shell.execute_reply.started":"2024-12-30T00:50:25.944383Z","shell.execute_reply":"2024-12-30T00:50:25.959213Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Custom Dataset Class for Loading DICOM Images","metadata":{}},{"cell_type":"code","source":"# class VinBigDataDataset(Dataset):\n#     def __init__(self, csv_file, img_dir, transform=None):\n#         self.annotations = pd.read_csv(csv_file)\n#         self.img_dir = img_dir\n#         self.transform = transform\n\n#     def __len__(self):\n#         return len(self.annotations)\n\n#     def __getitem__(self, idx):\n#         img_path = os.path.join(self.img_dir, self.annotations.iloc[idx, 0] + \".dicom\")\n#         dicom_image = pydicom.dcmread(img_path)\n        \n#         # Convert to uint8 by scaling the pixel values\n#         img = dicom_image.pixel_array\n#         img = img.astype(np.uint8)  # Convert uint16 to uint8 by truncation\n\n#         # Optionally normalize the pixel values to the [0, 255] range\n#         img = cv2.normalize(img, None, 0, 255, cv2.NORM_MINMAX)\n\n#         img = cv2.cvtColor(img, cv2.COLOR_GRAY2RGB)  # Convert to RGB\n#         img = cv2.resize(img, (224, 224))  # Resize for VGG16\n\n#         if self.transform:\n#             img = self.transform(img)\n\n#         label = int(self.annotations.iloc[idx, 2])  # class_id\n#         return img, label\n\nimport pandas as pd\nimport random\n\nclass VinBigDataDataset(Dataset):\n    def __init__(self, csv_file, img_dir, transform=None, num_samples=300):\n        self.annotations = pd.read_csv(csv_file)\n        self.img_dir = img_dir\n        self.transform = transform\n        \n        # Randomly sample the specified number of samples\n        self.annotations = self.annotations.sample(n=num_samples, random_state=42).reset_index(drop=True)\n\n    def __len__(self):\n        return len(self.annotations)\n\n    def __getitem__(self, idx):\n        img_path = os.path.join(self.img_dir, self.annotations.iloc[idx, 0] + \".dicom\")\n        dicom_image = pydicom.dcmread(img_path)\n        \n        # Convert to uint8 by scaling the pixel values\n        img = dicom_image.pixel_array\n        img = img.astype(np.uint8)  # Convert uint16 to uint8 by truncation\n\n        # Optionally normalize the pixel values to the [0, 255] range\n        img = cv2.normalize(img, None, 0, 255, cv2.NORM_MINMAX)\n\n        img = cv2.cvtColor(img, cv2.COLOR_GRAY2RGB)  # Convert to RGB\n        img = cv2.resize(img, (224, 224))  # Resize for VGG16\n\n        if self.transform:\n            img = self.transform(img)\n\n        label = int(self.annotations.iloc[idx, 2])  # class_id\n        return img, label\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:50:25.962345Z","iopub.execute_input":"2024-12-30T00:50:25.962731Z","iopub.status.idle":"2024-12-30T00:50:25.978659Z","shell.execute_reply.started":"2024-12-30T00:50:25.962639Z","shell.execute_reply":"2024-12-30T00:50:25.977595Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Hyperparameters and Dataset Paths","metadata":{}},{"cell_type":"code","source":"# Hyperparameters and configuration\nnum_classes = 15  # 14 abnormalities + \"No finding\"\nbatch_size = 16\nlearning_rate = 0.001\nnum_epochs = 2\n\n# Transformations for DICOM images\ntransform = transforms.Compose([\n    transforms.ToPILImage(),\n    transforms.ToTensor(),\n    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),\n])\n\n# Dataset paths\ntrain_csv = \"/kaggle/input/vinbigdata-chest-xray-abnormalities-detection/train.csv\"\ntrain_img_dir = \"/kaggle/input/vinbigdata-chest-xray-abnormalities-detection/train\"\ntest_img_dir = \"/kaggle/input/vinbigdata-chest-xray-abnormalities-detection/test\"","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:50:25.980227Z","iopub.execute_input":"2024-12-30T00:50:25.980514Z","iopub.status.idle":"2024-12-30T00:50:25.997062Z","shell.execute_reply.started":"2024-12-30T00:50:25.980485Z","shell.execute_reply":"2024-12-30T00:50:25.996161Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Load Dataset and DataLoader","metadata":{}},{"cell_type":"code","source":"# # Load dataset\n# train_dataset = VinBigDataDataset(csv_file=train_csv, img_dir=train_img_dir, transform=transform)\n# train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True)\ntrain_dataset = VinBigDataDataset(\n    csv_file='/kaggle/input/vinbigdata-chest-xray-abnormalities-detection/train.csv',\n    img_dir='/kaggle/input/vinbigdata-chest-xray-abnormalities-detection/train',\n    transform=transform,\n    num_samples=300  # Limit to 300 images\n)\n\ntrain_loader = DataLoader(train_dataset, batch_size=32, shuffle=True)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:50:25.998603Z","iopub.execute_input":"2024-12-30T00:50:25.999027Z","iopub.status.idle":"2024-12-30T00:50:26.110776Z","shell.execute_reply.started":"2024-12-30T00:50:25.998986Z","shell.execute_reply":"2024-12-30T00:50:26.109945Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"from torch.utils.data import random_split\nfull_dataset = VinBigDataDataset(\n    csv_file='/kaggle/input/vinbigdata-chest-xray-abnormalities-detection/train.csv',\n    img_dir='/kaggle/input/vinbigdata-chest-xray-abnormalities-detection/train',\n    transform=transform,\n    num_samples=300\n)\n\n# Split sizes for train and validation\ntrain_size = int(0.8 * len(full_dataset))\nval_size = len(full_dataset) - train_size\n\n# Split the dataset into training and validation sets\ntrain_dataset, val_dataset = random_split(full_dataset, [train_size, val_size])\n\n# DataLoaders for training and validation\ntrain_loader = DataLoader(train_dataset, batch_size=32, shuffle=True)\nval_loader = DataLoader(val_dataset, batch_size=32, shuffle=False)\n\n# Display dataset sizes\nprint(f\"Train dataset size: {len(train_dataset)}\")\nprint(f\"Validation dataset size: {len(val_dataset)}\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:50:26.114685Z","iopub.execute_input":"2024-12-30T00:50:26.115014Z","iopub.status.idle":"2024-12-30T00:50:26.203891Z","shell.execute_reply.started":"2024-12-30T00:50:26.114984Z","shell.execute_reply":"2024-12-30T00:50:26.202811Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Model, Loss Function, and Optimizer Setup","metadata":{}},{"cell_type":"code","source":"# Model, loss, and optimizer\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\nmodel = VGG16(num_classes=num_classes).to(device)\ncriterion = nn.CrossEntropyLoss()\noptimizer = optim.Adam(model.parameters(), lr=learning_rate)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:50:26.206457Z","iopub.execute_input":"2024-12-30T00:50:26.206775Z","iopub.status.idle":"2024-12-30T00:50:27.546363Z","shell.execute_reply.started":"2024-12-30T00:50:26.206742Z","shell.execute_reply":"2024-12-30T00:50:27.54538Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Training Function","metadata":{}},{"cell_type":"code","source":"# Training function\ndef train(model, train_loader, criterion, optimizer, num_epochs):\n    model.train()\n    for epoch in range(num_epochs):\n        print('epoch:',epoch)\n        running_loss = 0.0\n        for images, labels in train_loader:\n            images, labels = images.to(device), labels.to(device)\n\n            # Forward pass\n            outputs = model(images)\n            loss = criterion(outputs, labels)\n\n            # Backward and optimize\n            optimizer.zero_grad()\n            loss.backward()\n            optimizer.step()\n\n            running_loss += loss.item()\n\n        print(f'Epoch [{epoch + 1}/{num_epochs}], Loss: {running_loss / len(train_loader):.4f}')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:50:27.547751Z","iopub.execute_input":"2024-12-30T00:50:27.548074Z","iopub.status.idle":"2024-12-30T00:50:27.554233Z","shell.execute_reply.started":"2024-12-30T00:50:27.548042Z","shell.execute_reply":"2024-12-30T00:50:27.553275Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Train the Model","metadata":{}},{"cell_type":"code","source":"train(model, train_loader, criterion, optimizer, num_epochs)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:50:27.555582Z","iopub.execute_input":"2024-12-30T00:50:27.555906Z","iopub.status.idle":"2024-12-30T01:05:23.816825Z","shell.execute_reply.started":"2024-12-30T00:50:27.555877Z","shell.execute_reply":"2024-12-30T01:05:23.815325Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Save the trained model\ntorch.save(model.state_dict(), 'model.pth')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T01:05:23.819389Z","iopub.execute_input":"2024-12-30T01:05:23.819778Z","iopub.status.idle":"2024-12-30T01:05:24.815091Z","shell.execute_reply.started":"2024-12-30T01:05:23.819728Z","shell.execute_reply":"2024-12-30T01:05:24.813974Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"for epoch in range(2):\n    # Training loop\n    model.train()\n    running_loss = 0.0\n    for batch_idx, (inputs, labels) in enumerate(train_loader):\n        inputs, labels = inputs.to(device), labels.to(device)\n\n        optimizer.zero_grad()\n        outputs = model(inputs)\n        loss = criterion(outputs, labels)\n        loss.backward()\n        optimizer.step()\n\n        running_loss += loss.item()\n    print(f\"Epoch [{epoch+1}/{num_epochs}], Training Loss: {running_loss / len(train_loader):.4f}\")\n\n    # Validation loop\n    model.eval()\n    val_loss = 0.0\n    correct = 0\n    total = 0\n    with torch.no_grad():\n        for inputs, labels in val_loader:\n            inputs, labels = inputs.to(device), labels.to(device)\n            outputs = model(inputs)\n            loss = criterion(outputs, labels)\n            val_loss += loss.item()\n\n            _, predicted = torch.max(outputs, 1)\n            total += labels.size(0)\n            correct += (predicted == labels).sum().item()\n\n    val_accuracy = 100 * correct / total\n    print(f\"Epoch [{epoch+1}/{num_epochs}], Validation Loss: {val_loss / len(val_loader):.4f}, Validation Accuracy: {val_accuracy:.2f}%\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T01:05:24.816554Z","iopub.execute_input":"2024-12-30T01:05:24.816975Z","iopub.status.idle":"2024-12-30T01:22:37.073353Z","shell.execute_reply.started":"2024-12-30T01:05:24.816931Z","shell.execute_reply":"2024-12-30T01:22:37.070906Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"import torch\nimport os\nimport pydicom\nimport cv2\nfrom torch.utils.data import DataLoader, Dataset\nfrom torchvision import transforms\nimport numpy as np\n\n# Define the test dataset class for unlabeled data\nclass VinBigDataInferenceDataset(Dataset):\n    def __init__(self, img_dir, transform=None):\n        self.img_dir = img_dir\n        self.transform = transform\n        self.image_files = [f for f in os.listdir(img_dir) if f.endswith(\".dicom\")]\n\n    def __len__(self):\n        return len(self.image_files)\n\n    def __getitem__(self, idx):\n        img_path = os.path.join(self.img_dir, self.image_files[idx])\n        dicom_image = pydicom.dcmread(img_path)\n\n        # Convert to uint8 by scaling the pixel values\n        img = dicom_image.pixel_array\n        img = img.astype(np.uint8)\n\n        # Normalize the pixel values to the [0, 255] range\n        img = cv2.normalize(img, None, 0, 255, cv2.NORM_MINMAX)\n        img = cv2.cvtColor(img, cv2.COLOR_GRAY2RGB)  # Convert to RGB\n        img = cv2.resize(img, (224, 224))  # Resize for VGG16\n\n        if self.transform:\n            img = self.transform(img)\n\n        return img, self.image_files[idx]  # Returning the file name for reference\n\n# Transformations for DICOM images\ntransform = transforms.Compose([\n    transforms.ToPILImage(),\n    transforms.ToTensor(),\n    transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),\n])\n\n# Directory for test images\ntest_img_dir = \"/kaggle/input/vinbigdata-chest-xray-abnormalities-detection/test\"\ntest_dataset = VinBigDataInferenceDataset(img_dir=test_img_dir, transform=transform)\ntest_loader = DataLoader(test_dataset, batch_size=16, shuffle=False)\n\n# Load the model and set to evaluation mode\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\nmodel = VGG16(num_classes=15)  # Adjust num_classes if needed\nmodel.load_state_dict(torch.load('model.pth'))\nmodel = model.to(device)\nmodel.eval()\n\n# Inference function\ndef run_inference(model, test_loader):\n    predictions = {}\n\n    with torch.no_grad():  # Disable gradient calculation for inference\n        for images, filenames in test_loader:\n            images = images.to(device)\n            outputs = model(images)\n            _, predicted = torch.max(outputs, 1)  # Get class with highest score\n            \n            # Store predictions with file names\n            for filename, pred in zip(filenames, predicted.cpu().numpy()):\n                predictions[filename] = pred\n\n    # Display predictions\n    for filename, pred in predictions.items():\n        print(f'Image: {filename}, Predicted Label: {pred}')\n\n# Run inference\nrun_inference(model, test_loader)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T01:22:37.075649Z","iopub.execute_input":"2024-12-30T01:22:37.075988Z","iopub.status.idle":"2024-12-30T02:24:52.604098Z","shell.execute_reply.started":"2024-12-30T01:22:37.075953Z","shell.execute_reply":"2024-12-30T02:24:52.601737Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Another code","metadata":{}},{"cell_type":"code","source":"import torch\nimport torch.nn as nn\nimport torch.optim as optim\nimport torch.nn.functional as F\nfrom torch.utils.data import Dataset, DataLoader\nfrom torchvision import transforms as ts\nimport pydicom\nimport os\nimport numpy as np\nimport cv2\nimport pandas as pd\nfrom torch.autograd import Variable\nfrom matplotlib import pyplot as plt\nimport math","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T02:24:52.610332Z","iopub.execute_input":"2024-12-30T02:24:52.610881Z","iopub.status.idle":"2024-12-30T02:24:52.621298Z","shell.execute_reply.started":"2024-12-30T02:24:52.610804Z","shell.execute_reply":"2024-12-30T02:24:52.620052Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"class VinBigDataDataset(Dataset):\n    def __init__(self, csv_file, img_dir, transform=None, num_samples=300):\n        self.annotations = pd.read_csv(csv_file)\n        self.img_dir = img_dir\n        self.transform = transform\n        \n        # Randomly sample the specified number of samples\n        self.annotations = self.annotations.sample(n=num_samples, random_state=42).reset_index(drop=True)\n\n    def __len__(self):\n        return len(self.annotations)\n\n    def __getitem__(self, idx):\n        img_path = os.path.join(self.img_dir, self.annotations.iloc[idx, 0] + \".dicom\")\n        dicom_image = pydicom.dcmread(img_path)\n        \n        # Convert to uint8 by scaling the pixel values\n        img = dicom_image.pixel_array\n        img = img.astype(np.uint8)  # Convert uint16 to uint8 by truncation\n\n        # Optionally normalize the pixel values to the [0, 255] range\n        img = cv2.normalize(img, None, 0, 255, cv2.NORM_MINMAX)\n\n        img = cv2.cvtColor(img, cv2.COLOR_GRAY2RGB)  # Convert to RGB\n        img = cv2.resize(img, (224, 224))  # Resize for VGG16\n\n        if self.transform:\n            img = self.transform(img)\n\n        label = int(self.annotations.iloc[idx, 2])  # class_id\n        return img, label\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T02:24:52.623064Z","iopub.execute_input":"2024-12-30T02:24:52.623376Z","iopub.status.idle":"2024-12-30T02:24:52.647987Z","shell.execute_reply.started":"2024-12-30T02:24:52.623344Z","shell.execute_reply":"2024-12-30T02:24:52.646968Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"transform = ts.Compose(\n    [\n        ts.ToTensor(),\n        ts.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))  # Normalize to [-1, 1]\n    ]\n)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T02:24:52.650097Z","iopub.execute_input":"2024-12-30T02:24:52.650405Z","iopub.status.idle":"2024-12-30T02:24:52.664979Z","shell.execute_reply.started":"2024-12-30T02:24:52.650376Z","shell.execute_reply":"2024-12-30T02:24:52.663941Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"train_dataset = VinBigDataDataset(\n    csv_file='/kaggle/input/vinbigdata-chest-xray-abnormalities-detection/train.csv',\n    img_dir='/kaggle/input/vinbigdata-chest-xray-abnormalities-detection/train',\n    transform=transform,\n    num_samples=300  # Limit to 300 images\n)\n\ntrain_loader = DataLoader(train_dataset, batch_size=4, shuffle=True)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T02:24:52.6669Z","iopub.execute_input":"2024-12-30T02:24:52.667265Z","iopub.status.idle":"2024-12-30T02:24:52.870534Z","shell.execute_reply.started":"2024-12-30T02:24:52.667232Z","shell.execute_reply":"2024-12-30T02:24:52.869568Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# s","metadata":{}},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# # # # class VGGNet(nn.Module):\n# # # #     def __init__(self, num_classes):\n# # # #         super(VGGNet, self).__init__()\n# # # #         self.features = nn.Sequential(\n# # # #             nn.Conv2d(3, 32, 3, padding=1),  # Conv1\n# # # #             nn.ReLU(True),\n# # # #             nn.Conv2d(32, 32, 3, padding=1),  # Conv2\n# # # #             nn.ReLU(True),\n# # # #             nn.MaxPool2d(2, 2),  # Pool1\n# # # #             nn.Conv2d(32, 64, 3, padding=1),  # Conv3\n# # # #             nn.ReLU(True),\n# # # #             nn.Conv2d(64, 64, 3, padding=1),  # Conv4\n# # # #             nn.ReLU(True),\n# # # #             nn.MaxPool2d(2, 2),  # Pool2\n# # # #             nn.Conv2d(64, 128, 3, padding=1),  # Conv5\n# # # #             nn.ReLU(True),\n# # # #             nn.Conv2d(128, 128, 3, padding=1),  # Conv6\n# # # #             nn.ReLU(True),\n# # # #             nn.Conv2d(128, 128, 3, padding=1),  # Conv7\n# # # #             nn.ReLU(True),\n# # # #             nn.MaxPool2d(2, 2),  # Pool3\n# # # #             nn.Conv2d(128, 256, 3, padding=1),  # Conv8\n# # # #             nn.ReLU(True),\n# # # #             nn.Conv2d(256, 256, 3, padding=1),  # Conv9\n# # # #             nn.ReLU(True),\n# # # #             nn.Conv2d(256, 256, 3, padding=1),  # Conv10\n# # # #             nn.ReLU(True),\n# # # #             nn.MaxPool2d(2, 2),  # Pool4\n# # # #             nn.Conv2d(256, 256, 3, padding=1),  # Conv11\n# # # #             nn.ReLU(True),\n# # # #             nn.Conv2d(256, 256, 3, padding=1),  # Conv12\n# # # #             nn.ReLU(True),\n# # # #             nn.Conv2d(256, 256, 3, padding=1),  # Conv13\n# # # #             nn.ReLU(True),\n# # # #         )\n\n# # # #         self.classifier = nn.Sequential(\n# # # #             nn.Linear(2 * 2 * 256, 512),\n# # # #             nn.ReLU(True),\n# # # #             nn.Dropout(),\n# # # #             nn.Linear(512, 512),\n# # # #             nn.ReLU(True),\n# # # #             nn.Dropout(),\n# # # #             nn.Linear(512, num_classes),\n# # # #         )\n# # # #         self._initialize_weights()\n\n# # # #     def _initialize_weights(self):\n# # # #         for m in self.modules():\n# # # #             if isinstance(m, nn.Conv2d):\n# # # #                 n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels\n# # # #                 m.weight.data.normal_(0, math.sqrt(2. / n))\n# # # #                 if m.bias is not None:\n# # # #                     m.bias.data.zero_()\n# # # #             elif isinstance(m, nn.Linear):\n# # # #                 m.weight.data.normal_(0, 0.01)\n# # # #                 m.bias.data.zero_()\n\n# # # #     def forward(self, x):\n# # # #         x = self.features(x)\n# # # #         x = x.view(x.size(0), -1)\n# # # #         x = self.classifier(x)\n# # # #         return x\n\n# # # import torch\n# # # import torch.nn as nn\n# # # import torch.nn.functional as F\n# # # import math\n\n# # # class VGGNet(nn.Module):\n# # #     def __init__(self, num_classes):\n# # #         super(VGGNet, self).__init__()\n# # #         self.features = nn.Sequential(\n# # #             nn.Conv2d(3, 32, 3, padding=1),  # Conv1\n# # #             nn.ReLU(True),\n# # #             nn.Conv2d(32, 32, 3, padding=1),  # Conv2\n# # #             nn.ReLU(True),\n# # #             nn.MaxPool2d(2, 2),  # Pool1\n# # #             nn.Conv2d(32, 64, 3, padding=1),  # Conv3\n# # #             nn.ReLU(True),\n# # #             nn.Conv2d(64, 64, 3, padding=1),  # Conv4\n# # #             nn.ReLU(True),\n# # #             nn.MaxPool2d(2, 2),  # Pool2\n# # #             nn.Conv2d(64, 128, 3, padding=1),  # Conv5\n# # #             nn.ReLU(True),\n# # #             nn.Conv2d(128, 128, 3, padding=1),  # Conv6\n# # #             nn.ReLU(True),\n# # #             nn.Conv2d(128, 128, 3, padding=1),  # Conv7\n# # #             nn.ReLU(True),\n# # #             nn.MaxPool2d(2, 2),  # Pool3\n# # #             nn.Conv2d(128, 256, 3, padding=1),  # Conv8\n# # #             nn.ReLU(True),\n# # #             nn.Conv2d(256, 256, 3, padding=1),  # Conv9\n# # #             nn.ReLU(True),\n# # #             nn.Conv2d(256, 256, 3, padding=1),  # Conv10\n# # #             nn.ReLU(True),\n# # #             nn.MaxPool2d(2, 2),  # Pool4\n# # #             nn.Conv2d(256, 256, 3, padding=1),  # Conv11\n# # #             nn.ReLU(True),\n# # #             nn.Conv2d(256, 256, 3, padding=1),  # Conv12\n# # #             nn.ReLU(True),\n# # #             nn.Conv2d(256, 256, 3, padding=1),  # Conv13\n# # #             nn.ReLU(True),\n# # #         )\n\n# # #         # Initializing the classifier with the correct input size\n# # #         self.classifier = nn.Sequential(\n# # #             nn.Linear(2 * 2 * 256, 512),  # Adjust based on the output shape of features block\n# # #             nn.ReLU(True),\n# # #             nn.Dropout(),\n# # #             nn.Linear(512, 512),\n# # #             nn.ReLU(True),\n# # #             nn.Dropout(),\n# # #             nn.Linear(512, num_classes),\n# # #         )\n# # #         self._initialize_weights()\n\n# # #     def _initialize_weights(self):\n# # #         for m in self.modules():\n# # #             if isinstance(m, nn.Conv2d):\n# # #                 n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels\n# # #                 m.weight.data.normal_(0, math.sqrt(2. / n))\n# # #                 if m.bias is not None:\n# # #                     m.bias.data.zero_()\n# # #             elif isinstance(m, nn.Linear):\n# # #                 m.weight.data.normal_(0, 0.01)\n# # #                 m.bias.data.zero_()\n\n# # #     def forward(self, x):\n# # #         x = self.features(x)\n        \n# # #         # Print the shape of the output from the convolutional layers for debugging\n# # #         print(f\"Shape after convolutional layers: {x.shape}\")\n        \n# # #         # Flatten the tensor before passing to the fully connected layers\n# # #         x = x.view(x.size(0), -1)  # Flatten the tensor to (batch_size, features)\n        \n# # #         x = self.classifier(x)\n# # #         return x\n\n\n# # class VGGNet(nn.Module):\n# #     def __init__(self, num_classes):\n# #         super(VGGNet, self).__init__()\n# #         self.features = nn.Sequential(\n# #             nn.Conv2d(3, 32, 3, padding=1),  # Conv1\n# #             nn.ReLU(True),\n# #             nn.Conv2d(32, 32, 3, padding=1),  # Conv2\n# #             nn.ReLU(True),\n# #             nn.MaxPool2d(2, 2),  # Pool1\n# #             nn.Conv2d(32, 64, 3, padding=1),  # Conv3\n# #             nn.ReLU(True),\n# #             nn.Conv2d(64, 64, 3, padding=1),  # Conv4\n# #             nn.ReLU(True),\n# #             nn.MaxPool2d(2, 2),  # Pool2\n# #             nn.Conv2d(64, 128, 3, padding=1),  # Conv5\n# #             nn.ReLU(True),\n# #             nn.Conv2d(128, 128, 3, padding=1),  # Conv6\n# #             nn.ReLU(True),\n# #             nn.Conv2d(128, 128, 3, padding=1),  # Conv7\n# #             nn.ReLU(True),\n# #             nn.MaxPool2d(2, 2),  # Pool3\n# #             nn.Conv2d(128, 256, 3, padding=1),  # Conv8\n# #             nn.ReLU(True),\n# #             nn.Conv2d(256, 256, 3, padding=1),  # Conv9\n# #             nn.ReLU(True),\n# #             nn.Conv2d(256, 256, 3, padding=1),  # Conv10\n# #             nn.ReLU(True),\n# #             nn.MaxPool2d(2, 2),  # Pool4\n# #             nn.Conv2d(256, 256, 3, padding=1),  # Conv11\n# #             nn.ReLU(True),\n# #             nn.Conv2d(256, 256, 3, padding=1),  # Conv12\n# #             nn.ReLU(True),\n# #             nn.Conv2d(256, 256, 3, padding=1),  # Conv13\n# #             nn.ReLU(True),\n# #         )\n\n# #         # Adjust the input size of the first Linear layer based on the actual output size\n# #         self.classifier = nn.Sequential(\n# #             nn.Linear(256 * 14 * 14, 512),  # Adjusted input size for the first Linear layer\n# #             nn.ReLU(True),\n# #             nn.Dropout(),\n# #             nn.Linear(512, 512),\n# #             nn.ReLU(True),\n# #             nn.Dropout(),\n# #             nn.Linear(512, num_classes),\n# #         )\n# #         self._initialize_weights()\n\n# #     def _initialize_weights(self):\n# #         for m in self.modules():\n# #             if isinstance(m, nn.Conv2d):\n# #                 n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels\n# #                 m.weight.data.normal_(0, math.sqrt(2. / n))\n# #                 if m.bias is not None:\n# #                     m.bias.data.zero_()\n# #             elif isinstance(m, nn.Linear):\n# #                 m.weight.data.normal_(0, 0.01)\n# #                 m.bias.data.zero_()\n\n# #     def forward(self, x):\n# #         x = self.features(x)\n        \n# #         # Print the shape of the output from the convolutional layers for debugging\n# #         print(f\"Shape after convolutional layers: {x.shape}\")\n        \n# #         # Flatten the tensor before passing to the fully connected layers\n# #         x = x.view(x.size(0), -1)  # Flatten the tensor to (batch_size, features)\n        \n# #         x = self.classifier(x)\n# #         return x\n\n\n# import torch\n# import torch.nn as nn\n# import torch.optim as optim\n# import numpy as np\n# import math\n# import matplotlib.pyplot as plt\n\n# class VGGNet(nn.Module):\n#     def __init__(self, num_classes):\n#         super(VGGNet, self).__init__()\n#         self.features = nn.Sequential(\n#             nn.Conv2d(3, 32, 3, padding=1),  # Conv1\n#             nn.ReLU(True),\n#             nn.Conv2d(32, 32, 3, padding=1),  # Conv2\n#             nn.ReLU(True),\n#             nn.MaxPool2d(2, 2),  # Pool1\n#             nn.Conv2d(32, 64, 3, padding=1),  # Conv3\n#             nn.ReLU(True),\n#             nn.Conv2d(64, 64, 3, padding=1),  # Conv4\n#             nn.ReLU(True),\n#             nn.MaxPool2d(2, 2),  # Pool2\n#             nn.Conv2d(64, 128, 3, padding=1),  # Conv5\n#             nn.ReLU(True),\n#             nn.Conv2d(128, 128, 3, padding=1),  # Conv6\n#             nn.ReLU(True),\n#             nn.Conv2d(128, 128, 3, padding=1),  # Conv7\n#             nn.ReLU(True),\n#             nn.MaxPool2d(2, 2),  # Pool3\n#             nn.Conv2d(128, 256, 3, padding=1),  # Conv8\n#             nn.ReLU(True),\n#             nn.Conv2d(256, 256, 3, padding=1),  # Conv9\n#             nn.ReLU(True),\n#             nn.Conv2d(256, 256, 3, padding=1),  # Conv10\n#             nn.ReLU(True),\n#             nn.MaxPool2d(2, 2),  # Pool4\n#             nn.Conv2d(256, 256, 3, padding=1),  # Conv11\n#             nn.ReLU(True),\n#             nn.Conv2d(256, 256, 3, padding=1),  # Conv12\n#             nn.ReLU(True),\n#             nn.Conv2d(256, 256, 3, padding=1),  # Conv13\n#             nn.ReLU(True),\n#         )\n\n#         # Adjust the input size of the first Linear layer based on the actual output size\n#         self.classifier = nn.Sequential(\n#             nn.Linear(256 * 14 * 14, 512),  # Updated based on actual output size of convolutional layers\n#             nn.ReLU(True),\n#             nn.Dropout(),\n#             nn.Linear(512, 512),\n#             nn.ReLU(True),\n#             nn.Dropout(),\n#             nn.Linear(512, num_classes),\n#         )\n#         self._initialize_weights()\n\n#     def _initialize_weights(self):\n#         for m in self.modules():\n#             if isinstance(m, nn.Conv2d):\n#                 n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels\n#                 m.weight.data.normal_(0, math.sqrt(2. / n))\n#                 if m.bias is not None:\n#                     m.bias.data.zero_()\n#             elif isinstance(m, nn.Linear):\n#                 m.weight.data.normal_(0, 0.01)\n#                 m.bias.data.zero_()\n\n#     def forward(self, x):\n#         x = self.features(x)\n        \n#         # Print the shape of the output from the convolutional layers for debugging\n#         print(f\"Shape after convolutional layers: {x.shape}\")\n        \n#         # Flatten the tensor before passing to the fully connected layers\n#         x = x.view(x.size(0), -1)  # Flatten the tensor to (batch_size, features)\n        \n#         x = self.classifier(x)\n#         return x\n\n# # Device setup\n# device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n\n# # Initialize the model, loss function, and optimizer\n# model = VGGNet(num_classes=14)  # 14 classes for the VinBigData dataset\n# model = model.to(device)\n\n# # Hyperparameters\n# learning_rate = 1e-3\n# momentum = 0.9\n# num_epochs = 2\n\n# criterion = nn.CrossEntropyLoss()\n# optimizer = optim.SGD(model.parameters(), lr=learning_rate, momentum=momentum)\n\n# # Training loop\n# loss_p = np.array([])  # Store the loss for plotting\n# e = np.linspace(0, num_epochs, num_epochs)\n\n# for epoch in range(num_epochs):\n#     running_loss = 0\n#     total_images = 0\n#     correct_images = 0\n#     if epoch == 25:\n#         optimizer = optim.SGD(model.parameters(), lr=learning_rate / 10, momentum=momentum)  # Reduce lr after 25 epochs\n\n#     for i, (images, labels) in enumerate(train_loader, 0):\n#         images, labels = images.to(device), labels.to(device)\n\n#         optimizer.zero_grad()\n\n#         # Forward pass\n#         outputs = model(images)\n\n#         # Check for NaN values in outputs\n#         if torch.isnan(outputs).any():\n#             print(f\"NaN detected in outputs at batch {i}\")\n#             continue  # Skip this batch\n\n#         _, predicts = torch.max(outputs.data, 1)\n#         loss = criterion(outputs, labels)\n\n#         # Check for NaN values in loss\n#         if torch.isnan(loss).any():\n#             print(f\"NaN detected in loss at batch {i}\")\n#             continue  # Skip this batch\n\n#         # Backward pass and optimization\n#         loss.backward()\n#         optimizer.step()\n\n#         total_images += labels.size(0)\n#         correct_images += (predicts == labels).sum().item()\n#         running_loss += loss.item()\n\n#         if i % 2000 == 1999:  # Print every 2000 batches\n#             print(f'Epoch [{epoch + 1}/{num_epochs}], Batch [{i + 1}], Loss: {running_loss / 2000:.6f}, Accuracy: {100 * correct_images / total_images:.2f}%')\n#             running_loss = 0\n#             total_images = 0\n#             correct_images = 0\n\n#     loss_p = np.append(loss_p, running_loss)\n\n# print('Finished Training')\n\n# # Plot the loss\n# plt.plot(e, loss_p, color='red', linestyle='--', label='Training Loss')\n# plt.xlabel('Epoch')\n# plt.ylabel('Loss')\n# plt.legend()\n# plt.show()\nimport torch\nimport torch.nn as nn\nimport torch.optim as optim\nimport numpy as np\nimport math\nimport matplotlib.pyplot as plt\n\nclass VGGNet(nn.Module):\n    def __init__(self, num_classes):\n        super(VGGNet, self).__init__()\n        self.features = nn.Sequential(\n            nn.Conv2d(3, 32, 3, padding=1),  # Conv1\n            nn.ReLU(True),\n            nn.Conv2d(32, 32, 3, padding=1),  # Conv2\n            nn.ReLU(True),\n            nn.MaxPool2d(2, 2),  # Pool1\n            nn.Conv2d(32, 64, 3, padding=1),  # Conv3\n            nn.ReLU(True),\n            nn.Conv2d(64, 64, 3, padding=1),  # Conv4\n            nn.ReLU(True),\n            nn.MaxPool2d(2, 2),  # Pool2\n            nn.Conv2d(64, 128, 3, padding=1),  # Conv5\n            nn.ReLU(True),\n            nn.Conv2d(128, 128, 3, padding=1),  # Conv6\n            nn.ReLU(True),\n            nn.Conv2d(128, 128, 3, padding=1),  # Conv7\n            nn.ReLU(True),\n            nn.MaxPool2d(2, 2),  # Pool3\n            nn.Conv2d(128, 256, 3, padding=1),  # Conv8\n            nn.ReLU(True),\n            nn.Conv2d(256, 256, 3, padding=1),  # Conv9\n            nn.ReLU(True),\n            nn.Conv2d(256, 256, 3, padding=1),  # Conv10\n            nn.ReLU(True),\n            nn.MaxPool2d(2, 2),  # Pool4\n            nn.Conv2d(256, 256, 3, padding=1),  # Conv11\n            nn.ReLU(True),\n            nn.Conv2d(256, 256, 3, padding=1),  # Conv12\n            nn.ReLU(True),\n            nn.Conv2d(256, 256, 3, padding=1),  # Conv13\n            nn.ReLU(True),\n        )\n\n        # Adjust the input size of the first Linear layer based on the actual output size\n        self.classifier = nn.Sequential(\n            nn.Linear(256 * 14 * 14, 512),  # Updated based on actual output size of convolutional layers\n            nn.ReLU(True),\n            nn.Dropout(),\n            nn.Linear(512, 512),\n            nn.ReLU(True),\n            nn.Dropout(),\n            nn.Linear(512, num_classes),\n        )\n        self._initialize_weights()\n\n    def _initialize_weights(self):\n        for m in self.modules():\n            if isinstance(m, nn.Conv2d):\n                n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels\n                m.weight.data.normal_(0, math.sqrt(2. / n))\n                if m.bias is not None:\n                    m.bias.data.zero_()\n            elif isinstance(m, nn.Linear):\n                m.weight.data.normal_(0, 0.01)\n                m.bias.data.zero_()\n\n    def forward(self, x):\n        x = self.features(x)\n        \n        # Print the shape of the output from the convolutional layers for debugging\n        print(f\"Shape after convolutional layers: {x.shape}\")\n        \n        # Flatten the tensor before passing to the fully connected layers\n        x = x.view(x.size(0), -1)  # Flatten the tensor to (batch_size, features)\n        \n        x = self.classifier(x)\n        return x\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T02:24:52.872826Z","iopub.execute_input":"2024-12-30T02:24:52.87326Z","iopub.status.idle":"2024-12-30T02:24:52.896281Z","shell.execute_reply.started":"2024-12-30T02:24:52.873216Z","shell.execute_reply":"2024-12-30T02:24:52.895192Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n\n# Initialize the model, loss function, and optimizer\nmodel = VGGNet(num_classes=14)  # 14 classes for the VinBigData dataset\nmodel = model.to(device)\n\n# Hyperparameters\nlearning_rate = 1e-3\nmomentum = 0.9\nnum_epochs = 2\n\ncriterion = nn.CrossEntropyLoss()\noptimizer = optim.SGD(model.parameters(), lr=learning_rate, momentum=momentum)\n\n# Training loop\nloss_p = np.array([])  # Store the loss for plotting\ne = np.linspace(0, num_epochs, num_epochs)\n\nfor epoch in range(num_epochs):\n    running_loss = 0\n    total_images = 0\n    correct_images = 0\n    if epoch == 25:\n        optimizer = optim.SGD(model.parameters(), lr=learning_rate / 10, momentum=momentum)  # Reduce lr after 25 epochs\n\n    for i, (images, labels) in enumerate(train_loader, 0):\n        images, labels = images.to(device), labels.to(device)\n\n        optimizer.zero_grad()\n        outputs = model(images)\n        _, predicts = torch.max(outputs.data, 1)\n        loss = criterion(outputs, labels)\n\n        loss.backward()\n        optimizer.step()\n\n        total_images += labels.size(0)\n        correct_images += (predicts == labels).sum().item()\n        running_loss += loss.item()\n\n        if i % 2000 == 1999:  # Print every 2000 batches\n            print(f'Epoch [{epoch + 1}/{num_epochs}], Batch [{i + 1}], Loss: {running_loss / 2000:.6f}, Accuracy: {100 * correct_images / total_images:.2f}%')\n            running_loss = 0\n            total_images = 0\n            correct_images = 0\n\n    loss_p = np.append(loss_p, running_loss)\n\nprint('Finished Training')\n\n# Plot the loss\nplt.plot(e, loss_p, color='red', linestyle='--', label='Training Loss')\nplt.xlabel('Epoch')\nplt.ylabel('Loss')\nplt.legend()\nplt.show()\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T02:24:52.897811Z","iopub.execute_input":"2024-12-30T02:24:52.898137Z","iopub.status.idle":"2024-12-30T02:24:57.172494Z","shell.execute_reply.started":"2024-12-30T02:24:52.898102Z","shell.execute_reply":"2024-12-30T02:24:57.170343Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# train(model, train_loader, criterion, optimizer, num_epochs)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T02:24:57.173779Z","iopub.status.idle":"2024-12-30T02:24:57.174293Z","shell.execute_reply":"2024-12-30T02:24:57.174053Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# train","metadata":{}},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"code","source":"class VinBigDataTestDataset(Dataset):\n    def __init__(self, csv_file, img_dir, transform=None, num_samples=300):\n        self.annotations = pd.read_csv(csv_file)\n        self.img_dir = img_dir\n        self.transform = transform\n        \n        # Randomly sample the specified number of samples (same as in training)\n        self.annotations = self.annotations.sample(n=num_samples, random_state=42).reset_index(drop=True)\n\n    def __len__(self):\n        return len(self.annotations)\n\n    def __getitem__(self, idx):\n        img_path = os.path.join(self.img_dir, self.annotations.iloc[idx, 0] + \".dicom\")\n        dicom_image = pydicom.dcmread(img_path)\n        \n        # Convert to uint8 by scaling the pixel values\n        img = dicom_image.pixel_array\n        img = img.astype(np.uint8)  # Convert uint16 to uint8 by truncation\n\n        # Optionally normalize the pixel values to the [0, 255] range\n        img = cv2.normalize(img, None, 0, 255, cv2.NORM_MINMAX)\n\n        img = cv2.cvtColor(img, cv2.COLOR_GRAY2RGB)  # Convert to RGB\n        img = cv2.resize(img, (224, 224))  # Resize for VGG16\n\n        if self.transform:\n            img = self.transform(img)\n\n        label = int(self.annotations.iloc[idx, 2])  # class_id\n        return img, label\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T02:24:57.175725Z","iopub.status.idle":"2024-12-30T02:24:57.176232Z","shell.execute_reply":"2024-12-30T02:24:57.17599Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"transform = ts.Compose(\n    [\n        ts.ToTensor(),\n        ts.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))  # Normalize to [-1, 1]\n    ]\n)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T02:24:57.177341Z","iopub.status.idle":"2024-12-30T02:24:57.177827Z","shell.execute_reply":"2024-12-30T02:24:57.17755Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n\n# Load the pre-trained model\nmodel = VGGNet(num_classes=14)  # 14 classes for the VinBigData dataset\nmodel.load_state_dict(torch.load('model.pth'))  # Load the saved model weights\nmodel = model.to(device)\nmodel.eval()  # Set the model to evaluation mode\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T02:24:57.179002Z","iopub.status.idle":"2024-12-30T02:24:57.179437Z","shell.execute_reply":"2024-12-30T02:24:57.179208Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"test_dataset = VinBigDataTestDataset(\n    csv_file='/kaggle/input/vinbigdata-chest-xray-abnormalities-detection/test.csv',\n    img_dir='/kaggle/input/vinbigdata-chest-xray-abnormalities-detection/test',\n    transform=transform,\n    num_samples=300  # Number of test samples\n)\n\ntest_loader = DataLoader(test_dataset, batch_size=4, shuffle=False)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T02:24:57.180464Z","iopub.status.idle":"2024-12-30T02:24:57.18093Z","shell.execute_reply":"2024-12-30T02:24:57.180695Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"test_dataset = VinBigDataTestDataset(\n    csv_file='/kaggle/input/vinbigdata-chest-xray-abnormalities-detection/test.csv',\n    img_dir='/kaggle/input/vinbigdata-chest-xray-abnormalities-detection/test',\n    transform=transform,\n    num_samples=300  # Number of test samples\n)\n\ntest_loader = DataLoader(test_dataset, batch_size=4, shuffle=False)\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T02:24:57.182087Z","iopub.status.idle":"2024-12-30T02:24:57.182522Z","shell.execute_reply":"2024-12-30T02:24:57.182293Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"criterion = nn.CrossEntropyLoss()\n\n# Initialize variables for accuracy calculation\ncorrect_images = 0\ntotal_images = 0\ntest_loss = 0\n\n# Testing loop\nwith torch.no_grad():  # No need to compute gradients during testing\n    for images, labels in test_loader:\n        images, labels = images.to(device), labels.to(device)\n\n        # Forward pass\n        outputs = model(images)\n        _, predicts = torch.max(outputs.data, 1)\n\n        # Calculate loss\n        loss = criterion(outputs, labels)\n        test_loss += loss.item()\n\n        # Update accuracy\n        correct_images += (predicts == labels).sum().item()\n        total_images += labels.size(0)\n\n    avg_loss = test_loss / len(test_loader)\n    accuracy = 100 * correct_images / total_images\n\nprint(f\"Test Accuracy: {accuracy:.2f}%\")\nprint(f\"Test Loss: {avg_loss:.4f}\")\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T02:24:57.183749Z","iopub.status.idle":"2024-12-30T02:24:57.184186Z","shell.execute_reply":"2024-12-30T02:24:57.183952Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Save the trained model\ntorch.save(model.state_dict(), 'model.pth')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T02:24:57.185165Z","iopub.status.idle":"2024-12-30T02:24:57.185641Z","shell.execute_reply":"2024-12-30T02:24:57.185367Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Other method","metadata":{}},{"cell_type":"code","source":"# Built In Imports\nimport os\nfrom tqdm.auto import tqdm\nimport random\n\nimport numpy as np\nimport pandas as pd\n\n#image handles\nimport pydicom\nfrom pydicom.pixel_data_handlers.util import apply_voi_lut\nfrom PIL import Image\n\nimport matplotlib.pyplot as plt\nimport plotly.express as px\nimport seaborn as sns\n%matplotlib inline\nimport matplotlib.patches as patches\nimport matplotlib.image as mpimg\nimport cv2\nimport plotly.graph_objects as go","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T03:01:28.748524Z","iopub.execute_input":"2024-12-30T03:01:28.74899Z","iopub.status.idle":"2024-12-30T03:01:29.573891Z","shell.execute_reply.started":"2024-12-30T03:01:28.748932Z","shell.execute_reply":"2024-12-30T03:01:29.572746Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def read_xray(path, voi_lut = True, fix_monochrome = True):\n    dicom = pydicom.read_file(path)\n    \n    # VOI LUT (if available by DICOM device) is used to transform raw DICOM data to \"human-friendly\" view\n    if voi_lut:\n        data = apply_voi_lut(dicom.pixel_array, dicom)\n    else:\n        data = dicom.pixel_array\n               \n    # depending on this value, X-ray may look inverted - fix that:\n    if fix_monochrome and dicom.PhotometricInterpretation == \"MONOCHROME1\":\n        data = np.amax(data) - data\n        \n    data = data - np.min(data)\n    data = data / np.max(data)\n    data = (data * 255).astype(np.uint8) \n    return data","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T03:01:36.381603Z","iopub.execute_input":"2024-12-30T03:01:36.382007Z","iopub.status.idle":"2024-12-30T03:01:36.38821Z","shell.execute_reply.started":"2024-12-30T03:01:36.381972Z","shell.execute_reply":"2024-12-30T03:01:36.387259Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def resize(array, size, keep_ratio=False, resample=Image.LANCZOS):\n    im = Image.fromarray(array)\n    if keep_ratio:\n        im.thumbnail((size, size), resample)\n    else:\n        im = im.resize((size, size), resample)\n    return im","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T03:01:55.223074Z","iopub.execute_input":"2024-12-30T03:01:55.223445Z","iopub.status.idle":"2024-12-30T03:01:55.228692Z","shell.execute_reply.started":"2024-12-30T03:01:55.223409Z","shell.execute_reply":"2024-12-30T03:01:55.227652Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"dataset_dir = '../input/vinbigdata-chest-xray-abnormalities-detection'\n\n# Define the paths to the training and testing dicom folders respectively\ntrain_dir = os.path.join(dataset_dir, \"train\")\ntest_dir =  os.path.join(dataset_dir, \"test\")\n\n# get full path of images\ntrain_full_path = [os.path.join(train_dir, f_name) for f_name in os.listdir(train_dir)]\ntest_full_path  = [os.path.join(test_dir, f_name) for f_name in os.listdir(test_dir)]\n\n# get image id \ntrain_img_id = [os.path.splitext(f_name)[0] for f_name in os.listdir(train_dir)]\ntest_img_id = [os.path.splitext(f_name)[0] for f_name in os.listdir(test_dir)]\n\nprint(f\"The number of training files is {len(train_full_path)}\")\nprint(f\"The number of testing files is {len(test_full_path)}\")\n\n# create img_id and img_path\ntrain_img_id_df= pd.DataFrame(list(zip(train_img_id,train_full_path)),columns =['image_id', 'img_path'])\ntest_img_id_df = pd.DataFrame(list(zip(test_img_id,test_full_path)),columns =['image_id', 'img_path'])","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T03:02:58.107233Z","iopub.execute_input":"2024-12-30T03:02:58.107588Z","iopub.status.idle":"2024-12-30T03:02:58.523801Z","shell.execute_reply.started":"2024-12-30T03:02:58.107559Z","shell.execute_reply":"2024-12-30T03:02:58.522526Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"dataset_dir_org = '../input/vinbigdata-chest-xray-abnormalities-detection'\ntrain_df = pd.read_csv(os.path.join(dataset_dir_org, \"train.csv\"))\nss_df = pd.read_csv(os.path.join(dataset_dir_org, \"sample_submission.csv\"))\n\n# add img_path to data frame\ntrain_df['img_path'] = train_df.image_id.map(train_img_id_df.set_index('image_id')['img_path'])\nss_df['img_path'] =    ss_df.image_id.map(test_img_id_df.set_index('image_id')['img_path']) ","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T03:03:24.048564Z","iopub.execute_input":"2024-12-30T03:03:24.048992Z","iopub.status.idle":"2024-12-30T03:03:24.169003Z","shell.execute_reply.started":"2024-12-30T03:03:24.048953Z","shell.execute_reply":"2024-12-30T03:03:24.167808Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"train_df.head()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T03:03:34.097791Z","iopub.execute_input":"2024-12-30T03:03:34.098217Z","iopub.status.idle":"2024-12-30T03:03:34.116865Z","shell.execute_reply.started":"2024-12-30T03:03:34.09818Z","shell.execute_reply":"2024-12-30T03:03:34.115653Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"train_df.shape","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T03:04:15.06471Z","iopub.execute_input":"2024-12-30T03:04:15.065117Z","iopub.status.idle":"2024-12-30T03:04:15.071834Z","shell.execute_reply.started":"2024-12-30T03:04:15.06508Z","shell.execute_reply":"2024-12-30T03:04:15.070553Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"train_df_shape = pd.read_csv(os.path.join(dataset_dir, \"train.csv\"))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T03:05:07.125366Z","iopub.execute_input":"2024-12-30T03:05:07.125755Z","iopub.status.idle":"2024-12-30T03:05:07.217781Z","shell.execute_reply.started":"2024-12-30T03:05:07.125718Z","shell.execute_reply":"2024-12-30T03:05:07.216742Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# df = pd.merge(train_df, train_df_shape, on=\"image_id\")","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"df=train_df","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T03:06:12.633907Z","iopub.execute_input":"2024-12-30T03:06:12.634299Z","iopub.status.idle":"2024-12-30T03:06:12.638547Z","shell.execute_reply.started":"2024-12-30T03:06:12.634265Z","shell.execute_reply":"2024-12-30T03:06:12.637558Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# df['x_min'] = df.apply(lambda row: (row.x_min)/row.dim1, axis =1) \n# df['y_min'] = df.apply(lambda row: (row.y_min)/row.dim0, axis =1)\n\n# df['x_max'] = df.apply(lambda row: (row.x_max)/row.dim1, axis =1)\n# df['y_max'] = df.apply(lambda row: (row.y_max)/row.dim0, axis =1)\n\n# df['x_mid'] = df.apply(lambda row: (row.x_max+row.x_min)/2, axis =1)\n# df['y_mid'] = df.apply(lambda row: (row.y_max+row.y_min)/2, axis =1)\n\n# df['box_w'] = df.apply(lambda row: (row.x_max-row.x_min), axis =1)\n# df['box_h'] = df.apply(lambda row: (row.y_max-row.y_min), axis =1)\n\n# df['area'] = df['box_w']*df['box_h']\n# df.head()\nnormal_df = df.groupby(\"image_id\")[\"class_id\"].agg(lambda s:(s == 14).sum()).reset_index().rename({\"class_id\": \"num_normal_annotations\"},axis=1)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T03:10:30.245178Z","iopub.execute_input":"2024-12-30T03:10:30.245581Z","iopub.status.idle":"2024-12-30T03:10:33.713544Z","shell.execute_reply.started":"2024-12-30T03:10:30.245548Z","shell.execute_reply":"2024-12-30T03:10:33.712551Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"normal_df.head() \ndef change(x):\n    if (x==3):\n        x=1\n    return x\n\nnormal_df['target'] = normal_df['num_normal_annotations'].apply(lambda x: change(x))\n\nnormal_df['img_path'] = normal_df.image_id.map(train_img_id_df.set_index('image_id')['img_path'])\nnormal_df.head()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T03:11:21.317444Z","iopub.execute_input":"2024-12-30T03:11:21.317835Z","iopub.status.idle":"2024-12-30T03:11:21.354142Z","shell.execute_reply.started":"2024-12-30T03:11:21.317803Z","shell.execute_reply":"2024-12-30T03:11:21.352868Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"data = normal_df[['target','img_path']]\ndata.to_csv('classfication.csv', index=False)\ndata.head()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T03:11:54.550315Z","iopub.execute_input":"2024-12-30T03:11:54.550705Z","iopub.status.idle":"2024-12-30T03:11:54.620249Z","shell.execute_reply.started":"2024-12-30T03:11:54.550666Z","shell.execute_reply":"2024-12-30T03:11:54.618786Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"\n!nvidia-smi","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T02:42:02.239869Z","iopub.execute_input":"2024-12-30T02:42:02.240311Z","iopub.status.idle":"2024-12-30T02:42:03.495262Z","shell.execute_reply.started":"2024-12-30T02:42:02.240272Z","shell.execute_reply":"2024-12-30T02:42:03.49419Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"!pip install keras_efficientnets ","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T02:42:05.396913Z","iopub.execute_input":"2024-12-30T02:42:05.397279Z","iopub.status.idle":"2024-12-30T02:42:14.524866Z","shell.execute_reply.started":"2024-12-30T02:42:05.397247Z","shell.execute_reply":"2024-12-30T02:42:14.523478Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"!pip install Keras-Applications","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T02:42:25.452401Z","iopub.execute_input":"2024-12-30T02:42:25.452841Z","iopub.status.idle":"2024-12-30T02:42:32.162852Z","shell.execute_reply.started":"2024-12-30T02:42:25.452797Z","shell.execute_reply":"2024-12-30T02:42:32.161767Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"#importing other required libraries\nimport numpy as np\nimport pandas as pd\nfrom sklearn.utils.multiclass import unique_labels\nimport os\nimport matplotlib.pyplot as plt\nimport matplotlib.image as mpimg\nimport seaborn as sns\nimport itertools\nfrom tqdm import tqdm\nimport shutil\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.metrics import confusion_matrix\n\n#Onehot Encoding the labels.\nfrom sklearn.utils.multiclass import unique_labels\nimport albumentations\nfrom sklearn.metrics import classification_report, confusion_matrix\n\n\n\n# tensoflow models\nimport tensorflow as tf\nfrom keras import Sequential\nfrom tensorflow import keras\nfrom tensorflow.keras.preprocessing.image import ImageDataGenerator\nfrom tensorflow.keras.layers import Conv2D, MaxPooling2D\nfrom tensorflow.keras.layers import Dense, Dropout, Flatten, Activation\nfrom tensorflow.keras.models import Sequential\nfrom tensorflow.keras.callbacks import EarlyStopping, ReduceLROnPlateau, ModelCheckpoint\nfrom tensorflow.keras.optimizers import Adam\nfrom keras_efficientnets import EfficientNetB0\nfrom keras.utils import to_categorical","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T02:43:02.089223Z","iopub.execute_input":"2024-12-30T02:43:02.089647Z","iopub.status.idle":"2024-12-30T02:43:11.699043Z","shell.execute_reply.started":"2024-12-30T02:43:02.089585Z","shell.execute_reply":"2024-12-30T02:43:11.697928Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"os.makedirs(\"./self_cas_study\", exist_ok=True)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T02:43:11.701084Z","iopub.execute_input":"2024-12-30T02:43:11.701382Z","iopub.status.idle":"2024-12-30T02:43:11.711928Z","shell.execute_reply.started":"2024-12-30T02:43:11.701355Z","shell.execute_reply":"2024-12-30T02:43:11.710372Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"df_train = pd.read_csv('../input/vinbigdata-chest-xray-abnormalities-detection/train.csv') \nprint(df_train.shape)\ndf_train.head()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T02:43:40.803833Z","iopub.execute_input":"2024-12-30T02:43:40.804236Z","iopub.status.idle":"2024-12-30T02:43:40.932881Z","shell.execute_reply.started":"2024-12-30T02:43:40.804203Z","shell.execute_reply":"2024-12-30T02:43:40.931914Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Specify image size\nIMG_WIDTH = 224\nIMG_HEIGHT = 224\nCHANNELS = 3","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T02:44:27.298593Z","iopub.execute_input":"2024-12-30T02:44:27.299025Z","iopub.status.idle":"2024-12-30T02:44:27.303235Z","shell.execute_reply.started":"2024-12-30T02:44:27.298985Z","shell.execute_reply":"2024-12-30T02:44:27.30225Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"test_df = pd.read_csv('../input/vinbigdata-chest-xray-abnormalities-detection/sample_submission.csv')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T02:45:59.025314Z","iopub.execute_input":"2024-12-30T02:45:59.025715Z","iopub.status.idle":"2024-12-30T02:45:59.041386Z","shell.execute_reply.started":"2024-12-30T02:45:59.025677Z","shell.execute_reply":"2024-12-30T02:45:59.040244Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# add .png to images id\ndef append_ext(fn):\n  return fn+\".png\"\ntest_df[\"img_id\"]=test_df[\"image_id\"].apply(append_ext)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T02:46:04.489887Z","iopub.execute_input":"2024-12-30T02:46:04.490242Z","iopub.status.idle":"2024-12-30T02:46:04.497543Z","shell.execute_reply.started":"2024-12-30T02:46:04.490213Z","shell.execute_reply":"2024-12-30T02:46:04.496417Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def append_ext(fn):\n  return fn+\".png\"\ntest_df[\"img_id\"]=test_df[\"image_id\"].apply(append_ext)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T02:46:29.128435Z","iopub.execute_input":"2024-12-30T02:46:29.12883Z","iopub.status.idle":"2024-12-30T02:46:29.137217Z","shell.execute_reply.started":"2024-12-30T02:46:29.128798Z","shell.execute_reply":"2024-12-30T02:46:29.136101Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"test_df.head()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T02:46:49.612834Z","iopub.execute_input":"2024-12-30T02:46:49.613184Z","iopub.status.idle":"2024-12-30T02:46:49.623956Z","shell.execute_reply.started":"2024-12-30T02:46:49.613155Z","shell.execute_reply":"2024-12-30T02:46:49.62299Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"df = pd.read_csv('../working/classfication.csv') \nprint(df.shape)\ndf.head()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T03:13:19.297972Z","iopub.execute_input":"2024-12-30T03:13:19.298379Z","iopub.status.idle":"2024-12-30T03:13:19.337707Z","shell.execute_reply.started":"2024-12-30T03:13:19.298346Z","shell.execute_reply":"2024-12-30T03:13:19.336771Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"df['img_id']= df.img_path.str.split('/',expand=True)[4]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T03:13:23.116258Z","iopub.execute_input":"2024-12-30T03:13:23.116652Z","iopub.status.idle":"2024-12-30T03:13:23.173481Z","shell.execute_reply.started":"2024-12-30T03:13:23.1166Z","shell.execute_reply":"2024-12-30T03:13:23.172238Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# we are applying data image augmentation\ndatagen= ImageDataGenerator(rotation_range=20,\n                                   width_shift_range=0.2,\n                                   height_shift_range=0.2,\n                                   zoom_range=0.2,\n                                   featurewise_center=True,\n                                   featurewise_std_normalization=True,\n                                   brightness_range=[0.3,0.6],\n                                   horizontal_flip=True,\n                                   vertical_flip=False,\n                                   fill_mode=\"wrap\",rescale=1/255., validation_split=0.2)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T03:13:42.530605Z","iopub.execute_input":"2024-12-30T03:13:42.531059Z","iopub.status.idle":"2024-12-30T03:13:42.536878Z","shell.execute_reply.started":"2024-12-30T03:13:42.531012Z","shell.execute_reply":"2024-12-30T03:13:42.535693Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"train_generator = datagen.flow_from_dataframe(dataframe=df, directory='../input/vinbigdata/train',\n                                             x_col='img_id',\n                                             y_col='target',\n                                             class_mode='raw',\n                                             target_size=(IMG_WIDTH, IMG_HEIGHT),\n                                             batch_size=64,\n                                             subset='training',seed=42,shuffle=True)\nvalidation_generator = datagen.flow_from_dataframe(dataframe=df, directory='../input/vinbigdata/train',\n                                             x_col='img_id',\n                                             y_col='target',\n                                             class_mode='raw',\n                                             target_size=(IMG_WIDTH, IMG_HEIGHT),\n                                             batch_size=64,\n                                             subset='validation',seed=42,shuffle=False)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T03:13:50.776106Z","iopub.execute_input":"2024-12-30T03:13:50.776448Z","iopub.status.idle":"2024-12-30T03:13:50.871254Z","shell.execute_reply.started":"2024-12-30T03:13:50.77642Z","shell.execute_reply":"2024-12-30T03:13:50.869876Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"test_datagen = ImageDataGenerator(rescale=1/255)\ntest_generator=test_datagen.flow_from_dataframe(dataframe=test_df,directory=\"../input/vinbigdata/test\",\n                                           x_col=\"img_id\",\n                                           y_col=None,\n                                           batch_size=1,seed=42,shuffle=False,\n                                           target_size=(IMG_WIDTH, IMG_HEIGHT),\n                                           class_mode=None)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T03:13:56.773079Z","iopub.execute_input":"2024-12-30T03:13:56.773423Z","iopub.status.idle":"2024-12-30T03:13:56.80204Z","shell.execute_reply.started":"2024-12-30T03:13:56.773395Z","shell.execute_reply":"2024-12-30T03:13:56.801079Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"NUM_CLASSES = 2\n#Defining the model\nmodel= Sequential()\nbase_model = EfficientNetB0(include_top=False, weights=\"imagenet\", input_shape=(IMG_WIDTH, IMG_HEIGHT, 3),classes=NUM_CLASSES)\nmodel.add(base_model)\nmodel.add(Flatten())\nmodel.add(Dense(256,activation=('relu'))) \nmodel.add(Dropout(.3))\nmodel.add(Dense(128,activation=('relu')))\nmodel.add(Dropout(.2))\nmodel.add(Dense(2,activation=('softmax'))) \n\n#Model summary\nmodel.summary()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T03:14:01.394266Z","iopub.execute_input":"2024-12-30T03:14:01.394629Z","iopub.status.idle":"2024-12-30T03:14:05.668871Z","shell.execute_reply.started":"2024-12-30T03:14:01.394584Z","shell.execute_reply":"2024-12-30T03:14:05.667658Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"#Defining the parameters\nbatch_size= 100\n#learn_rate=.001\n#sgd=SGD(lr=learn_rate,momentum=.9,nesterov=False) \nmodel.compile(optimizer=\"adam\", loss=\"sparse_categorical_crossentropy\", metrics=[\"acc\"])","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T03:14:05.671219Z","iopub.execute_input":"2024-12-30T03:14:05.671494Z","iopub.status.idle":"2024-12-30T03:14:05.692785Z","shell.execute_reply.started":"2024-12-30T03:14:05.671467Z","shell.execute_reply":"2024-12-30T03:14:05.691588Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"train_steps = train_generator.n//train_generator.batch_size\nvalidation_steps = validation_generator.n//validation_generator.batch_size","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T03:14:06.329701Z","iopub.execute_input":"2024-12-30T03:14:06.330104Z","iopub.status.idle":"2024-12-30T03:14:06.334724Z","shell.execute_reply.started":"2024-12-30T03:14:06.330067Z","shell.execute_reply":"2024-12-30T03:14:06.333585Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"os.makedirs(\"self_cas_study/models\", exist_ok=True)\ntrain_steps = train_generator.n//train_generator.batch_size\nvalidation_steps = validation_generator.n//validation_generator.batch_size\n\n# check points\ncheckpointer = ModelCheckpoint('self_cas_study/models/best_model.h5',monitor='val_acc',verbose=1,save_best_only=True,save_weights_only=True)\n\nhistory = model.fit(train_generator,steps_per_epoch=train_steps,verbose =2,epochs=20,validation_data=validation_generator,validation_steps=validation_steps,callbacks = [checkpointer])","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def plot_hist(hist):\n  plt.plot(hist.history[\"acc\"])\n  plt.plot(hist.history[\"val_acc\"])\n  plt.title(\"model accuracy\")\n  plt.ylabel(\"accuracy\")\n  plt.xlabel(\"epoch\")\n  plt.legend([\"train\", \"validation\"], loc=\"upper left\")\n  plt.show()\nplot_hist(history)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"loss, acc = model.evaluate_generator(validation_generator,validation_steps)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print(\"Restored model, accuracy: {:5.2f}%\".format(100 * acc))\nprint(\"Restored model, log-loss {}:\".format(loss))","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"true_lables = validation_generator.labels\nbatch_size = 32\nnb_validation_samples = 3000\nsteps = nb_validation_samples / batch_size\npredictions = model.predict_generator(validation_generator, steps,verbose=1)\ny_pred = np.array([np.argmax(x) for x in predictions])","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"val_files = validation_generator.filenames\nval_compare = pd.DataFrame({\"Filename\":val_files,\"Abnormal\":predictions[:,0],\"Normal\":predictions[:,1],\"Ture_lab\":t","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# lets plot confusion matrix\ndef plot_confusion_matrix(test_y, predict_y):\n  C = confusion_matrix(test_y, predict_y)\n  A =(((C.T)/(C.sum(axis=1))).T)\n  #divid each element of the confusion matrix with the sum of elements in that column \n  B =(C/C.sum(axis=0))\n  labels = [0,1]\n  # representing A in heatmap format\n  print(\"-\"*20, \"Confusion matrix\", \"-\"*20)\n  plt.figure(figsize=(8,8))\n  sns.heatmap(C, annot=True, cmap=\"YlGnBu\", fmt=\".3f\", xticklabels=labels, yticklabels=labels)\n  plt.xlabel('Predicted Class')\n  plt.ylabel('Original Class')\n  plt.show()","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"plot_confusion_matrix(true_lables, y_pred)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Let see visual image how model is performing \ndef Human_evaluation(img_id):\n  dataset_dir = '../input/vinbigdata/train'\n  # Define the paths to the training and testing dicom folders respectively\n  ture_data = df_train[df_train['image_id'] ==img_id]\n  img_path = append_ext(img_id)\n  pre_data = val_compare[val_compare['Filename'] == img_path ]\n  ture_class = ture_data.class_name.unique().tolist() \n  print(\"Ture image class are {}\".format(ture_class))\n  print(\"Probability that a person with Abnormal lung condition: {:5.2f}%\".format(pre_data.Abnormal.iloc[0]*100))\n  print(\"Probability that a person with Normal lung condition: {:5.2f}%\".format(pre_data.Normal.iloc[0]*100)) \n  path = os.path.join(dataset_dir,img_path)\n  img = mpimg.imread(path)\n  plt.imshow(img,cmap=\"gray\")","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"Human_evaluation(\"0007d316f756b3fa0baea2ff514ce945\")","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"model.load_weights('./self_cas_study/models/best_model.h5')","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Let's predict on  test data \nfilenames = test_generator.filenames\npredict = model.predict_generator(test_generator,steps =len(filenames),verbose=1) ","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"results=pd.DataFrame({\"Filename\":filenames,\"Abnormal\":predict[:,0],\"Normal\":predict[:,1]}) \nresults.to_csv('predection.csv')","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def image_prediction(img_id):\n  dataset_dir = '../input/vinbigdata/test'\n  img_path = append_ext(img_id)\n  pre_data = results[results['Filename'] == img_path ]\n  print(\"Probability that a person with Abnormal lung condition: {:5.2f}%\".format(pre_data.Abnormal.iloc[0]*100))\n  print(\"Probability that a person with Normal lung condition: {:5.2f}%\".format(pre_data.Normal.iloc[0]*100)) \n  path = os.path.join(dataset_dir,img_path)\n  img = mpimg.imread(path)\n  plt.imshow(img,cmap=\"Spectral\")","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"image_prediction('00a2145de1886cb9eb88869c85d74080')","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null}]}