{"cells":[{"metadata":{},"cell_type":"markdown","source":"# This notebook uses the new Fastai 2 medical library to train a classification model\n\nSome remarks:\n\n* This notebook is for training (and learning) only and it is not ready for submission. \n* I tried to make a quick overview of the DataBlock api. For more information you can take a look at this post https://towardsdatascience.com/how-to-create-a-datablock-for-multispectral-satellite-image-segmentation-with-the-fastai-v2-bc5e82f4eb5 or the Fastai documentation. \n* This version loads a pre-trained model and train with 500k images\n* The problem with this dataset is that is is unbalanced\n* If something is not clear enough or wrong, or you just have a suggestion on how to improve the notebook, please let us all know in the comments. \n\nThanks,"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"# Importing our dependencies\nimport torch\nimport fastai\nfrom fastai.medical import *\nfrom fastai.medical.imaging import *\nfrom fastai.torch_core import *\nimport matplotlib.pyplot as plt\nimport pandas as pd\nfrom fastai.vision.all import *\nfrom pathlib import Path\nimport pydicom\n\n!conda install -c conda-forge gdcm -y\nimport gdcm","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Taking a look at data using the PILdicom class\nThis section is just to check if we can open the images correctly"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Just for visualization purpose, we will grab the files contained in just 1 study (each study has many images/slices)\n# For that, we will use Fastai 2 get_dicom_files, that just maps all dcm files within recursive subdirs\nsample_files = get_dicom_files('../input/rsna-str-pulmonary-embolism-detection/train/0003b3d648eb')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Let's grab the first file of this study and display its metadata\ndicom = dcmread(sample_files[0])\ndicom","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Just for information: another way to open the dcm file is just using:<br> `dicom = files[idx].dcmread()`<br>\nAs we used the Fastai 2 function `get_dicom_files` to grab the files, it added automatically the dcmread() method to our path items."},{"metadata":{"trusted":true},"cell_type":"code","source":"# using a snippet from fastai medical tutorial, we will display the images with different scales\nscales = False, True, dicom_windows.brain, dicom_windows.subdural\ntitles = 'raw','normalized','brain windowed','subdural windowed'\nfor s,a,t in zip(scales, subplots(2,2,imsize=4)[1].flat, titles):\n    dicom.show(scale=s, ax=a, title=t)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"dicom.show(cmap=plt.cm.gist_ncar, figsize=(6,6))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Subsampling the training set\nThe training set is unbalanced with much more negative results and that interferes with our training. It's much easier to just guess all negative than trying to predict anything. \n"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Initially, we will create our Pandas Dataframes with the CSV diles.\n# The train dataframe contains all the information to get to the images, so it will be passed \n# as the source of our dataset and the datablock will be in charge of transforming it into \n# inputs and targets (x, y)\n\ntrain = pd.read_csv('../input/rsna-str-pulmonary-embolism-detection/train.csv', low_memory=False)\ntest = pd.read_csv('../input/rsna-str-pulmonary-embolism-detection/test.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# We will separate in train_pos and train_neg. Afterwards, we will grab 100k images from each dataframe and join then to do our training\nnegatives = train['negative_exam_for_pe'] == 1\ntrain_neg = train[negatives]\ntrain_pos = train[~negatives]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"balanced = pd.concat([train_neg[:250000], train_pos[:250000]], axis=0)\nbalanced","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Preparing the DataBlock\nThe first step in fastai is to create the DataBlock, that holds all the necessary transformations to get the item, put it in the correct format, and so on...\n\nAfter the DataBlock is prepared, we will easily create datasets and dataloaders"},{"metadata":{},"cell_type":"markdown","source":"Each row of the source dataframe will be passed to each block of the DataBlock, so we need to create functions that manipulate each row and transform it into inputs (X) or targets (Y).<br>\nFor that, we will define `get_x` and `get_y` functions.\n\nKeep in mind that the `get_x` will not apply the necessary transformations. Instead it will be responsible just for transforming the rows into paths to the files. \n\nThe `get_y` will return the necessary columns. As it is a multicategory classification, where they occur at the same time, our targets cannot be just a single number. We can return a list of occurrences and the DataBlock will take care of transforming it into hot-encoded. The problem of this approach is that the Categorize function takes too long, because it has to look at all possible targets to create it's internal vocab. <br>\nTo overcome this situation, I created the `get_encoded_y` that returns it as hot-encoded and the vocab are the columns that we want to extract. This way, we can bypass the Categorize function using encoded=True in the MultiCategoryBlock.\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"vocab = ['negative_exam_for_pe', 'pe_present_on_image', 'rv_lv_ratio_gte_1', 'rv_lv_ratio_lt_1',\n         'leftsided_pe', 'chronic_pe', 'rightsided_pe', 'acute_and_chronic_pe',\n         'central_pe', 'indeterminate']\nvocab.sort()\n\ndef get_x(row):\n    base_path = Path('../input/rsna-str-pulmonary-embolism-detection/train')\n    file_path = f\"{row['StudyInstanceUID']}/{row['SeriesInstanceUID']}/{row['SOPInstanceUID']}.dcm\"\n    return base_path/file_path\n\n# def get_y(row):\n#     labels = row[vocab]\n    \n#     return list(labels.index[labels==1])\n\ndef get_encoded_y(row):\n    return row[vocab].values.squeeze().astype('long')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# we will test our functions by passing an arbitrary row\nr = train.iloc[3]\nget_x(r), get_encoded_y(r)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The DataBlock will be then created using the `get_x` and `get_y` functions we just created. <br>\nThe transformation that will be applied to the inputs (Xs) is ImageBlock(cls=PILDicom), that gets the file path and create a PILDicom instance.<br>\nFor the targets (Ys) we will pass our hot-encoded to the MultiCategoryBlock block, passing encoded=True and the vocab.\n\nJust after the dblock creation, we will create a dataset to test it.\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"dblock = DataBlock(#blocks=(ImageBlock(cls=PILDicom), MultiCategoryBlock(encoded=True, vocab=vocab)),\n                   blocks=(TransformBlock([PILDicom.create, ToTensor]), MultiCategoryBlock(encoded=True, vocab=vocab)),\n                   get_x=get_x,\n                   get_y=get_encoded_y,\n                  )\ndsets = dblock.datasets(balanced, verbose=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# If we index the dataset, we get x and y as return\ndsets[150000]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# to check the sanity of our dblock, we could also call the `.summary()` function\n# dblock.summary(train.iloc[:100])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Creating a dataloader\nAs noted before, we will use just a subset of the training set for learning purposes. <br>\nOur dataloader will be created with 20000 images and batch size of 32."},{"metadata":{"trusted":true},"cell_type":"code","source":"# dls = dblock.dataloaders(train.iloc[:20000], bs=16, num_workers=0)\n# To check our dataloader we can either create an item or a full batch\ndls = dsets.dataloaders(bs=64, num_workers=0)\ndls.create_item(1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"dls.show_batch()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Great!!! It seems to be working. Let's pass to the training phase."},{"metadata":{},"cell_type":"markdown","source":"## Training a simple model\nFastai provides a wide range of CNN models to be used (https://docs.fast.ai/vision.models.xresnet)\n\nWe will use a simple resnet18, pretrained with ImageNet (don't know if it makes sense) that is fast to train, just to see if everything is working"},{"metadata":{"trusted":true},"cell_type":"code","source":"# Will create a multicategorical accuracy\ndef accuracy_multi(inp, targ, thresh=0.5, sigmoid=True):\n    \"Compute accuracy when `inp` and `targ` are the same size.\"\n    if sigmoid: inp = inp.sigmoid()\n    return ((inp>thresh)==targ.bool()).float().mean()\n\n# accuracy_multi(y, activs, thresh=0.5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn = cnn_learner(dls, resnet18, n_in=1, metrics=accuracy_multi)\nlearn.model_dir = '.'\n\ntry:\n    learn.load('../input/fastai2-medical-simple-training/resnet18-v3')\n    print('Model Loaded Successfully')\nexcept:\n    print('Could not load model. Content of added data:')\n\n    !ls ../input/fastai2-medical-simple-training/\n    print('Content of working directory')\n    !ls ../working","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#testing one pass through the model\n# x,y = to_cpu(dls.train.one_batch())\n# activs = learn.model(x)\n# activs.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# apply the new metrics and look for the best learning rate\n# learn.metrics=accuracy_multi\n# learn.lr_find()\n\n# I noticed a problem when training with more data. \n# I also noticed that some dcm cannot be opened correctly, have to check further.\n# It seems to be a good idea to iterate through all dcms and check if thay are \n# opening correctly and that they can be cast to PILDicom","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.fine_tune(1, base_lr=2e-2, freeze_epochs=1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.model_dir = '.'\nlearn.save('./resnet18-v4')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"item = dsets.valid[1500]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"learn.predict(item[0])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"item[1]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# interp = ClassificationInterpretation.from_learner(learn)\n# interp.plot_confusion_matrix(figsize=(12,12), dpi=60)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# interp.plot_top_losses(6)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"That's all for now!"}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}