{"cells":[{"metadata":{},"cell_type":"markdown","source":"<h1>**Basic CNN Starter Code V0.2\n**</h1>\nIn this small notebook I have created a basic sketch of a very simple CNN network. This is just bare bones and is meant to be just a schematic example of how to get about to implementing category identification using CNN. I have used multiple functions from various notebooks and I will add references. If i have missed some, please comment below. I will gladly add them. \n\nv 0.2 == Data Loader function added + Added a graph of model performance stats\n\nCOMMENTS WELCOME. CHEERS :)"},{"metadata":{},"cell_type":"markdown","source":"data-processing references -\nhttps://www.kaggle.com/xhlulu/rsna-intracranial-simple-densenet-in-keras\n"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import numpy as np\nfrom os import listdir\nfrom os.path import isfile, join\nimport os\nimport pandas as pd\nimport pydicom\nimport tensorflow as tf\nfrom tensorflow.keras.preprocessing.image import ImageDataGenerator\nfrom tensorflow.keras.models import Sequential\nfrom tensorflow.keras.layers import Dense, Dropout, Activation, Flatten\nfrom tensorflow.keras.layers import Conv2D, MaxPooling2D\nfrom keras.utils import to_categorical\nfrom keras.callbacks import ModelCheckpoint\nfrom tqdm import tqdm\nfrom sklearn.utils import shuffle\nimport cv2\nimport pickle\nimport gc\nimport matplotlib.pyplot as plt\nimport keras\nfrom tensorflow.python.keras.utils.data_utils import Sequence\nfrom tensorflow.python.ops import array_ops\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Declaration of various global quantities"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"batch_size=32\nvalidation_ratio=0.1\nsample_size=2000\nepochs=3\nimg_size=512","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Reading in the data"},{"metadata":{"trusted":true},"cell_type":"code","source":"train = pd.read_csv('../input/rsna-intracranial-hemorrhage-detection/stage_1_train.csv')\ntrain.shape\nread_data = train.copy()\ntrain.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Some quality of life improvements"},{"metadata":{"trusted":true},"cell_type":"code","source":"train['filename'] = train['ID'].apply(lambda x: \"ID_\" + x.split('_')[1] + \".dcm\")\ntrain['type'] = train['ID'].apply(lambda x: x.split('_')[2])\ntrain.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now we combine all the similar data-points in a categorical way. This will enable us to easily extract the category vector."},{"metadata":{"trusted":true},"cell_type":"code","source":"train = train[['Label', 'filename', 'type']].drop_duplicates().pivot(\n    index='filename', columns='type', values='Label').reset_index()\ntrain.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We shuffle our data and take first <sample_size> data points."},{"metadata":{"trusted":true},"cell_type":"code","source":"train = shuffle(train)\ntrain_sample=train\ntrain_sample2=train_sample.reset_index(drop=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_sample2.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"len(train_sample2)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now that we gave our input data set we can extract the categories out"},{"metadata":{"trusted":true},"cell_type":"code","source":"yvals=pd.DataFrame(train_sample2,columns=['any','epidural','intraparenchymal','intraventricular','subarachnoid','subdural'])\nyvals.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Similarly we will extract the image names of the relevant samples out."},{"metadata":{"trusted":true},"cell_type":"code","source":"xhead=pd.DataFrame(train_sample2,columns=['filename'])\nxhead.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Some fucntions to extract the correct window-image. These have been adapted from https://www.kaggle.com/allunia/rsna-ih-detection-eda and references therin"},{"metadata":{"trusted":true},"cell_type":"code","source":"def window_image(img, window_center,window_width, intercept, slope, rescale=True):\n\n    img = (img*slope +intercept)\n    img_min = window_center - window_width//2\n    img_max = window_center + window_width//2\n    img[img<img_min] = img_min\n    img[img>img_max] = img_max\n    \n    if rescale:\n        # Extra rescaling to 0-1, not in the original notebook\n        img = (img - img_min) / (img_max - img_min)\n    \n    return img\n    \ndef get_first_of_dicom_field_as_int(x):\n    #get x[0] as in int is x is a 'pydicom.multival.MultiValue', otherwise get int(x)\n    if type(x) == pydicom.multival.MultiValue:\n        return int(x[0])\n    else:\n        return int(x)\n\ndef get_windowing(data):\n    dicom_fields = [data[('0028','1050')].value, #window center\n                    data[('0028','1051')].value, #window width\n                    data[('0028','1052')].value, #intercept\n                    data[('0028','1053')].value] #slope\n    return [get_first_of_dicom_field_as_int(x) for x in dicom_fields]","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Since our data-set is too big to fit memory we will construct a function to use the data from the disk. This implementation is based on code written by,\nhttps://stanford.edu/~shervine/blog/keras-how-to-generate-data-on-the-fly"},{"metadata":{"trusted":true},"cell_type":"code","source":"class DataGenerator(Sequence):\n    'Generates data for Keras'\n    def __init__(self, list_IDs_labels, batch_size=100, dim=(512,512)):\n        'Initialization'\n        self.dim = dim\n        self.batch_size = batch_size\n        self.list_IDs = list_IDs_labels\n        self.on_epoch_end()\n\n    def __len__(self):\n        'Denotes the number of batches per epoch'\n        return int(np.floor(len(self.list_IDs) / self.batch_size))\n\n    def __getitem__(self, index):\n        'Generate one batch of data'\n        # Generate indexes of the batch\n        indexes = self.indexes[index*self.batch_size:(index+1)*self.batch_size]\n\n        # Find list of IDs\n        list_IDs_temp = [self.list_IDs['filename'][k] for k in indexes]\n        list_label_temp=[[int(self.list_IDs['any'][i]),int(self.list_IDs['epidural'][i]),int(self.list_IDs['intraparenchymal'][i]),int(self.list_IDs['intraventricular'][i]),int(self.list_IDs['subarachnoid'][i]),int(self.list_IDs['subdural'][i])] for i in indexes]\n        # Generate data\n        X, y = self.__data_generation(list_IDs_temp,list_label_temp)\n\n        return X, y\n\n    def on_epoch_end(self):\n        'Updates indexes after each epoch'\n        self.indexes = np.arange(len(self.list_IDs))\n\n    def __data_generation(self, list_IDs_temp,list_label_temp):\n        'Generates data containing batch_size samples' # X : (n_samples, *dim, n_channels)\n        # Initialization\n        X = []\n        y = []\n\n        # Generate data\n        for i, ID in enumerate(list_IDs_temp):\n            # Store sample\n            ds=pydicom.dcmread('/kaggle/input/rsna-intracranial-hemorrhage-detection/stage_1_train_images/' +list_IDs_temp[i] )\n            temp=ds.pixel_array\n            window_center , window_width, intercept, slope = get_windowing(ds)\n            img = window_image(temp, 50, 100, intercept, slope)\n            resized = cv2.resize(img, (200, 200))\n            X.append(resized)       \n        X=np.array(X).reshape(-1,200,200,1)\n        y_train=np.asarray(list_label_temp) \n        return X,y_train","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"For now we will train on a sample of 100000(16% of actual data) and use 20000 as validation"},{"metadata":{"trusted":true},"cell_type":"code","source":"valid=train_sample2[100000:120000]\nvalid=valid.reset_index(drop=True)\nvalid.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"traingen=DataGenerator(train_sample2[0:100000])\nvalidgen=DataGenerator(valid)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Some old code that we wont need anymore"},{"metadata":{"trusted":true},"cell_type":"code","source":"'''\nX=[]\nfor i in tqdm(xhead.index):\n    ds=pydicom.dcmread('/kaggle/input/rsna-intracranial-hemorrhage-detection/stage_1_train_images/' +xhead['filename'][i] )\n    temp=ds.pixel_array\n    window_center , window_width, intercept, slope = get_windowing(ds)\n    img = window_image(temp, 50, 100, intercept, slope)\n    resized = cv2.resize(img, (100, 100))\n    X.append(resized)\npickle_out=open(\"/kaggle/X.pickle\",\"wb\")\npickle.dump(X,pickle_out)\npickle_out.close()\nX=pickle.load(open(\"/kaggle/X.pickle\",\"rb\"))\nX=np.array(X).reshape(-1,100,100,1)\ny_train=[]\nfor i in tqdm(yvals.index):\n    y_train.append([int(yvals['any'][i]),int(yvals['epidural'][i]),int(yvals['intraparenchymal'][i]),int(yvals['intraventricular'][i]),int(yvals['subarachnoid'][i]),int(yvals['subdural'][i])])\ny_train=np.asarray(y_train) \n'''","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"For an unbalanced data-set it is good to define a custom loss function like the focal loss. We will borrowed the code for this from https://www.kaggle.com/xhlulu/rsna-intracranial-simple-densenet-in-keras"},{"metadata":{"trusted":true},"cell_type":"code","source":"def focal_loss(prediction_tensor, target_tensor, weights=None, alpha=0.25, gamma=2):\n    r\"\"\"Compute focal loss for predictions.\n        Multi-labels Focal loss formula:\n            FL = -alpha * (z-p)^gamma * log(p) -(1-alpha) * p^gamma * log(1-p)\n                 ,which alpha = 0.25, gamma = 2, p = sigmoid(x), z = target_tensor.\n    Args:\n     prediction_tensor: A float tensor of shape [batch_size, num_anchors,\n        num_classes] representing the predicted logits for each class\n     target_tensor: A float tensor of shape [batch_size, num_anchors,\n        num_classes] representing one-hot encoded classification targets\n     weights: A float tensor of shape [batch_size, num_anchors]\n     alpha: A scalar tensor for focal loss alpha hyper-parameter\n     gamma: A scalar tensor for focal loss gamma hyper-parameter\n    Returns:\n        loss: A (scalar) tensor representing the value of the loss function\n    \"\"\"\n    sigmoid_p = tf.nn.sigmoid(prediction_tensor)\n    zeros = array_ops.zeros_like(sigmoid_p, dtype=sigmoid_p.dtype)\n    \n    # For poitive prediction, only need consider front part loss, back part is 0;\n    # target_tensor > zeros <=> z=1, so poitive coefficient = z - p.\n    pos_p_sub = array_ops.where(target_tensor > zeros, target_tensor - sigmoid_p, zeros)\n    \n    # For negative prediction, only need consider back part loss, front part is 0;\n    # target_tensor > zeros <=> z=1, so negative coefficient = 0.\n    neg_p_sub = array_ops.where(target_tensor > zeros, zeros, sigmoid_p)\n    per_entry_cross_ent = - alpha * (pos_p_sub ** gamma) * tf.log(tf.clip_by_value(sigmoid_p, 1e-8, 1.0)) \\\n                          - (1 - alpha) * (neg_p_sub ** gamma) * tf.log(tf.clip_by_value(1.0 - sigmoid_p, 1e-8, 1.0))\n    return tf.reduce_sum(per_entry_cross_ent)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now that we have both Input=X , Category=y_trest we can train a basic barebones model.\nHere we have a model which is like ==   CNN -> CNN->  Flatten --> Dense ---> Dense(output)"},{"metadata":{"trusted":true},"cell_type":"code","source":"model = Sequential()\n\nmodel.add(Conv2D(32, (3, 3), input_shape=(200, 200,1)))\nmodel.add(Activation('relu'))\nmodel.add(MaxPooling2D(pool_size=(2, 2)))\n\nmodel.add(Conv2D(32,(3,3)))\nmodel.add(Activation('relu'))\nmodel.add(MaxPooling2D(pool_size=(5,5)))\n\nmodel.add(Flatten())  \n\n# model.add(Dense(100))\n# model.add(Activation('relu'))\n\nmodel.add(Dense(50))\nmodel.add(Activation('relu'))\n\nmodel.add(Dense(6))\nmodel.add(Activation('sigmoid'))\n\n# model.compile(loss='categorical_crossentropy',\n#               optimizer='adam',\n#               metrics=['accuracy'])\nmodel.compile(loss=focal_loss,\n               optimizer='adam',\n               metrics=['accuracy'])\n\n#model.fit(X,y_train,batch_size=32,epochs=6,validation_split=0.5)\nhistory=model.fit_generator(generator=traingen,validation_data=validgen,use_multiprocessing=True,\n                    workers=-1,epochs=2)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#history = model.fit(X, y_train, validation_split=0.25, epochs=20, batch_size=32, verbose=1)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now that our result are looking half-decent we can improve the model further. To study any possible improvements, we need to first check out the statistics of our model and epochs. We will plot the accuracy and loss values."},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.plot(history.history['acc'])\nplt.plot(history.history['val_acc'])\nplt.title('Model accuracy')\nplt.ylabel('Accuracy')\nplt.xlabel('Epoch')\nplt.legend(['Train', 'Test'], loc='upper left')\nplt.show()\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.plot(history.history['loss'])\nplt.plot(history.history['val_loss'])\nplt.title('Model loss')\nplt.ylabel('Loss')\nplt.xlabel('Epoch')\nplt.legend(['Train', 'Test'], loc='upper left')\nplt.show()","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}