{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"This competition provides an exciting and challenging task of doing multi-label classification on a dataset with well over half a million images. There are multiple very nice notebooks which perform only 2 or 3 epochs with all the training data. In this notebook I will try out and see what the effect is of using more epochs but less steps per epoch. By averaging the predictions made during the last few epochs we should be able to achieve a nice LB score. This also should provide some alternative ways to experiment for the Kagglers that don't have the adequate computing resources available and are dependent on Kaggle Kernels.\n\nAs model I will be using the EfficientNet B2 model. It should be able to provide highly accurate predictions while still being able to run within the kernel limits. With 9 hours max time for a GPU kernel you have to make some trade-offs ;-)\n\nI hope this kernel will be usefull and may'be will provide you with some new and alternative ideas to try out. If you like it..then please upvote it ;-)\nAny feedback or remarks are appreciated.\n\nLets start by importing all the necessary modules.\n\nNote!! This kernel is now updated for Stage2 Training and Test data..altough with less epochs because of the increase in train and test data.","metadata":{}},{"cell_type":"code","source":"!pip -q install mlflow","metadata":{"execution":{"iopub.status.busy":"2021-09-20T13:41:36.752043Z","iopub.execute_input":"2021-09-20T13:41:36.752599Z","iopub.status.idle":"2021-09-20T13:41:38.736703Z","shell.execute_reply.started":"2021-09-20T13:41:36.752553Z","shell.execute_reply":"2021-09-20T13:41:38.735865Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import mlflow.tensorflow\nmlflow.tensorflow.autolog()","metadata":{"execution":{"iopub.status.busy":"2021-09-20T13:41:38.740274Z","iopub.execute_input":"2021-09-20T13:41:38.7405Z","iopub.status.idle":"2021-09-20T13:41:38.746393Z","shell.execute_reply.started":"2021-09-20T13:41:38.740474Z","shell.execute_reply":"2021-09-20T13:41:38.74563Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport pydicom\nimport os\nimport collections\nimport sys\nimport glob\nimport random\nimport cv2\nimport tensorflow as tf\nimport multiprocessing\n\nfrom math import ceil, floor\nfrom copy import deepcopy\nfrom tqdm import tqdm_notebook as tqdm\nfrom imgaug import augmenters as iaa\n\nimport tensorflow.keras\nimport tensorflow.keras.backend as K\nfrom tensorflow.keras.callbacks import Callback, ModelCheckpoint\nfrom tensorflow.keras.layers import Dense, Flatten, Dropout\nfrom tensorflow.keras.models import Model, load_model\nfrom tensorflow.keras.utils import Sequence\nfrom tensorflow.keras.losses import binary_crossentropy\nfrom tensorflow.keras.optimizers import Adam\n\ndef calculating_class_weights(y_true):\n    from sklearn.utils.class_weight import compute_class_weight\n    number_dim = np.shape(y_true)[1]\n    weights = np.empty([number_dim, 2])\n    for i in range(number_dim):\n        weights[i] = compute_class_weight('balanced', [0.,1.], y_true[:, i])\n    return weights","metadata":{"execution":{"iopub.status.busy":"2021-09-20T13:41:38.748057Z","iopub.execute_input":"2021-09-20T13:41:38.748435Z","iopub.status.idle":"2021-09-20T13:41:38.761531Z","shell.execute_reply.started":"2021-09-20T13:41:38.748401Z","shell.execute_reply":"2021-09-20T13:41:38.760834Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Install and import the efficientnet and iterative-stratification packages from the internet. The iterative-stratification package provides a very nice implementation of multi-label stratification. I've used it in a few competitions now with good results. There are offcourse more packages that provide implementations for it.","metadata":{}},{"cell_type":"code","source":"# Install Modules from internet\n!pip install efficientnet\n!pip install iterative-stratification","metadata":{"execution":{"iopub.status.busy":"2021-09-20T13:41:38.764123Z","iopub.execute_input":"2021-09-20T13:41:38.764339Z","iopub.status.idle":"2021-09-20T13:41:43.448232Z","shell.execute_reply.started":"2021-09-20T13:41:38.764311Z","shell.execute_reply":"2021-09-20T13:41:43.447008Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Import Custom Modules\nimport efficientnet.tfkeras as efn \nfrom iterstrat.ml_stratifiers import MultilabelStratifiedShuffleSplit","metadata":{"execution":{"iopub.status.busy":"2021-09-20T13:41:43.450035Z","iopub.execute_input":"2021-09-20T13:41:43.450655Z","iopub.status.idle":"2021-09-20T13:41:43.456057Z","shell.execute_reply.started":"2021-09-20T13:41:43.450608Z","shell.execute_reply":"2021-09-20T13:41:43.455185Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Next we will set the random_state, some constants and folders that will be used later on. I've specified a rather small test size as I want to maximize the training time available and minimize the time used for validation. I'am not using methods like early stopping...when the kernel time limit is approaching we could still increase the results on the LB if we were allowed to continue.","metadata":{}},{"cell_type":"code","source":"# Seed\nSEED = 12345\nnp.random.seed(SEED)\n# tf.set_random_seed(SEED)\n\n# Constants\nTEST_SIZE = 0.1\nHEIGHT = 256\nWIDTH = 256\nCHANNELS = 3\nTRAIN_BATCH_SIZE = 32\nVALID_BATCH_SIZE = 64\nSHAPE = (HEIGHT, WIDTH, CHANNELS)\n\n# Folders\nDATA_DIR = '/kaggle/input/rsna-intracranial-hemorrhage-detection/rsna-intracranial-hemorrhage-detection/'\nTEST_IMAGES_DIR = DATA_DIR + 'stage_2_test/'\nTRAIN_IMAGES_DIR = DATA_DIR + 'stage_2_train/'","metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","execution":{"iopub.status.busy":"2021-09-20T13:41:43.457867Z","iopub.execute_input":"2021-09-20T13:41:43.458208Z","iopub.status.idle":"2021-09-20T13:41:43.466732Z","shell.execute_reply.started":"2021-09-20T13:41:43.458165Z","shell.execute_reply":"2021-09-20T13:41:43.465997Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Next the code for the DICOM windowing and the Data Generators. After seeing the effect of different versions of windowing as presented in this very nice [kernel](https://www.kaggle.com/akensert/inceptionv3-prev-resnet50-keras-baseline-model) I decided to also update my kernel with it. Lets see what the effect will be.","metadata":{}},{"cell_type":"code","source":"def correct_dcm(dcm):\n    x = dcm.pixel_array + 1000\n    px_mode = 4096\n    x[x>=px_mode] = x[x>=px_mode] - px_mode\n    dcm.PixelData = x.tobytes()\n    dcm.RescaleIntercept = -1000\n\ndef window_image(dcm, window_center, window_width):    \n    if (dcm.BitsStored == 12) and (dcm.PixelRepresentation == 0) and (int(dcm.RescaleIntercept) > -100):\n        correct_dcm(dcm)\n    img = dcm.pixel_array * dcm.RescaleSlope + dcm.RescaleIntercept\n    \n    # Resize\n    img = cv2.resize(img, SHAPE[:2], interpolation = cv2.INTER_LINEAR)\n   \n    img_min = window_center - window_width // 2\n    img_max = window_center + window_width // 2\n    img = np.clip(img, img_min, img_max)\n    return img\n\ndef bsb_window(dcm):\n    brain_img = window_image(dcm, 40, 80)\n    subdural_img = window_image(dcm, 80, 200)\n    soft_img = window_image(dcm, 40, 380)\n    \n    brain_img = (brain_img - 0) / 80\n    subdural_img = (subdural_img - (-20)) / 200\n    soft_img = (soft_img - (-150)) / 380\n    bsb_img = np.array([brain_img, subdural_img, soft_img]).transpose(1,2,0)\n    return bsb_img\n\ndef _read(path, SHAPE):\n    dcm = pydicom.dcmread(path)\n    try:\n        img = bsb_window(dcm)\n    except:\n        img = np.zeros(SHAPE)\n    return img","metadata":{"execution":{"iopub.status.busy":"2021-09-20T13:41:43.469456Z","iopub.execute_input":"2021-09-20T13:41:43.469642Z","iopub.status.idle":"2021-09-20T13:41:43.483627Z","shell.execute_reply.started":"2021-09-20T13:41:43.469621Z","shell.execute_reply":"2021-09-20T13:41:43.482812Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"I'll specify some light image augmentation. Some horizontal and vertical flipping and some cropping. I haven't yet tried out more augmentation but will do so in future versions of the kernel. Also the code for Data Generators for train and test data.","metadata":{}},{"cell_type":"code","source":"# Image Augmentation\nsometimes = lambda aug: iaa.Sometimes(0.25, aug)\naugmentation = iaa.Sequential([ iaa.Fliplr(0.25),\n                                iaa.Flipud(0.10),\n                                sometimes(iaa.Crop(px=(0, 25), keep_size = True, sample_independently = False))   \n                            ], random_order = True)       \n        \n# Generators\nclass TrainDataGenerator(tensorflow.keras.utils.Sequence):\n    def __init__(self, dataset, labels, batch_size = 16, img_size = SHAPE, img_dir = TRAIN_IMAGES_DIR, augment = False, *args, **kwargs):\n        self.dataset = dataset\n        self.ids = dataset.index\n        self.labels = labels\n        self.batch_size = batch_size\n        self.img_size = img_size\n        self.img_dir = img_dir\n        self.augment = augment\n        self.on_epoch_end()\n\n    def __len__(self):\n        return int(ceil(len(self.ids) / self.batch_size))\n\n    def __getitem__(self, index):\n        indices = self.indices[index*self.batch_size:(index+1)*self.batch_size]\n        X, Y = self.__data_generation(indices)\n        return X, Y\n\n    def augmentor(self, image):\n        augment_img = augmentation        \n        image_aug = augment_img.augment_image(image)\n        return image_aug\n\n    def on_epoch_end(self):\n        self.indices = np.arange(len(self.ids))\n        np.random.shuffle(self.indices)\n\n    def __data_generation(self, indices):\n        X = np.empty((self.batch_size, *self.img_size))\n        Y = np.empty((self.batch_size, 1), dtype=np.float32)\n        \n        for i, index in enumerate(indices):\n            ID = self.ids[index]\n            image = _read(self.img_dir+ID+\".dcm\", self.img_size)\n            if self.augment:\n                X[i,] = self.augmentor(image)\n            else:\n                X[i,] = image\n            Y[i,] = self.labels.iloc[index].values        \n        return X, Y\n    \nclass TestDataGenerator(tensorflow.keras.utils.Sequence):\n    def __init__(self, dataset, labels, batch_size = 16, img_size = SHAPE, img_dir = TEST_IMAGES_DIR, *args, **kwargs):\n        self.dataset = dataset\n        self.ids = dataset.index\n        self.labels = labels\n        self.batch_size = batch_size\n        self.img_size = img_size\n        self.img_dir = img_dir\n        self.on_epoch_end()\n\n    def __len__(self):\n        return int(ceil(len(self.ids) / self.batch_size))\n\n    def __getitem__(self, index):\n        indices = self.indices[index*self.batch_size:(index+1)*self.batch_size]\n        X = self.__data_generation(indices)\n        return X\n\n    def on_epoch_end(self):\n        self.indices = np.arange(len(self.ids))\n    \n    def __data_generation(self, indices):\n        X = np.empty((self.batch_size, *self.img_size))\n        \n        for i, index in enumerate(indices):\n            ID = self.ids[index]\n            image = _read(self.img_dir+ID+\".dcm\", self.img_size)\n            X[i,] = image              \n        return X","metadata":{"execution":{"iopub.status.busy":"2021-09-20T13:41:43.485246Z","iopub.execute_input":"2021-09-20T13:41:43.485777Z","iopub.status.idle":"2021-09-20T13:41:43.509238Z","shell.execute_reply.started":"2021-09-20T13:41:43.485741Z","shell.execute_reply":"2021-09-20T13:41:43.508295Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Import the training and test datasets.","metadata":{"trusted":true}},{"cell_type":"code","source":"def read_testset(filename = DATA_DIR + \"stage_2_sample_submission.csv\"):\n    df = pd.read_csv(filename)\n    df[\"Image\"] = df[\"ID\"].str.slice(stop=12)\n    df[\"Diagnosis\"] = df[\"ID\"].str.slice(start=13)\n    df = df.loc[:, [\"Label\", \"Diagnosis\", \"Image\"]]\n    df = df.set_index(['Image', 'Diagnosis']).unstack(level=-1)\n    return df\n\ndef read_trainset(filename = DATA_DIR + \"stage_2_train.csv\"):\n    df = pd.read_csv(filename)\n    df[\"Image\"] = df[\"ID\"].str.slice(stop=12)\n    df[\"Diagnosis\"] = df[\"ID\"].str.slice(start=13)\n    duplicates_to_remove = [56346, 56347, 56348, 56349,\n                            56350, 56351, 1171830, 1171831,\n                            1171832, 1171833, 1171834, 1171835,\n                            3705312, 3705313, 3705314, 3705315,\n                            3705316, 3705317, 3842478, 3842479,\n                            3842480, 3842481, 3842482, 3842483 ]\n    df = df.drop(index = duplicates_to_remove)\n    df = df.reset_index(drop = True)    \n    df = df.loc[:, [\"Label\", \"Diagnosis\", \"Image\"]]\n    df = df.set_index(['Image', 'Diagnosis']).unstack(level=-1)\n    return df\n\n# Read Train and Test Datasets\ntest_df = read_testset()\ntrain_df = read_trainset()","metadata":{"execution":{"iopub.status.busy":"2021-09-20T13:41:43.510652Z","iopub.execute_input":"2021-09-20T13:41:43.510944Z","iopub.status.idle":"2021-09-20T13:41:43.875311Z","shell.execute_reply.started":"2021-09-20T13:41:43.510911Z","shell.execute_reply":"2021-09-20T13:41:43.871473Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df = train_df.iloc[:]\ntrain_df","metadata":{"execution":{"iopub.status.busy":"2021-09-20T13:41:43.876121Z","iopub.status.idle":"2021-09-20T13:41:43.876437Z","shell.execute_reply.started":"2021-09-20T13:41:43.876282Z","shell.execute_reply":"2021-09-20T13:41:43.876301Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"The training data contains some class inbalance. Multiple kernels explored the use of undersampling..so let's try the opposite and oversample the minority class 'epidural' one additional time.","metadata":{}},{"cell_type":"code","source":"# Oversampling\nepidural_df = train_df[train_df.Label['epidural'] == 1]\ntrain_oversample_df = pd.concat([train_df, epidural_df])\ntrain_df = train_oversample_df\n\n# Summary\nprint('Train Shape: {}'.format(train_df.shape))\nprint('Test Shape: {}'.format(test_df.shape))","metadata":{"execution":{"iopub.status.busy":"2021-09-20T13:41:43.878092Z","iopub.status.idle":"2021-09-20T13:41:43.879042Z","shell.execute_reply.started":"2021-09-20T13:41:43.8787Z","shell.execute_reply":"2021-09-20T13:41:43.878724Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"weights = calculating_class_weights(train_df.values)\nweights","metadata":{"execution":{"iopub.status.busy":"2021-09-20T13:41:43.880102Z","iopub.status.idle":"2021-09-20T13:41:43.880873Z","shell.execute_reply.started":"2021-09-20T13:41:43.880617Z","shell.execute_reply":"2021-09-20T13:41:43.880641Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Some methods for predictions on the test data, a callback method and a method to create the EfficientNet B2 model. For the EfficientNet we use the pretrained imagenet weights. Also a Dropout layer is added with a small value to prevent some overfitting. ","metadata":{}},{"cell_type":"code","source":"def predictions(test_df, model):    \n    test_preds = model.predict_generator(TestDataGenerator(test_df, None, 8, SHAPE, TEST_IMAGES_DIR), verbose = 1)\n    return test_preds[:test_df.iloc[range(test_df.shape[0])].shape[0]]\n\ndef ModelCheckpointFull(model_name):\n    return ModelCheckpoint(model_name, \n                            monitor = 'val_AUC_full', \n                            verbose = 1, \n                            save_best_only = True, \n                            save_weights_only = True, \n                            mode = 'max', \n                            period = 1)\n\n# Create Model\ndef create_model():\n    K.clear_session()\n    \n    base_model =  efn.EfficientNetB2(weights = 'imagenet', include_top = False, pooling = 'avg', input_shape = SHAPE)\n    x = base_model.output\n    x = Dropout(0.15)(x)\n    y_pred = Dense(1, activation = 'sigmoid')(x)\n\n    return Model(inputs = base_model.input, outputs = y_pred)","metadata":{"execution":{"iopub.status.busy":"2021-09-20T13:41:43.882213Z","iopub.status.idle":"2021-09-20T13:41:43.883188Z","shell.execute_reply.started":"2021-09-20T13:41:43.882809Z","shell.execute_reply":"2021-09-20T13:41:43.882832Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Next we setup the multi label stratification. I've specified multiple splits but only using the first one for train data and validation data. Optionally you can also loop through the different splits and use a different train and validation set for each epoch. ","metadata":{}},{"cell_type":"code","source":"# Submission Placeholder\nsubmission_predictions = []\n\n# Multi Label Stratified Split stuff...\nmsss = MultilabelStratifiedShuffleSplit(n_splits = 10, test_size = TEST_SIZE, random_state = SEED)\nX = train_df.index\nY = train_df.Label.values","metadata":{"execution":{"iopub.status.busy":"2021-09-20T13:41:43.884225Z","iopub.status.idle":"2021-09-20T13:41:43.88524Z","shell.execute_reply.started":"2021-09-20T13:41:43.884821Z","shell.execute_reply":"2021-09-20T13:41:43.884845Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Get train and test index\nmsss_splits = next(msss.split(X, Y))\ntrain_idx = msss_splits[0]\nvalid_idx = msss_splits[1]","metadata":{"execution":{"iopub.status.busy":"2021-09-20T13:41:43.886259Z","iopub.status.idle":"2021-09-20T13:41:43.887111Z","shell.execute_reply.started":"2021-09-20T13:41:43.886743Z","shell.execute_reply":"2021-09-20T13:41:43.886767Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now we can train the model for a number of epochs. All epochs we train the full model but each time on only 1/6 of the train data. With each epoch only a subset of the train data will allow us to make more epochs and allows todo averaging over more then just 1 or 2 epochs (compared to using all data every epoch).\n\nNote that I recreate the data generators and model on each epoch. This is only necessary when using the different Multi-label stratified splits since the data generators will get a totally different set of data on each epoch then. I left it in so that you can try it out.\n\nStarting with the 6th epoch a prediction for the test set is made on each epoch. In total predictions from the last 6 epochs will be averaged this way for the final submission.","metadata":{}},{"cell_type":"code","source":"np.random.shuffle(train_idx)\nprint(train_idx[:5])    \nprint(valid_idx[:5])\n\ndata_generator_train = TrainDataGenerator(train_df.iloc[train_idx,[0]], \n                                            train_df.iloc[train_idx,[0]], \n                                            TRAIN_BATCH_SIZE, \n                                            SHAPE,\n                                            augment = True)\ndata_generator_val = TrainDataGenerator(train_df.iloc[valid_idx,[0]], \n                                        train_df.iloc[valid_idx,[0]], \n                                        VALID_BATCH_SIZE, \n                                        SHAPE,\n                                        augment = False)\n\nTRAIN_STEPS = int(len(data_generator_train) / 10)\nLR = 0.000125","metadata":{"execution":{"iopub.status.busy":"2021-09-20T13:41:43.888324Z","iopub.status.idle":"2021-09-20T13:41:43.889292Z","shell.execute_reply.started":"2021-09-20T13:41:43.888911Z","shell.execute_reply":"2021-09-20T13:41:43.888996Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"(train_df.iloc[valid_idx].values[:,:]==1).sum(axis=0)/10","metadata":{"execution":{"iopub.status.busy":"2021-09-20T13:41:43.89033Z","iopub.status.idle":"2021-09-20T13:41:43.891216Z","shell.execute_reply.started":"2021-09-20T13:41:43.890832Z","shell.execute_reply":"2021-09-20T13:41:43.890856Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"AUC = tf.keras.metrics.AUC\nRECALL = tf.keras.metrics.Recall\nPRECISION = tf.keras.metrics.Precision","metadata":{"execution":{"iopub.status.busy":"2021-09-20T13:41:43.892393Z","iopub.status.idle":"2021-09-20T13:41:43.89339Z","shell.execute_reply.started":"2021-09-20T13:41:43.893112Z","shell.execute_reply":"2021-09-20T13:41:43.893139Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Create Model\nMetrics = [AUC(name = 'AUC_full'),\n           \n           RECALL(thresholds=0.7,name='REC_full'),\n          \n           PRECISION(thresholds=0.7, name='PRE_full')]\n\ndef get_weighted_loss(weights):\n    def weighted_loss(y_true, y_pred):\n        return K.mean((weights[:,0]**(1-y_true))*(weights[:,1]**(y_true))*K.binary_crossentropy(y_true, y_pred), axis=-1)\n    return weighted_loss\n\nmodel = create_model()   \nmodel.compile(optimizer = Adam(learning_rate = LR), \n                  loss = get_weighted_loss(weights),\n                  metrics = Metrics)","metadata":{"execution":{"iopub.status.busy":"2021-09-20T13:41:47.442499Z","iopub.execute_input":"2021-09-20T13:41:47.443099Z","iopub.status.idle":"2021-09-20T13:41:49.797034Z","shell.execute_reply.started":"2021-09-20T13:41:47.44306Z","shell.execute_reply":"2021-09-20T13:41:49.796254Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# def main():\nwith mlflow.start_run():\n    model.fit_generator(generator = data_generator_train,\n                            validation_data = data_generator_val,\n                            steps_per_epoch = TRAIN_STEPS,\n                            epochs = 10,\n                            callbacks = [ModelCheckpointFull('model.h5')],\n                            verbose = 1,workers=4)","metadata":{"execution":{"iopub.status.busy":"2021-09-20T13:41:52.868836Z","iopub.execute_input":"2021-09-20T13:41:52.869756Z","iopub.status.idle":"2021-09-20T13:43:44.927998Z","shell.execute_reply.started":"2021-09-20T13:41:52.869719Z","shell.execute_reply":"2021-09-20T13:43:44.925118Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# if __name__ == \"__main__\":\n#     main()","metadata":{"execution":{"iopub.status.busy":"2021-09-20T13:43:44.929123Z","iopub.status.idle":"2021-09-20T13:43:44.929416Z","shell.execute_reply.started":"2021-09-20T13:43:44.929265Z","shell.execute_reply":"2021-09-20T13:43:44.929283Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# test_df0 = test_df.iloc[:]\n\n# val_df0 = train_df.iloc[valid_idx]\n# val_df0 = val_df0.iloc[:]\n\n# train_df0 = train_df.iloc[train_idx]\n# train_df0 = train_df0.iloc[:]","metadata":{"execution":{"iopub.status.busy":"2021-09-20T13:43:44.936259Z","iopub.status.idle":"2021-09-20T13:43:44.936552Z","shell.execute_reply.started":"2021-09-20T13:43:44.936401Z","shell.execute_reply":"2021-09-20T13:43:44.93642Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# i,j = next(iter(data_generator_train))\n# i.shape,j.shape","metadata":{"execution":{"iopub.status.busy":"2021-09-20T13:43:44.937602Z","iopub.status.idle":"2021-09-20T13:43:44.937908Z","shell.execute_reply.started":"2021-09-20T13:43:44.937739Z","shell.execute_reply":"2021-09-20T13:43:44.937759Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# def calculating_class_weights(y_true):\n#     from sklearn.utils.class_weight import compute_class_weight\n#     number_dim = np.shape(y_true)[1]\n#     weights = np.empty([number_dim, 2])\n#     for i in range(number_dim):\n#         weights[i] = compute_class_weight('balanced', [0.,1.], y_true[:, i])\n#     return weights","metadata":{"execution":{"iopub.status.busy":"2021-09-20T13:43:44.940628Z","iopub.status.idle":"2021-09-20T13:43:44.941064Z","shell.execute_reply.started":"2021-09-20T13:43:44.940848Z","shell.execute_reply":"2021-09-20T13:43:44.940868Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# arr = [[1.0, 2.0], [3.0, 4.0]]\n# def numpy_to_tensor(arr):\n#     arg = tf.constant(arr)\n#     arg = tf.convert_to_tensor(arg, dtype=tf.float32)\n#     return arg\n# numpy_to_tensor(arr)","metadata":{"execution":{"iopub.status.busy":"2021-09-20T13:43:44.944231Z","iopub.status.idle":"2021-09-20T13:43:44.944543Z","shell.execute_reply.started":"2021-09-20T13:43:44.944386Z","shell.execute_reply":"2021-09-20T13:43:44.944405Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"","metadata":{}},{"cell_type":"code","source":"# sample_p =np.array([[0.5,0.5,0.5],\n#                     [0.5,0.5,0.5],\n#                     [0.5,0.5,0.5],\n#                     [0.5,0.5,0.5],\n#                     [0.5,0.5,0.5],\n#                     [0.5,0.5,0.5]],'float32')\n# y_true =  np.array([[0,0,0],\n#                     [1,0,0],\n#                     [1,0,0],\n#                     [1,0,0],\n#                     [1,1,0],\n#                     [0,1,1]],'float32')\n# weights = calculating_class_weights(y_true)\n# weights","metadata":{"execution":{"iopub.status.busy":"2021-09-20T13:43:44.949702Z","iopub.status.idle":"2021-09-20T13:43:44.950443Z","shell.execute_reply.started":"2021-09-20T13:43:44.950223Z","shell.execute_reply":"2021-09-20T13:43:44.950246Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# def get_weighted_loss(weights):\n#     def weighted_loss(y_true, y_pred):\n#         return K.mean((weights[:,0]**(1-y_true))*(weights[:,1]**(y_true))*K.binary_crossentropy(y_true, y_pred), axis=-1)\n#     return weighted_loss","metadata":{"execution":{"iopub.status.busy":"2021-09-20T13:43:44.951863Z","iopub.status.idle":"2021-09-20T13:43:44.952515Z","shell.execute_reply.started":"2021-09-20T13:43:44.952297Z","shell.execute_reply":"2021-09-20T13:43:44.95232Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# def weighted_loss(y_true, y_pred):\n#     return K.mean((weights[:,0]**(1-y_true))*(weights[:,1]**(y_true))*K.binary_crossentropy(y_true, y_pred), axis=-1)\n# #     return weighted_loss","metadata":{"execution":{"iopub.status.busy":"2021-09-20T13:43:44.953817Z","iopub.status.idle":"2021-09-20T13:43:44.954409Z","shell.execute_reply.started":"2021-09-20T13:43:44.95419Z","shell.execute_reply":"2021-09-20T13:43:44.954223Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# loss = get_weighted_loss(weights)\n# # loss = weighted_loss(numpy_to_tensor(y_true), numpy_to_tensor(sample_p))\n# loss","metadata":{"execution":{"iopub.status.busy":"2021-09-20T13:43:44.95595Z","iopub.status.idle":"2021-09-20T13:43:44.956528Z","shell.execute_reply.started":"2021-09-20T13:43:44.956317Z","shell.execute_reply":"2021-09-20T13:43:44.956339Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# w_weight = 'model'\n\n# model.load_weights(f'../input/keras-efficientnet-b2-from-start-to-output/{w_weight}.h5')\n# test_preds = model.predict_generator(TestDataGenerator(test_df0, None, 8, SHAPE, TEST_IMAGES_DIR), verbose = 1)\n# preds = test_preds[:test_df0.iloc[range(test_df0.shape[0])].shape[0]]\n# np.savez_compressed(f'pred_test_{w_weight}.npz',data = preds)\n# print(preds.shape)\n\n# test_preds = model.predict_generator(TestDataGenerator(train_df0, None, 8, SHAPE, TRAIN_IMAGES_DIR), verbose = 1)\n# preds = test_preds[:train_df0.iloc[range(train_df0.shape[0])].shape[0]]\n# np.savez_compressed(f'pred_train_{w_weight}.npz',data = preds)\n# print(preds.shape)\n\n# test_preds = model.predict_generator(TestDataGenerator(val_df0, None, 8, SHAPE, TRAIN_IMAGES_DIR), verbose = 1)\n# preds = test_preds[:val_df0.iloc[range(val_df0.shape[0])].shape[0]]\n# np.savez_compressed(f'pred_val_{w_weight}.npz',data = preds)\n# print(preds.shape)\n##################################################################################################################\n# w_weight = 'Final'\n\n# model.load_weights(f'../input/keras-efficientnet-b2-from-start-to-output/{w_weight}.h5')\n# test_preds = model.predict_generator(TestDataGenerator(test_df0, None, 8, SHAPE, TEST_IMAGES_DIR), verbose = 1)\n# preds = test_preds[:test_df0.iloc[range(test_df0.shape[0])].shape[0]]\n# np.savez_compressed(f'pred_test_{w_weight}.npz',data = preds)\n# print(preds.shape)\n\n# test_preds = model.predict_generator(TestDataGenerator(train_df0, None, 8, SHAPE, TRAIN_IMAGES_DIR), verbose = 1)\n# preds = test_preds[:train_df0.iloc[range(train_df0.shape[0])].shape[0]]\n# np.savez_compressed(f'pred_train_{w_weight}.npz',data = preds)\n# print(preds.shape)\n\n# test_preds = model.predict_generator(TestDataGenerator(val_df0, None, 8, SHAPE, TRAIN_IMAGES_DIR), verbose = 1)\n# preds = test_preds[:val_df0.iloc[range(val_df0.shape[0])].shape[0]]\n# np.savez_compressed(f'pred_val_{w_weight}.npz',data = preds)\n# print(preds.shape)","metadata":{"execution":{"iopub.status.busy":"2021-09-20T13:43:44.962002Z","iopub.status.idle":"2021-09-20T13:43:44.962399Z","shell.execute_reply.started":"2021-09-20T13:43:44.962204Z","shell.execute_reply":"2021-09-20T13:43:44.962225Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# sub 1\n# submission_predictions = [np.load('./pred_test_model.npz')['data']]\n# test_df0.iloc[:, :] = np.average(submission_predictions, axis = 0, weights = [1])\n# test_df0 = test_df0.stack().reset_index()\n# test_df0.insert(loc = 0, column = 'ID', value = test_df0['Image'].astype(str) + \"_\" + test_df0['Diagnosis'])\n# test_df0 = test_df0.drop([\"Image\", \"Diagnosis\"], axis=1)\n# test_df0.to_csv('submission.csv', index = False)\n# print(test_df0.head(12))\n\n# # sub 2\n# test_df0 = test_df.iloc[:1600]\n# submission_predictions = [np.load('./pred_test_Final.npz')['data']]\n# test_df0.iloc[:, :] = np.average(submission_predictions, axis = 0, weights = [1])\n# test_df0 = test_df0.stack().reset_index()\n# test_df0.insert(loc = 0, column = 'ID', value = test_df0['Image'].astype(str) + \"_\" + test_df0['Diagnosis'])\n# test_df0 = test_df0.drop([\"Image\", \"Diagnosis\"], axis=1)\n# test_df0.to_csv('submission_2.csv', index = False)\n# print(test_df0.head(12))\n\n# # sub 3\n# test_df0 = test_df.iloc[:1600]\n# submission_predictions = [np.load('./pred_test_model.npz')['data'],np.load('./pred_test_Final.npz')['data']]\n# test_df0.iloc[:, :] = np.average(submission_predictions, axis = 0, weights = [1,1])\n# test_df0 = test_df0.stack().reset_index()\n# test_df0.insert(loc = 0, column = 'ID', value = test_df0['Image'].astype(str) + \"_\" + test_df0['Diagnosis'])\n# test_df0 = test_df0.drop([\"Image\", \"Diagnosis\"], axis=1)\n# test_df0.to_csv('submission_3.csv', index = False)\n# print(test_df0.head(12))","metadata":{"execution":{"iopub.status.busy":"2021-09-20T13:43:44.963891Z","iopub.status.idle":"2021-09-20T13:43:44.964415Z","shell.execute_reply.started":"2021-09-20T13:43:44.964197Z","shell.execute_reply":"2021-09-20T13:43:44.964218Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# test_df.iloc[:, :] = np.average(submission_predictions, axis = 0, weights = [2**i for i in range(len(submission_predictions))])\n# test_df = test_df.stack().reset_index()\n# test_df.insert(loc = 0, column = 'ID', value = test_df['Image'].astype(str) + \"_\" + test_df['Diagnosis'])\n# test_df = test_df.drop([\"Image\", \"Diagnosis\"], axis=1)\n# test_df.to_csv('submission.csv', index = False)\n# print(test_df.head(12))","metadata":{"execution":{"iopub.status.busy":"2021-09-20T13:41:43.921775Z","iopub.status.idle":"2021-09-20T13:41:43.922239Z","shell.execute_reply.started":"2021-09-20T13:41:43.921989Z","shell.execute_reply":"2021-09-20T13:41:43.922011Z"},"trusted":true},"execution_count":null,"outputs":[]}]}