{"cells":[{"metadata":{"_uuid":"77d14b1acbbe49c4cbe7bd20dfed89059d2dd686"},"cell_type":"markdown","source":"# Table of Contents\n1. [The Problem](#ch1) <br>\n    1.1 [Importing libraries](#ch1.1) <br>\n    1.2 [Data Description](#ch1.2) <br>\n    1.3 [EDA for all categories](#ch1.3) <br>\n    1.4 [Training Data](#ch1.4) <br>\n2. [Modelling](#ch2)<br>\n    2.1 [Data pre-processing](#ch2.1)<br>\n    2.2 [Models](#ch2.2)<br>\n     2.2.1 [CNN](#ch2.2.1)<br>\n     2.2.2 [VGG16](#ch2.2.2)<br>\n     2.2.3 [ResNet](#ch2.2.3)<br>\n3. [Results](#ch3)"},{"metadata":{"_uuid":"0b97fc14033746686a9ca74019741cf513bff24e"},"cell_type":"markdown","source":"<a id=\"ch1\"></a>\n# 1. The problem\n\n*\"Quick, Draw!\"* was released as a mobile game. It prompts users to draw an image depicting a certain category, such as ”banana,” “table,” etc. The game generated more than 1B drawings, of which a subset was publicly released.\n\nThe competition contains *50M drawings encompassing 340 label* categories.\n\n*The challenge*: since the training data comes from the game itself, drawings can be incomplete or may not match the label. Your task is to build a classifier for the  Quick, Draw! dataset.\n\nOther details - \n1. GPU - NVIDIA TITAN X (1 out of 4 cores) on Kaggle Kernel"},{"metadata":{"_uuid":"9942748ba4e8e5a265a38642f544dd0aee84477d"},"cell_type":"markdown","source":"<a id=\"ch1.1\"></a>\n## 1.1 Importing libraries"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"scrolled":true},"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport os\nprint(os.listdir(\"../input/quickdraw-doodle-recognition\"))\nimport matplotlib.pyplot as plt\nimport ast\nimport pandas as pd\nimport cv2\nfrom skimage.io import imread, imshow\nimport seaborn as sns","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"b81efc0083dfcb0f10bfad08ecdad83a87987b12"},"cell_type":"markdown","source":"### Importing Data modelling libraries"},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"from sklearn.preprocessing import LabelEncoder\nfrom keras.utils.np_utils import to_categorical\nfrom keras import layers\nfrom keras.layers import Input, Add, Dense, Activation, BatchNormalization, Conv2D, AveragePooling2D, MaxPooling2D, Flatten, LSTM, Dropout, Flatten\nfrom keras.models import Model, load_model\nfrom keras.metrics import top_k_categorical_accuracy\nfrom keras.callbacks import ModelCheckpoint, LearningRateScheduler, EarlyStopping, ReduceLROnPlateau\nfrom tensorflow.keras.applications.mobilenet import preprocess_input\nfrom keras.applications.mobilenet_v2 import MobileNetV2\nfrom keras.applications.vgg16 import VGG16\nfrom keras.applications.resnet50 import ResNet50\nfrom keras.utils import plot_model","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"67dec682b1087b53e6f830a11080470a88429f61"},"cell_type":"markdown","source":"<a id=\"ch1.2\"></a>\n## 1.2 Data Description\n\nThe drawings are captured as timestamped vectors, tagged with metadata including what the player was asked to draw and in which country the player was located.\n\nTwo versions of the data are given - <br>\n**Raw data** - exact input recorded from the user drawing <br>\n**Simplified version** -  removes unnecessary points from the vector information. (For example, a straight line may have been recorded with 8 points, but since you only need 2 points to uniquely identify a line, 6 points can be dropped.) The simplified files are much smaller and provide effectively the same information.\n\n**File descriptions**\n\ntest_raw.csv - *580MB*<br>\ntest_simplified.csv - *59MB*<br>\ntrain_raw.zip - *66GB*<br>\ntrain_simplified.zip - *7.4GB*<br>\n\n### 1.2.1 Reading data for owl category"},{"metadata":{"trusted":true,"_uuid":"38c6e75bcb6742b3869156067f32b204353497a1","scrolled":true},"cell_type":"code","source":"owls = pd.read_csv('../input/quickdraw-doodle-recognition/train_simplified/owl.csv')\nrecog_counts = owls['recognized'].value_counts()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"a21a9cdd4f64c0da4ecdc0226442e1ab247e9324"},"cell_type":"code","source":"owls = owls[owls.recognized]\nowls['timestamp'] = pd.to_datetime(owls.timestamp)\nowls = owls.sort_values(by='timestamp', ascending=False)\nowls['drawing'] = owls['drawing'].apply(ast.literal_eval)\nowls.head()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"fe045592c1a5cadc0011d70bfb2149620ce52c92"},"cell_type":"markdown","source":"Above data shows some drawings data for owl category. Next we would take some samples and draw them - "},{"metadata":{"trusted":true,"_uuid":"f12d19d86fdf7116434cc171786440d89b5397d5"},"cell_type":"code","source":"n = 10\nfig, axs = plt.subplots(nrows=n, ncols=n, sharex=True, sharey=True, figsize=(16, 10))\nfor i, drawing in enumerate(owls.drawing[-100:]):\n    ax = axs[i // n, i % n]\n    for x, y in drawing:\n        ax.plot(x, -np.array(y), lw=3)\n    ax.axis('off')\nfig.savefig('owls.png', dpi=200)\nplt.show();","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"c60b09ebe6caf354b315beaba3679b812e0df1a9"},"cell_type":"markdown","source":"Below is the vector data for a single owl drawing."},{"metadata":{"trusted":true,"_uuid":"b88e6f97f33cf8bc7a9a9995adb1be294f99dd16"},"cell_type":"code","source":"for x,y in owls.drawing[51970]:\n    print(\"x:\",x,\"  y:\",y)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ec961c24560d2460a903a03516e29e307c26d838"},"cell_type":"markdown","source":"Here is the country wise distribution of owl drawings - "},{"metadata":{"_uuid":"5e0ed6c204db651a62e59b673a32a7bf38134f93"},"cell_type":"markdown","source":"Top 5 states with owl drawings - "},{"metadata":{"trusted":true,"_uuid":"4677065ae57975e5a75aa6cc38fce8289fb00ce1"},"cell_type":"code","source":"country_counts = owls['countrycode'].value_counts()\npd.DataFrame(country_counts).head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ddcb796b30b8f37033c3081c43b499344a493b9c"},"cell_type":"code","source":"top_10_states = list(country_counts[:10].index)\nowls_top_10 = owls[owls['countrycode'].isin(top_10_states)]\ng = sns.catplot(x=\"countrycode\", data=owls_top_10, kind=\"count\",height=5, aspect=4)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"28fbd0f99c5170dea173bdd394e52ab0bb38cbfb"},"cell_type":"markdown","source":"Count of recognized doodles in owl category - "},{"metadata":{"trusted":true,"_uuid":"95c46d933f9147c3d24086ae5d4f7deafd2f59af"},"cell_type":"code","source":"sns.barplot(x=recog_counts.index, y=recog_counts)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"1f830c704838573d5bbb4b1a23219b7138771947"},"cell_type":"markdown","source":"<a id=\"ch1.3\"></a>\n## 1.3 EDA for all categories\n\n1. Below, we will explore classified category and its relation with various x-variables."},{"metadata":{"trusted":true,"_uuid":"6eff532c4fb35002f9a727ecdef7ef20b2cffb64"},"cell_type":"code","source":"word = []\ncount = []\ncount_recog = []\nfor f in os.listdir(\"../input/quickdraw-doodle-recognition/train_simplified\"):\n    df = pd.read_csv(\"../input/quickdraw-doodle-recognition/train_simplified/\" + f)\n    word.append(df['word'][0])\n    count.append(df.shape[0])\n    count_recog.append(len(df[df['recognized'] == True]))\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"8b84f64716e1ef4a48ac6578e5f798fb156cfd2d"},"cell_type":"code","source":"summary = pd.DataFrame({'word':word,'count':count,'count_recog':count_recog})\nsummary.head()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"2acbf1b99eea5770e963dd120c4e3f77002bea3c"},"cell_type":"markdown","source":"![](http://)![](http://)### Most drawed and least drawed categories"},{"metadata":{"trusted":true,"_uuid":"cb21c15a39cd81eac3dbd43b677fcb31ea133275"},"cell_type":"code","source":"summary_top = summary.sort_values(by='count')[-10:]\nsummary_top.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"b21fa97b06fa8c7f72acb21b3ae23e38c93a5546"},"cell_type":"code","source":"summary_less = pd.concat([summary.sort_values(by='count')[-10:],summary.sort_values(by='count')[:10]])\ng = sns.catplot(x=\"word\", y=\"count\", data=summary_less, kind=\"bar\",height=5, aspect=4)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"9f7829d420d6ef6373ce7ec4c0c3d630149c888f"},"cell_type":"markdown","source":"<a id=\"ch1.4\"></a>\n## 1.4 Training Data\n\nFor convenience, training data is chosen by sampling category files for fixed number of rows and merging them. The final data is a set of 100 train files each of which has randomized data for all categories."},{"metadata":{"trusted":true,"_uuid":"6a4bd5139f9e498a56b326422bc8fd4c1f8e20ee"},"cell_type":"code","source":"CSV_DIR = '../input/doodle-detection-dataprep'\nfilename = CSV_DIR + '/train_k0.csv.gz'\ntrain_sample = pd.read_csv(filename)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"3c3198de142e33781d5e8c91dd9d40d0a561f284"},"cell_type":"code","source":"train_sample.head()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d5671d9cf458ebce82eec64ea91efa794c0bac50"},"cell_type":"markdown","source":"### Total train data size "},{"metadata":{"trusted":true,"_uuid":"e88a96baedcbf1037ee238ab025a96db7ad7c577"},"cell_type":"code","source":"# lens = []\n# CSV_DIR = '../input/doodle-detection-dataprep'\n# for k in range(100):\n#     filename = os.path.join(CSV_DIR, 'train_k{}.csv.gz'.format(k))\n#     for df in pd.read_csv(filename):\n#         lens.append(len(df))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ef1aed7a2dcd39c80541f3539aa16e90006e8103"},"cell_type":"markdown","source":"<a id=\"ch2\"></a>\n# 2 Modelling \n<a id=\"ch2.1\"></a>\n## 2.1 Data pre-processing\n\nVariables initialize - "},{"metadata":{"trusted":true,"_uuid":"e0cdaae2818c982291af12f4e0f7b8eee3d09d28"},"cell_type":"code","source":"BATCH_SIZE = 128\nMAX_TRAIN_EPOCHS = 20\nSTEPS_PER_EPOCH = 900\nNCSVS = 100\nCSV_DIR = '../input/doodle-detection-dataprep'\nBASE_SIZE = 256\nsize = 128","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ada03f60f175af6f5a510badfef6ec371267aa31"},"cell_type":"markdown","source":"Converting 340 given categories into one hot encoding"},{"metadata":{"trusted":true,"_uuid":"6b99b35c3ca668abc8146f5834dcf7812ca94173"},"cell_type":"code","source":"word_encoder = LabelEncoder()\ncategories = [word.split('.')[0] for word in os.listdir(os.path.join(\"../input/quickdraw-doodle-recognition/train_simplified/\"))]\nword_encoder.fit(categories)\nprint('words', len(word_encoder.classes_), '=>', ', '.join([x for x in word_encoder.classes_[:50]]))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8219be51092567d508d24e8760cc9fee2b965aa4"},"cell_type":"markdown","source":"draw_cv2 :\nfunction for converting sketches into images\n\nimage_generator_xd:\ntraining data image generator takes 100 compressed csvs formed from 340 category csvs in doodle detection dataprep"},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":false,"trusted":true,"_uuid":"be20609155410564ad38b38ccdfc9f52f9832180"},"cell_type":"code","source":"def draw_cv2(raw_strokes, size=256, lw=6, time_color=True):\n    img = np.zeros((BASE_SIZE, BASE_SIZE), np.uint8)\n    for t, stroke in enumerate(raw_strokes):\n        for i in range(len(stroke[0]) - 1):\n            color = 255 - min(t, 10) * 13 if time_color else 255\n            _ = cv2.line(img, (stroke[0][i], stroke[1][i]),\n                         (stroke[0][i + 1], stroke[1][i + 1]), color, lw)\n    if size != BASE_SIZE:\n        return cv2.resize(img, (size, size))\n    else:\n        return img","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"2573f7f61dd47008d7f8bde76188e214009d431c"},"cell_type":"code","source":"def image_generator_xd(size, batchsize, ks, lw=6, time_color=True):\n    while True:\n        for k in np.random.permutation(ks):\n            filename = os.path.join(CSV_DIR, 'train_k{}.csv.gz'.format(k))\n            for df in pd.read_csv(filename, chunksize=batchsize):\n                df['drawing'] = df['drawing'].apply(ast.literal_eval)\n                x = np.zeros((len(df), size, size, 3))\n                for i, raw_strokes in enumerate(df.drawing.values):\n                    x[i, :, :, 0] = draw_cv2(raw_strokes, size=size, lw=lw,\n                                             time_color=time_color)\n                    x[i, :, :, 1] = draw_cv2(raw_strokes, size=size, lw=lw,\n                                             time_color=time_color)\n                    x[i, :, :, 2] = draw_cv2(raw_strokes, size=size, lw=lw,\n                                             time_color=time_color)\n                x = preprocess_input(x).astype(np.float32)\n                y = to_categorical(word_encoder.transform(df[\"word\"].values),num_classes=340).astype(np.int32)\n                yield x, y","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"74b3838375876346e75249611161b7b0ae738101"},"cell_type":"markdown","source":"df_to_image_array_xd:\nreads dataframe and returns array of images from drawing column"},{"metadata":{"trusted":true,"_uuid":"1f5859d5cceeda9283e2409feed041acbca538d4"},"cell_type":"code","source":"def df_to_image_array_xd(df, size, lw=6, time_color=True):\n    df['drawing'] = df['drawing'].apply(ast.literal_eval)\n    x = np.zeros((len(df), size, size, 3))\n    for i, raw_strokes in enumerate(df.drawing.values):\n        x[i, :, :, 0] = draw_cv2(raw_strokes, size=size, lw=lw, time_color=time_color)\n        x[i, :, :, 1] = draw_cv2(raw_strokes, size=size, lw=lw, time_color=time_color)\n        x[i, :, :, 2] = draw_cv2(raw_strokes, size=size, lw=lw, time_color=time_color)\n    x = preprocess_input(x).astype(np.float32)\n    return x","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"6eb24647a229fc622351fab5b86d4aeee4f7d73c"},"cell_type":"code","source":"train_datagen = image_generator_xd(batchsize=BATCH_SIZE, ks=range(NCSVS - 1), size=size)\ntrain_x, train_y = next(train_datagen)\nprint ('train x shape:{}'.format(train_x.shape))\nprint ('train y shape:{}'.format(train_y.shape))\nprint('train_x', train_x.dtype, train_x.min(), train_x.max())\nprint('train_y', train_y.dtype, train_y.min(), train_y.max())","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"4c54b0af486cae44a16340f0423f8a7455addecf"},"cell_type":"markdown","source":"Forming validation dataset"},{"metadata":{"trusted":true,"_uuid":"b45cf8fb7c32bb7ede2abb3d3d047c2c4c95ec1f"},"cell_type":"code","source":"valid_set = pd.read_csv(os.path.join(CSV_DIR, 'train_k{}.csv.gz'.format(NCSVS - 1)), nrows=1000)\nvalid_x = df_to_image_array_xd(valid_set, size)\nvalid_y = to_categorical(word_encoder.transform(valid_set[\"word\"].values),num_classes=340).astype(np.int32)\nprint ('valid x shape:{}'.format(valid_x.shape))\nprint ('valid y shape:{}'.format(valid_y.shape))\nprint('valid_x', valid_x.dtype, valid_x.min(), valid_x.max())\nprint('valid_y', valid_y.dtype, valid_y.min(), valid_y.max())","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5cdc8513c400f0ac5d49b56b66e1b9ba4829389b"},"cell_type":"markdown","source":"Train data visualization"},{"metadata":{"trusted":true,"_uuid":"dcfa18ec49972098a6f0336bbc7d7a015b112616"},"cell_type":"code","source":"fig, m_axs = plt.subplots(4,4, figsize = (8, 8))\nrand_idxs = np.random.choice(range(train_x.shape[0]), size = 16, replace=False)\nfor c_id, c_ax in zip(rand_idxs, m_axs.flatten()):\n    test_arr = train_x[c_id, :, :, 0]  \n    c_ax.imshow(test_arr, cmap=plt.cm.gray)\n    c_ax.axis('off')\n    c_ax.set_title(word_encoder.classes_[np.argmax(train_y[c_id])])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"e682260fd5b4d3f697200e8ecc61a09ee82ff3df"},"cell_type":"markdown","source":"<a id=\"ch2.2\"></a>\n## 2.2 Models"},{"metadata":{"trusted":true,"_uuid":"ddc594ac3f30f71ab414cc4ac126ea25ea50e5a0"},"cell_type":"code","source":"def cnn(input_shape):\n    input_img = Input(input_shape)\n    conv0= Conv2D(256, (3, 3), activation='relu', padding='valid')(input_img) \n    pool0 = MaxPooling2D(pool_size=(2, 2))(conv0)\n    conv1= Conv2D(128, (3, 3), activation='relu', padding='valid')(pool0)\n    pool1 = MaxPooling2D(pool_size=(2, 2))(conv1)\n    conv2= Conv2D(64, (3, 3), activation='relu', padding='valid')(pool1) \n    pool2 = MaxPooling2D(pool_size=(2, 2))(conv2)\n    conv3 = Conv2D(32, (3, 3), activation='relu', padding='valid')(pool2) \n    pool3 = MaxPooling2D(pool_size=(2, 2))(conv3) \n    flat = Flatten()(pool3)\n    dense1 = Dense(680, activation='relu')(flat)\n    dense2 = Dense(len(word_encoder.classes_), activation = 'softmax')(dense1)\n    \n    model =  Model(inputs = input_img, outputs = dense2, name = 'Doodle_model')    \n    return model\n\ndef MobileNetV2_model():\n    base_model = MobileNetV2(include_top=False, weights='imagenet', input_shape=(128,128,3), pooling='avg')\n    x = base_model.output\n    x = Dense(512, activation='relu')(x)\n    output = Dense(340, activation='softmax')(x)\n    model = Model(inputs=base_model.input, outputs=output)\n    for layer in base_model.layers:\n        layer.trainable = False\n    \n    return model\n\ndef vgg16_model():\n    base_model = VGG16(include_top=False, weights='imagenet', input_shape=(128,128,3), pooling='avg')\n    x = base_model.output\n    x = Dense(512, activation='relu')(x)\n    output = Dense(340, activation='softmax')(x)\n    \n    model = Model(inputs=base_model.input, outputs=output)\n    \n    for layer in base_model.layers:\n        layer.trainable = False\n    \n    return model\n\ndef resnet_model():\n    base_model = ResNet50(include_top=False, weights='imagenet', input_shape=(128,128,3), pooling='avg')\n    x = base_model.output\n    x = Dense(512, activation='relu')(x)\n    output = Dense(340, activation='softmax')(x)\n    \n    model = Model(inputs=base_model.input, outputs=output)\n    \n    for layer in base_model.layers:\n        layer.trainable = False\n    \n    return model\n","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"453cef80f296f1c065e60f2c03f0148bf658507b"},"cell_type":"code","source":"def top_3_accuracy(y_true, y_pred):\n    return top_k_categorical_accuracy(y_true, y_pred, k=3)\n\ndef run_model(model):\n    model.compile(optimizer = 'adam', loss='categorical_crossentropy', metrics = ['categorical_accuracy', top_3_accuracy])\n    \n    checkpoint = ModelCheckpoint(\"model_weights.best.hdf5\", monitor='val_top_3_accuracy', verbose=1, save_best_only=True, mode='max', save_weights_only=True, period=1)\n    early = EarlyStopping(monitor=\"val_top_3_accuracy\", mode=\"max\", verbose=2,patience=8)\n    callbacks_list = [checkpoint, early]\n    \n    loss_history = [model.fit_generator(train_datagen,epochs=15,steps_per_epoch=STEPS_PER_EPOCH,\n                                    validation_data=(valid_x, valid_y),callbacks=callbacks_list,workers=1)]\n    model.load_weights(\"model_weights.best.hdf5\")\n    model.save('model.h5')\n\n    return loss_history\n\ndef display_plots(loss_history):    \n    epochs = np.concatenate([mh.epoch for mh in loss_history])\n    loss = np.concatenate([mh.history['loss'] for mh in loss_history])\n    val_loss  = np.concatenate([mh.history['val_loss'] for mh in loss_history])\n    \n    train_accuracy = np.concatenate([mh.history['top_3_accuracy'] for mh in loss_history])\n    test_accuracy = np.concatenate([mh.history['val_top_3_accuracy'] for mh in loss_history])\n    print ('train accuray: {}'.format(max(train_accuracy)))\n    print ('test accuray: {}'.format(max(test_accuracy)))\n    fig, (ax1, ax2) = plt.subplots(1,2, figsize = (30,10))\n\n    ax1.plot(epochs,train_accuracy, epochs,test_accuracy)\n    ax1.legend(['Training', 'Validation'])\n    ax1.set_xlabel('epoch')\n    ax1.set_ylabel('accuracy')\n    ax1.set_title('accuracy train vs validation')\n\n    ax2.plot(epochs,loss, epochs,val_loss)\n    ax2.legend(['Training', 'Validation'])\n    ax2.set_xlabel('epoch')\n    ax2.set_ylabel('loss')\n    ax2.set_title('loss train vs validation')\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"f78c4739a7652007b7ca121b5cb5f17f342b2988"},"cell_type":"code","source":"valid_set = pd.read_csv(os.path.join(CSV_DIR, 'train_k{}.csv.gz'.format(NCSVS - 1)), nrows=16)\nvalid_x = df_to_image_array_xd(valid_set, size)\nvalid_y = to_categorical(word_encoder.transform(valid_set[\"word\"].values),num_classes=340).astype(np.int32)\n\ndef validation_and_display(model):\n    valid_img_label = model.predict(valid_x, verbose=True)\n    top_3_pred_valid = [word_encoder.classes_[np.argsort(-1*c_pred)[:3]] for c_pred in valid_img_label]\n    top_3_pred_valid = [' '.join([col.replace(' ', '_') for col in row]) for row in top_3_pred_valid]\n    \n    fig, m_axs = plt.subplots(4,4, figsize = (20, 20))\n    rand_idxs = np.random.choice(range(valid_x.shape[0]), size = 16, replace=False)\n    for c_id, c_ax in zip(rand_idxs, m_axs.flatten()):\n        test_arr = valid_x[c_id, :, :, 0]\n        c_ax.imshow(test_arr,cmap=plt.cm.gray)\n        c_ax.axis('off')\n        c_ax.set_title(('pred:'+top_3_pred_valid[c_id],'gt:'+valid_set[\"word\"].iloc[c_id]))\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"79a38954b751679f996dcbfd8cc1bdac8fdc68ec"},"cell_type":"markdown","source":"<a id=\"ch2.2.1\"></a>\n## 2.2.1 CNN"},{"metadata":{"trusted":true,"_uuid":"7836a5aee5a04b7b6ef1cf1fc92b3eff2993f7ec"},"cell_type":"code","source":"model = cnn(input_shape = train_x.shape[1:])\nloss_history = run_model(model)\ndisplay_plots(loss_history)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"4307bec489ba69a6dd589ed181663134fb1b0ab7"},"cell_type":"code","source":"plot_model(model, to_file='model_cnn.png')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5a5d6b2843449fe6e1201b924ac61288f8100f01"},"cell_type":"markdown","source":"[](http://)![CNN Model](model_cnn.png)"},{"metadata":{"_uuid":"427e92310868ffa5ecbaf08158b96064dd562c03"},"cell_type":"markdown","source":"<a id=\"ch2.2.2\"></a>\n## 2.2.2 VGG16"},{"metadata":{"trusted":true,"_uuid":"cf93deb24bac8592886abead1f1072f8d20d30f7"},"cell_type":"code","source":"model = vgg16_model()\nloss_history = run_model(model)\ndisplay_plots(loss_history)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"6d0cf59eb75229f2889c3e85bf7eef9becc575e5"},"cell_type":"code","source":"plot_model(model, to_file='model_vgg16.png')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"c132c9c9efec652ddca86eb458634840190cf622"},"cell_type":"markdown","source":"<a id=\"ch2.2.3\"></a>\n## 2.2.3 MobileNet"},{"metadata":{"trusted":true,"_uuid":"295b00adfd743531cd2e5a6ff96e15747f9154fe"},"cell_type":"code","source":"model = MobileNetV2_model()\nloss_history = run_model(model)\ndisplay_plots(loss_history)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"655d5a40b2199720c012d462c80e3d24e2eed55c"},"cell_type":"code","source":"plot_model(model, to_file='model_mobilenet.png')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"384e0b7f0d62fe0b94cf1dfab218b75b0e3d66c8"},"cell_type":"markdown","source":"<a id=\"ch2.2.4\"></a>\n## 2.2.4 Resnet"},{"metadata":{"trusted":true,"_uuid":"0c9aea6582cb06477303c0c120e876d3d47c60ca"},"cell_type":"code","source":"model = resnet_model()\nloss_history = run_model(model)\ndisplay_plots(loss_history)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"11a89efc13fc7cd214bb8c30452d6ea67b1d9963"},"cell_type":"markdown","source":"Validation"},{"metadata":{"trusted":true,"_uuid":"1b3240f065823874af93d891afdeffd055388c5d"},"cell_type":"code","source":"validation_and_display(model)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"169a8f1ea6ce32ee30c08fe5330907a4d02f1ee8"},"cell_type":"markdown","source":"<a id=\"ch3\"></a>\n### 3 Results \n\nThe results indicate that the CNN model has the best accuracy. Hence, this was used to submit the solution in the competition.\n\n![](http://)"}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}