{"cells":[{"metadata":{"_uuid":"77d14b1acbbe49c4cbe7bd20dfed89059d2dd686"},"cell_type":"markdown","source":"- by Kapil Yadav, ShriramPrabhu, Shweta Baranwal\n\n# Table of Contents\n1. [The Problem](#ch1) <br>\n    1.1 [Importing libraries](#ch1.1) <br>\n    1.2 [Data Description](#ch1.2) <br>\n    1.3 [EDA for all categories](#ch1.3) <br>\n    1.4 [Training Data](#ch1.4) <br>\n2. [Modelling](#ch2)<br>\n    2.1 [Data pre-processing](#ch2.1)<br>\n    2.2 [Models](#ch2.2)<br>\n     2.2.1 [CNN](#ch2.2.1)<br>\n     2.2.2 [VGG16](#ch2.2.2)<br>\n     2.2.3 [ResNet](#ch2.2.3)<br>\n3. [Results](#ch3)"},{"metadata":{"_uuid":"0b97fc14033746686a9ca74019741cf513bff24e"},"cell_type":"markdown","source":"<a id=\"ch1\"></a>\n# 1. The problem\n\n*\"Quick, Draw!\"* was released as a mobile game. It prompts users to draw an image depicting a certain category, such as ”banana,” “table,” etc. The game generated more than 1B drawings, of which a subset was publicly released.\n\nThe competition contains *50M drawings encompassing 340 label* categories.\n\n*The challenge*: since the training data comes from the game itself, drawings can be incomplete or may not match the label. Your task is to build a classifier for the  Quick, Draw! dataset.\n\nOther details - \n1. GPU - NVIDIA TITAN X (1 out of 4 cores) on Kaggle Kernel"},{"metadata":{"_uuid":"9942748ba4e8e5a265a38642f544dd0aee84477d"},"cell_type":"markdown","source":"<a id=\"ch1.1\"></a>\n## 1.1 Importing libraries"},{"metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_kg_hide-input":true,"_kg_hide-output":true,"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","scrolled":true,"trusted":true},"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport os\nprint(os.listdir(\"../input/quickdraw-doodle-recognition\"))\nimport matplotlib.pyplot as plt\nimport ast\nimport pandas as pd\nimport cv2\nfrom skimage.io import imread, imshow\nimport seaborn as sns\nfrom IPython.display import Image","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"b81efc0083dfcb0f10bfad08ecdad83a87987b12"},"cell_type":"markdown","source":"### Importing Data modelling libraries"},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_kg_hide-input":true,"_kg_hide-output":true,"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":false},"cell_type":"code","source":"from sklearn.preprocessing import LabelEncoder\nfrom keras.utils.np_utils import to_categorical\nfrom keras import layers\nfrom keras.layers import Input, Add, Dense, Activation, BatchNormalization, Conv2D, AveragePooling2D, MaxPooling2D, Flatten, LSTM, Dropout, Flatten\nfrom keras.models import Model, load_model\nfrom keras.metrics import top_k_categorical_accuracy\nfrom keras.callbacks import ModelCheckpoint, LearningRateScheduler, EarlyStopping, ReduceLROnPlateau\nfrom tensorflow.keras.applications.mobilenet import preprocess_input\nfrom keras.applications.mobilenet_v2 import MobileNetV2\nfrom keras.applications.vgg16 import VGG16\nfrom keras.applications.resnet50 import ResNet50\nfrom keras.utils import plot_model","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"67dec682b1087b53e6f830a11080470a88429f61"},"cell_type":"markdown","source":"<a id=\"ch1.2\"></a>\n## 1.2 Data Description\n\nThe drawings are captured as timestamped vectors, tagged with metadata including what the player was asked to draw and in which country the player was located.\n\nTwo versions of the data are given - <br>\n**Raw data** - exact input recorded from the user drawing <br>\n**Simplified version** -  removes unnecessary points from the vector information. (For example, a straight line may have been recorded with 8 points, but since you only need 2 points to uniquely identify a line, 6 points can be dropped.) The simplified files are much smaller and provide effectively the same information.\n\n**File descriptions**\n\ntest_raw.csv - *580MB*<br>\ntest_simplified.csv - *59MB*<br>\ntrain_raw.zip - *66GB*<br>\ntrain_simplified.zip - *7.4GB*<br>\n\n### 1.2.1 Reading data for owl category"},{"metadata":{"_uuid":"38c6e75bcb6742b3869156067f32b204353497a1","scrolled":true,"trusted":false},"cell_type":"code","source":"owls = pd.read_csv('../input/quickdraw-doodle-recognition/train_simplified/owl.csv')\nrecog_counts = owls['recognized'].value_counts()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"9626b1f4085c8cd37c14fe1dbc85334203f6d455"},"cell_type":"markdown","source":"Below is the vector data for a single owl drawing."},{"metadata":{"_uuid":"a21a9cdd4f64c0da4ecdc0226442e1ab247e9324","trusted":false},"cell_type":"code","source":"owls = owls[owls.recognized]\nowls['timestamp'] = pd.to_datetime(owls.timestamp)\nowls = owls.sort_values(by='timestamp', ascending=False)\nowls['drawing'] = owls['drawing'].apply(ast.literal_eval)\nfor x,y in owls.drawing[51970]:\n    print(\"x:\",x,\"  y:\",y)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7d23f0fc54267b390b6c9888ce30ce7d62e44342"},"cell_type":"markdown","source":"Above data shows some drawings data for owl category. draw them - "},{"metadata":{"_uuid":"9fe28ad4464a26ed691203432d95fa92b2dd572e","trusted":false},"cell_type":"code","source":"n = 10\nfig, axs = plt.subplots(nrows=n, ncols=n, sharex=True, sharey=True, figsize=(16, 10))\nfor i, drawing in enumerate(owls.drawing[-100:]):\n    ax = axs[i // n, i % n]\n    for x, y in drawing:\n        ax.plot(x, -np.array(y), lw=3)\n    ax.axis('off')\nfig.savefig('owls.png', dpi=200)\nplt.show();","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"6b516d713e7ba7c442ac4c34c0d99c27b2e7ec3d","trusted":false},"cell_type":"code","source":"owls.head()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"da021aec19874efa91588ec9808df18c7109db08"},"cell_type":"markdown","source":"Some of the output categories - "},{"metadata":{"_uuid":"facdd04140fb1ba71fce86c8c9df319c3c702ddf","trusted":false},"cell_type":"code","source":"word_encoder = LabelEncoder()\ncategories = [word.split('.')[0] for word in os.listdir(os.path.join(\"../input/quickdraw-doodle-recognition/train_simplified/\"))]\nword_encoder.fit(categories)\nprint('words', len(word_encoder.classes_), '=>', ', '.join([x for x in word_encoder.classes_[:50]]))\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ec961c24560d2460a903a03516e29e307c26d838"},"cell_type":"markdown","source":"Here is the country wise distribution of owl drawings - "},{"metadata":{"_uuid":"5e0ed6c204db651a62e59b673a32a7bf38134f93"},"cell_type":"markdown","source":"Top 5 states with owl drawings - "},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"_uuid":"4677065ae57975e5a75aa6cc38fce8289fb00ce1","trusted":false},"cell_type":"code","source":"country_counts = owls['countrycode'].value_counts()\n# pd.DataFrame(country_counts).head()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ddcb796b30b8f37033c3081c43b499344a493b9c","trusted":false},"cell_type":"code","source":"top_10_states = list(country_counts[:10].index)\nowls_top_10 = owls[owls['countrycode'].isin(top_10_states)]\ng = sns.catplot(x=\"countrycode\", data=owls_top_10, kind=\"count\",height=5, aspect=4)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"28fbd0f99c5170dea173bdd394e52ab0bb38cbfb"},"cell_type":"markdown","source":"Count of recognized doodles in owl category - "},{"metadata":{"_uuid":"95c46d933f9147c3d24086ae5d4f7deafd2f59af","trusted":false},"cell_type":"code","source":"sns.barplot(x=recog_counts.index, y=recog_counts)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"1f830c704838573d5bbb4b1a23219b7138771947"},"cell_type":"markdown","source":"<a id=\"ch1.3\"></a>\n## 1.3 EDA for all categories\n\n1. Below, we will explore classified category and its relation with various x-variables."},{"metadata":{"_uuid":"6eff532c4fb35002f9a727ecdef7ef20b2cffb64","trusted":false},"cell_type":"code","source":"word = []\ncount = []\ncount_recog = []\nfor f in os.listdir(\"../input/quickdraw-doodle-recognition/train_simplified\"):\n    df = pd.read_csv(\"../input/quickdraw-doodle-recognition/train_simplified/\" + f)\n    word.append(df['word'][0])\n    count.append(df.shape[0])\n    count_recog.append(len(df[df['recognized'] == True]))\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8b84f64716e1ef4a48ac6578e5f798fb156cfd2d","trusted":false,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"summary = pd.DataFrame({'word':word,'count':count})\nsummary.head()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"2acbf1b99eea5770e963dd120c4e3f77002bea3c"},"cell_type":"markdown","source":"![](http://)![](http://)### Most drawed and least drawed categories"},{"metadata":{"_uuid":"cb21c15a39cd81eac3dbd43b677fcb31ea133275","trusted":false},"cell_type":"code","source":"# summary_top = summary.sort_values(by='count')[-10:]\n# summary_top.head()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"b21fa97b06fa8c7f72acb21b3ae23e38c93a5546","trusted":false},"cell_type":"code","source":"summary_less = pd.concat([summary.sort_values(by='count')[-10:],summary.sort_values(by='count')[:10]])\ng = sns.catplot(x=\"word\", y=\"count\", data=summary_less, kind=\"bar\",height=5, aspect=4)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"9f7829d420d6ef6373ce7ec4c0c3d630149c888f"},"cell_type":"markdown","source":"<a id=\"ch1.4\"></a>\n## 1.4 Training Data\n\nThe final data is a set of 100 train files each of which has randomized data for all categories. For convenience, training data is chosen by sampling category files for fixed number of rows and merging them.\n\nTraining data size - 1.4 GB <br>\nTraining data rows - 3.5 million"},{"metadata":{"_uuid":"6a4bd5139f9e498a56b326422bc8fd4c1f8e20ee","trusted":false,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"CSV_DIR = '../input/doodle-detection-dataprep'\nfilename = CSV_DIR + '/train_k0.csv.gz'\ntrain_sample = pd.read_csv(filename)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"3c3198de142e33781d5e8c91dd9d40d0a561f284","trusted":false},"cell_type":"code","source":"train_sample.head()","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"_uuid":"e88a96baedcbf1037ee238ab025a96db7ad7c577","trusted":false},"cell_type":"code","source":"# lens = []\n# CSV_DIR = '../input/doodle-detection-dataprep'\n# for k in range(100):\n#     filename = os.path.join(CSV_DIR, 'train_k{}.csv.gz'.format(k))\n#     for df in pd.read_csv(filename):\n#         lens.append(len(df))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ef1aed7a2dcd39c80541f3539aa16e90006e8103"},"cell_type":"markdown","source":"<a id=\"ch2\"></a>\n# 2 Modelling \n<a id=\"ch2.1\"></a>\n## 2.1 Data pre-processing\n\nVariables initialize - "},{"metadata":{"_uuid":"e0cdaae2818c982291af12f4e0f7b8eee3d09d28","trusted":false},"cell_type":"code","source":"BATCH_SIZE = 128\nMAX_TRAIN_EPOCHS = 20\nNCSVS = 100\nCSV_DIR = '../input/doodle-detection-dataprep'\nBASE_SIZE = 256\nsize = 128","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true,"_uuid":"34faa8ea50bdf19908bdaeaeed4f00e6c6b0567a"},"cell_type":"code","source":"STEPS_PER_EPOCH = 900","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"6b99b35c3ca668abc8146f5834dcf7812ca94173","trusted":false,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"word_encoder = LabelEncoder()\ncategories = [word.split('.')[0] for word in os.listdir(os.path.join(\"../input/quickdraw-doodle-recognition/train_simplified/\"))]\nword_encoder.fit(categories)\nprint('words', len(word_encoder.classes_), '=>', ', '.join([x for x in word_encoder.classes_[:50]]))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8219be51092567d508d24e8760cc9fee2b965aa4"},"cell_type":"markdown","source":"draw_cv2 :\nfunction for converting sketches into images\n\nimage_generator_xd:\ntraining data image generator takes 100 compressed csvs formed from 340 category csvs in doodle detection dataprep"},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"_uuid":"be20609155410564ad38b38ccdfc9f52f9832180","trusted":false},"cell_type":"code","source":"def draw_cv2(raw_strokes, size=256, lw=6, time_color=True):\n    img = np.zeros((BASE_SIZE, BASE_SIZE), np.uint8)\n    for t, stroke in enumerate(raw_strokes):\n        for i in range(len(stroke[0]) - 1):\n            color = 255 - min(t, 10) * 13 if time_color else 255\n            _ = cv2.line(img, (stroke[0][i], stroke[1][i]),\n                         (stroke[0][i + 1], stroke[1][i + 1]), color, lw)\n    if size != BASE_SIZE:\n        return cv2.resize(img, (size, size))\n    else:\n        return img","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"_uuid":"2573f7f61dd47008d7f8bde76188e214009d431c","trusted":false},"cell_type":"code","source":"def image_generator_xd(size, batchsize, ks, lw=6, time_color=True):\n    while True:\n        for k in np.random.permutation(ks):\n            filename = os.path.join(CSV_DIR, 'train_k{}.csv.gz'.format(k))\n            for df in pd.read_csv(filename, chunksize=batchsize):\n                df['drawing'] = df['drawing'].apply(ast.literal_eval)\n                x = np.zeros((len(df), size, size, 3))\n                for i, raw_strokes in enumerate(df.drawing.values):\n                    x[i, :, :, 0] = draw_cv2(raw_strokes, size=size, lw=lw,\n                                             time_color=time_color)\n                    x[i, :, :, 1] = draw_cv2(raw_strokes, size=size, lw=lw,\n                                             time_color=time_color)\n                    x[i, :, :, 2] = draw_cv2(raw_strokes, size=size, lw=lw,\n                                             time_color=time_color)\n                x = preprocess_input(x).astype(np.float32)\n                y = to_categorical(word_encoder.transform(df[\"word\"].values),num_classes=340).astype(np.int32)\n                yield x, y","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"74b3838375876346e75249611161b7b0ae738101"},"cell_type":"markdown","source":"df_to_image_array_xd:\nreads dataframe and returns array of images from drawing column"},{"metadata":{"_uuid":"1f5859d5cceeda9283e2409feed041acbca538d4","trusted":false,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"def df_to_image_array_xd(df, size, lw=6, time_color=True):\n    df['drawing'] = df['drawing'].apply(ast.literal_eval)\n    x = np.zeros((len(df), size, size, 3))\n    for i, raw_strokes in enumerate(df.drawing.values):\n        x[i, :, :, 0] = draw_cv2(raw_strokes, size=size, lw=lw, time_color=time_color)\n        x[i, :, :, 1] = draw_cv2(raw_strokes, size=size, lw=lw, time_color=time_color)\n        x[i, :, :, 2] = draw_cv2(raw_strokes, size=size, lw=lw, time_color=time_color)\n    x = preprocess_input(x).astype(np.float32)\n    return x","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"6eb24647a229fc622351fab5b86d4aeee4f7d73c","trusted":false},"cell_type":"code","source":"train_datagen = image_generator_xd(batchsize=BATCH_SIZE, ks=range(NCSVS - 1), size=size)\ntrain_x, train_y = next(train_datagen)\nprint ('train x shape:{}'.format(train_x.shape))\nprint ('train y shape:{}'.format(train_y.shape))\nprint('train_x', train_x.dtype, train_x.min(), train_x.max())\nprint('train_y', train_y.dtype, train_y.min(), train_y.max())","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"4c54b0af486cae44a16340f0423f8a7455addecf"},"cell_type":"markdown","source":"Forming validation dataset"},{"metadata":{"_kg_hide-input":false,"_kg_hide-output":true,"_uuid":"b45cf8fb7c32bb7ede2abb3d3d047c2c4c95ec1f","trusted":false},"cell_type":"code","source":"valid_set = pd.read_csv(os.path.join(CSV_DIR, 'train_k{}.csv.gz'.format(NCSVS - 1)))\nvalid_x = df_to_image_array_xd(valid_set, size)\nvalid_y = to_categorical(word_encoder.transform(valid_set[\"word\"].values),num_classes=340).astype(np.int32)\nprint ('valid x shape:{}'.format(valid_x.shape))\nprint ('valid y shape:{}'.format(valid_y.shape))\nprint('valid_x', valid_x.dtype, valid_x.min(), valid_x.max())\nprint('valid_y', valid_y.dtype, valid_y.min(), valid_y.max())","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"dcfa18ec49972098a6f0336bbc7d7a015b112616","trusted":false,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"fig, m_axs = plt.subplots(4,4, figsize = (8, 8))\nrand_idxs = np.random.choice(range(train_x.shape[0]), size = 16, replace=False)\nfor c_id, c_ax in zip(rand_idxs, m_axs.flatten()):\n    test_arr = train_x[c_id, :, :, 0]  \n    c_ax.imshow(test_arr, cmap=plt.cm.gray)\n    c_ax.axis('off')\n    c_ax.set_title(word_encoder.classes_[np.argmax(train_y[c_id])])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"e682260fd5b4d3f697200e8ecc61a09ee82ff3df"},"cell_type":"markdown","source":"<a id=\"ch2.2\"></a>\n## 2.2 Models"},{"metadata":{"_uuid":"ddc594ac3f30f71ab414cc4ac126ea25ea50e5a0","trusted":false,"_kg_hide-input":true,"_kg_hide-output":true},"cell_type":"code","source":"def cnn(input_shape):\n    input_img = Input(input_shape)\n    conv0= Conv2D(256, (3, 3), activation='relu', padding='valid')(input_img) \n    pool0 = MaxPooling2D(pool_size=(2, 2))(conv0)\n    conv1= Conv2D(128, (3, 3), activation='relu', padding='valid')(pool0)\n    pool1 = MaxPooling2D(pool_size=(2, 2))(conv1)\n    conv2= Conv2D(64, (3, 3), activation='relu', padding='valid')(pool1) \n    pool2 = MaxPooling2D(pool_size=(2, 2))(conv2)\n    conv3 = Conv2D(32, (3, 3), activation='relu', padding='valid')(pool2) \n    pool3 = MaxPooling2D(pool_size=(2, 2))(conv3) \n    flat = Flatten()(pool3)\n    dense1 = Dense(680, activation='relu')(flat)\n    dense2 = Dense(len(word_encoder.classes_), activation = 'softmax')(dense1)\n    \n    model =  Model(inputs = input_img, outputs = dense2, name = 'Doodle_model')    \n    return model\n\ndef vgg16_model():\n    base_model = VGG16(include_top=False, weights='imagenet', input_shape=(128,128,3), pooling='avg')\n    x = base_model.output\n    x = Dense(512, activation='relu')(x)\n    output = Dense(340, activation='softmax')(x)\n    \n    model = Model(inputs=base_model.input, outputs=output)\n    \n    for layer in base_model.layers:\n        layer.trainable = False\n    \n    return model\n\ndef resnet_model():\n    base_model = ResNet50(include_top=False, weights='imagenet', input_shape=(128,128,3), pooling='avg')\n    x = base_model.output\n    x = Dense(512, activation='relu')(x)\n    output = Dense(340, activation='softmax')(x)\n    \n    model = Model(inputs=base_model.input, outputs=output)\n    \n    for layer in base_model.layers:\n        layer.trainable = False\n    \n    return model\n","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_uuid":"453cef80f296f1c065e60f2c03f0148bf658507b","trusted":false},"cell_type":"code","source":"def top_3_accuracy(y_true, y_pred):\n    return top_k_categorical_accuracy(y_true, y_pred, k=3)\n\ndef run_model(model):\n    model.compile(optimizer = 'adam', loss='categorical_crossentropy', metrics = ['categorical_accuracy', top_3_accuracy])\n    \n    checkpoint = ModelCheckpoint(\"model_weights.best.hdf5\", monitor='val_top_3_accuracy', verbose=1, save_best_only=True, mode='max', save_weights_only=True, period=1)\n    early = EarlyStopping(monitor=\"val_top_3_accuracy\", mode=\"max\", verbose=2,patience=8)\n    callbacks_list = [checkpoint, early]\n    \n    loss_history = [model.fit_generator(train_datagen,epochs=12,steps_per_epoch=STEPS_PER_EPOCH,\n                                    validation_data=(valid_x, valid_y),callbacks=callbacks_list,workers=1)]\n    model.load_weights(\"model_weights.best.hdf5\")\n    model.save('model.h5')\n\n    return loss_history\n\ndef display_plots(loss_history):    \n    epochs = np.concatenate([mh.epoch for mh in loss_history])\n    loss = np.concatenate([mh.history['loss'] for mh in loss_history])\n    val_loss  = np.concatenate([mh.history['val_loss'] for mh in loss_history])\n    \n    train_accuracy = np.concatenate([mh.history['top_3_accuracy'] for mh in loss_history])\n    test_accuracy = np.concatenate([mh.history['val_top_3_accuracy'] for mh in loss_history])\n    print ('train accuray: {}'.format(max(train_accuracy)))\n    print ('test accuray: {}'.format(max(test_accuracy)))\n    fig, (ax1, ax2) = plt.subplots(1,2, figsize = (30,10))\n\n    ax1.plot(epochs,train_accuracy, epochs,test_accuracy)\n    ax1.legend(['Training', 'Validation'])\n    ax1.set_xlabel('epoch')\n    ax1.set_ylabel('accuracy')\n    ax1.set_title('accuracy train vs validation')\n\n    ax2.plot(epochs,loss, epochs,val_loss)\n    ax2.legend(['Training', 'Validation'])\n    ax2.set_xlabel('epoch')\n    ax2.set_ylabel('loss')\n    ax2.set_title('loss train vs validation')\n    \ndef display_plots1(model = 'cnn'):\n    if model == 'cnn':\n        print(Image(filename='../input/cnn-plot/plot_cnn.png'))\n    elif model == 'resnet':\n        Image(filename='../input/cnn-plot/plot_cnn.png')\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"79a38954b751679f996dcbfd8cc1bdac8fdc68ec"},"cell_type":"markdown","source":"<a id=\"ch2.2.1\"></a>\n## 2.2.1 CNN"},{"metadata":{"_uuid":"3b5ee34578bcb942960073ee5715c53148a97c4b","trusted":false},"cell_type":"code","source":"def cnn(input_shape):\n    input_img = Input(input_shape)\n    conv0= Conv2D(256, (3, 3), activation='relu', padding='valid')(input_img) \n    pool0 = MaxPooling2D(pool_size=(2, 2))(conv0)\n    conv1= Conv2D(128, (3, 3), activation='relu', padding='valid')(pool0)\n    pool1 = MaxPooling2D(pool_size=(2, 2))(conv1)\n    conv2= Conv2D(64, (3, 3), activation='relu', padding='valid')(pool1) \n    pool2 = MaxPooling2D(pool_size=(2, 2))(conv2)\n    conv3 = Conv2D(32, (3, 3), activation='relu', padding='valid')(pool2) \n    pool3 = MaxPooling2D(pool_size=(2, 2))(conv3) \n    flat = Flatten()(pool3)\n    dense1 = Dense(680, activation='relu')(flat)\n    dense2 = Dense(len(word_encoder.classes_), activation = 'softmax')(dense1)\n    \n    model =  Model(inputs = input_img, outputs = dense2, name = 'Doodle_model')    \n    return model\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"7836a5aee5a04b7b6ef1cf1fc92b3eff2993f7ec","trusted":false},"cell_type":"code","source":"model = cnn(input_shape = train_x.shape[1:])\n# loss_history = run_model(model)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"_uuid":"14969e2f7fad8002e9f60145c5c298e205699460","trusted":false},"cell_type":"code","source":"print(\"Epoch 11/15 \\n 900/900 [==============================] - 512s 569ms/step - loss: 1.3648 - categorical_accuracy: 0.6381 - top_3_accuracy: 0.8320 - val_loss: 1.4366 - val_categorical_accuracy: 0.6875 - val_top_3_accuracy: 0.8750\")","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"_uuid":"241587f3cc66b50c02b1cd8ac7d51356a5d5d267","trusted":false},"cell_type":"code","source":"print(\"Train top_3_accuracy: 0.8320\")\nprint(\"Val top_3_accuracy: 0.8380\")","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_uuid":"7b90b80d9381b18a6efd71c44498a1c0acdaca03","trusted":false},"cell_type":"code","source":"Image(filename='../input/cnn-plot/plot_cnn.png')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"3f2cb760eb80652fa665e2df50c1bd33c3c1f0ed"},"cell_type":"markdown","source":"<a id=\"ch2.2.1\"></a>\n## 2.2.1 VGG16"},{"metadata":{"_uuid":"7b7fa224816e2673b23106fa9df58f0f0b65b08f","trusted":false},"cell_type":"code","source":"def vgg16_model():\n    base_model = VGG16(include_top=False, weights='imagenet', input_shape=(128,128,3), pooling='avg')\n    x = base_model.output\n    x = Dense(512, activation='relu')(x)\n    output = Dense(340, activation='softmax')(x)\n    \n    model = Model(inputs=base_model.input, outputs=output)\n    \n    for layer in base_model.layers:\n        layer.trainable = False\n    \n    return model","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"08ec40fb5c9b81d91c0e6e2621ea8286d78e7341","trusted":false},"cell_type":"code","source":"model = vgg16_model()\n# loss_history = run_model(model)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"_uuid":"0dffe386b3d911012670a89cfbc83f80524c32d7","trusted":false},"cell_type":"code","source":"print(\"Epoch 00015: val_top_3_accuracy did not improve\\n900/900 [==============================] - 120s 134ms/step - loss: 1.3381 - categorical_accuracy: 0.6430 - top_3_accuracy: 0.8329 - val_loss: 1.3772 - val_categorical_accuracy: 0.6500 - val_top_3_accuracy: 0.8240\")","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"26aa8a240c4066a10ad1aff0748ef97b18efc27a","trusted":false},"cell_type":"code","source":"print(\"Train top_3_accuracy: 0.8329\")\nprint(\"Val top_3_accuracy: 0.8240\")","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_uuid":"f0442fa7690268a315ccb6cc381e8078defb5dcc","trusted":false},"cell_type":"code","source":"Image(filename='../input/cnn-plot/vgg_plot.png')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"211de1cc1109a3de6937e3833e390568ce7ea664"},"cell_type":"markdown","source":"## 2.2.2 ResNet"},{"metadata":{"_uuid":"dc9c7652e5efd3d844e56dc455ea37892cddfba9","trusted":false},"cell_type":"code","source":"def resnet_model():\n    base_model = ResNet50(include_top=False, weights='imagenet', input_shape=(128,128,3), pooling='avg')\n    x = base_model.output\n    x = Dense(512, activation='relu')(x)\n    output = Dense(340, activation='softmax')(x)\n    \n    model = Model(inputs=base_model.input, outputs=output)\n    \n    for layer in base_model.layers:\n        layer.trainable = False\n    \n    return model\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"037a3572246e7f34e87e89a15c5a091d00b19ade","trusted":false},"cell_type":"code","source":"model = resnet_model()\n# loss_history = run_model(model)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"_uuid":"af5f1a3deb45c0f3602196a4e321677ec1ba3cdc","trusted":false},"cell_type":"code","source":"print(\"Epoch 00015: \\n 900/900 [==============================] - 327s 363ms/step - loss: 0.9652 - categorical_accuracy: 0.7304 - top_3_accuracy: 0.8939 - val_loss: 1.1290 - val_categorical_accuracy: 0.6570 - val_top_3_accuracy: 0.8710\")","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"_uuid":"78cb5b0cad2df944cf5929b3d39f722fd5501889","trusted":false},"cell_type":"code","source":"print(\"Train top_3_accuracy: 0.8939\")\nprint(\"Val top_3_accuracy: 0.8710\")","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"_uuid":"fa147f9c126f2bed3992125566dd44a249681c3a","trusted":true},"cell_type":"code","source":"Image(filename='../input/resnet-plot/resnet_plot.png')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"de3c7d1295c67845d306165d2c380536cf48b840"},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}