{"cells":[{"metadata":{"id":"qGBNi1zci-w_","colab_type":"text","_uuid":"8af5ff55a54d92ce5a375bfb8d31912ce4e46a29"},"cell_type":"markdown","source":"# **Quick Draw: Catch Doodle!**"},{"metadata":{"id":"h03Ib2BBlii5","colab_type":"text","_uuid":"d59111411a80503e248d9e7772f3d51e204a8f1a"},"cell_type":"markdown","source":"βQuick, Draw!β is a game created by Google. It's a game where one player is prompted to draw a picture of an object, and the other player needs to guess what it is. More details can be found [this post](https://towardsdatascience.com/quick-draw-the-worlds-largest-doodle-dataset-823c22ffce6b). \n\nThis project is for building an image classifier model that can handle noisy and sometimes incomplete drawings and perform well on classifying 50 different animals."},{"metadata":{"_uuid":"70c625e6175bd430e5f1d5e293593302d9cdb38f"},"cell_type":"markdown","source":"Table of contents:\n- [1. Load data](# 1. Load data)\n- [2. Let's draw doodle](# 2. Let's draw doodle)\n- [3. From strokes to Image](# 3. From strokes to Image)\n- [4. Let's call all friends here!](# 4. Let's call all friends here!)\n- [5. Modeling- CNN](# 5. Modeling- CNN)\n- [6. Plot the result](# 6. Plot the result)\n- [7. Modeling with ResNet50](# 7. Modeling with ResNet50)"},{"metadata":{"id":"y0Pwfsq6lt2t","colab_type":"code","colab":{},"trusted":true,"_uuid":"def8f0ba6328789bdec6ef879bf570976279a175"},"cell_type":"code","source":"import os\nimport pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nsns.set(style='white', context='notebook')\n\nnp.random.seed(36)","execution_count":null,"outputs":[]},{"metadata":{"id":"I30upr_MmGzH","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":34},"outputId":"7a9a0fe6-ff79-41fb-9595-61706a5d0dce","trusted":true,"_uuid":"ba13919e11b4c715882aa6762785deb90851fee7"},"cell_type":"code","source":"import ast\nimport cv2\nimport dask.bag as db\n\nfrom keras.models import Sequential\nfrom keras.layers import Conv2D, MaxPool2D, Flatten, Dense, Dropout\nfrom keras.callbacks import EarlyStopping, ReduceLROnPlateau \n\nfrom keras.applications.resnet50 import ResNet50\nfrom tensorflow.keras.applications.vgg19 import VGG19","execution_count":null,"outputs":[]},{"metadata":{"id":"fnMpTSjYmgmj","colab_type":"text","_uuid":"a2c243f13db6795ef2cfa4efa679100c3baa5e36"},"cell_type":"markdown","source":"# 1. Load data "},{"metadata":{"id":"pgnNKsYOqRMq","colab_type":"text","_uuid":"c75fce815002c9474c159ac8897a7c47ddfed2fd"},"cell_type":"markdown","source":"I'm going to use only the train_simplified.zip file for training. And we will use only the animal drawings among them. Cause...they are soooo cute :-) To load all 50 animals files automatically, I'll make a list for filenames. "},{"metadata":{"id":"LTVwyW6Dm6wS","colab_type":"code","colab":{},"trusted":true,"_uuid":"fb26e57e7097d2281a35c05e2003939a9e200ce7"},"cell_type":"code","source":"# list of animals \nanimals = ['ant', 'bat', 'bear', 'bee', 'bird', 'butterfly', 'camel', 'cat', 'cow',\n 'crab', 'crocodile', 'dog', 'dolphin', 'dragon', 'duck', 'elephant', 'fish',\n 'flamingo', 'frog', 'giraffe', 'hedgehog', 'horse', 'kangaroo', 'lion',\n 'lobster', 'monkey', 'mosquito', 'mouse', 'octopus', 'owl', 'panda',\n 'parrot', 'penguin', 'pig', 'rabbit', 'raccoon', 'rhinoceros', 'scorpion',\n 'sea turtle', 'shark', 'sheep', 'snail', 'snake', 'spider', 'squirrel',\n 'swan', 'teddy-bear', 'tiger', 'whale', 'zebra']","execution_count":null,"outputs":[]},{"metadata":{"id":"ljFCMf9Mm6sE","colab_type":"text","_uuid":"43fe0abd514d2a73eefff2d702dfaf14162f29bc"},"cell_type":"markdown","source":"Before uploading all the files at once, let's take a test with .csv first."},{"metadata":{"id":"VQMqjMQirD6G","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":195},"outputId":"5438a40b-f4a6-4c60-bf45-53e805daf6b9","trusted":true,"_uuid":"db5a777b1c9fdbd402468739be53efd2b52d3418"},"cell_type":"code","source":"dir_path = '../input/train_simplified/'\ndf = pd.read_csv(dir_path + animals[0] + '.csv')\ndf.head()","execution_count":null,"outputs":[]},{"metadata":{"id":"zhFzN0JsrDNU","colab_type":"text","_uuid":"d02386d8b5c533d5b61697897167da0bf085feb9"},"cell_type":"markdown","source":"`drawing` is the stroke values, which is telling the drawing of animals. We need to exchage this data into image data later. `word` indicates the result of drawings or animals. `recognized` means whether the drawing was understood as a certain object or not. Let's take only the 10 rows per animals and filter the unrecogizable drawing out. "},{"metadata":{"id":"qq39wshOm6n1","colab_type":"code","colab":{},"trusted":true,"_uuid":"fbac2d9fb250d4afd706baf6a65aea0e5ed4a8f6"},"cell_type":"code","source":"am = pd.DataFrame(columns = df.columns)\n\nfor i in range(len(animals)):\n filename = dir_path + animals[i] + '.csv'\n df = pd.read_csv(filename, nrows = 100)\n df = df[df.recognized == True]\n am = am.append(df)","execution_count":null,"outputs":[]},{"metadata":{"id":"swIHz_Qpm6ja","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":34},"outputId":"0a37eb3b-2ec6-44e8-9eb2-7c0231d69c94","trusted":true,"_uuid":"c21294dfeada9ffa5d862401323274958e3d6c3b"},"cell_type":"code","source":"# Check\nam.word.nunique()","execution_count":null,"outputs":[]},{"metadata":{"id":"_VOhJVPwm59l","colab_type":"text","_uuid":"75e9a3017842222be34a2954a84342dac4fcdacc"},"cell_type":"markdown","source":"# 2. Let's draw doodle "},{"metadata":{"id":"SmP5Gq8Lm56G","colab_type":"text","_uuid":"7f7bc0ee9297276efbfd66f2d575b785c999c950"},"cell_type":"markdown","source":"Before data proprocessing and modeling, let's see how people drew animals. The image information can be found at `drawing` but in order to make it visual, we need some steps of processing. Let's take only 100 data for an example."},{"metadata":{"id":"rmSZKbZcm52N","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":195},"outputId":"bb34cea2-a746-4e51-b774-023d918db418","trusted":true,"_uuid":"7a31cea7c59e0f7d94db29ac853fca05f8d90ba2"},"cell_type":"code","source":"# Sampling only 100 examples\nex = am.sample(100)\nex.head()","execution_count":null,"outputs":[]},{"metadata":{"id":"z1l33IzMxIDE","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":70},"outputId":"cf6c769c-ca01-46aa-f109-8809c8ea2fe6","trusted":true,"_uuid":"0e5a856825f0d251cb994fb3e91cf8d7bfae59dc"},"cell_type":"code","source":"ex.drawing.head(1).values # -> strings","execution_count":null,"outputs":[]},{"metadata":{"id":"GCASPbq05mIb","colab_type":"text","_uuid":"8647b9acc6c4d81fbd7bd0ad29933addcc1a0833"},"cell_type":"markdown","source":"Take a note at the front of the result. \n
\narray(**[ ' [[ 34, 41 ... ]]]) **\n
\nThis indicates this data is strings, not list."},{"metadata":{"id":"0GIIyiSSxH-f","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":70},"outputId":"805b4869-e583-4a24-83c6-b34e110d9c86","trusted":true,"_uuid":"e3c3f9586135a7376f73e957ba93243ebf4b2911"},"cell_type":"code","source":"ex.drawing.head(1).map(ast.literal_eval).values # -> list","execution_count":null,"outputs":[]},{"metadata":{"id":"N9VHnQ3NxH6l","colab_type":"text","_uuid":"8405df4cb8781b08ecfb8a2857846fcc0df02f82"},"cell_type":"markdown","source":"We can see that the dypes changed. Now we are able to use it for plotting the strokes. Let's convert the other data as well."},{"metadata":{"id":"T2YiIlqMxH2P","colab_type":"code","colab":{},"trusted":true,"_uuid":"554b1a219b0b0332b6a870be5a96ae2be09b3fa9"},"cell_type":"code","source":"# Convert to list\nex['drawing'] = ex.drawing.map(ast.literal_eval)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"c475abe5dfb098ce1a01a9f1362637fa4265d649"},"cell_type":"markdown","source":"Now we are going to meet our lovely cats, dogs and pandas. Take a look at the code below."},{"metadata":{"id":"M14T8IziBckQ","colab_type":"code","colab":{},"trusted":true,"_uuid":"cc55db7fb9d020c0342ed1fd7b27e62c59c7779c"},"cell_type":"code","source":"# Plot the strokes \n# fig, axs = plt.subplots(nrows = 10, ncols = 10, figsize = (10, 8))\n\n# for index, col in enumerate(ex.drawing):\n# ax = axs[index//10, index%10]\n# for x, y in col:\n# ax.plot(x,-np.array(y), lw = 3)\n# ax.axis('off')\n \n# plt.show()","execution_count":null,"outputs":[]},{"metadata":{"id":"B6VQKKfK7pax","colab_type":"text","_uuid":"4302e429e2c41cdbf4306f5e9b611509a19f95da"},"cell_type":"markdown","source":"The concept of visualization can be seen complex at the first sight but it's not true. First we will get 100 grids and put the drawing in the grids one by one. `enumerate()` will return the index and column values. Let's take one example to understand step by step. "},{"metadata":{"id":"3G_bbFNd8fdN","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":843},"outputId":"f6dbac47-ff68-4aea-f730-d946538f5795","trusted":true,"_uuid":"482196af9e93bcd5a5a3e3963db59751989b454e","_kg_hide-output":true},"cell_type":"code","source":"# Understanding enumerate\nfor index, col in enumerate(ex.drawing[:12]):\n print('The index is ', index)\n print('Position will be ({}, {})'.format(index//10, index%10))\n print('The strokes are ', col)\n print('===========')","execution_count":null,"outputs":[]},{"metadata":{"id":"6uYaV-tY8fWz","colab_type":"text","_uuid":"68d24d213a15d83c1e2458462f78a9199ee32316"},"cell_type":"markdown","source":"As you can see above `enumerate()` brings us the index and the values one by one. So we are going to plot the values at the given values by col. "},{"metadata":{"id":"08eWLxW5_3nx","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":706},"outputId":"a20958b9-71e6-4cb1-f23c-5a46b48a1eac","trusted":true,"_uuid":"a20de768efa2ac1235544d2b3d8a6a9f1108c385","_kg_hide-output":true},"cell_type":"code","source":"for index, col in enumerate(ex.drawing[:2]):\n print('==================================')\n for x, y in col:\n print('X is {}'.format(x))\n print('Y is {}'.format(y))\n print('-----------------------')","execution_count":null,"outputs":[]},{"metadata":{"id":"ZPanWv4R_3hj","colab_type":"text","_uuid":"c11b12bdbee8d00a80f74775b0f841692bdd1ea4"},"cell_type":"markdown","source":"So what we are going to do is ploting these x, y values just like what we've been doing with graphs. Now let's apply all this into one shot and finally meet our lovely friends."},{"metadata":{"id":"JaZkNy1t7eaM","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":485},"outputId":"323deb5b-c10b-4f13-b773-d0a3e1364042","trusted":true,"_uuid":"9865f5c3f1b02ca0cd6beca58961e27ec3f6bd84"},"cell_type":"code","source":"# Plot the strokes \nfig, axs = plt.subplots(nrows = 10, ncols = 10, figsize = (10, 8))\n\nfor index, col in enumerate(ex.drawing):\n ax = axs[index//10, index%10]\n for x, y in col:\n ax.plot(x,-np.array(y), lw = 3)\n ax.axis('off')\n \nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"id":"PhScLnjcxHyZ","colab_type":"text","_uuid":"f417fb5fbebf8b13dae39e9978219ada0f6ab108"},"cell_type":"markdown","source":"OMG πππ.....This is so funny."},{"metadata":{"id":"DtUmBUVIxHuP","colab_type":"text","_uuid":"d62fca572c925ecf32e767e04c4cb09b02c1a47b"},"cell_type":"markdown","source":"# 3. From strokes to Image "},{"metadata":{"id":"ju5Yzsn4xHqO","colab_type":"text","_uuid":"a86c08561e61c82a48da4d5804cd8b46baf62329"},"cell_type":"markdown","source":"Now the next step is transforming all these drawings into image data. Like I said above, the data isn't in the form of image data. We have to covert it into numpy array format. I'm going to make a function for this. "},{"metadata":{"id":"e2DbtfLTCXpa","colab_type":"code","colab":{},"trusted":true,"_uuid":"878b19c8f097eae82ecde61db1b83b3b70cf6607"},"cell_type":"code","source":"im_size = 64\nn_class = len(animals)","execution_count":null,"outputs":[]},{"metadata":{"id":"78QkBhf1CXj9","colab_type":"code","colab":{},"trusted":true,"_uuid":"96b61211dedee37dd1ee80b25aa0e38a5f12bdbf"},"cell_type":"code","source":"# define a function converting drawing to image data\ndef draw_to_img(strokes, im_size = im_size):\n\n fig, ax = plt.subplots() # plot the drawing as we did above\n for x, y in strokes:\n ax.plot(x, -np.array(y), lw = 10)\n ax.axis('off')\n \n fig.canvas.draw() # update a figure that has been altered\n A = np.array(fig.canvas.renderer._renderer) # converting them into array\n \n plt.close('all')\n plt.clf()\n \n A = (cv2.resize(A, (im_size, im_size)) / 255.) # image resizing to uniform format\n \n return A","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a81b778e825c58c11d0aa21742b7649f8b941c47"},"cell_type":"markdown","source":"All the things we discussed at the second section are put inside the `draw_to_img()` function. Let's try it with an example."},{"metadata":{"id":"DlXMbPs9CXfb","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":364},"outputId":"a516e24e-c781-4911-d739-b4a74ffc7c8f","trusted":true,"_uuid":"ae97a1a6d5fe78bdd0be3c4e666825db920b76af"},"cell_type":"code","source":"X = ex.drawing.values\nimage = draw_to_img(X[1])\nplt.imshow(image)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"a7ab77bad64db95585bacc1972b9ef1f33c3b3b2"},"cell_type":"code","source":"image.shape","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"1e2f90721327d383c45beed09a0a4b6263fbb290"},"cell_type":"markdown","source":"The image has 4 channels and we can also check each channels separately."},{"metadata":{"id":"x3fYgj_4CXbd","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":171},"outputId":"66696ffd-1899-4a11-8616-749e4189d0af","trusted":true,"_uuid":"2afe3dc4799f0dea23c96d8dc70e27e285f07f2e"},"cell_type":"code","source":"# Channel selection \nfig, axs = plt.subplots(nrows = 1, ncols = 4, figsize = (10, 10))\n\nfor i in range(4):\n ax = axs[i]\n ax.imshow(image[:, :, i])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8fcbc2f2e1b3e253f33b813ada96c182ffe08147"},"cell_type":"markdown","source":"We will make the input image shape as `(im_size, im_size, 3)`, which means it has only one channels. Therefore we''ll take only the last channel here."},{"metadata":{"id":"uIWOUSbZCXXN","colab_type":"code","colab":{},"trusted":true,"_uuid":"1fb76b40bd0cba7a12bada160164e55d0f197696"},"cell_type":"code","source":"# redefine\ndef draw_to_img(strokes, im_size = im_size):\n fig, ax = plt.subplots() # plot the drawing as we did above\n for x, y in strokes:\n ax.plot(x, -np.array(y), lw = 10)\n ax.axis('off')\n \n fig.canvas.draw() # update a figure that has been altered\n A = np.array(fig.canvas.renderer._renderer) # converting them into array\n \n plt.close('all')\n plt.clf()\n \n A = (cv2.resize(A, (im_size, im_size)) / 255.) # image resizing to uniform format\n\n return A[:, :, :3] # drop the last one ","execution_count":null,"outputs":[]},{"metadata":{"id":"RuXkXu_bENWY","colab_type":"code","colab":{"base_uri":"https://localhost:8080/","height":34},"outputId":"490b62b3-5787-434f-eea0-4bbf8ab4766c","trusted":true,"_uuid":"f1a8ecd878d9edf34701f59277ea8e523f05e19d"},"cell_type":"code","source":"image = draw_to_img(X[1])\nplt.imshow(image)\nprint(image.shape)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"b9e89023778406824a120da79dbe2c929a560a36"},"cell_type":"markdown","source":"# 4. Let's call all friends here! "},{"metadata":{"id":"eNgCGy54CXOz","colab_type":"text","_uuid":"ab2dcc870c10bac4e64075b4c1b6b1e9dcf215b6"},"cell_type":"markdown","source":"Now we are ready to apply what we've been doing so far into the entire dataset."},{"metadata":{"id":"-KZdE5n5Dq-3","colab_type":"code","colab":{},"trusted":true,"_uuid":"437b5006b7feaee3695f3d4d88d63d2219ea68a4","_kg_hide-output":true,"scrolled":true},"cell_type":"code","source":"n_samples = 500\nX_train = np.zeros((1, im_size, im_size, 3))\ny = []\n\nfor a in animals:\n print(a)\n filename = dir_path + a + '.csv'\n df = pd.read_csv(filename, usecols=['drawing', 'word'], nrows=n_samples) # import the data in chunks\n df['drawing'] = df.drawing.map(ast.literal_eval) # convert strings into list\n X = df.drawing.values\n \n img_bag = db.from_sequence(X).map(draw_to_img) # covert strokes into array\n X = np.array(img_bag.compute()) \n X_train = np.vstack((X_train, X)) # concatenate to get X_train \n \n y.append(df.word)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"515c9b01fdafa85dcc720b6a2d3367a95ab43ac5"},"cell_type":"markdown","source":"As I just stack the array, the dimension of `X_train` has one more values than it's expected. Therefore we'll drop the first layer. "},{"metadata":{"trusted":true,"_uuid":"990b0e37b40a2be48e9c5146248ce6f251f1b453"},"cell_type":"code","source":"# The dimension of X_train\nX_train.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"3d8f467246600fb70210e88e038561a30de25e74"},"cell_type":"code","source":"# Drop the first layer\nX_train = X_train[1:, :, :, :]\nX_train.shape","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"89015d966e7ed0521830c7b101b82d2964632762"},"cell_type":"markdown","source":"Don't forget to encoding the categorical data before modeling fitting"},{"metadata":{"id":"74wyWChzCXBH","colab_type":"code","colab":{},"trusted":true,"_uuid":"e81926d46ac436e94f4c80efee3599b05e612062"},"cell_type":"code","source":"# Encoding \ny = pd.DataFrame(y)\ny = pd.get_dummies(y)\ny_train = np.array(y).transpose()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"6a22cb11ccabf2f604f3702b2f60d27b10df0ff4"},"cell_type":"code","source":"# Check the result\nprint(\"The input shape is {}\".format(X_train.shape))\nprint(\"The output shape is {}\".format(y_train.shape))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a0c61725fd502d9f802c7da8e04a8430443d267b"},"cell_type":"markdown","source":"Now let's combine the X_train and y_train again. This is for splitting the data into the trainning set and validation set. "},{"metadata":{"trusted":true,"_uuid":"1efe23a79ec02b7e96a1917d834fa9664deeaa07"},"cell_type":"code","source":"# Reshape X_train\nX_train_2 = X_train.reshape((X_train.shape[0], im_size*im_size*3))\n\n# Concatenate X_train and y_train\nX_y_train = np.hstack((X_train_2, y_train))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"b1028de50a79b7db8eb89e795c785e2554406293"},"cell_type":"markdown","source":""},{"metadata":{"trusted":true,"_uuid":"a4289ade8d6559b358b72b7644b70da1538a7869"},"cell_type":"code","source":"# Random shuffle\nnp.random.shuffle(X_y_train)\na = im_size*im_size*3\ncut = int(len(X_y_train) * .1)\nX_val = X_y_train[:cut, :a]\ny_val = X_y_train[:cut, a:]\nX_train = X_y_train[cut:, :a]\ny_train = X_y_train[cut:, a:]\n\n# Reshape X_train back to (64, 64)\nX_train = X_train.reshape((X_train.shape[0], im_size, im_size, 3))\nX_val = X_val.reshape((X_val.shape[0], im_size, im_size, 3))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"108eb8ac5eb9f7dc34cc46ffde99a357cf5479f7"},"cell_type":"markdown","source":"Check the final shape of train and validation set."},{"metadata":{"id":"4p817k7QCV6z","colab_type":"code","colab":{},"trusted":true,"_uuid":"3ad96c39343320d23901bb909629ea040dfca73c"},"cell_type":"code","source":"# Check the result\nprint(\"The input shape of train set is {}\".format(X_train.shape))\nprint(\"The input shape of validation set is {}\".format(X_val.shape))\nprint(\"The output shape of train set is {}\".format(y_train.shape))\nprint(\"The output shape of validation set is {}\".format(y_val.shape))","execution_count":null,"outputs":[]},{"metadata":{"id":"Rpa3P_jKCVpg","colab_type":"code","colab":{},"trusted":false,"_uuid":"a6651a7b452a9298a551e740bc78cde522d82fc5"},"cell_type":"markdown","source":"# 5. Modeling- CNN "},{"metadata":{"_uuid":"e23ac8274fb90307281afdb01b4f166aa1986549"},"cell_type":"markdown","source":"I'm going to start with the basic CNN model as a baseline. And then compare the results with ResNet and VGG19"},{"metadata":{"trusted":true,"_uuid":"832e0fec930b6cdd58b03bda3391eaecf2a114fe"},"cell_type":"code","source":"n_epochs = 10\nbatch_size = 500\n\n# Initialize\nmodel = Sequential()\n\n# ConvNet_1\nmodel.add(Conv2D(32, kernel_size = 3, input_shape = (im_size, im_size, 3), padding = 'same', activation = 'relu'))\nmodel.add(MaxPool2D(2, strides = 2))\n# Dropout\nmodel.add(Dropout(.2))\n\n# ConvNet_2\nmodel.add(Conv2D(64, kernel_size = 3, activation = 'relu'))\nmodel.add(MaxPool2D(2, strides = 2))\n# Dropout\nmodel.add(Dropout(.2))\n\n# ConvNet_3\nmodel.add(Conv2D(64, kernel_size = 3, activation = 'relu'))\nmodel.add(MaxPool2D(2, strides = 2))\n# Dropout\nmodel.add(Dropout(.2))\n\n# Flattening\nmodel.add(Flatten())\n\n# Fully connected\nmodel.add(Dense(680, activation = 'relu'))\n\n# Dropout\nmodel.add(Dropout(.5))\n\n# Final layer\nmodel.add(Dense(n_class, activation = 'softmax'))\n\n# Compile\nmodel.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"3ceeef819897da0259b1c2dca1997d89dd3e0aab"},"cell_type":"code","source":"model.summary()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"cc9e075eca123bb4b6f03697be65e6e89859b437"},"cell_type":"markdown","source":"I'll also add callbacks not to get overfitting. "},{"metadata":{"trusted":true,"_uuid":"94a111971516477feaf78a01d2a73bf18b0e7d2e"},"cell_type":"code","source":"# Early stopper\nstopper = EarlyStopping(monitor='val_top_3_accuracy', mode='max', patience = 3)\n\n# Learning rate reducer\nreducer = ReduceLROnPlateau(monitor = 'val_acc',\n patience = 3,\n verbose = 1,\n factor = .5,\n min_lr = 0.00001)\n\ncallbacks = [stopper, reducer]","execution_count":null,"outputs":[]},{"metadata":{"id":"nPfeaYhfCVkm","colab_type":"code","colab":{},"trusted":true,"_uuid":"a420ec319be5663e01e0dbb30e8fea1cb6f26381"},"cell_type":"code","source":"# Fitting baseline\nhistory = model.fit(X_train, y_train, epochs = n_epochs, batch_size = batch_size, \n validation_split = .2, verbose = True)","execution_count":null,"outputs":[]},{"metadata":{"id":"w0StYUBYxHkW","colab_type":"code","colab":{},"trusted":false,"_uuid":"5f8c827ab4bd1e93af1b53a664d9eb1b6f74a098"},"cell_type":"markdown","source":"# 6. Plot the result "},{"metadata":{"_uuid":"4377a949b150373befb3c0d45673735e7289c23f"},"cell_type":"markdown","source":"Let's see how well our model is trained."},{"metadata":{"trusted":true,"_uuid":"d795c4e74c419f09ae9b46b7609ee9b9bf9a253c"},"cell_type":"code","source":"# Train and validation curves\nfig, (ax1, ax2) = plt.subplots(2, 1)\nax1.plot(history.history['loss'], color = 'b', label = 'Train Loss')\nax1.plot(history.history['val_loss'], color = 'm', label = 'Valid Loss')\nax1.legend(loc = 'best')\n\nax2.plot(history.history['acc'], color = 'b', label = 'Train Accuracy')\nax2.plot(history.history['val_acc'], color = 'm', label = 'Valid Accuracy')\nax2.legend(loc = 'best')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"23a668596ce0da09b9d57c3cce6ade92aa08a43e"},"cell_type":"markdown","source":"# 7. Modeling with ResNet50"},{"metadata":{"_uuid":"c5d091b9a251a705c875b550a789397042ac9634"},"cell_type":"markdown","source":"It's seem not good. Let's try other pre-trained model."},{"metadata":{"trusted":true,"_uuid":"d9d4083205548c4162460544b39a08279dee1b8c"},"cell_type":"code","source":"# ResNet50 Application \nmodel_r = ResNet50(include_top = True, weights= None, input_shape=(im_size, im_size, 3), classes = n_class)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-output":true,"trusted":true,"_uuid":"25512dfaac5dac4777681aaa5f1a5432126ad691"},"cell_type":"code","source":"model_r.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])\nmodel_r.summary()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"307d4b7935f00685f7a3678cafca542db41bbfe6"},"cell_type":"code","source":"n_epochs = 5\nbatch_size = 50","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"0ef2c6768a0de3326abd20b78dd8f5a841ab235e"},"cell_type":"code","source":"# Fitting ResNet50\nhistory_r = model_r.fit(X_train, y_train, epochs = n_epochs, batch_size = batch_size, \n validation_split = .2, verbose = True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"958f952437dafe0a06142e5006de2c01d130b741"},"cell_type":"code","source":"# Train and validation curves with ResNet50\nfig, (ax1, ax2) = plt.subplots(2, 1)\nax1.plot(history_r.history['loss'], color = 'b', label = 'Train Loss')\nax1.plot(history_r.history['val_loss'], color = 'm', label = 'Valid Loss')\nax1.legend(loc = 'best')\n\nax2.plot(history_r.history['acc'], color = 'b', label = 'Train Accuracy')\nax2.plot(history_r.history['val_acc'], color = 'm', label = 'Valid Accuracy')\nax2.legend(loc = 'best')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"9f22b47875dc2bb90f00aa5e229d7fb8cda66d62"},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"55a155f41b8bb1ef3cf8ab344528565606d37a63"},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"colab":{"name":"Doodle.ipynb","version":"0.3.2","provenance":[],"toc_visible":true},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"accelerator":"GPU","language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}