{"cells":[{"metadata":{"_uuid":"331fbee8c5b4b49ee05eefc98bba3b60fa3a2073"},"cell_type":"markdown","source":"> **Problem overview**\n\n\"Quick, Draw!\" was released as an experimental game to educate the public in a playful way about how AI works. The game prompts users to draw an image depicting a certain category, such as ”banana,” “table,” etc. The game generated more than 1B drawings, of which a subset was publicly released as the basis for this competition’s training set. That subset contains 50M drawings encompassing 340 label categories.\n\nSounds fun, right? Here's the challenge: since the training data comes from the game itself, drawings can be incomplete or may not match the label. You’ll need to build a recognizer that can effectively learn from this noisy data and perform well on a manually-labeled test set from a different distribution.\n\nYour task is to build a better classifier for the existing Quick, Draw! dataset. By advancing models on this dataset, Kagglers can improve pattern recognition solutions more broadly. This will have an immediate impact on handwriting recognition and its robust applications in areas including OCR (Optical Character Recognition), ASR (Automatic Speech Recognition) & NLP (Natural Language Processing)."},{"metadata":{"trusted":true,"_uuid":"9da7fb8f7ae88df5934a3aff7811972fcd139542"},"cell_type":"code","source":"# import python standard library\nimport json, gc, os\n\n# import data manipulation library\nimport numpy as np\nimport pandas as pd\n\n# import data visualization library\nimport matplotlib.pyplot as plt\nfrom tqdm import tqdm\n\n# import image processing library\nimport cv2\n\n# import tensorflow model class\nfrom tensorflow import keras\nfrom tensorflow.keras.applications import MobileNet\nfrom tensorflow.keras.layers import Conv2D, Dense, Dropout, Flatten, MaxPooling2D\nfrom tensorflow.keras.models import load_model, Sequential\n\n# import sklearn model selection\nfrom sklearn.model_selection import train_test_split\n\n# import tensorflow model evaluation classification metrics\nfrom tensorflow.keras.metrics import top_k_categorical_accuracy","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# numpy options\nnp.random.seed(seed=58)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"1b1e96984cf9718ba06e595da35f4d0df3d37955"},"cell_type":"markdown","source":"> **Acquiring training and testing data**\n\nWe start by acquiring the training and testing datasets into Pandas DataFrames."},{"metadata":{"trusted":true},"cell_type":"code","source":"# list training and testing data directory\nos.listdir('../input/')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# acquiring training and testing data\ndf_train = pd.read_csv('../input/quick-draw-doodle-recognition-challenge-shufflecsv/train_k0.csv.gz', nrows=100)\ndf_test = pd.read_csv('../input/quickdraw-doodle-recognition/test_simplified.csv', nrows=2)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# visualize head of the training data\ndf_train.head(n=5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"f7b57430b942c3e431767d5a4fda64f901ce7de7"},"cell_type":"code","source":"# visualize tail of the testing data\ndf_test.tail(n=5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# dataframe columns name\nnames = ['countrycode', 'drawing', 'key_id', 'recognized', 'timestamp', 'word']\n\n# class files and dictionary\nfiles = sorted(os.listdir('../input/quickdraw-doodle-recognition/train_simplified/'), reverse=False)\nclass_dict = {file[:-4].replace(\" \", \"_\"): i for i, file in enumerate(files)}\nclassreverse_dict = {v: k for k, v in class_dict.items()}\n\n# combine training and testing dataframe\ndf_train = df_train.drop(['shuffle'], axis=1)\ndf_train['datatype'], df_test['datatype'] = 'training', 'testing'\ndf_train = df_train[['key_id', 'countrycode', 'drawing', 'datatype', 'word', 'recognized']]\ndf_test['word'], df_test['recognized'] = '', True\ndf_data = pd.concat([df_train, df_test], ignore_index=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"6e42537a2084ef606afd908c407f8a08a3ca377e"},"cell_type":"code","source":"# data dimensions\nchunksize = 680\nimg_size = 64\nnum_channels = 1\nnum_classes = 340\nnum_shuffles = 50\n\n# flat dimensions\nimg_size_flat = img_size * img_size * num_channels","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"e864d4e8e05dfdb7dd2187b8063708ae0b1069d0"},"cell_type":"markdown","source":"> **Feature exploration, engineering and cleansing**\n\nHere we generate descriptive statistics that summarize the central tendency, dispersion and shape of a dataset’s distribution together with exploring some data."},{"metadata":{"trusted":true,"_uuid":"1a1211c5b0be9137f42b569c4f382f59092e3ffb"},"cell_type":"code","source":"def drawplot(draw: list, label: list, figsize: tuple = (4, 3), ncols: int = 5, nrows: int = None) -> plt.figure:\n    \"\"\" Return a draw image plot applied for an image data in vector format.\n    \n    Args:\n        draw (list): The draw image data.\n        label (list): The label of an image data.\n        figsize (tuple): The matplotlib figure size width and height in inches. Default to (4, 3).\n        ncols (int): The number of columns for axis in the figure. Default to 5.\n        nrows (int): The number of rows for axis in the figure. Default to None.\n    \n    Returns:\n        plt.figure: The plot figure.\n    \"\"\"\n    \n    if nrows is None: nrows = (len(label) - 1) // ncols + 1\n    \n    fig, axes = plt.subplots(figsize=(figsize[0]*ncols , figsize[1]*nrows), ncols=ncols, nrows=nrows)\n    axes = axes.flatten()\n    for i in label.index:\n        for j in range(len(draw[i])): _ = axes[i - label.index[0]].plot(draw[i][j][0], draw[i][j][1])\n        axes[i - label.index[0]].invert_yaxis()\n        axes[i - label.index[0]].set_title(label[i])\n    return fig","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def imageplot(image: list, label: list, size: tuple, figsize: tuple = (4, 3), ncols: int = 5, nrows: int = None) -> plt.figure:\n    \"\"\" Return an image plot applied for an image data in grayscale picture (m, n) format, RGB picture (m, n, 3) format and RGBA picture (m, n, 4) format.\n    \n    Args:\n        image (list): The image data.\n        label (list): The label of an image data.\n        size (tuple): The tuple of an image size.\n        figsize (tuple): The matplotlib figure size width and height in inches. Default to (4, 3).\n        ncols (int): The number of columns for axis in the figure. Default to 5.\n        nrows (int): The number of rows for axis in the figure. Default to None.\n    \n    Returns:\n        plt.figure: The plot figure.\n    \"\"\"\n    \n    if nrows is None: nrows = (len(label) - 1) // ncols + 1\n    \n    fig, axes = plt.subplots(figsize=(figsize[0]*ncols , figsize[1]*nrows), ncols=ncols, nrows=nrows)\n    axes = axes.flatten()\n    _ = [axes[i].imshow(image[i].reshape(size), interpolation='spline16') for i in range(len(label))]\n    return fig","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def draw2pixel(draw: list) -> np.ndarray:\n    \"\"\" Return a draw image to pixel image data.\n    \n    Args:\n        draw (list): The draw image data.\n    \n    Returns:\n        np.ndarray: The draw image to pixel image data.\n    \"\"\"\n    \n    image, xmin, xmax, ymin, ymax = np.zeros((256, 256)), 255, 0, 255, 0\n    for k, stroke in enumerate(draw):\n        for i in range(len(stroke[0])-1):\n            xmin, xmax = min(xmin, stroke[0][i], stroke[0][i + 1]), max(xmax, stroke[0][i], stroke[0][i + 1])\n            ymin, ymax = min(ymin, stroke[1][i], stroke[1][i + 1]), max(ymax, stroke[1][i], stroke[1][i + 1])\n            color = (255.0 - min(k, 10) * 13) / 255.0\n            _ = cv2.line(image, (stroke[0][i], stroke[1][i]), (stroke[0][i + 1], stroke[1][i + 1]), color=color, thickness=5)\n    if xmin == xmax: xmin, xmax = 0, 255\n    if ymin == ymax: ymin, ymax = 0, 255\n    return cv2.resize(image[ymin:ymax, xmin:xmax], (img_size, img_size))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def feature_extraction(df_data: pd.DataFrame) -> pd.DataFrame:\n    \"\"\" Return the feature exploration, engineering and cleansing.\n    \n    Args:\n        df_data (pd.DataFrame): The data to extract features.\n    \n    Returns:\n        pd.DataFrame: The extracted features dataframe.\n    \"\"\"\n    \n    # feature extraction: drawing\n    df_data['drawing'] = df_data['drawing'].apply(lambda x: json.loads(x))\n    \n    # feature extraction: word\n    df_data['word'] = df_data['word'].apply(lambda x: -1 if x == '' else class_dict[x.replace(' ', '_')])\n    \n    # feature extraction: drawing to pixel\n    df_data['pixel'] = df_data['drawing'].apply(lambda x: draw2pixel(x))\n    \n    return df_data","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def feature_extraction2(df_data: pd.DataFrame) -> pd.DataFrame:\n    \"\"\" Return the feature exploration, engineering and cleansing.\n    \n    Args:\n        df_data (pd.DataFrame): The data to extract features.\n    \n    Returns:\n        pd.DataFrame: The extracted features dataframe.\n    \"\"\"\n    \n    # feature extraction: remove countrycode, drawing and datatype\n    df_data = df_data.drop(['countrycode', 'drawing', 'datatype'], axis=1)\n    \n    return df_data","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# feature extraction: step 1\ndf_data = feature_extraction(df_data)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"08a1a92fd1e687febb5b5c764905d922b0e8c9a9"},"cell_type":"code","source":"# feature exploration: image\n_ = drawplot(df_data.loc[:19, 'drawing'], df_data.loc[:19, 'word'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# feature exploration: image\n_ = imageplot(df_data.loc[:19, 'pixel'], df_data.loc[:19, 'word'], (img_size, img_size))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"42a0501ba107e48239a34f1a2efae102feb96088"},"cell_type":"markdown","source":"After extracting all features, it is required to convert category features to numerics features, a format suitable to feed into our Machine Learning models."},{"metadata":{"trusted":true},"cell_type":"code","source":"# feature extraction: step 2\ndf_data = feature_extraction2(df_data)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"0d6fbde07625fa3208ac73aa0382c9112258bb94"},"cell_type":"code","source":"# describe data dataframe\ndf_data.describe(include='all')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"25789bb4ebeab293d0e777fe11ad3895eb4041f5"},"cell_type":"code","source":"# verify dtypes object\ndf_data.info()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# memory clean-up\ndel df_data, df_train, df_test\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true,"_uuid":"b326915278461c00eada39393214cc5e523f4052"},"cell_type":"markdown","source":"> **Model, predict and solve the problem**\n\nNow, it is time to feed the features to Machine Learning models."},{"metadata":{"trusted":true},"cell_type":"code","source":"def train_generator() -> tuple:\n    \"\"\" Return training data generator.\n    \n    Returns:\n        tuple: The training data tuple.\n    \"\"\"\n    \n    while True:\n        for k in np.random.permutation(range(num_shuffles - 1)):\n            for df_data in pd.read_csv('../input/quick-draw-doodle-recognition-challenge-shufflecsv/train_k%d.csv.gz' %k, chunksize=chunksize):\n                # feature extraction: drawing\n                df_data['drawing'] = df_data['drawing'].apply(lambda x: json.loads(x))\n                \n                # feature extraction: word\n                df_data['word'] = df_data['word'].apply(lambda x: -1 if x == '' else class_dict[x.replace(' ', '_')])\n                \n                # feature extraction: drawing to pixel\n                x = np.zeros((df_data.shape[0], img_size, img_size, 1))\n                for i, drawing in enumerate(df_data['drawing'].values): x[i, :, :, 0] = draw2pixel(drawing)\n                y = keras.utils.to_categorical(df_data['word'], num_classes=num_classes)\n                yield x, y\n\n# training data generator\ngen_train = train_generator()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# testing (validating) data\ndf_data = pd.read_csv('../input/quick-draw-doodle-recognition-challenge-shufflecsv/train_k%d.csv.gz' %(num_shuffles - 1), nrows=34000)\n\n# feature extraction: drawing\ndf_data['drawing'] = df_data['drawing'].apply(lambda x: json.loads(x))\n\n# feature extraction: word\ndf_data['word'] = df_data['word'].apply(lambda x: -1 if x == '' else class_dict[x.replace(' ', '_')])\n\n# feature extraction: drawing to pixel\nx_validate = np.zeros((df_data.shape[0], img_size, img_size, 1))\nfor i, drawing in enumerate(df_data['drawing'].values): x_validate[i, :, :, 0] = draw2pixel(drawing)\ny_validate = keras.utils.to_categorical(df_data['word'], num_classes=num_classes)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# memory clean-up\ndel df_data\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"aabeca409f35daa3f52d693417faf7150598ebf4"},"cell_type":"markdown","source":"A TensorFlow graph consists of the following parts which will be detailed below:\n\n* Placeholder variables used for inputting data to the graph.\n* Variables that are going to be optimized so as to make the convolutional network perform better.\n* The mathematical formulas for the convolutional network.\n* A loss measure that can be used to guide the optimization of the variables.\n* An optimization method which updates the variables."},{"metadata":{"trusted":true},"cell_type":"code","source":"def top_3_categorical_accuracy(y_true: np.ndarray, y_pred: np.ndarray) -> float:\n    \"\"\" Return top 3 categorical accuracy.\n    \n    Args:\n        y_true (np.ndarray): The ground truth (correct) labels.\n        y_pred (np.ndarray): The predicted labels.\n    \n    Returns:\n        float: The top 3 categorical accuracy.\n    \"\"\"\n    \n    return top_k_categorical_accuracy(y_true, y_pred, k=3)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# mobilenet model setup\nmodel_mobilenet = MobileNet(input_shape=(img_size, img_size, 1), alpha=1.0, dropout=1e-3, weights=None, classes=num_classes)\nmodel_mobilenet.summary()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# mobilenet model setup\nmodel_mobilenet.compile(optimizer='adam', loss='categorical_crossentropy', metrics=[top_3_categorical_accuracy])\n\n# mobilenet model fit\nhist = model_mobilenet.fit_generator(gen_train, steps_per_epoch=800, epochs=96, verbose=2, validation_data=(x_validate, y_validate))\nmodel_hist = pd.DataFrame(hist.history)\n\n# mobilenet model metrics\nmodel_mobilenet_score = model_mobilenet.evaluate(x_validate, y_validate, verbose=1)\nprint('mobilenet\\n  top 3 categorical accuracy score: %0.4f' %model_mobilenet_score[1])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# plot the model history\nfig, axes = plt.subplots(figsize=(20, 10), ncols=1, nrows=2)\naxes = axes.flatten()\nmodel_hist.plot(y='top_3_categorical_accuracy', kind='line', ax=axes[0])\nmodel_hist.plot(y='val_top_3_categorical_accuracy', kind='line', ax=axes[0])\nmodel_hist.plot(y='loss', kind='line', ax=axes[1])\nmodel_hist.plot(y='val_loss', kind='line', ax=axes[1])\nfor axis in axes: axis.set_xlabel('epoch')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# mobilenet model save\nmodel_mobilenet.save('model_mobilenet.h5')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# memory clean-up\ndel x_validate, y_validate\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"42f6db4b24c5365f339bc6e5f642af1a2b5791f7"},"cell_type":"markdown","source":"> **Supply or submit the results**\n\nOur submission to the competition site Kaggle is ready. Any suggestions to improve our score are welcome."},{"metadata":{"trusted":true},"cell_type":"code","source":"# acquiring testing data\ndf_test = pd.read_csv('../input/quickdraw-doodle-recognition/test_simplified.csv')\n\n# feature extraction: drawing\ndf_test['drawing'] = df_test['drawing'].apply(lambda x: json.loads(x))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"c79fee3dc126cc6801bd32fa49fa2a4cab6948f9","trusted":true},"cell_type":"code","source":"# prepare testing data and compute the observed value\nx_test = np.zeros((df_test.shape[0], img_size, img_size, 1))\nfor i, drawing in enumerate(df_test['drawing'].values): x_test[i, :, :, 0] = draw2pixel(drawing)\ny_test = np.argsort(-model_mobilenet.predict(x_test, verbose=1))[:, 0:3]\ndf_word = pd.DataFrame({'top 1': y_test[:, 0], 'top 2': y_test[:, 1], 'top 3': y_test[:, 2]})\ndf_word = df_word.replace(classreverse_dict)\ndf_word['submission'] = df_word['top 1'] + ' ' + df_word['top 2'] + ' ' + df_word['top 3']","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"9585c714a5bd80669d713892673f6239181285f2","trusted":true},"cell_type":"code","source":"# submit the results\nout = pd.DataFrame({'key_id': df_test['key_id'], 'word': df_word['submission']})\nout.to_csv('submission.csv', index=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# visualize head of the submitted results\nout.head(n=5)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}