{"cells":[{"metadata":{"_uuid":"8d661cf624e064e02de14b0fc5929a1346da0733"},"cell_type":"markdown","source":"**Quick Draw!\n**\nThis note book is just for tracking my own progress. Sharing with the though it might be helpful for other starters like me.  If you are looking for serious calculation please jump to the section with heading \"Serious stuff\""},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport matplotlib.pyplot as plt\nimport ast\nimport cv2\nimport gc\n%matplotlib inline\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory\n\nimport os\nprint(os.listdir(\"../input\"))\n# Any results you write to the current directory are saved as output.\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"61bf1cc9d27c678665ce001e82b2695755bfc431"},"cell_type":"markdown","source":"**Attributes and their explanation**: Modified from https://github.com/googlecreativelab/quickdraw-dataset#the-raw-moderated-dataset \n1. key_id: Just a ID to identify each image. \n1. countrycode: two letter country code. More: https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2 \n1. drawing: vectors of the doodle.\n1. recognized: Whether the word was recognized by the game. If it is not recongnized by the game we can still use this labelled data for training the model.\n1. time-stamp: Time \n1. word: the response class\n\n[ \n  [  // First stroke \n    [x0, x1, x2, x3, ...],\n    [y0, y1, y2, y3, ...],\n    [t0, t1, t2, t3, ...]\n  ],\n  [  // Second stroke\n    [x0, x1, x2, x3, ...],\n    [y0, y1, y2, y3, ...],\n    [t0, t1, t2, t3, ...]\n  ],\n  ... // Additional strokes\n]\n\nWhere x and y are the pixel coordinates, and t is the time in milliseconds since the first point. x and y are real-valued while t is an integer. The raw drawings can have vastly different bounding boxes and number of points due to the different devices used for display and input.\n\nIn simplified data time was removed. "},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"# Exploring train csv files to know the structure/dimensions of the data\n\ndata_train = pd.read_csv('../input/train_simplified/sleeping bag.csv',\n                   index_col='key_id',\n                   nrows=5)\ndata_train.head(5)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"608bf9beb5a50ce9e90438b6ebbc2a65da4e39be"},"cell_type":"markdown","source":"**Test Data Set**"},{"metadata":{"trusted":true,"_uuid":"53ab4ef1484a47519e7565ab702e300108fa9b5f"},"cell_type":"code","source":"# Exploring test csv files to know the structure/dimensions of the data\n\ndata_test = pd.read_csv('../input/test_simplified.csv',\n                   index_col='key_id',\n                   nrows=5)\ndata_test.head(5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"23c7c13302800bec5163a7d6a0f331af52708e3b"},"cell_type":"code","source":"# Looking at one Training data in details\n# max value is 255. \n# import ast\nwords = data_train['word'].tolist()\ndrawings = [ast.literal_eval(pts) for pts in data_train[0:4]['drawing'].values] # python list","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"44c1111e1e4aaa7dc3a916c30bb5e3ebf9df60dd","scrolled":true},"cell_type":"code","source":"plt.figure(figsize=(10, 10))\nfor i, drawing in enumerate(drawings):\n    plt.subplot(220 + (i+1))\n    for x, y in drawing:\n        plt.plot(x, y, marker='.')\n        plt.title(words[i])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"4f1e578a351d8672f9f15eb06402c1a6d26c8bbf"},"cell_type":"markdown","source":"**Creating Binary images from drawings:**\nAbove images are still strokes, not images. So now we will convert these strokes to images. We will make images of size 255 by 255 for at least 100. One important resource is: Bresenham's Line Algorithm\n http://www.roguebasin.com/index.php?title=Bresenham%27s_Line_Algorithm#Python, I modified the code from Miha Skalic for converting image. Thanks Miha."},{"metadata":{"trusted":true,"_uuid":"dc7127453e940d861aa26802b7115471d2f78576"},"cell_type":"code","source":"# import cv2\ndef draw2img(drawing, Xsize=256, Ysize=256, lw = 4):\n    '''\n    converts drawing to image\n    input:\n    drawing: the drawing, it takes one at a time.\n    Xsize, Ysize are the pixel size of the image.\n    lw is the line width of the output image strokes.\n    Note: This function uses cv2, so import cv2\n    Example: >>>a = draw2img(drawings[3], 256, 256)\n             >>>plt.imshow(a)\n    '''\n    fig, ax = plt.subplots()\n    for x,y in drawing:\n        ax.plot(x, y ,linewidth=lw) #  marker='.', See which line width is better, <4 might be good\n    ax.axis('off')\n    fig.canvas.draw()    \n    X = np.array(fig.canvas.renderer._renderer)\n    plt.close(fig)\n    # image resizing. Original X is of various size due to strokes variable's length\n    return (cv2.resize(X, (Xsize,Ysize)) / 255.)[::-1]\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"2fe2cfc7f4071084b6dc948d3c518f7ee8df8270"},"cell_type":"code","source":"image = draw2img(drawings[3], 256, 256)\nplt.imshow(image)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c4f06c48bdb7d62fb41b19f599979d2987620692"},"cell_type":"code","source":"plt.figure(figsize=(10, 10))\nfor i in range(4):\n    plt.subplot(140+i+1)\n    plt.imshow(image[:, :, i])\n    plt.title(i)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5732ed2eaa3d4c3c8dde841b63c945af86ee2de4"},"cell_type":"markdown","source":"From the above we see that color information is kept on the third dimension. Which is not necessary. So we will make a black and white one or at least a 2D one. Image is becoming a 3D one because we have used plotting function earlier. During plot function it created color images. Each stroke got a different color. to get rid of it. we can suppress it to black and white color.  Here in this constructed binary image '1' is bright spot and \"0\" is for dark spot. So our final b/w image we shoud be able to see only 1/0 in a 2D tensor. "},{"metadata":{"trusted":true,"_uuid":"a8fe6873e53abcbc6b24752e63c5c144e2456c70"},"cell_type":"code","source":"img = image[:,:,1] # taking only green channel\nplt.imshow(img)\nimg.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"0ec2bbba55cc1d834c4c46388cff1aaca8f949ac"},"cell_type":"code","source":"# modifying the drawing to image function\n\ndef draw2img(drawing, shape =(32,32), lw = 4):\n    '''\n    converts drawing to image\n    input:\n    drawing: the drawing, it takes one at a time.\n    Xsize, Ysize are the pixel size of the image.\n    lw is the line width of the output image strokes.\n    Note: This function uses cv2, so import cv2\n    Example: >>>a = draw2img(drawings[3], 256, 256)\n             >>>plt.imshow(a)\n    '''\n    fig, ax = plt.subplots()\n    #drawing = eval(drawing)\n    for x,y in drawing:\n        ax.plot(x, y,'g',  marker='.', linewidth = lw) #  marker='.', See which line width is better, <4 might be good\n    ax.axis('off')\n    fig.canvas.draw()    \n    X = np.array(fig.canvas.renderer._renderer)\n    plt.close(fig)\n    # image resizing. Original X is of various size due to strokes variable's length\n    temp = (cv2.resize(X, shape) / 255.)[::-1]\n    return temp[:,:,1] # only green channel, as we have drawn with green","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"44a958d2cd41d55515e05d76e17810a3c465f5bf"},"cell_type":"code","source":"img = draw2img(drawings[3], (256, 256), lw =1)\nplt.imshow(img)\nimg.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"c40e078f15072b1294a3b7a3f87423626d7c4546"},"cell_type":"code","source":"'''\nfilename = list()\nshape = list()\nfor file in os.listdir('../input/train_simplified/'):\n    train = pd.read_csv('../input/train_simplified/' + file, index_col='key_id')\n    filename.append(file.split('.')[0].replace(\" \", \"_\"))\n    shape.append(train.shape[0]) \nsize_train = pd.DataFrame(data = shape, index =filename , columns  = ['entry'])\n\nsize_train.to_csv('training_sample_per_class.csv')\nsize_train.sort_values(by = 'entry')\n'''\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"5b229a08fedb01802ec53bfe74225b58b723cf56"},"cell_type":"markdown","source":"Panda has 113613 entry and snow man has 340029 entry. Rest of the classes are with sample size in between these two numbers. \n"},{"metadata":{"_uuid":"39f8c70da664843041d0fe4acea670ad4c709fe2"},"cell_type":"markdown","source":"**So far, and what next<br>**\nNow that we have a way to convert our strokes to images of any size we want, we will focus on how to use them for the next steps for image classification. Since all the files are training folder coming seperately for each class of image, we need to concatenate them to a master trainig data. Also before jmping into the serious business lets, make a toy training set of. In this toy we will not train on the whole trainng set, rather a subset of the trainig set. But we will train for all the classes. This toy will give us opportunity to simulate the whole thing without too much computations. It will be faster way of developing the whole system. In the final state we can train on whole data set with longer time. <br>\nSteps we will follow here, <br>\n* Cleaning\n* Making Trainin set\n* Build model using tensorflow convnet\n* Train the model\n* Evaluate the model\n\nFew things we will keep in mind in this immediate next step. These are to save time and resource at this development stage.\n* drawing to image: we will use low resolution as 32*32 pixels jus to see.\n* Training sample size 1000 per class, just to play\n* We will take all 340 classes of images. "},{"metadata":{"_uuid":"d207cae81aa922ea161e16dffa4ca9509d946c3e"},"cell_type":"markdown","source":"* Cleaning and preparing training data frame: "},{"metadata":{"trusted":true,"_uuid":"3e1397cc6a5e8f2b51a059cb395353b50b02ff80"},"cell_type":"code","source":"# this is defined again because of reading training set again, where we will not use ast.literal_eval\ndef draw2img(drawing, shape =(32,32), lw = 1):\n    '''\n    converts drawing to image\n    input:\n    drawing: the drawing, it takes one at a time.\n    Xsize, Ysize are the pixel size of the image.\n    lw is the line width of the output image strokes.\n    Note: This function uses cv2, so import cv2\n    Example: >>>a = draw2img(drawings[3], 256, 256)\n             >>>plt.imshow(a)\n    '''\n    fig, ax = plt.subplots()\n    drawing = eval(drawing)\n    for x,y in drawing:\n        ax.plot(x, y,'g',  marker='.', linewidth = lw) #  marker='.', See which line width is better, <4 might be good\n    ax.axis('off')\n    fig.canvas.draw()    \n    X = np.array(fig.canvas.renderer._renderer)\n    plt.close(fig)\n    # image resizing. Original X is of various size due to strokes variable's length\n    temp = (cv2.resize(X, shape) / 255.)[::-1]\n    return temp[:,:,1] # only green channel, as we have drawn with green","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true,"_uuid":"c0c0b20d0871dd1d69ec45d7dfa0d11e9f21a542"},"cell_type":"code","source":"'''\nimport time\nstart_time = time.time()\nprint(f\"--- {(time.time() - start_time)} seconds ---\") \n'''\n# It took me 26 secs to load 1000 *340 images (1000 per class), possibly 100MB of data\ntrain = pd.DataFrame()\nfor file in os.listdir('../input/train_simplified/'):\n    train = train.append(pd.read_csv('../input/train_simplified/' + file, index_col='key_id', nrows=10))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"ebfa3b574568a1bfc97cd742a115d4b21a32cca4"},"cell_type":"code","source":"train.head(5)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"da3c4692e6cfffa49e0a4a8efc151634db35dc84"},"cell_type":"markdown","source":"**Cleaning and mapping** : word column need to be cleaned with '_' and drawing need to be transformed to 32*32 pixels. To save on memory I will remove the orignal drawing. "},{"metadata":{"trusted":true,"_uuid":"bd2e1cfc3d64e8ab9ab221819093d00f8fe476b1"},"cell_type":"code","source":"train['word']= train['word'].apply(lambda x: x.replace(' ', '_'))\ntrain.head(4)\n#train['word'].values","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"e697f6dd2b44ebf580769cef6e488628fd0374f7"},"cell_type":"code","source":"# most time consuing, part, think of getting rid of it. \ntrain['drawing'] = train['drawing'].apply(draw2img) # for this training set eval() necessary in draw2img","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"25e92fa8c133a7ae6486d0d138cfb2e4a814294b"},"cell_type":"code","source":"plt.imshow(train['drawing'].values[0])","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"174c7c7baeee1f7cd662d2767168506c35b34b0c"},"cell_type":"markdown","source":"So far it is clear that a lot data processing require. And here the size of trainig set is HUGE. It is good for the Neural Nets but constrain is that we don't have enough memory to process them all. Thinking what can be done for this. May be small batch generator function, which will randomly choose a small set of training set and then proces on them before learning anything. I will try this for now.   In doing differet proces steps, we may need to use some of the disk space for svaing some files. In this case /working directory might be helpful."},{"metadata":{"_uuid":"b318fcb440605987379f5eaca2e95d92abd0d628"},"cell_type":"markdown","source":"**Serious Stuff**"},{"metadata":{"_uuid":"1b807e9dd71b631b90cbd06fa3e171d4c113da81"},"cell_type":"markdown","source":"Following is a list of parameters, by chaning them we can scale up our calculation. I have tested up to 3 classes with large data set. Please change these following parameter as you feel comfortable with your memory. If you find any better solution for memory please share in the comment. "},{"metadata":{"trusted":true,"_uuid":"f646d035621979ff641ff577c185dd191da80cba"},"cell_type":"code","source":"# parameters\nshape = (32, 32)\ntraining_classes = 2 # how many class we are training now\ntrain_size = 60 # how many images per class considering for training+evaluation","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"7dd27cd00dd4ba5b8beb34af9bf22104c26ead34"},"cell_type":"code","source":"# modifying the drawing to image function\n# use it as train['drawing'] = train['drawing'].apply(draw2img)\ndef draw2img(drawing, shape = shape):\n    fig, ax = plt.subplots()\n    #drawing = eval(drawing)\n    for x,y in drawing:\n        ax.plot(x, y,'g',  marker='.') #  marker='.',\n    ax.axis('off')\n    fig.canvas.draw()    \n    X = np.array(fig.canvas.renderer._renderer)\n    plt.close(fig)\n    # image resizing. Original X is of various size due to strokes variable's length\n    temp = (cv2.resize(X, shape) / 255.)[::-1]\n    return temp[:,:,1].astype('int8') # only green channel, as we have drawn with green, try bool\n\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"ce46bcfc195f05adce4a123cfa0835d5c5eea450"},"cell_type":"markdown","source":"Reading and processing simultaneously. This is to save memory we will selectively read columns from the csv files. And at the same time data type will be assigned as needed. For example default data type for number is float64 which cost 8 bytes. Just to save a non negative integer of the range 0 to 255 we need int8. So reading and saving only required column will give better edge on memory. "},{"metadata":{"trusted":true,"_uuid":"b2997099c2108ad8969692bf6e8238022c5b5e23"},"cell_type":"code","source":"import time\nstart_time = time.time()\n\ntrain = pd.DataFrame()\ni = 0\nlabels = dict()\nfor file in os.listdir('../input/train_simplified/'):\n    print(f\"Reading...{file}.....{i*100/340}% complete\")\n    label = file.split('.')[0].replace(' ', '_')\n    labels.update({i:label})\n    \n    temp = pd.read_csv('../input/train_simplified/' + file, nrows=train_size, \n                                    usecols = ['drawing', 'word'])\n    # processing data\n    temp['drawing'] = [ast.literal_eval(pts) for pts in temp['drawing'].values]   \n    temp['drawing'] = temp['drawing'].apply(draw2img)\n    \n    #global label encoding\n    temp['word']    = np.int16(i)\n    train = train.append(temp)\n    \n    i = i+1\n    if i==training_classes: \n        break\n    if i%10==0:\n        print(f\"Time elasped in reading: {(time.time() - start_time)} seconds ---\") \n\n\nprint(f\"Total Time elasped in reading: {(time.time() - start_time)} seconds ---\") \n\n# labesl is the dictionary to convert lebels from number to the actual words\ndf_labels = pd.DataFrame(index = list(labels.keys()), \n                         data = list(labels.items()),\n                         columns = ['Class Code', 'Word'] )\ndf_labels.to_csv('labels.csv')\ndel labels","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"e89ad523aa98f6e6d726fa152f82ced0f86ca7d6"},"cell_type":"markdown","source":"Preparing data"},{"metadata":{"trusted":true,"_uuid":"c92a6866b84ef6e0233ac1e7d4fe33006e9d5077"},"cell_type":"code","source":"# preparing x_train and y_train\nx = np.array(train['drawing'])\ny = np.array(train['word'])\n# each row of x, y is a input \n\n# y_train to onehot encoding for making them as useful for output softmax layer\n'''\narray([[1., 0., 0., 0., 0., 0., 0., 0., 0., 0.],\n       [1., 0., 0., 0., 0., 0., 0., 0., 0., 0.],\n'''\ny =y.reshape(-1, 1)  # making it a 2d array like [[1], [1], ]\nfrom sklearn import preprocessing\nenc = preprocessing.OneHotEncoder()\nenc.fit(y)\ny = enc.transform(y).toarray()\n\n#del train\ngc.collect()\n\n# test train split\n\nfrom sklearn.model_selection import train_test_split\nx_train, x_test, y_train, y_test = train_test_split(\n    x, y, test_size=0.1, random_state=101)\n\nprint(\"Taking care of dimensions--------------------\")\nprint(f\"shape of x_train: {x_train.shape}\")\nprint(f\"shape of x_test: {x_test.shape}\")\nprint(f\"shape of y_train: {y_train.shape}\")\nprint(f\"shape of y_test: {y_test.shape}\")\nprint(f\"shape of image: {x_train[1].shape}\")\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"1fee93964f4625df3738555a3a94e13319fea0fb"},"cell_type":"markdown","source":"Genrating class for batch wise processing of data"},{"metadata":{"trusted":true,"_uuid":"41ce21c6c60adc39d5dcc26b4ea5f50da007dea2"},"cell_type":"code","source":"# class for batch processing\nclass qdHelper():\n    def __init__(self):\n        self.i= 0\n        self.x_train = x_train\n        self.x_test = x_test        \n        self.y_train = y_train\n        self.y_test = y_test\n    def setupimage(self):\n        self.x_train = np.vstack([a for a in x_train]).reshape(x_train.shape[0],32,32,1)\n        self.x_test = np.vstack([a for a in x_test]).reshape(x_test.shape[0],32,32,1)\n\n    def next_batch(self, batch_size):\n        x = self.x_train[self.i:self.i+batch_size]\n        y = self.y_train[self.i:self.i+batch_size]\n        self.i = (self.i + batch_size) % len(self.x_train)\n        return x, y\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"2ccdc4d355dd9e8de084de4a955e018e1aac4eae"},"cell_type":"code","source":"# instantiation\nqd = qdHelper() \nqd.setupimage()","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"a3b34832d8cfb1a81c04f8e9bd8753c3eed0bcbb"},"cell_type":"markdown","source":"**TENSORFLOW VARIABLE AND GRAPH SETUP**"},{"metadata":{"trusted":true,"_uuid":"25ac1236e522947a1696391515f05df19d216b62"},"cell_type":"code","source":"import tensorflow as tf\n# creating placeholder\nx = tf.placeholder(tf.float32, shape = [None, 32,32,1])\ny_true = tf.placeholder(tf.int8, shape = [None, training_classes])\nhold_prob = tf.placeholder(tf.float32)\n\n# Helper Function\ndef init_weights(shape):\n    init_random_dist = tf.truncated_normal(shape, stddev=0.1)\n    return tf.Variable(init_random_dist)\n\ndef init_bias(shape):\n    init_bias_vals = tf.constant(0.1, shape = shape)\n    return tf.Variable(init_bias_vals)\n\n# 2d conv without with strides 1 pixel in each dimension\n# with padding same to keep the image dimension same.\ndef conv2d(x,W):\n    return tf.nn.conv2d(x,W, strides = [1,1,1,1],padding= 'SAME')\n# defining a 2 by 2 maximum poooling\ndef max_pool(x):\n    return tf.nn.max_pool(x, ksize= [1,2,2,1],\n                         strides = [1,2,2,1],\n                         padding = 'SAME')\n# this layer is Wx+b , b is of of the last dimension\ndef convlayer(input_x, shape):\n    W = init_weights(shape)\n    b = init_bias([shape[3]])\n    return tf.nn.relu(conv2d(input_x, W)+b)\n\ndef full_layer(input_layer, output_size):\n    input_size = int(input_layer.get_shape()[1])\n    W = init_weights([input_size, output_size])\n    b = init_bias([output_size])\n    return tf.matmul(input_layer, W)+b\n\n# creating graph\n# shape of W, width, height, channel_in, channel_out\nconv1 = convlayer(x, shape = [4,4,1,32])\nconv1_pool = max_pool(conv1)\n\n# previous layer ouput 32 channels, so it will be the input\n# in this layere, and we want 64 output. \nconv2 = convlayer(conv1_pool, shape = [4,4,32,64])\nconv2_pool = max_pool(conv2)\n\n# creatiing flattened features\n# images got 2 max pool layer of 2 by 2, so got a shape of 32 by 32 now\nconv2_flat = tf.reshape(conv2_pool, [-1, 8*8*64])\n\n# fully connected layer\nlayer1 = tf.nn.relu(full_layer(conv2_flat, 512))\n# drop out layer\nlayer1_drop = tf.nn.dropout(layer1, keep_prob= hold_prob)\n\ny_pred = full_layer(layer1_drop,training_classes )\n\n# loss\ncross_entropy = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(\n    labels=y_true,logits=y_pred))\n\n# optimizer\noptimizer = tf.train.AdamOptimizer(learning_rate=0.001)\ntrain = optimizer.minimize(cross_entropy)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d7fe7ea3b2659dd2531bfc81e10bdc3e0b1c9fd6"},"cell_type":"markdown","source":"Computing the Graph"},{"metadata":{"trusted":true,"_uuid":"5b515d0eb511ccc49185f7f314c27ea5c8cc9cbd"},"cell_type":"code","source":"# variable initilazation\ninit = tf.global_variables_initializer()\n\n#Graph session\nwith tf.Session() as sess:\n    sess.run(init)\n\n    for i in range(10000):\n        batch = qd.next_batch(100)\n        sess.run(train, feed_dict={x: batch[0], y_true: batch[1], hold_prob: 0.5})\n        \n        # PRINT OUT A MESSAGE EVERY 100 STEPS\n        if i%100 == 0:            \n            print('Currently on step {}'.format(i))\n            print('Accuracy is:')\n            # Test the Train Model\n            matches = tf.equal(tf.argmax(y_pred,1),tf.argmax(y_true,1))\n\n            acc = tf.reduce_mean(tf.cast(matches,tf.float32))\n            print(f\"done on {i}\")\n\n            print(sess.run(acc,feed_dict={x:qd.x_test,y_true:qd.y_test,hold_prob:1.0}))\n            print('\\n')              \n\n","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"decc1f572ff1192bb9e971c56f5519e6fcc56f85"},"cell_type":"markdown","source":"This is for today. I will be keep updating this kernel daily basis. Please stay tuned to get update. "}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}