{"cells":[{"metadata":{"_uuid":"4a33e396d43e595c4d98bfeb76321a43ae81a058"},"cell_type":"markdown","source":"I wrote this kernel to mess around with the sketch data and share my experience so far.  After exploring the website and the data, I'll use a very basic CNN to classify sketches. This model gets 0.60 on the Public LB when run with 6000 recognized images per class.\n\n## Quick, Draw\nOk - I have to say this is kind of a fun thing. You go to the [Quick Draw](https://quickdraw.withgoogle.com/#)  website and agree to sketch several common objects. The host then gives you the object labels one by one, and you have 20 seconds to draw each one. If the AI guesses your sketch (that is, associates it with training set items of the same label) you get a check mark and move on. At the end you get something like this:\n\n<img src=\"https://s3.amazonaws.com/nonwebstorage/Screenshot+from+2018-09-26+22-44-21.png\" alt=\"drawing1\" width=\"600\"/>\n  \n<p><br></p>\nNotice that it didn't like my bird. Apparently it looked more like a dragon or a diving board(??) or a mosquito.  They're nice enough to show you how other people draw birds so you maybe learn how to draw better.\n<img src=\"https://s3.amazonaws.com/nonwebstorage/Screenshot+from+2018-09-26+22-45-21.png\" alt=\"drawing1\" width=\"600\"/>"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"markdown","source":"## Training Images\nWe can look at a few sketches and then see what the training data contains overall. I'll use an adaptation of Inversion's 'Getting Started' kernel."},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"import os\nimport re\nfrom glob import glob\nfrom tqdm import tqdm\nimport numpy as np\nimport pandas as pd\nimport ast\nimport matplotlib.pyplot as plt\n%matplotlib inline","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"376d2827a7f11e80f6591bfc439a9d5558ef10fc"},"cell_type":"markdown","source":"Here's what the training data looks like. This data frame is actually a concatenation of two rows from each of 6 csv files in the training set."},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"5090c701f9121207b67e13a0ae3f239514dff217","scrolled":true},"cell_type":"code","source":"fnames = glob('../input/train_simplified/*.csv')\ncnames = ['countrycode', 'drawing', 'key_id', 'recognized', 'timestamp', 'word']\ndrawlist = []\nfor f in fnames[0:6]:\n    first = pd.read_csv(f, nrows=10) # make sure we get a recognized drawing\n    first = first[first.recognized==True].head(2)\n    drawlist.append(first)\ndraw_df = pd.DataFrame(np.concatenate(drawlist), columns=cnames)\ndraw_df","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"60f35cc05283cc4f4871cec37bc873f5c9706042"},"cell_type":"markdown","source":"And here are some nice sketches..."},{"metadata":{"_kg_hide-input":true,"trusted":true,"_uuid":"2be57621533d0e0aa75108febde01261d2444c37"},"cell_type":"code","source":"evens = range(0,11,2)\nodds = range(1,12, 2)\ndf1 = draw_df[draw_df.index.isin(evens)]\ndf2 = draw_df[draw_df.index.isin(odds)]\n\nexample1s = [ast.literal_eval(pts) for pts in df1.drawing.values]\nexample2s = [ast.literal_eval(pts) for pts in df2.drawing.values]","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true,"scrolled":false,"_uuid":"0a1aaca9129ad3748e597993c90ab65c5fdefd5b"},"cell_type":"code","source":"labels = df2.word.tolist()\nfor i, example in enumerate(example1s):\n    plt.figure(figsize=(6,3))\n    \n    for x,y in example:\n        plt.subplot(1,2,1)\n        plt.plot(x, y, marker='.')\n        plt.axis('off')\n\n    for x,y, in example2s[i]:\n        plt.subplot(1,2,2)\n        plt.plot(x, y, marker='.')\n        plt.axis('off')\n        label = labels[i]\n        plt.title(label, fontsize=10)\n\n    plt.show()  ","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"cf8bc6e13cba236c6632cff33d9729d6eff728f4"},"cell_type":"markdown","source":"## The Quick, Draw Data Repository\nThere is a direct data source outside of Kaggle that seems pretty useful. The main link is a [GitHub Repository](https://github.com/googlecreativelab/quickdraw-dataset) that leads to the data in several formats, including Numpy bitmap files. Each file in the dataset covers a specific type of sketch and is in the shape of a long 1d array. Here are a few.\n\nUpdate: The test set for this project doesn't have premade files in the numpy bitmap format. When you apply your model to test arrays converted from matplotlib, there is a loss due to the different conversion process. The best results for an image-based model are had by using the same processing for train and test (no surprise there I guess). So I'd say these are good for exploration but consider the other files for better results."},{"metadata":{"trusted":true,"_uuid":"73e317f188154d2b908f75ee8aeafc06da86db42","scrolled":true,"_kg_hide-input":false},"cell_type":"code","source":"# # commented out to save memory\n# import urllib\n\n# LABELS = np.array(['baseball', 'bowtie', 'clock', 'hand', 'hat'])\n# for b in LABELS:\n#     url = \"https://storage.googleapis.com/quickdraw_dataset/full/numpy_bitmap/{}.npy\".format(b)\n#     urllib.request.urlretrieve(url, \"{}.npy\".format(b))\n#     nb = np.load(\"{}.npy\".format(b))\n#     print(\"\\n Class '{0}' has {1} examples of {2}x{2} images\".format(b, nb.shape[0], int(nb.shape[1]**0.5)))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8b9f5b85771add0d7672f7611a62e4668a28453c"},"cell_type":"markdown","source":"## Convolutional Neural Network (CNN)\nNext we'll build an image classifier. There are some resources in the repository I mentioned earlier showing how people have used the data. One of those resources is a CNN just like what you see here. \n\nThe biggest usage of resources seems to be converting the drawings to images. You can stick with a stroke-based model or go down the conversion path. Going that way requires watching data usage and managing space limits - deep learning on Kaggle can be like building a ship in a bottle:) \n\nUPDATE: I'll be experimenting with a separate kernel to use a fit_generator like in Beluga's kernel and others. This kernel has the most I could squeeze into memory with simple fit and predict."},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true,"_uuid":"adc697e3e58ef810c0576614ae95fa889bfb3345"},"cell_type":"code","source":"%reset -f ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"779113b4b0ef5fbe50ddda59813b019b3b769b58"},"cell_type":"code","source":"#%% import\nimport os\nfrom glob import glob\nimport re\nimport ast\nimport numpy as np \nimport pandas as pd\nfrom PIL import Image, ImageDraw \nfrom tqdm import tqdm\nfrom dask import bag\n\nimport tensorflow as tf\nfrom tensorflow import keras\nfrom tensorflow.keras.models import Sequential\nfrom tensorflow.keras.layers import Dense, Dropout, Flatten\nfrom tensorflow.keras.layers import Conv2D, MaxPooling2D, AveragePooling2D, BatchNormalization, concatenate, Input\nfrom tensorflow.keras.metrics import top_k_categorical_accuracy\nfrom tensorflow.keras.callbacks import ModelCheckpoint, ReduceLROnPlateau, EarlyStopping","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"44e5965471a53f3ca6d87a1df8c0e1b2538cd34c"},"cell_type":"code","source":"#%% set label dictionary and params\nclassfiles = os.listdir('../input/train_simplified/')\nnumstonames = {i: v[:-4].replace(\" \", \"_\") for i, v in enumerate(classfiles)} #adds underscores\n\nnum_classes = 20    #340 max \nimheight, imwidth = 32, 32  \nims_per_class = 10000  #max?","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"1ce1101eea3b2a0787c1dbe6b675e2a481802f67","_kg_hide-output":false,"scrolled":true},"cell_type":"code","source":"# faster conversion function\ndef draw_it(strokes):\n    image = Image.new(\"P\", (256,256), color=255)\n    image_draw = ImageDraw.Draw(image)\n    for stroke in ast.literal_eval(strokes):\n        for i in range(len(stroke[0])-1):\n            image_draw.line([stroke[0][i], \n                             stroke[1][i],\n                             stroke[0][i+1], \n                             stroke[1][i+1]],\n                            fill=0, width=5)\n    image = image.resize((imheight, imwidth))\n    return np.array(image)/255.\n\n#%% get train arrays\ntrain_grand = []\nnum_classes = 20\nclass_paths = glob('../input/train_simplified/*.csv')\nfor i,c in enumerate(tqdm(class_paths[0: num_classes])):\n    train = pd.read_csv(c, usecols=['drawing', 'recognized'], nrows=ims_per_class*5//4)\n    train = train[train.recognized == True].head(ims_per_class)\n    imagebag = bag.from_sequence(train.drawing.values).map(draw_it) \n    trainarray = np.array(imagebag.compute())  # PARALLELIZE\n    trainarray = np.reshape(trainarray, (ims_per_class, -1))    \n    labelarray = np.full((train.shape[0], 1), i)\n    trainarray = np.concatenate((labelarray, trainarray), axis=1)\n    train_grand.append(trainarray)\n    \n\ntrain_grand = np.array([train_grand.pop() for i in np.arange(num_classes)]) #less memory than np.concatenate\ntrain_grand = train_grand.reshape((-1, (imheight*imwidth+1)))\ndel trainarray\ndel train","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"6788c33f4a9dd552d473e3667387491c40dbfe4b","_kg_hide-input":false,"scrolled":true},"cell_type":"code","source":"# memory-friendly alternative to train_test_split?\nvalfrac = 0.1\ncutpt = int(valfrac * train_grand.shape[0])\n\nnp.random.shuffle(train_grand)\ny_train, X_train = train_grand[cutpt: , 0], train_grand[cutpt: , 1:]\ny_val, X_val = train_grand[0:cutpt, 0], train_grand[0:cutpt, 1:] #validation set is recognized==True\n\n# del train_grand\ny_train = keras.utils.to_categorical(y_train, num_classes)\nX_train = X_train.reshape(X_train.shape[0], imheight, imwidth, 1)\ny_val = keras.utils.to_categorical(y_val, num_classes)\nX_val = X_val.reshape(X_val.shape[0], imheight, imwidth, 1)\nprint(y_train.shape, \"\\n\",\n      X_train.shape, \"\\n\",\n      y_val.shape, \"\\n\",\n      X_val.shape)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"24ef673ac1b339a6449a2e20ecf946b40dbc0ca3"},"cell_type":"markdown","source":"Here's the architecture for the CNN. It's fairly simple compared to what else you can do."},{"metadata":{"trusted":true,"_uuid":"002b62eec9f0b5c139c4c08f82476433ccc11026"},"cell_type":"code","source":"model = Sequential()\nmodel.add(Conv2D(32, kernel_size=(3, 3), padding='same', activation='relu', input_shape=(imheight, imwidth, 1)))\nmodel.add(MaxPooling2D(pool_size=(2, 2)))\nmodel.add(Conv2D(64, kernel_size=(3, 3), padding='same', activation='relu'))\nmodel.add(MaxPooling2D(pool_size=(2, 2)))\nmodel.add(Dropout(0.2))\nmodel.add(Flatten())\nmodel.add(Dense(680, activation='relu'))\nmodel.add(Dropout(0.5))\nmodel.add(Dense(num_classes, activation='softmax'))\nmodel.summary()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"6464e84f6f0c7303110db9e26c74a5c90a0a9ab9","_kg_hide-output":true,"scrolled":false},"cell_type":"code","source":"def top_3_accuracy(x,y): \n    t3 = top_k_categorical_accuracy(x,y, 3)\n    return t3","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"scrolled":true,"_uuid":"bf4cc2d39633f9cef4ff9dc9a5f88b8e6f283f51"},"cell_type":"code","source":"model_1 = Sequential()\nmodel_1.add(Conv2D(12, kernel_size = (3,3), padding='same', activation='tanh', strides=1, input_shape=(32,32,1)))\nmodel_1.add(AveragePooling2D(pool_size=(2, 2), strides=2))\nmodel_1.add(Conv2D(32, kernel_size = (3,3), padding='same', activation='tanh', strides=1))\nmodel_1.add(AveragePooling2D(pool_size=(2, 2), strides=2))\nmodel_1.add(Conv2D(32, kernel_size = (3,3), padding='same', activation='tanh', strides=1))\nmodel_1.add(AveragePooling2D(pool_size=(2, 2), strides=2))\nmodel_1.add(Conv2D(240, kernel_size = (3,3), padding='same', activation='tanh', strides=1))\nmodel_1.add(Flatten())\nmodel_1.add(Dense(84, activation='tanh'))\nmodel_1.add(Dense(num_classes, activation='softmax'))\nmodel_1.summary()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"344d6b1dc1fe32272f08ad59c3be01426dce66e5"},"cell_type":"code","source":"model_1.compile(loss='categorical_crossentropy',\n              optimizer='adam',\n              metrics=['accuracy', top_3_accuracy])\n\nmodel_1.fit(x=X_train, y=y_train,\n          batch_size = 32,\n          epochs = 20,\n          validation_data = (X_val, y_val),\n#           callbacks = callbacks,\n          verbose = 1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"5ccbed8d03b3734f114c8c68ea9a6c3473751549"},"cell_type":"code","source":"model_2 = Sequential()\nmodel_2.add(Conv2D(32, kernel_size = (3,3), padding='same', activation='relu', strides=1, input_shape=(32,32,1)))\nmodel_2.add(MaxPooling2D(pool_size=(2, 2), strides=1))\nmodel_2.add(BatchNormalization(axis=1))\n\nmodel_2.add(Conv2D(86, kernel_size = (3,3), padding='same', activation='relu', strides=1))\nmodel_2.add(MaxPooling2D(pool_size=(3, 3), strides=2))\nmodel_2.add(BatchNormalization(axis=1))\n\nmodel_2.add(Conv2D(128, kernel_size = (3,3), padding='same', activation='relu', strides=1))\nmodel_2.add(Conv2D(128, kernel_size = (3,3), padding='same', activation='relu', strides=1))\nmodel_2.add(Conv2D(128, kernel_size = (3,3), padding='same', activation='relu', strides=1))\nmodel_2.add(MaxPooling2D(pool_size=(3, 3), strides=2))\nmodel_2.add(Flatten())\n\nmodel_2.add(Dense(64, activation='relu'))\nmodel_2.add(Dense(64, activation='relu'))\nmodel_2.add(Dense(num_classes, activation='softmax'))\nmodel_2.summary()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"7a548c63534f9eef6a9d0e521323b70971b565f1"},"cell_type":"code","source":"model_2.compile(loss='categorical_crossentropy',\n              optimizer='adam',\n              metrics=['accuracy', top_3_accuracy])\n\nmodel_2.fit(x=X_train, y=y_train,\n          batch_size = 32,\n          epochs = 20,\n          validation_data = (X_val, y_val),\n#           callbacks = callbacks,\n          verbose = 1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"79ae35ee126800f1f5ecfcbdc1fb824f80379492"},"cell_type":"code","source":"from keras.layers import merge\nfrom keras.models import Model\nfrom keras.layers import Dense, Dropout, Flatten\nfrom keras.layers import Conv2D, MaxPooling2D, AveragePooling2D, BatchNormalization, concatenate, Input\nfrom keras.metrics import top_k_categorical_accuracy\nfrom keras.callbacks import ModelCheckpoint, ReduceLROnPlateau, EarlyStopping\n\ninput_layer = Input(shape=(32,32,1))\n\nstem = Conv2D(32, kernel_size = (4,4), padding='same', activation='relu', strides=1)(input_layer)\nstem = MaxPooling2D(pool_size=(3, 3), strides=2)(stem)\nstem = BatchNormalization(axis=1)(stem)\n\nstem = Conv2D(32, kernel_size = (1,1), padding='same', activation='relu', strides=1)(stem)\nstem = Conv2D(32, kernel_size = (3,3), padding='same', activation='relu', strides=1)(stem)\nstem = BatchNormalization(axis=1)(stem)\n\ntower1_1 = Conv2D(64, kernel_size = (1,1), padding='same', activation='relu', strides=1)(stem)\n\ntower1_2 = Conv2D(64, kernel_size = (1,1), padding='same', activation='relu', strides=1)(stem)\ntower1_2 = Conv2D(64, kernel_size = (3,3), padding='same', activation='relu', strides=1)(tower1_2)\n\ntower1_3 = Conv2D(64, kernel_size = (1,1), padding='same', activation='relu', strides=1)(stem)\ntower1_3 = Conv2D(64, kernel_size = (5,5), padding='same', activation='relu', strides=1)(tower1_3)\n\ntower1_4 = MaxPooling2D(pool_size=(3, 3), strides=1, padding='same')(stem)\ntower1_4 = Conv2D(64, kernel_size = (1,1), padding='same', activation='relu', strides=1)(tower1_4)\n\noutput_inception_1 = concatenate([tower1_1, tower1_2, tower1_3, tower1_4])\n\n\n# INCEPTION MODULE 2\ntower2_1 = Conv2D(64, kernel_size = (1,1), padding='same', activation='relu', strides=1)(output_inception_1)\n\ntower2_2 = Conv2D(64, kernel_size = (1,1), padding='same', activation='relu', strides=1)(output_inception_1)\ntower2_2 = Conv2D(64, kernel_size = (3,3), padding='same', activation='relu', strides=1)(tower1_2)\n\ntower2_3 = Conv2D(64, kernel_size = (1,1), padding='same', activation='relu', strides=1)(output_inception_1)\ntower2_3 = Conv2D(64, kernel_size = (5,5), padding='same', activation='relu', strides=1)(tower1_3)\n\ntower2_4 = MaxPooling2D(pool_size=(3, 3), strides=1, padding='same')(output_inception_1)\ntower2_4 = Conv2D(64, kernel_size = (1,1), padding='same', activation='relu', strides=1)(tower1_4)\n\noutput_inception_2 = concatenate([tower2_1, tower2_2, tower2_3, tower2_4])\n\n#FINAL OUTPUT APEX\napex =  AveragePooling2D(pool_size=(5,5), strides=1)(output_inception_2)\napex = Flatten()(apex)\napex = Dropout(0.5)(apex)\napex = Dense(num_classes, activation='softmax')(apex)\n\nmodel_3 = Model([input_layer], outputs = apex)\nmodel_3.summary()\n\nmodel_3.compile(loss='categorical_crossentropy',\n              optimizer='adam',\n              metrics=['accuracy', top_3_accuracy])\n\nmodel_3.fit(x=X_train, y=y_train,\n          batch_size = 32,\n          epochs = 20,\n          validation_data = (X_val, y_val),\n#           callbacks = callbacks,\n          verbose = 1)","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"4125359749cd237de207eca0ab62074431df8323"},"cell_type":"markdown","source":"## Predicting on the Test data\nThe CNN does OK on the validation data, even with a basic model and limited training data. Let's generate predictions on the test set and submit."},{"metadata":{"trusted":true,"_uuid":"2828f46cdf053261a1435e6eca1a4aea55195273","_kg_hide-input":false,"_kg_hide-output":true},"cell_type":"code","source":"# #%% get test set\n# ttvlist = []\n# reader = pd.read_csv('../input/test_simplified.csv', index_col=['key_id'],\n#     chunksize=2048)\n# for chunk in tqdm(reader, total=55):\n#     imagebag = bag.from_sequence(chunk.drawing.values).map(draw_it)\n#     testarray = np.array(imagebag.compute())\n#     testarray = np.reshape(testarray, (testarray.shape[0], imheight, imwidth, 1))\n#     testpreds = model.predict(testarray, verbose=0)\n#     ttvs = np.argsort(-testpreds)[:, 0:3]  # top 3\n#     ttvlist.append(ttvs)\n    \n# ttvarray = np.concatenate(ttvlist)\ntest_images = 20000\ntest_grand_x = list()\ntest_grand_y = list()\nfor i,c in enumerate(tqdm(class_paths[0: num_classes])):\n    test = pd.read_csv(c, usecols=['drawing', 'recognized'], nrows=(test_images+ims_per_class)*5//4)\n    test = test.tail(test_images)\n    imagebag = bag.from_sequence(test.drawing.values).map(draw_it) \n    testarray = np.array(imagebag.compute())  # PARALLELIZE\n    testarray = np.reshape(testarray, (test_images, -1))    \n    labelarray = np.full((test.shape[0], 1), i)\n    test_grand_x.append(testarray)\n    test_grand_y.append(labelarray)\n    \ntest_grand_x = np.concatenate(test_grand_x)\ntest_grand_y = np.concatenate(test_grand_y)\ntest_grand_x = test_grand_x.reshape(test_grand_x.shape[0], imheight, imwidth, 1)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"5ece1155ccb01347d9420d3ac0066d89e5ba441d"},"cell_type":"code","source":"test_predicted_1 = model_1.predict(test_grand_x, verbose=0)\nprint('done prediction 1')\ntest_predicted_2 = model_2.predict(test_grand_x, verbose=0)\nprint('done prediction 2')\ntest_predicted_3 = model_3.predict(test_grand_x, verbose=0)\nprint('done prediction 3')\n\ntop_three_classes_1 = np.argsort(-test_predicted_1)[:, 0:3]\ntop_three_classes_2 = np.argsort(-test_predicted_2)[:, 0:3]\ntop_three_classes_3 = np.argsort(-test_predicted_3)[:, 0:3]\n\ncount_1 = 0\ncount_2 = 0\ncount_3 = 0\nfor i, o_1 in enumerate(tqdm(top_three_classes_1)):\n    if test_grand_y[i] in top_three_classes_1[i]:\n        count_1 = count_1+1\n    if test_grand_y[i] in top_three_classes_2[i]:\n        count_2 = count_2+1 \n    if test_grand_y[i] in top_three_classes_3[i]:\n        count_3 = count_3+1\n        \nprint(count_1/top_three_classes_1.shape[0])\nprint(count_2/top_three_classes_2.shape[0])\nprint(count_3/top_three_classes_3.shape[0])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"5c650c6f9a0ec6f99042e6132ffc4a25b2b2f0b3","_kg_hide-input":false,"_kg_hide-output":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"494ef170cb02e2425e82a897a0ca6c794c676b5a"},"cell_type":"markdown","source":" A full version with 6000 images per class at 28x28 gets just under 0.60 on the public LB.  "},{"metadata":{"trusted":true,"_uuid":"f75a34af14218daf6865999b19d2766b48f339a2"},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_uuid":"68bc0f3273c80e2ab7abb10c29c4741d19fb941a"},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}