{"cells":[{"metadata":{},"cell_type":"markdown","source":"분석 순서\n1. 데이터 살펴보기\n2. CNN 적용\n"},{"metadata":{},"cell_type":"markdown","source":"# CNN을 선택한 이유\n  \nCNN은 이미지 인식과 음석 인식 등 다양한 곳에서 사용되는데, 특히 이미지 인식 분야에서 딥러닝을 활용한 기법은 거의 다 CNN을 기초로 한다.\n지금까지는 완전연결(Affine) 신경망을 사용했다.\n\n완전연결 계층에서는 인접하는 계층의 뉴런이 모두 연결되고 출력의 수는 임의로 정할 수 있다.\n\n완전연경 계층을 사용하면 되지 않는가?\n\n완전연결 계층의 문제점은 무엇인가?\n\n바로 '데이터의 형상이 무시'된다는 사실이다.\n\n예를 들어보면, 이미지의 경우 (가로,세로,색상)으로 구성된 3차원 데이터이다.\n\n그러나 완전연결 계층에 입력할 때는 3차원 데이터를 1차원으로 평탄화(flatten=true)를 해줘야 한다.\n\n이미지의 3차원 형상에는 중요한 공간적 정보가 있다. \n\n예를들어 공간적으로 가까운 픽셀은 값이 비슷하거나, 색상이 밀접하게 관련되어있다.\n\n\n이러한 3차원 속에서의 패턴이 완전연결 계층에서는 모두 무시하고 동등한 뉴련으로 취급하게 된다.\n한편 합성곱 계층은 이 형상을 유지하므로, 이미지를 제대로 이해할 수 있다.\n\nCNN에서는 합성곱 계층의 입출력 데이터를 특징 맵(feature map)이라고도 한다.\n\n합성곱 연산은 이미지 처리에서 말하는 필터연산에 해당한다.\n\n필터는 커널이라고도 한다.\n\n\n\n\n결과 = 가중치\n\n결과의 크기 = 입력-커널+1 \n\n\n\n\n합성곱 연산은 필터의 윈도우(window)를 일정 간격으로 이동해가며 입력 데이터에 적용한다.\n\n대응하는 원소끼리 곱한 후 그 총합을 구한다.(단일 곱센-누산 FMA)\n\n그 결과를 출력의 해당 장소에 저장한다.\n\nCNN에서는 필터의 매개변수가 그동안의 '가중치'에 해당한다.\n\n물론 CNN에도 편향이 존재한다.\n\n편향을 필터를 적용한 후의 데이터에 더해진다.\n\n편향은 항상 하나(1*1)만 존재한다.\n\n패딩이란? 합성곱 연산을 수행하기 전에 입력 데이터 주변을 특정값(예를 들어 0)으로 채운다. 이를 패딩이라고 한다.\n\n패딩은 주로 출력 크기를 조정할 목적으로 사용한다.\n\n\n예를 들어 (4,4)입력데이터에 (3,3)필터를 적용하면 (2,2)가 된다.\n\n이렇게 필터를 적용하다 보면, 어느 시점에서는 출력 크기가 1이되어버리고, 합성곱 연산을 적용할 수 없다.\n\n\n예를 들어 (4,4)입력데이터에 패딩 1을 적용하여 (3,3)필터를 적용하면 (4,4)가 된다.\n\n한 마디로 입력 데이터의 공간적 크기를 고정한 채로 다음 계층에 전달 할 수 있다.\n\n\n\n\n필터를 적용하는 위치의 간격을 스트라이드(stride)라고 한다.\n\n지금까지는 모두 한 칸씩 이동하였지만, 스트라이드를 2로하면 아래와 같이 2칸씩 이동한다.\n\n그런데 위에서 보면 (7,7)인 입력 데이터에 스트라이드 2로 설정한 필터를 적용하니 출력이 (3,3)이 되었다.\n\n이처럼 스트라이드를 키우면 출력 크기는 작아진다.\n\n[출처] [파이썬][딥러닝] CNN 합성곱 계층|작성자 하쿠나마타타\nhttps://blog.naver.com/ssdyka/221364894122\n"},{"metadata":{},"cell_type":"markdown","source":"epochs는 전체 샘플 데이터를 이용하여 한바퀴 돌며 학습하는 것을 1회 epoch라고한다. 반복 횟수를 100, 150, 200, 300회로 각각 적합 해보며 적절한 epoch를 찾으려고 노력 해야하지만 이 데이터는 너무나도 많기 때문에 1회 epoch로 설정하였다.\n\nbatch_size(1회 step에 사용한 데이터의 수)는 1000으로 설정하였다.\n\nkeras를 활용하여 신경망을 구축해준다. 이 때, 은닉층은 16개이며 활성화함수는 ‘relu’ 함수를 사용한다.\n마지막 출력층에서의 활성화 함수는 softmax를 사용하였다.\n\nmodel.compile()에서는 optimizer를 rmsprop, adagrad, adadelta, adam, adamax, nadam(참고: https://keras.io/optimizers/)을 선택 할 수 있으며 여기서는 'adagrad'를 활용하여 신경망을 구축하였다.\nmetrics는 정확도를 기준으로 하였으며, 손실함수로는 categorical_crossentropy를 사용한다.\n\nmodel.compile(loss='categorical_crossentropy',\n                  optimizer=adagrad,\n                  metrics=['accuracy'])\n                  \n따라서 categorical_crossentropy방식으로 손실 함수에 넣어 loss score를 구한 뒤, optimizer adagrad를 사용하여 가중치 update를 해서 값을 도출 한 뒤, 훈련하는 동안 모니터링으로 metrics함수 를 사용하는데 accuracy로 예측 값을 측정한다.****"},{"metadata":{},"cell_type":"markdown","source":"# <한계점>\n사실상 이 분석은 epoch를 200회 이상으로 하여 분석하는것이 옳지만 컴퓨터의 사양과 시간 부족으로 인해 그리드서치 또는 랜덤서치를 할 수가 없었다. \n그랬기 때문에 정확성이 현저히 떨어진다.\n\n"},{"metadata":{},"cell_type":"markdown","source":"# 1. 데이터 살펴보기\n"},{"metadata":{},"cell_type":"markdown","source":"########################################################################################\n# 어떤 데이터들이 존재하는지 살펴본다.\n########################################################################################"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory\n\nimport os\nprint(os.listdir(\"../input\"))\n\n# Any results you write to the current directory are saved as output.","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"########################################################################################\n# 라이브러리를 사용하기 위해 여러 라이브러리 import 및 살펴보기\n########################################################################################"},{"metadata":{"trusted":true},"cell_type":"code","source":"#setup\nimport numpy as np\nimport pandas as pd\nimport os\nimport matplotlib.pyplot as plt\nimport ast\nimport json\nfrom PIL import Image, ImageDraw \n","execution_count":null,"outputs":[]},{"metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"cell_type":"code","source":"test_simplified = pd.read_csv(\"../input/test_simplified.csv\")\ntest_simplified.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"display_test=pd.DataFrame()\ndisplay_test=display_test.append(pd.read_csv(\"../input/test_simplified.csv\",usecols=['drawing'],nrows=50))\ndisplay_test.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"display_test['drawing'] = display_test['drawing'].apply(json.loads)\ndisplay_test.shape","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"########################################################################################\n# 숫자로 표현된 좌표들은 그림으로 표현 해보았다\n########################################################################################"},{"metadata":{"trusted":true},"cell_type":"code","source":"figrows=10\nfigcols=5\nfig, axs = plt.subplots(nrows=figrows, ncols=figcols, sharex=True, sharey=True, figsize=(16, 10))\nfor i, drawing in enumerate(display_test.drawing):\n    ax = axs[i // figcols, i % figcols]\n    for x, y in drawing:\n        ax.plot(x, -np.array(y), lw=3)\n    ax.axis('off')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sample_submission = pd.read_csv(\"../input/sample_submission.csv\")\nsample_submission.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"path_train = '../input/train_simplified/'\ntrain0 = pd.read_csv(path_train+(os.listdir(path_train)[0]))\ntrain0.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"path_train = '../input/train_simplified/'\ndisplay_samples=pd.DataFrame()\ndisplay_samples=display_samples.append(pd.read_csv(path_train+(os.listdir(path_train)[0]),usecols=['drawing', 'word'],nrows=50))\ndisplay_samples.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"display_samples['drawing'] = display_samples['drawing'].apply(json.loads)\ndisplay_samples.shape","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"########################################################################################\n# 하나의 데이터 sleeping bag의 그림을 그려봄\n########################################################################################"},{"metadata":{"trusted":true},"cell_type":"code","source":"figrows=10\nfigcols=5\nfig, axs = plt.subplots(nrows=figrows, ncols=figcols, sharex=True, sharey=True, figsize=(16, 10))\nfor i, drawing in enumerate(display_samples.drawing):\n    ax = axs[i // figcols, i % figcols]\n    for x, y in drawing:\n        ax.set_title(display_samples.word.iloc[i])\n        ax.plot(x, -np.array(y), lw=3)\n    ax.axis('off')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"**모든 학습데이터 merge**"},{"metadata":{"trusted":true},"cell_type":"code","source":"len(os.listdir(path_train))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# 2. 케라스 CNN 사용"},{"metadata":{},"cell_type":"markdown","source":"다음에는 이미지 분류기를 만들 것이다. 앞서 언급한 저장소에 사람들이 데이터를 어떻게 사용해 왔는지를 보여주는 몇 가지 리소스가 있다.\n\n그 자원들 중 하나는 여기 보이는 것과 같은 CNN이다.\n\n자원의 가장 큰 용도는 도면을 이미지로 변환하는 것 같다. 스트로크 기반 모델을 고수하거나 변환 경로를 따라 이동할 수 있다. \n\n그렇게 하려면 데이터 사용량 감시와 공간 제한 관리 필요 - 카글에 대한 깊은 학습은 병 안에 배를 만드는 것과 같을 수 있다:)\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport matplotlib.pyplot as plt\nimport ast\nimport os\nfrom glob import glob\nfrom tqdm import tqdm\nfrom dask import bag\nimport cv2\nimport tensorflow as tf\nfrom tensorflow import keras\nfrom tensorflow import keras\nfrom keras.models import Sequential\nfrom keras.layers import Dense, Dropout, Flatten\nfrom keras.layers import Conv2D, MaxPooling2D\nfrom keras.metrics import top_k_categorical_accuracy\nfrom keras.metrics import sparse_top_k_categorical_accuracy\nfrom keras.callbacks import ModelCheckpoint, ReduceLROnPlateau, EarlyStopping","execution_count":1,"outputs":[{"output_type":"stream","text":"Using TensorFlow backend.\n","name":"stderr"}]},{"metadata":{"trusted":true},"cell_type":"code","source":"classfiles=os.listdir('../input/train_simplified/')\nnumstonames={i : v[:-4].replace(' ','_') for i , v in enumerate(classfiles)}\n\nnum_class=340\nimheight,imwidth=32,32\nims_per_class=2000","execution_count":8,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"PIL(Python Imaging Library)은 파이썬을 이용해서 쉽게 이미지 프로세싱을 할 수 있게 주는 라이브러리이다. PIL에 대한 자세한 문서는 PIL의 공식 Documentation 사이트(http://www.pythonware.com/library/pil/handbook/)를 확인할 수있다.\n\nImage.new(mode, size) -> image Image.new(mode, size, color) -> image\n\n이 함수는 주어진 형식의 새로운 이미지를 생성한다. mode에는 \"P\",\"RGB\", \"CMYK\", \"L\"(흑백 모드) 등이 사용될 수 있다. size에는 가로, 세로 크기가 정수로 주어진 튜플이 주어지며, color는 RGB 모드의 경우 0~255의 값을 가지는 R, G, B 성분으로 이루어진 튜플이 전달된다. color 인수가 주어지지 않으면 검정 바탕의 이미지가 생성되고, color 인수가 주어지면 주어진 색을 바탕색으로 하는 이미지가 생성된다. 리턴값으로는 image 객체의 인스턴스가 주어진다.\n\n(예) 256x256사이즈의 흰색 바탕의 이미지를 생성하는 코드 image = Image.new(\"P\", (256,256), color=255)"},{"metadata":{"trusted":true},"cell_type":"code","source":"def stroke_to_img(strokes):\n    img=np.zeros((256,256))\n    for each in ast.literal_eval(strokes):\n        for i in range(len(each[0])-1):\n            cv2.line(img,(each[0][i],each[1][i]),(each[0][i+1],each[1][i+1]),255,5)\n    img=cv2.resize(img,(32,32))\n    img=img/255\n    return img","execution_count":9,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"rd=np.random.randint(340)\nranclass=numstonames[rd]\nranclass=ranclass.replace('_',' ')\nrdpath='../input/train_simplified/'+ranclass+'.csv'\none=pd.read_csv(rdpath,usecols=['drawing','recognized','word'],nrows=10)\none=one[one.recognized==True].head(2)\nname=one['word'].head(1)\nstrk=one['drawing']\npic=[]\nfor s in strk:\n    pic.append(stroke_to_img(s))\nname=name.values","execution_count":10,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train_grand=[]\nclass_paths = glob('../input/train_simplified/*.csv')\nfor i , c in enumerate(tqdm(class_paths[0:num_class])):\n    train=pd.read_csv(c,usecols=['drawing','recognized'],nrows=ims_per_class*2)\n    train=train[train.recognized==True].head(ims_per_class)\n    imagebag=bag.from_sequence(train.drawing.values).map(stroke_to_img)\n    trainarray=np.array(imagebag.compute())\n    trainarray=np.reshape(trainarray,(ims_per_class,-1))\n    labelarray=np.full((train.shape[0],1),i)\n    trainarray=np.concatenate((labelarray,trainarray),axis=1)\n    train_grand.append(trainarray)\n\ntrain_grand=np.array([train_grand.pop() for i in np.arange(num_class)])\ntrain_grand=train_grand.reshape((-1,(imheight*imwidth+1)))\n\ndel trainarray\ndel train","execution_count":11,"outputs":[{"output_type":"stream","text":"100%|██████████| 340/340 [07:58<00:00,  1.43s/it]\n","name":"stderr"}]},{"metadata":{"trusted":true},"cell_type":"code","source":"valfrac=0.2\ncutpt=int(valfrac*train_grand.shape[0])\n\nnp.random.shuffle(train_grand)\ny_train, x_train=train_grand[cutpt:,0],train_grand[cutpt:,1:]\ny_val,x_val=train_grand[0:cutpt,0], train_grand[0:cutpt,1:]\n\ndel train_grand\n\nx_train=x_train.reshape(x_train.shape[0],imheight,imwidth,1)\nx_val=x_val.reshape(x_val.shape[0],imheight,imwidth,1)","execution_count":12,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model =Sequential()\nmodel.add(Conv2D(32,kernel_size=(3,3),padding='same',activation='relu',input_shape=(imheight,imwidth,1)))\nmodel.add(MaxPooling2D(pool_size=(2,2)))\nmodel.add(Conv2D(64,kernel_size=(3,3),padding='same',activation='relu'))\nmodel.add(MaxPooling2D(pool_size=(2,2)))\nmodel.add(Conv2D(64,kernel_size=(3,3),padding='same',activation='relu'))\nmodel.add(MaxPooling2D(pool_size=(2,2)))\nmodel.add(Flatten())\nmodel.add(Dropout(0.5))\nmodel.add(Dense(680,activation='relu'))\nmodel.add(Dense(num_class,activation='softmax'))\nmodel.summary()","execution_count":13,"outputs":[{"output_type":"stream","text":"WARNING:tensorflow:From /opt/conda/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\nInstructions for updating:\nColocations handled automatically by placer.\nWARNING:tensorflow:From /opt/conda/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:3445: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.\nInstructions for updating:\nPlease use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.\n_________________________________________________________________\nLayer (type)                 Output Shape              Param #   \n=================================================================\nconv2d_1 (Conv2D)            (None, 32, 32, 32)        320       \n_________________________________________________________________\nmax_pooling2d_1 (MaxPooling2 (None, 16, 16, 32)        0         \n_________________________________________________________________\nconv2d_2 (Conv2D)            (None, 16, 16, 64)        18496     \n_________________________________________________________________\nmax_pooling2d_2 (MaxPooling2 (None, 8, 8, 64)          0         \n_________________________________________________________________\nconv2d_3 (Conv2D)            (None, 8, 8, 64)          36928     \n_________________________________________________________________\nmax_pooling2d_3 (MaxPooling2 (None, 4, 4, 64)          0         \n_________________________________________________________________\nflatten_1 (Flatten)          (None, 1024)              0         \n_________________________________________________________________\ndropout_1 (Dropout)          (None, 1024)              0         \n_________________________________________________________________\ndense_1 (Dense)              (None, 680)               697000    \n_________________________________________________________________\ndense_2 (Dense)              (None, 340)               231540    \n=================================================================\nTotal params: 984,284\nTrainable params: 984,284\nNon-trainable params: 0\n_________________________________________________________________\n","name":"stdout"}]},{"metadata":{},"cell_type":"markdown","source":"epochs는 전체 샘플 데이터를 이용하여 한바퀴 돌며 학습하는 것을 1회 epoch라고한다. 반복 횟수를 100, 150, 200, 300,500회로 각각 적합 해보며 적절한 epoch를 찾으려고 노력 해야하지만 이 데이터는 너무나도 많기 때문에 500회 epoch로 시도해보려고한다.\n\nbatch_size(1회 step에 사용한 데이터의 수)는 150으로 설정하였다.\n\nkeras를 활용하여 신경망을 구축해준다. 이 때, 활성화함수는 ‘relu’ 함수를 사용한다. 마지막 출력층에서의 활성화 함수는 softmax를 사용하였다.\n\nmodel.compile()에서는 optimizer를 rmsprop, adagrad, adadelta, adam, adamax, nadam(참고: https://keras.io/optimizers/)을 선택 할 수 있으며 여기서는 'adam'를 활용하여 신경망을 구축하였다. metrics는 정확도를 기준으로 하였으며, 손실함수로는 sparse_categorical_crossentropy를 사용한다.\n\nmodel.compile(loss='sparse_categorical_crossentropy',optimizer='adam',metrics=['accuracy',top_3_accuracy])\n\n따라서 sparse_categorical_crossentropy방식으로 손실 함수에 넣어 loss score를 구한 뒤, optimizer adam를 사용하여 가중치 update를 해서 값을 도출 한 뒤, 훈련하는 동안 모니터링으로 metrics함수 를 사용하는데 accuracy,top_3_accuracy로 예측 값을 측정한다."},{"metadata":{"trusted":true},"cell_type":"code","source":"def top_3_accuracy(x,y):\n    t3=sparse_top_k_categorical_accuracy(x,y,3)\n    return t3\n\nreduceLROnPlat=ReduceLROnPlateau(monitor='val_loss',factor=0.3,patience=5,verbose=1,mode='auto',min_delta=0.005,cooldown=5,min_lr=0.001)\nearlystop=EarlyStopping(monitor='val_acc',mode='max',patience=5)\ncallbacks=[reduceLROnPlat,earlystop]\n\nmodel.compile(loss='sparse_categorical_crossentropy',optimizer='adam',metrics=['accuracy',top_3_accuracy])\n\nhistory=model.fit(x=x_train,y=y_train,batch_size=150,epochs=500,validation_data=(x_val,y_val),callbacks=callbacks,verbose=1)","execution_count":14,"outputs":[{"output_type":"stream","text":"WARNING:tensorflow:From /opt/conda/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\nInstructions for updating:\nUse tf.cast instead.\nTrain on 544000 samples, validate on 136000 samples\nEpoch 1/50\n544000/544000 [==============================] - 31s 57us/step - loss: 2.4487 - acc: 0.4200 - top_3_accuracy: 0.6237 - val_loss: 1.6301 - val_acc: 0.5809 - val_top_3_accuracy: 0.7844\nEpoch 2/50\n544000/544000 [==============================] - 27s 50us/step - loss: 1.7251 - acc: 0.5524 - top_3_accuracy: 0.7656 - val_loss: 1.4177 - val_acc: 0.6288 - val_top_3_accuracy: 0.8239\nEpoch 3/50\n544000/544000 [==============================] - 26s 49us/step - loss: 1.5525 - acc: 0.5901 - top_3_accuracy: 0.7973 - val_loss: 1.3166 - val_acc: 0.6511 - val_top_3_accuracy: 0.8412\nEpoch 4/50\n544000/544000 [==============================] - 26s 49us/step - loss: 1.4525 - acc: 0.6115 - top_3_accuracy: 0.8159 - val_loss: 1.2706 - val_acc: 0.6600 - val_top_3_accuracy: 0.8483\nEpoch 5/50\n544000/544000 [==============================] - 27s 50us/step - loss: 1.3838 - acc: 0.6271 - top_3_accuracy: 0.8278 - val_loss: 1.2477 - val_acc: 0.6656 - val_top_3_accuracy: 0.8528\nEpoch 6/50\n544000/544000 [==============================] - 26s 49us/step - loss: 1.3356 - acc: 0.6368 - top_3_accuracy: 0.8358 - val_loss: 1.2122 - val_acc: 0.6726 - val_top_3_accuracy: 0.8571\nEpoch 7/50\n544000/544000 [==============================] - 26s 49us/step - loss: 1.2985 - acc: 0.6450 - top_3_accuracy: 0.8426 - val_loss: 1.1859 - val_acc: 0.6795 - val_top_3_accuracy: 0.8612\nEpoch 8/50\n544000/544000 [==============================] - 27s 50us/step - loss: 1.2662 - acc: 0.6520 - top_3_accuracy: 0.8482 - val_loss: 1.1695 - val_acc: 0.6842 - val_top_3_accuracy: 0.8650\nEpoch 9/50\n544000/544000 [==============================] - 26s 49us/step - loss: 1.2429 - acc: 0.6563 - top_3_accuracy: 0.8524 - val_loss: 1.1537 - val_acc: 0.6865 - val_top_3_accuracy: 0.8666\nEpoch 10/50\n544000/544000 [==============================] - 27s 50us/step - loss: 1.2226 - acc: 0.6612 - top_3_accuracy: 0.8547 - val_loss: 1.1455 - val_acc: 0.6893 - val_top_3_accuracy: 0.8678\nEpoch 11/50\n544000/544000 [==============================] - 27s 51us/step - loss: 1.2045 - acc: 0.6649 - top_3_accuracy: 0.8583 - val_loss: 1.1420 - val_acc: 0.6896 - val_top_3_accuracy: 0.8689\nEpoch 12/50\n544000/544000 [==============================] - 26s 49us/step - loss: 1.1892 - acc: 0.6684 - top_3_accuracy: 0.8609 - val_loss: 1.1288 - val_acc: 0.6934 - val_top_3_accuracy: 0.8711\nEpoch 13/50\n544000/544000 [==============================] - 26s 49us/step - loss: 1.1761 - acc: 0.6718 - top_3_accuracy: 0.8629 - val_loss: 1.1238 - val_acc: 0.6929 - val_top_3_accuracy: 0.8718\nEpoch 14/50\n544000/544000 [==============================] - 27s 49us/step - loss: 1.1629 - acc: 0.6754 - top_3_accuracy: 0.8655 - val_loss: 1.1214 - val_acc: 0.6938 - val_top_3_accuracy: 0.8719\nEpoch 15/50\n544000/544000 [==============================] - 26s 49us/step - loss: 1.1538 - acc: 0.6765 - top_3_accuracy: 0.8660 - val_loss: 1.1184 - val_acc: 0.6957 - val_top_3_accuracy: 0.8729\nEpoch 16/50\n544000/544000 [==============================] - 27s 49us/step - loss: 1.1426 - acc: 0.6786 - top_3_accuracy: 0.8684 - val_loss: 1.1156 - val_acc: 0.6964 - val_top_3_accuracy: 0.8733\nEpoch 17/50\n544000/544000 [==============================] - 27s 50us/step - loss: 1.1341 - acc: 0.6805 - top_3_accuracy: 0.8702 - val_loss: 1.1160 - val_acc: 0.6956 - val_top_3_accuracy: 0.8730\nEpoch 18/50\n544000/544000 [==============================] - 26s 48us/step - loss: 1.1261 - acc: 0.6820 - top_3_accuracy: 0.8712 - val_loss: 1.1187 - val_acc: 0.6964 - val_top_3_accuracy: 0.8731\nEpoch 19/50\n544000/544000 [==============================] - 27s 49us/step - loss: 1.1193 - acc: 0.6833 - top_3_accuracy: 0.8727 - val_loss: 1.1092 - val_acc: 0.6962 - val_top_3_accuracy: 0.8738\nEpoch 20/50\n544000/544000 [==============================] - 27s 49us/step - loss: 1.1126 - acc: 0.6851 - top_3_accuracy: 0.8737 - val_loss: 1.1075 - val_acc: 0.6969 - val_top_3_accuracy: 0.8747\nEpoch 21/50\n544000/544000 [==============================] - 26s 48us/step - loss: 1.1069 - acc: 0.6862 - top_3_accuracy: 0.8745 - val_loss: 1.1201 - val_acc: 0.6953 - val_top_3_accuracy: 0.8721\nEpoch 22/50\n544000/544000 [==============================] - 27s 50us/step - loss: 1.0996 - acc: 0.6882 - top_3_accuracy: 0.8755 - val_loss: 1.1124 - val_acc: 0.6977 - val_top_3_accuracy: 0.8735\nEpoch 23/50\n544000/544000 [==============================] - 27s 49us/step - loss: 1.0965 - acc: 0.6892 - top_3_accuracy: 0.8764 - val_loss: 1.1121 - val_acc: 0.6975 - val_top_3_accuracy: 0.8737\nEpoch 24/50\n544000/544000 [==============================] - 26s 48us/step - loss: 1.0912 - acc: 0.6907 - top_3_accuracy: 0.8766 - val_loss: 1.0988 - val_acc: 0.6996 - val_top_3_accuracy: 0.8762\nEpoch 25/50\n544000/544000 [==============================] - 26s 48us/step - loss: 1.0870 - acc: 0.6912 - top_3_accuracy: 0.8777 - val_loss: 1.1073 - val_acc: 0.6964 - val_top_3_accuracy: 0.8747\nEpoch 26/50\n544000/544000 [==============================] - 27s 49us/step - loss: 1.0827 - acc: 0.6918 - top_3_accuracy: 0.8788 - val_loss: 1.1019 - val_acc: 0.6996 - val_top_3_accuracy: 0.8754\nEpoch 27/50\n544000/544000 [==============================] - 26s 48us/step - loss: 1.0788 - acc: 0.6927 - top_3_accuracy: 0.8788 - val_loss: 1.1143 - val_acc: 0.6965 - val_top_3_accuracy: 0.8736\nEpoch 28/50\n544000/544000 [==============================] - 26s 48us/step - loss: 1.0733 - acc: 0.6937 - top_3_accuracy: 0.8798 - val_loss: 1.1042 - val_acc: 0.7007 - val_top_3_accuracy: 0.8756\nEpoch 29/50\n544000/544000 [==============================] - 27s 49us/step - loss: 1.0710 - acc: 0.6940 - top_3_accuracy: 0.8801 - val_loss: 1.1080 - val_acc: 0.6990 - val_top_3_accuracy: 0.8746\n\nEpoch 00029: ReduceLROnPlateau reducing learning rate to 0.0003000000142492354.\nEpoch 30/50\n544000/544000 [==============================] - 26s 48us/step - loss: 0.9777 - acc: 0.7171 - top_3_accuracy: 0.8948 - val_loss: 1.0546 - val_acc: 0.7129 - val_top_3_accuracy: 0.8829\nEpoch 31/50\n544000/544000 [==============================] - 27s 49us/step - loss: 0.9595 - acc: 0.7215 - top_3_accuracy: 0.8977 - val_loss: 1.0539 - val_acc: 0.7125 - val_top_3_accuracy: 0.8835\nEpoch 32/50\n544000/544000 [==============================] - 27s 49us/step - loss: 0.9497 - acc: 0.7239 - top_3_accuracy: 0.8990 - val_loss: 1.0497 - val_acc: 0.7137 - val_top_3_accuracy: 0.8839\nEpoch 33/50\n544000/544000 [==============================] - 26s 49us/step - loss: 0.9441 - acc: 0.7251 - top_3_accuracy: 0.9003 - val_loss: 1.0544 - val_acc: 0.7136 - val_top_3_accuracy: 0.8830\nEpoch 34/50\n544000/544000 [==============================] - 27s 50us/step - loss: 0.9378 - acc: 0.7271 - top_3_accuracy: 0.9011 - val_loss: 1.0482 - val_acc: 0.7135 - val_top_3_accuracy: 0.8846\nEpoch 35/50\n544000/544000 [==============================] - 27s 49us/step - loss: 0.9329 - acc: 0.7282 - top_3_accuracy: 0.9020 - val_loss: 1.0532 - val_acc: 0.7122 - val_top_3_accuracy: 0.8833\nEpoch 36/50\n544000/544000 [==============================] - 26s 49us/step - loss: 0.9277 - acc: 0.7292 - top_3_accuracy: 0.9025 - val_loss: 1.0539 - val_acc: 0.7124 - val_top_3_accuracy: 0.8835\nEpoch 37/50\n544000/544000 [==============================] - 26s 48us/step - loss: 0.9268 - acc: 0.7299 - top_3_accuracy: 0.9021 - val_loss: 1.0491 - val_acc: 0.7135 - val_top_3_accuracy: 0.8842\n","name":"stdout"}]},{"metadata":{},"cell_type":"markdown","source":"# epoch에 따른 정확도와 손실함수 그림"},{"metadata":{"trusted":true},"cell_type":"code","source":"acc=history.history['acc']\nval_acc=history.history['val_acc']\nloss= history.history['loss']\nval_loss=history.history['val_loss']\n\nepochs=range(1,len(acc)+1)\n\nplt.plot(epochs,acc,label='Training acc')\nplt.plot(epochs,val_acc,label='Validation acc')\nplt.title('Training and validation accuracy')\nplt.legend()\n\nplt.figure()\n\nplt.plot(epochs,loss,label='Training loss')\nplt.plot(epochs,val_loss,label='Validation loss')\nplt.title('Training and validation loss')\nplt.legend()\n\nplt.show()","execution_count":15,"outputs":[{"output_type":"display_data","data":{"text/plain":"<Figure size 432x288 with 1 Axes>","image/png":"iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xl8VNX5+PHPk33fE8CEfREQWSOooIJb0Sq0SlX82mqttdq61NZulrrbzdbaWr9WarHttyq1WhVa0FLEHyoqhB0CsguBkH3fJjNzfn/cmzCELEMyyUxmnvfrdV9z93nmJnnm5NxzzhVjDEoppUJDmL8DUEop1Xc06SulVAjRpK+UUiFEk75SSoUQTfpKKRVCNOkrpVQI0aQfgkQkXERqRWSIL/f1JxEZJSI+b38sIpeKyCGP5U9F5AJv9u3Ge70gIg9093ilvBHh7wBU10Sk1mMxDmgCXPbyN4wxL53O+YwxLiDB1/uGAmPMmb44j4jcBtxkjJntce7bfHFupTqjSb8fMMa0Jl27JHmbMea/He0vIhHGGGdfxKZUV/T3MbBo9U4QEJHHReTvIvKKiNQAN4nIeSLysYhUikihiPxORCLt/SNExIjIMHv5b/b2lSJSIyIficjw093X3n6FiOwRkSoReUZEPhSRWzqI25sYvyEi+0SkQkR+53FsuIj8RkTKROQAMLeT6/NjEVnaZt2zIvKUPX+biOyyP89+uxTe0bkKRGS2PR8nIv9nx7YTmNZm30UicsA+704RmWevPxv4PXCBXXVW6nFtH/Y4/g77s5eJyJsiMsiba3M617klHhH5r4iUi8hxEfm+x/v8xL4m1SKSJyJntFeVJiIftPyc7eu51n6fcmCRiIwWkTX2e5Ta1y3Z4/ih9mcssbf/VkRi7JjHeew3SETqRSS9o8+rumCM0akfTcAh4NI26x4HHMDVWF/kscA5wAys/+ZGAHuAu+z9IwADDLOX/waUArlAJPB34G/d2DcLqAHm29u+AzQDt3TwWbyJ8S0gGRgGlLd8duAuYCeQA6QDa61f53bfZwRQC8R7nLsYyLWXr7b3EeBioAGYaG+7FDjkca4CYLY9/yvgPSAVGArkt9n3OmCQ/TO50Y5hgL3tNuC9NnH+DXjYnr/cjnEyEAP8L/CuN9fmNK9zMlAE3AtEA0nAdHvbj4CtwGj7M0wG0oBRba818EHLz9n+bE7gTiAc6/dxDHAJEGX/nnwI/Mrj8+ywr2e8vf9Me9ti4AmP9/ku8Ia//w778+T3AHQ6zR9Yx0n/3S6Oux/4hz3fXiL/g8e+84Ad3dj3VuB9j20CFNJB0vcyxnM9tv8TuN+eX4tVzdWy7cq2iajNuT8GbrTnrwA+7WTffwHfsuc7S/qHPX8WwDc9923nvDuAz9vzXSX9vwA/9diWhHUfJ6era3Oa1/nLwIYO9tvfEm+b9d4k/QNdxLCg5X2BC4DjQHg7+80EDgJiL28BrvH131UoTVq9EzyOeC6IyFgR+bf973o18CiQ0cnxxz3m6+n85m1H+57hGYex/koLOjqJlzF69V7AZ53EC/AysNCev9FebonjKhH5xK56qMQqZXd2rVoM6iwGEblFRLbaVRSVwFgvzwvW52s9nzGmGqgAsj328epn1sV1HoyV3NvT2bautP19HCgir4rIUTuGP7eJ4ZCxGg2cxBjzIdZ/DbNEZAIwBPh3N2NSaJ1+MGnbXPF5rJLlKGNMEvAgVsm7NxVilUQBEBHh5CTVVk9iLMRKFi26alL6KnCpiGRjVT+9bMcYC7wG/Ayr6iUF+I+XcRzvKAYRGQE8h1XFkW6fd7fHebtqXnoMq8qo5XyJWNVIR72Iq63OrvMRYGQHx3W0rc6OKc5j3cA2+7T9fL/AanV2th3DLW1iGCoi4R3E8VfgJqz/Sl41xjR1sJ/ygib94JUIVAF19o2wb/TBe/4LmCoiV4tIBFY9cWYvxfgq8G0RybZv6v2gs52NMcexqiD+jFW1s9feFI1Vz1wCuETkKqy6Z29jeEBEUsTqx3CXx7YErMRXgvX993Wskn6LIiDH84ZqG68AXxORiSISjfWl9L4xpsP/nDrR2XVeBgwRkbtEJFpEkkRkur3tBeBxERkplskikob1ZXccq8FAuIjcjscXVCcx1AFVIjIYq4qpxUdAGfBTsW6Ox4rITI/t/4dVHXQj1heA6gFN+sHru8DNWDdWn8e64dqrjDFFwPXAU1h/xCOBzVglPF/H+BywGtgObMAqrXflZaw6+taqHWNMJXAf8AbWzdAFWF9e3ngI6z+OQ8BKPBKSMWYb8Ayw3t7nTOATj2NXAXuBIhHxrKZpOf5trGqYN+zjhwD/42VcbXV4nY0xVcBlwLVYX0R7gIvszU8Cb2Jd52qsm6oxdrXd14EHsG7qj2rz2drzEDAd68tnGfC6RwxO4CpgHFap/zDWz6Fl+yGsn3OTMWbdaX521UbLzRGlfM7+d/0YsMAY876/41H9l4j8Fevm8MP+jqW/085ZyqdEZC5WS5kGrCZ/zVilXaW6xb4/Mh8429+xBAOt3lG+Ngs4gFWX/Tngi3rjTXWXiPwMq6/AT40xh/0dTzDQ6h2llAohWtJXSqkQEnB1+hkZGWbYsGH+DkMppfqVjRs3lhpjOmsiDQRg0h82bBh5eXn+DkMppfoVEemqVzqg1TtKKRVSNOkrpVQI0aSvlFIhRJO+UkqFEE36SikVQjTpK6VUCNGkr5RSISTg2ukrpVSwMcbQ0OyittFJTZOT2kYntU1OauzX2sZmapucpMVHc+OMrp4H1DOa9JVS6jQ1u9yU1zkorW2irNZBWZ31WlHvoKK+mcp6BxV1zfaytc7hdHd53qlDUjTpK6VUX6tqaOZASS0HS+s4UFLHwbI6SqqbKLWTe1VDc7vHhYcJqXGRpMRFkRoXyeC0OCbmJJMaH0VKbBRJsREkRHtMMREkRkeSEBNBfHQ40REdPTHSdzTpK6VCitttqGxoprS2yZ4cHK9q4ECJleAPlNZSWuto3T88TBicGsvA5BjGDUoiIz6K9IRo0hOiSI+PJiPBWk6LjyIpJgLr0dCBS5O+UirolNU2sft4DbsKq9l9vIai6kZKapooq3NQXufA5T51SPmMhCiGZ8RzydgBjMiMZ0RmAiMy4xmcGkdURPC0edGkr5Tqt9xuw57iGnYXWgl+1/EadhdWU1xz4rk9mYnR5KTGkpMax5QhKSeVzjMSrPmspBiSYzt6Rn1w0aSvlOpX3G5D3mcVrNheyModhRRVWwk+KjyM0QMSuGB0JuMGJTJuUBJjByaSnhDt54gDiyZ9pVTAc7kNeYfK7UR/nOKaJqIjwph9ZiaXjx/I2TnJDM+IJzI8eKpheosmfaVUwNpVWM3S9YdZseM4JXain3NmFldOHMTFY7NIiNYUdrr0iimlAlJRdSPXPrcOl9u0JvpLxmYRr4m+R/TqKaUC0i/e3o3TZVj1nQsZmh7fvZM4m6CuFNzNgICEgUj782Hh9rqW+XCP1z6qNnI1Q3jv3lDWpK+UCjhbj1TyxqYjLBn9MUPXvgXhURARbb/GeMzbN2nry6CuxErwdSUn5puqfRSRQFwaJAyExAGQYE+JA0+8RidaSdvttF5djhPz7mZwOqChAhrKrXhPmsqtadAk+No7Poq5fZr0lVIBxRjDo8t38tPYl5lzZAVUZVvJ09lkJVJnI5g2QxpIGMRlQHwmxGfAGVOt1/gMa314FGDAGPvYdubdLmvZuOx5F7jtZVezlZxri6H2OJTsgdoi+z+IbohJgbh0a0rKgYETrS+V9NE9vHpd06SvlAooy7cVMvvYYhZGrIBzvwmf+6ldDePB5QRXk/VFAFYS7asqmBZut1Vyry2yvgiaaq2qmfBICLNfw6MgLOLEfGyqFWu4/1KvJn2lVMBocLg4uuwJ7o54E/eUrxDWXsIHK2mGR0BUN+v6fSEsDOLTrWnAeP/FcZq0UatSKmB88vefcafrb5QOn0fY1U+3n/BVj2jSV0oFhKp1f2b2/ifZGj+TjJuWWC1nlM95lfRFZK6IfCoi+0Tkh+1s/42IbLGnPSJS6bHtZhHZa083+zJ4pVSQ2PkGif+5jw/cZ5N28996vdliKOuyTl9EwoFngcuAAmCDiCwzxuS37GOMuc9j/7uBKfZ8GvAQkAsYYKN9bIVPP4VSqv/a8w7mtdvIc49mw4xnmJWV5u+Igpo3N3KnA/uMMQcARGQpMB/I72D/hViJHuBzwCpjTLl97CpgLvBKT4JWSnWiqdZqZhiT3P1zNFRazRfjupGAqwpg9wr4dAU46iB1KKQMPfGaMgSSc6zS/MG1mL9/mf3hw/h++I/516Vndz9m5RVvkn42cMRjuQCY0d6OIjIUGA6828mx2e0cdztwO8CQIb37qDClgoIxVgekkk+hdM+JqWQPVBdY+2SNh6Hnw5DzrNekMzo+X20xfLbOnj6Eop2AsdqNDznXns6DtBGn3lw1Bop3we5/w+5/QeEWa33GmZCQBUc+gR3/tL6IWkg4JGVDXQk1cYNZUPI9HlgwVcfS6QO+vsI3AK8Z4/nT7ZoxZjGwGCA3N/fUpxso1RG3Cxy10NwI0QkQGXf6LT7cLqvnZlOtdfMwLMKePOcjrA5ALoe1X1M1NNVY791Uc2JyO61SbMoQSB4MMUkdv68xUFMIx3dA0Xbr9fh2qD5mNUcM82zz7bEsApWHobHqxLki4yFjNAybab0aA4c/gq1LYcML1j4pQ098CQyaaH1BfPahlejL9trniYPB02HOA9ZnPvIJ7FoOm//P2h6faX0BDD7Xep+Da61kX3HQ2p4zHS59BMZ+3trewuWE6qNQ+RlUfNb66nS5uHHfFeRkD2TB1JzT+7mpbvEm6R8FBnss59jr2nMD8K02x85uc+x73oenQlJzA5QfhPL9ULbfeq0pOpFgHbVW4nXUQnP9ycdKuFWtEZNsJdyW+ehkq7qiscpjqrRefdZVvx0xKZAy2Eq4yYMhORtqjlvJvWiH1cuzRcoQq2fm6MtP9AJ1N9td+lvmndYXS845Vkk6YzRkngmJZ7TfOcnltL5QPvvISvB7V8FWj9rV6GQriU+5CYbNsoYBaHsT1e22/os4/JH1JXD4I+uLAKwOR8Mvgpn3wplXWMMRtCc8wqreSR1q1QXYnlm1hx3Ve3l14VmEhWnzzL4gxnResBaRCGAPcAlWEt8A3GiM2dlmv7HA28BwY5/UvpG7EZhq77YJmNZSx9+e3Nxck5eX171Po/qW2w0lu6wkFBELkR5TROzJScjttpJr24TbWGWNOVJx0E7wB6w6YTx+L+MzIXGQNbZJdCJEJVil+qiEE8uRMVb9cWMVNFa3Se72FOb5heA5pVivUfHWF4PbaZX+3U67K7697Gq2xnppiaM1liQrnuhEQE6UaCuPWCXyKvu18gg011ljx2SNgwETrCQ/cAIMOKtndfDeMgZK98LxbZAxxnrf7jSNrDluVS2dMQVikjDGUNPkpLKumfJ6BxX1DirqHNQ7XDhdbpxuY00uN80ug9Ptxuky/OWjQ1wybgDP3ji1y7dUnRORjcaY3K7267Kkb4xxishdwDtAOLDEGLNTRB4F8owxy+xdbwCWGo9vEWNMuYg8hvVFAfBoZwlf9QP15bD/Xdj7H9i3GupLO943PNpKxmAlYjopYMSkQPooq/ohbSSkj7Tqj9NH9k0y9KWkQZDTzt+eMVa3/egk/3XDF4HMMdbUhtPlps7hot7hpK7JRU1jM1UNHlP9ifnKhmaqGqKoqNtMRX0zlfUOnO08d7Yj4WFCRJiQnRLLD+eO9eUnVF3osqTf17SkH2DcbijcDHv/C/tWwdGNVmk4Ng1GXQIjL7aSmLPRqmpptl89lzEnStOxKSeXrj3XqW4xxlBe5+B4dSPHqxoprGqkos5Bo9NFg8NNQ7OLpmYXDc0uGltf3a3Jvd7hpM7hwuF0d/lesZHhJMdGtk6p8ZGkxUeREhdFWlwUKXEnllPjIkmIjrASfHgYkeFCRFgYEWGiVTm9wGclfRVkWm4eHt0EhVutkmfLwFWtU+OJ0QwrPrNL8wLZU+HC78Poy6x/67XHZK9xuQ1VDc2t1STldVaVSXldM+V1TRRVN7Um+ePVje0m7PAwITYynJjIcGIiw1rnYyPDSYyJYGBSDHHR4cRHRZx4jQonITqCuOgIEqLDSY6Nak3wSbERREfoz7y/06Qf7GpL4Nhme9pkvdYWWdsk3LrZ2To+ebT1GhFtrYtNtep9R8yxSvXxGf79LEGmsdnFobI69hfXcaCklv0ltewvqaOgop7KhmY6+ic8KiKMgUkxDEyKYfLgFAYlxzAwOcZ+jWVQcgypcVFERegoK+pUmvSDhctp3QQt2mG1sS7Ot1qIVLV0kxCrlcfIi61S+hlTrRuIkbF+DTsYud2G6sZmyuoclNU6KK9roqzOQXmtg7I6B4fK6jhQUseRivqTEnt2SiwjMuOZmDOI9PgoUuOjSIuPIjXOnuyqlNjIcEQHIlPdpEm/P3K7oWADHM2zEnzRDijebVXTgFWCzxhttbeecYeV5AdNtFuXqJ5yuQ2FVQ0cLq/nSHk9h8vrOVxuLR+rbKC8zoGrg5uaidER5KTFMTEnmS9OyWZkVgIjMuIZkRlPXJT+Oarep79l/YUxVnLf/g/Y/vqJXpfxWVazu+lft5oADjjLqpJpaTWjTktdk5PimiaKqxsprmmiqLqRkpoma11NI0crGiioaDippUpEmHBGSixD0uK4+MwsMhKjSIuPJt0uqafFR5GeYL1qnbjyN036ga78IOx4Dba/BiW7rVL8qEvg0odgxGyrm7vqUks78sLKRgqrGii0W7kUVjZwvNqaP17VSG2T85Rjo8LDyEyMJispmgnZyVx59iCGpMUxJC2OwWlxDEqOISJc689V/6BJPxA1Vlvd57f/AwrWW+uGnAef/zWM/4LeUO1EXZOTg6V1J00HSus4WFJLdePJCV0EshKjGZgcy+isBGaNymBAUgxZdoLPSoxhQFI0ybGRWoeugoYm/UDS3GCNk/L+U9BQblXXXPowTLjW6qKvAKtO/VhlA/tKajlQUsf+kloOlNRysLSOouqmk/bNTolleEY88ydnMzgtlkF265ZBKbFkJUYTqSV0FWI06QcCVzNs+iusfdJqQz/yYpizCHKm+Tsyv6h3OCmrdVBa20RZrYOyuiYKKhpOJPjSupPapafERTIyM4ELR2cyPDOeERnxDMuIZ1h6PDGRWoeulCdN+v7kdllVOO/9DCoOWSMXXvuCNfBVkDPGsKeolvf3lrD+YDlFNU2U2Um+ofnUQVrDw4QhaXGMzIznwjGZjMyMZ2RmAiMyE0iLj/LDJ1Cqf9Kk7w/GWOOOv/u4dXN24Nlw4z+snq5BXHdcXNPIh/tKeX9PKR/sK6W4xqqKGZ4Rz+C0OEZmxJOeEEV6gtXyJSMhunU5MyFaOxsp5QOa9PuSMXBgDax+zOodmz4avvRnGDe//WFx+7naJicbDpbz4T4rye8+XgNAWnwUM0dlcMGoDGaOziA7RTuIKdVXNOn3lSPrYfWjcOh9a1z1+c/CxBv8N9piL2hwuNj4WQUfHShl3f4ythVU4XIboiLCOGdYKj+YO5YLRmcwflCSDrillJ8ET8YJVMd3wLuPwZ63rY5UVzwJ0262xrfp5xxON5sPV7BufxkfHShjy+FKHC43EWHCxJxk7rxoJOeNTGfa0FS9oapUgNCk31vK9sOan1odq2KS4ZIHrSERouL9HVm3ud2G/MJq1u0v5cN9Zaw/WE5Ds4swgQnZyXx15jDOG5nOOcPSiNdnnSoVkPQv09dK98G638Hmv1ml+Qu+C+ffbY1Y2c8YYzhcXs8H+0pZt6+MdftLqahvBmBUVgLXnzOY80emM2NEOsmxkV2cTSkVCDTp+4LbbT1g5JPnYf9q67mh59wGF97f74ZJqGlsZt3+MtbuKWHt3hKOlDcAMDAphovHDmDmqHRm2j1XlVL9jyb9nmiotEr0G/5otbNPGAhzfgxTb4bEAf6Ozitut2H70arWJL/pcCUutyE+KpzzRmZw26wRzBqdwYiMeB2KQKkgoEm/O4ryYf1i2PZ365GAg8+16uzHzYPwwK/mcLkNH+wr5c3NR3nv0+LWKpsJ2Ul848IRXDgmk6lDUrVdvFJBSJP+6aguhBX3Wx2rwqPh7C/BjNth0CR/R+aVvUU1vLapgDc3H6Wouonk2EguGZvFhWMymTU6g4yE/t+iSCnVOU363jDGKtWv/L71DNk5P4bcr0F8ur8j61JFnYNlW4/x+qYCthVUER4mzB6TyUNX53DJuCwd312pEKNJvys1x+Ff98GnK2DwDJj/v5Axyt9Rdaquycl7n5awfOsxVu8uotllGDcoiUWfH8f8ydlkJmqJXqlQpUm/I8ZYg6Gt+B44G+HyJ+DcOyEsMEvGVQ3NrN5VxModx1m7p4Qmp5uMhCi+fO4wrp2WzVlnJPs7RKVUAPAq6YvIXOC3QDjwgjHm5+3scx3wMGCArcaYG+31LmC7vdthY8w8H8Tdu2qK7NL9vyHnHPjCc9YzZwNMaW0Tq/KtRL9uXylOt2FgUgwLpw9h7oSBnDMsjXAd7kAp5aHLpC8i4cCzwGVAAbBBRJYZY/I99hkN/AiYaYypEBHPxukNxpjJPo6792x/zbpZ66iHyx6D874VcKX7LUcqeWrVHj7YW4LbwJC0OL42azhzJwxkUk6KjmujlOqQNyX96cA+Y8wBABFZCswH8j32+TrwrDGmAsAYU+zrQPvEnnfg9a9Bdq5Vus8c4++ITrKvuIZfvbOHt3ceJz0+im/NGcUVEwYxblCitqFXSnnFm6SfDRzxWC4AZrTZZwyAiHyIVQX0sDHmbXtbjIjkAU7g58aYN9u+gYjcDtwOMGSInx4L2NwIK38AGWPgqyshInAezHG0soGnV+3h9U0FxEVFcN+lY/jaBcNJ0PFtlFKnyVdZIwIYDcwGcoC1InK2MaYSGGqMOSoiI4B3RWS7MWa/58HGmMXAYoDc3Fzjo5hOz0fPQMVB+PIbAZPwy2qb+N/39vN/H30GArfOHM4354zSJ0UppbrNm6R/FBjssZxjr/NUAHxijGkGDorIHqwvgQ3GmKMAxpgDIvIeMAXYTyCpPAJrf231qB15sb+jod7h5I9rD/LH9w9Q73CyYFoO9146Rh82opTqMW+S/gZgtIgMx0r2NwA3ttnnTWAh8KKIZGBV9xwQkVSg3hjTZK+fCfzSZ9H7yn9+bL1+7gn/xgH8N7+Ih5bt5GhlA1dMGMh3Lx/DqKxEf4ellAoSXSZ9Y4xTRO4C3sGqr19ijNkpIo8CecaYZfa2y0UkH3AB3zPGlInI+cDzIuIGwrDq9PM7eCv/2L8G8t+COYsgxU/3E4BjlQ08snwn7+wsYsyABP5xx3mcMyzNb/EopYKTGOOfKvSO5Obmmry8vL55M6cD/jATXM3wzY8hsu+HC3a63Px53SGeWrUHtzHce8kYvjZruA52ppQ6LSKy0RiT29V+od3845M/QOkeuPFVvyT8zYcreOCNHewqrObisVk8Mu8sBqfF9XkcSqnQEbpJv7oQ/t8vYMxcGPO5Pn3rqvpmfvnObl5ef5gBiTH84aapfO6sgdrWXinV60I36a96EFwOmPuzPn3bjw+Uce/SzZTUNHHrzOHcd9kYbW+vlOozoZltDn0I21+FC78HaSP65C2dLjfPvLuPZ97dy9D0eN761jmcnaODoCml+lboJX2X0xoXP3kwzPpOn7xlYVUD9y7dwvqD5VwzNZtH50/Q0r1Syi9CL/PkLYGiHXDdXyGq92+a/je/iPtf24rD6ebXX5rEtdNyev09lVKqI6GV9GtLYM3jMGK21fu2FzU5Xfx85W5e/PAQ4wcl8fsbpzAiM6FX31MppboSWkn//V+Dow6u+CX0YkuZg6V13PXyJnYeq+aW84fxoyvH6mMJlVIBIXSSvjHWQ1FGXw6ZZ/ba22w4VM4tS9YTGRHG4i9P4/KzBvbaeyml1OkKnaRfth8qD8P59/TaWxwqreP2v+aRlRTDS7fN4AwdIE0pFWBCp6///net11GX9Mrpq+qbufXPGzDAklvO0YSvlApIIZT0V0PqsF5pl+9wuvnG3/I4UlHP4i/nMjwj3ufvoZRSvhAaSd/pgIPvw0jfl/KNMfz4je18fKCcXy6YyPThOjKmUipwhUbSP/IxNNf1StXO/763n39sLOCeS0bzxSnaBl8pFdhCI+nvfxfCImDYBT497b+3FfLkO58yb9IZ3HfpaJ+eWymlekNoJP19qyFnOsQk+eyUmw5X8J1XtzBtaCq/XDBRR8hUSvULwZ/0a4vh+DYY5btn3x4pr+f2v+YxICmGxV+eRkykdrxSSvUPwd9O/8B71quPbuJWN1pNM5ucbpbefg7pCdE+Oa9SSvWF4C/p71sNsWkwaLJPTve9f2zlYGkdz980jVFZOpaOUqp/Ce6k73ZbN3FHzoGwnn/Ud3Ye552dRdz/uTM5f1SGDwJUSqm+FdxJv2gH1BX7pGqnrsnJw8t2MnZgIl+bNdwHwSmlVN8L7jr9lqEXRvb8Ju5vVu2hsKqR3984lcjw4P6uVEoFr+DOXvtXQ9ZZkDSoR6fZeayKF9cdYuH0IUwbmuqj4JRSqu95lfRFZK6IfCoi+0Tkhx3sc52I5IvIThF52WP9zSKy155u9lXgXXLUweGPe9xU0+02/PiNHaTERvKDub03JLNSSvWFLqt3RCQceBa4DCgANojIMmNMvsc+o4EfATONMRUikmWvTwMeAnIBA2y0j63w/Udp49AH4HL0uGrn5fWH2XKkkt9cP4mUuCgfBaeUUv7hTUl/OrDPGHPAGOMAlgLz2+zzdeDZlmRujCm2138OWGWMKbe3rQLm+ib0Lux/FyJiYcj53T5FSU0Tv3h7N+ePTOcLk7N9GJxSSvmHN0k/GzjisVxgr/M0BhgjIh+KyMciMvc0jkVEbheRPBHJKykp8T76zuxbDcNmQmRMt0/x+L/zaWp289gXJugwC0qpoOCrG7kRwGhgNrAQ+KOIpHh7sDFmsTEm1xiTm5mZ2fNoKg9D2d4eNdX8YG8pb205xh37xBjfAAAYEElEQVSzRzJSH2iulAoS3iT9o8Bgj+Uce52nAmCZMabZGHMQ2IP1JeDNsb63b7X12s36/MZmFz95awfD0uP45uyRPgxMKaX8y5ukvwEYLSLDRSQKuAFY1mafN7FK+YhIBlZ1zwHgHeByEUkVkVTgcntd79r/LiRld/sB6M+9t5+DpXU89oUJOpiaUiqodNl6xxjjFJG7sJJ1OLDEGLNTRB4F8owxyziR3PMBF/A9Y0wZgIg8hvXFAfCoMaa8Nz5IK5cTDvw/GD8PulEPf6Cklufe28+8SWdwwWgfVDUppVQA8apHrjFmBbCizboHPeYN8B17anvsEmBJz8I8DUc3QlNVt56SZYxh0Zs7iI4MY9FV43ohOKWU8q/g65G7fzVIGAy/6LQP/eRgOev2l3H/5WeSldj9Vj9KKRWogjDpvwtnTIW4039A+VtbjhEXFc6XcvVZt0qp4BRcSb+hwqre6UbVjsPpZsX2Qi4fP4C4qOAeh04pFbqCK+kfeA+Mu1vt89fuKaGqoZn52vNWKRXEgivp71sN0cmQPe20D31r6zFS4yKZNVofjqKUCl7Bk/SNgf1rYMSFEH561TN1TU5W5R/n8xMH6Vj5SqmgFjwZrvIzqD7araqdVflFNDa7tWpHKRX0gueOZeow+P4BCDv9j/TWlqNkp8QybYg+IEUpFdyCp6QPVjPNmKTTOqSstom1e0u5etIZhIXpSJpKqeAWXEm/G1bsOI7LbZg/+Qx/h6KUUr0u5JP+si1HGTMggbEDE/0dilJK9bqQTvoFFfVsOFTB/MnZ+pAUpVRICOmkv3xrIQDzJmnVjlIqNIR00n9ry1GmDklhcFqcv0NRSqk+EbJJ/9PjNew+XqNt85VSISVkk/6yrUcJDxOuPHuQv0NRSqk+E5JJ3xjDW1uOMXNUBpmJ0f4ORyml+kxIJv1NhyspqGhgvt7AVUqFmJBM+su2HCU6IozLzxrg71CUUqpPhVzSd7rc/GtbIZeOG0BiTKS/w1FKqT4Vckn/w/1llNU5mKfDLiilQlDIJf23thwlMSaC2Wdm+jsUpZTqcyGV9BubXbyz4zhXThhEdES4v8NRSqk+51XSF5G5IvKpiOwTkR+2s/0WESkRkS32dJvHNpfH+mW+DP50rd5VTJ3DpSNqKqVCVpdPHBGRcOBZ4DKgANggIsuMMfltdv27Meaudk7RYIyZ3PNQe+7d3cWkx0cxY0S6v0NRSim/8KakPx3YZ4w5YIxxAEuB+b0bVu8orGpgaHoc4fqwFKVUiPIm6WcDRzyWC+x1bV0rIttE5DURGeyxPkZE8kTkYxH5QntvICK32/vklZSUeB/9aSquaWJAUkyvnV8ppQKdr27kLgeGGWMmAquAv3hsG2qMyQVuBJ4WkZFtDzbGLDbG5BpjcjMze69VTVF1oyZ9pVRI8ybpHwU8S+459rpWxpgyY0yTvfgCMM1j21H79QDwHjClB/F2W4PDRU2jU8faUUqFNG+S/gZgtIgMF5Eo4AbgpFY4IuI5VOU8YJe9PlVEou35DGAm0PYGcJ8ormkE0JK+Uiqkddl6xxjjFJG7gHeAcGCJMWaniDwK5BljlgH3iMg8wAmUA7fYh48DnhcRN9YXzM/bafXTJ4qqrX9EsrSkr5QKYV0mfQBjzApgRZt1D3rM/wj4UTvHrQPO7mGMPlFUrSV9pZQKmR65xTVWSX9Akpb0lVKhK3SSfnUjURFhJMfqyJpKqdAVOkm/pomsxGhEtGOWUip0hUzS1zb6SikVYklfW+4opUJdyCR9HYJBKaVCJOnXO5zUNDrJ0pY7SqkQFxJJv7i1Y5aW9JVSoS00kr620VdKKSBEkn5Lb1wt6SulQl1IJH0t6SullCU0kr72xlVKKSBEkn5LG33tjauUCnUhkfS1jb5SSllCIulbQzBofb5SSoVE0i+ubtKWO0opRQgk/XqHk5om7Y2rlFIQAklfe+MqpdQJwZ/0tY2+Ukq1Cvqkr8/GVUqpE0Im6etY+kopFQJJv6SmSXvjKqWULeiTfksbfe2Nq5RSXiZ9EZkrIp+KyD4R+WE7228RkRIR2WJPt3lsu1lE9trTzb4M3htF2kZfKaVaRXS1g4iEA88ClwEFwAYRWWaMyW+z69+NMXe1OTYNeAjIBQyw0T62wifRe6G4ppEzByb21dsppVRA86akPx3YZ4w5YIxxAEuB+V6e/3PAKmNMuZ3oVwFzuxdq92hvXKWUOsGbpJ8NHPFYLrDXtXWtiGwTkddEZPDpHCsit4tInojklZSUeBl617Q3rlJKncxXN3KXA8OMMROxSvN/OZ2DjTGLjTG5xpjczMxMH4V0ojfuAC3pK6UU4F3SPwoM9ljOsde1MsaUGWOa7MUXgGneHtubWtvoa0lfKaUA75L+BmC0iAwXkSjgBmCZ5w4iMshjcR6wy55/B7hcRFJFJBW43F7XJ04MwaAlfaWUAi9a7xhjnCJyF1ayDgeWGGN2isijQJ4xZhlwj4jMA5xAOXCLfWy5iDyG9cUB8KgxprwXPke7Wodg0OodpZQCvEj6AMaYFcCKNuse9Jj/EfCjDo5dAizpQYzdVmz3xk2K9epjKqVU0AvqHrnF2htXKaVOEtRJv6i6Sat2lFLKQ1An/eKaRm25o5RSHoI76WtvXKWUOknQJn3tjauUUqcK2qSvvXGVUupUQZv09TGJSil1quBN+nZvXK3eUUqpE4I26Rdrb1yllDpF8Cb9miaitTeuUkqdJHiTfrXVRl974yql1AlBm/S1N65SSp0qeJO+9sZVSqlTBG3SL9HeuEopdYqgTPp1TVZvXG2jr5RSJwvKpN/yxKysRK3eUUopT8GZ9LU3rlJKtSsoG7EXtT4bV0v6Sp2O5uZmCgoKaGxs9HcoqgMxMTHk5OQQGRnZreODMum3lPT1Rq5Sp6egoIDExESGDRumfVwCkDGGsrIyCgoKGD58eLfOEZzVO9obV6luaWxsJD09XRN+gBIR0tPTe/SfWFAm/SLtjatUt+nfTWDr6c8nKJN+sfbGVUqpdgVl0i+qadSWO0r1Q2VlZUyePJnJkyczcOBAsrOzW5cdDodX5/jqV7/Kp59+2uk+zz77LC+99JIvQu53vKr0FpG5wG+BcOAFY8zPO9jvWuA14BxjTJ6IDAN2AS0/gY+NMXf0NOiuFFc3ceFobbmjVH+Tnp7Oli1bAHj44YdJSEjg/vvvP2kfYwzGGMLC2i+zvvjii12+z7e+9a2eB9tPdZn0RSQceBa4DCgANojIMmNMfpv9EoF7gU/anGK/MWayj+LtUl2Tk1rtjatUjz2yfCf5x6p9es7xZyTx0NVnnfZx+/btY968eUyZMoXNmzezatUqHnnkETZt2kRDQwPXX389Dz74IACzZs3i97//PRMmTCAjI4M77riDlStXEhcXx1tvvUVWVhaLFi0iIyODb3/728yaNYtZs2bx7rvvUlVVxYsvvsj5559PXV0dX/nKV9i1axfjx4/n0KFDvPDCC0yefHI6e+ihh1ixYgUNDQ3MmjWL5557DhFhz5493HHHHZSVlREeHs4///lPhg0bxk9/+lNeeeUVwsLCuOqqq3jiiSd8cm295U31znRgnzHmgDHGASwF5rez32PALwC/NvAt1jb6SgWl3bt3c99995Gfn092djY///nPycvLY+vWraxatYr8/PxTjqmqquKiiy5i69atnHfeeSxZsqTdcxtjWL9+PU8++SSPPvooAM888wwDBw4kPz+fn/zkJ2zevLndY++99142bNjA9u3bqaqq4u233wZg4cKF3HfffWzdupV169aRlZXF8uXLWblyJevXr2fr1q1897vf9dHV8Z431TvZwBGP5QJghucOIjIVGGyM+beIfK/N8cNFZDNQDSwyxrzf9g1E5HbgdoAhQ4acRvin0jb6SvlGd0rkvWnkyJHk5ua2Lr/yyiv86U9/wul0cuzYMfLz8xk/fvxJx8TGxnLFFVcAMG3aNN5//5T0A8A111zTus+hQ4cA+OCDD/jBD34AwKRJkzjrrPavx+rVq3nyySdpbGyktLSUadOmce6551JaWsrVV18NWB2qAP773/9y6623EhsbC0BaWlp3LkWP9Lghu4iEAU8Bt7SzuRAYYowpE5FpwJsicpYx5qT/GY0xi4HFALm5uaYn8WhvXKWCU3x8fOv83r17+e1vf8v69etJSUnhpptuarftelRUVOt8eHg4Tqez3XNHR0d3uU976uvrueuuu9i0aRPZ2dksWrQo4Hsze1O9cxQY7LGcY69rkQhMAN4TkUPAucAyEck1xjQZY8oAjDEbgf3AGF8E3pHWkr7W6SsVtKqrq0lMTCQpKYnCwkLeeecdn7/HzJkzefXVVwHYvn17u9VHDQ0NhIWFkZGRQU1NDa+//joAqampZGZmsnz5csDq9FZfX89ll13GkiVLaGhoAKC8vNzncXfFm5L+BmC0iAzHSvY3ADe2bDTGVAEZLcsi8h5wv916JxMoN8a4RGQEMBo44MP4T9HaGzdGe+MqFaymTp3K+PHjGTt2LEOHDmXmzJk+f4+7776br3zlK4wfP751Sk5OPmmf9PR0br75ZsaPH8+gQYOYMeNEzfdLL73EN77xDX784x8TFRXF66+/zlVXXcXWrVvJzc0lMjKSq6++mscee8znsXdGjOm6NkVErgSexmqyucQY84SIPArkGWOWtdn3PU4k/WuBR4FmwA08ZIxZ3tl75ebmmry8vG59GIB7l25m8+FK1n5/TrfPoVSo2rVrF+PGjfN3GAHB6XTidDqJiYlh7969XH755ezdu5eICP8XKNv7OYnIRmNMbgeHtPIqemPMCmBFm3UPdrDvbI/514HXvXkPXymqbtRx9JVSPVZbW8sll1yC0+nEGMPzzz8fEAm/p/r/J2ijuKaJcQOT/B2GUqqfS0lJYePGjf4Ow+eCbhiG4uomfSC6Ukp1IKiSfktvXG2jr5RS7QuqpK+9cZVSqnNBlfSL9Nm4SinVqaBK+i0lfW29o1T/NGfOnFM6Wj399NPceeednR6XkJAAwLFjx1iwYEG7+8yePZuumoM//fTT1NfXty5feeWVVFZWehN6vxFcSV974yrVry1cuJClS5eetG7p0qUsXLjQq+PPOOMMXnvttW6/f9ukv2LFClJSUrp9vkAUVE02i6obtTeuUr6y8odwfLtvzznwbLii3cdxALBgwQIWLVqEw+EgKiqKQ4cOcezYMS644AJqa2uZP38+FRUVNDc38/jjjzN//skD/h46dIirrrqKHTt20NDQwFe/+lW2bt3K2LFjW4c+ALjzzjvZsGEDDQ0NLFiwgEceeYTf/e53HDt2jDlz5pCRkcGaNWsYNmwYeXl5ZGRk8NRTT7WO0nnbbbfx7W9/m0OHDnHFFVcwa9Ys1q1bR3Z2Nm+99VbrgGotli9fzuOPP47D4SA9PZ2XXnqJAQMGUFtby913301eXh4iwkMPPcS1117L22+/zQMPPIDL5SIjI4PVq1f77EcQVNmxuKaJAUkx+oxPpfqptLQ0pk+fzsqVK5k/fz5Lly7luuuuQ0SIiYnhjTfeICkpidLSUs4991zmzZvX4d/7c889R1xcHLt27WLbtm1MnTq1ddsTTzxBWloaLpeLSy65hG3btnHPPffw1FNPsWbNGjIyMk4618aNG3nxxRf55JNPMMYwY8YMLrroIlJTU9m7dy+vvPIKf/zjH7nuuut4/fXXuemmm046ftasWXz88ceICC+88AK//OUv+fWvf81jjz1GcnIy27dbX64VFRWUlJTw9a9/nbVr1zJ8+HCfj88TVEm/qLpRW+4o5SudlMh7U0sVT0vS/9Of/gRYY94/8MADrF27lrCwMI4ePUpRUREDBw5s9zxr167lnnvuAWDixIlMnDixddurr77K4sWLcTqdFBYWkp+ff9L2tj744AO++MUvto70ec011/D+++8zb948hg8f3vpgFc+hmT0VFBRw/fXXU1hYiMPhYPjw4YA11LJndVZqairLly/nwgsvbN3H18MvB1mdfpO20Veqn5s/fz6rV69m06ZN1NfXM23aNMAawKykpISNGzeyZcsWBgwY0K1hjA8ePMivfvUrVq9ezbZt2/j85z/fo+GQW4Zlho6HZr777ru566672L59O88//7xfh18OrqRfo71xlervEhISmDNnDrfeeutJN3CrqqrIysoiMjKSNWvW8Nlnn3V6ngsvvJCXX34ZgB07drBt2zbAGpY5Pj6e5ORkioqKWLlyZesxiYmJ1NTUnHKuCy64gDfffJP6+nrq6up44403uOCCC7z+TFVVVWRnZwPwl7/8pXX9ZZddxrPPPtu6XFFRwbnnnsvatWs5ePAg4Pvhl4Mm6euzcZUKHgsXLmTr1q0nJf3/+Z//IS8vj7PPPpu//vWvjB07ttNz3HnnndTW1jJu3DgefPDB1v8YJk2axJQpUxg7diw33njjScMy33777cydO5c5c04epXfq1KnccsstTJ8+nRkzZnDbbbcxZcoUrz/Pww8/zJe+9CWmTZt20v2CRYsWUVFRwYQJE5g0aRJr1qwhMzOTxYsXc8011zBp0iSuv/56r9/HG14NrdyXuju0ckWdgweX7eRL03K4cExmL0SmVPDToZX7h14fWrk/SI2P4pmF3n/zKqVUKAqa6h2llFJd06SvlDpJoFX5qpP19OejSV8p1SomJoaysjJN/AHKGENZWRkxMd1vsBI0dfpKqZ7LycmhoKCAkpISf4eiOhATE0NOTk63j9ekr5RqFRkZ2doTVAUnrd5RSqkQoklfKaVCiCZ9pZQKIQHXI1dESoCOBtXIAEr7MJzu6i9xQv+JVeP0vf4Sq8bpnaHGmC6HIwi4pN8ZEcnzppuxv/WXOKH/xKpx+l5/iVXj9C2t3lFKqRCiSV8ppUJIf0v6i/0dgJf6S5zQf2LVOH2vv8SqcfpQv6rTV0op1TP9raSvlFKqBzTpK6VUCOk3SV9E5orIpyKyT0R+6O94OiIih0Rku4hsEZHTfwRYLxKRJSJSLCI7PNalicgqEdlrv6b6M0Y7pvbifFhEjtrXdYuIXOnPGO2YBovIGhHJF5GdInKvvT6grmkncQbUNRWRGBFZLyJb7TgfsdcPF5FP7L/9v4tIlD/j7CLWP4vIQY9rOtnfsZ7CGBPwExAO7AdGAFHAVmC8v+PqINZDQIa/4+ggtguBqcAOj3W/BH5oz/8Q+EWAxvkwcL+/Y2sT5yBgqj2fCOwBxgfaNe0kzoC6poAACfZ8JPAJcC7wKnCDvf4PwJ0BHOufgQX+jq+zqb+U9KcD+4wxB4wxDmApMN/PMfU7xpi1QHmb1fOBv9jzfwG+0KdBtaODOAOOMabQGLPJnq8BdgHZBNg17STOgGIstfZipD0Z4GLgNXu9368ndBprwOsvST8bOOKxXEAA/tLaDPAfEdkoIrf7OxgvDDDGFNrzx4EB/gymC3eJyDa7+sfv1VCeRGQYMAWrxBew17RNnBBg11REwkVkC1AMrML6D7/SGOO0dwmYv/22sRpjWq7pE/Y1/Y2IRPsxxHb1l6Tfn8wyxkwFrgC+JSIX+jsgbxnrf9VALa08B4wEJgOFwK/9G84JIpIAvA582xhT7bktkK5pO3EG3DU1xriMMZOBHKz/8Mf6OaQOtY1VRCYAP8KK+RwgDfiBH0NsV39J+keBwR7LOfa6gGOMOWq/FgNvYP3iBrIiERkEYL8W+zmedhljiuw/MjfwRwLkuopIJFYifckY8097dcBd0/biDNRrCmCMqQTWAOcBKSLS8sCngPvb94h1rl2VZowxTcCLBNA1bdFfkv4GYLR9Fz8KuAFY5ueYTiEi8SKS2DIPXA7s6Pwov1sG3GzP3wy85cdYOtSSRG1fJACuq4gI8CdglzHmKY9NAXVNO4oz0K6piGSKSIo9HwtchnX/YQ2wwN7N79cTOox1t8eXvWDde/D772lb/aZHrt2c7GmsljxLjDFP+DmkU4jICKzSPViPonw5kOIUkVeA2VhDwBYBDwFvYrWOGII1pPV1xhi/3kTtIM7ZWNUQBquF1Dc86s39QkRmAe8D2wG3vfoBrPrygLmmncS5kAC6piIyEetGbThWgfRVY8yj9t/VUqzqks3ATXZJ2m86ifVdIBOrdc8W4A6PG74Bod8kfaWUUj3XX6p3lFJK+YAmfaWUCiGa9JVSKoRo0ldKqRCiSV8ppUKIJn2llAohmvSVUiqE/H+ULZh6BMJeIwAAAABJRU5ErkJggg==\n"},"metadata":{}},{"output_type":"display_data","data":{"text/plain":"<Figure size 432x288 with 1 Axes>","image/png":"iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xl8VOXZ8PHflclM9j0BJCEEXICwCBiXFhFwq0uVYtG6oNVqqT62trW+r9S3rVt9HrU+arVWa1u11q3WfaeLVLRaFBDZ90XDlgWyr5Nc7x/nJATMMkkmmczk+n4+8zlnZu5zzjUncM0997nPfYuqYowxJrJEhToAY4wxwWfJ3RhjIpAld2OMiUCW3I0xJgJZcjfGmAhkyd0YYyKQJXfTLhHxiEiViOQGs2woicgRIhL0vr8icqqIbG/zfIOITA+kbA+O9QcRuamn23ey31+KyBPB3q8JnehQB2CCQ0Sq2jyNB+qBJvf591T16e7sT1WbgMRglx0MVHVMMPYjIlcB81R1Zpt9XxWMfZvIZ8k9Qqhqa3J1a4ZXqeo/OiovItGq6u+P2Iwx/c+aZQYJ92f3X0TkWRGpBOaJyFdE5D8iUiYiu0XkARHxuuWjRURFJM99/pT7/tsiUikiH4nIqO6Wdd8/U0Q2iki5iDwoIv8Wkcs7iDuQGL8nIptFZL+IPNBmW4+I3CcipSKyFTijk/Pz/0TkuUNee0hE7nXXrxKRde7n2eLWqjvaV6GIzHTX40Xkz25sa4BjDin7MxHZ6u53jYic674+EfgNMN1t8ippc25vabP91e5nLxWRV0TksEDOTVdEZI4bT5mIvCsiY9q8d5OI7BKRChFZ3+azniAiy93X94rIrwI9nukDqmqPCHsA24FTD3ntl0ADcA7Ol3occCxwPM4vuNHARuD7bvloQIE89/lTQAlQAHiBvwBP9aDsEKASmO2+dz3QCFzewWcJJMZXgRQgD9jX8tmB7wNrgBwgA1js/JNv9zijgSogoc2+i4AC9/k5bhkBTgZqgUnue6cC29vsqxCY6a7fA/wLSANGAmsPKXsBcJj7N7nYjWGo+95VwL8OifMp4BZ3/XQ3xslALPBb4N1Azk07n/+XwBPu+jg3jpPdv9FNwAZ3fTywAxjmlh0FjHbXPwEucteTgOND/X9hMD+s5j64fKCqr6tqs6rWquonqrpEVf2quhV4FJjRyfYvqOpSVW0EnsZJKt0t+3Vghaq+6r53H84XQbsCjPF/VLVcVbfjJNKWY10A3KeqhapaCtzZyXG2AqtxvnQATgP2q+pS9/3XVXWrOt4F/gm0e9H0EBcAv1TV/aq6A6c23va4z6vqbvdv8gzOF3NBAPsFuAT4g6quUNU6YAEwQ0Ry2pTp6Nx05kLgNVV91/0b3YnzBXE84Mf5IhnvNu1tc88dOF/SR4pIhqpWquqSAD+H6QOW3AeXL9o+EZGxIvKmiOwRkQrgNiCzk+33tFmvofOLqB2VHd42DlVVnJpuuwKMMaBj4dQ4O/MMcJG7frH7vCWOr4vIEhHZJyJlOLXmzs5Vi8M6i0FELheRz9zmjzJgbID7Befzte5PVSuA/UB2mzLd+Zt1tN9mnL9RtqpuAH6C83cocpv5hrlFrwDygQ0i8rGInBXg5zB9wJL74HJoN8Df4dRWj1DVZOAXOM0OfWk3TjMJACIiHJyMDtWbGHcDI9o876qr5vPAqSKSjVODf8aNMQ54AfgfnCaTVOBvAcaxp6MYRGQ08DBwDZDh7nd9m/121W1zF05TT8v+knCaf3YGEFd39huF8zfbCaCqT6nqNJwmGQ/OeUFVN6jqhThNb/8LvCgisb2MxfSQJffBLQkoB6pFZBzwvX445hvAVBE5R0SigR8CWX0U4/PAj0QkW0QygBs7K6yqe4APgCeADaq6yX0rBvABxUCTiHwdOKUbMdwkIqni3Afw/TbvJeIk8GKc77nv4tTcW+wFclouILfjWeBKEZkkIjE4SfZ9Ve3wl1A3Yj5XRGa6x/4/ONdJlojIOBGZ5R6v1n0043yAS0Uk063pl7ufrbmXsZgesuQ+uP0E+DbOf9zf4Vz47FOquhf4FnAvUAocDnyK0y8/2DE+jNM2vgrnYt8LAWzzDM4F0tYmGVUtA34MvIxzUXIuzpdUIG7G+QWxHXgbeLLNflcCDwIfu2XGAG3bqf8ObAL2ikjb5pWW7d/BaR552d0+F6cdvldUdQ3OOX8Y54vnDOBct/09Brgb5zrJHpxfCv/P3fQsYJ04vbHuAb6lqg29jcf0jDhNnsaEhoh4cJoB5qrq+6GOx5hIYTV30+9E5Ay3mSIG+DlOL4uPQxyWMRHFkrsJhROBrTg/+b8GzFHVjppljDE9YM0yxhgTgazmbowxEShkA4dlZmZqXl5eqA5vjDFhadmyZSWq2ln3YSCEyT0vL4+lS5eG6vDGGBOWRKSrO60Ba5YxxpiIZMndGGMikCV3Y4yJQDYTkzGDRGNjI4WFhdTV1YU6FBOA2NhYcnJy8Ho7Glqoc5bcjRkkCgsLSUpKIi8vD2cwTjNQqSqlpaUUFhYyatSorjdohzXLGDNI1NXVkZGRYYk9DIgIGRkZvfqVZcndmEHEEnv46O3fKuyS+4Y9lfxq4XrKamwkUWOM6UjYJfftpdU8tGgLhftrQx2KMaYbSktLmTx5MpMnT2bYsGFkZ2e3Pm9oCKyydsUVV7Bhw4ZOyzz00EM8/fTTwQiZE088kRUrVgRlX/0t7C6oZibGAFBcZYMIGhNOMjIyWhPlLbfcQmJiIjfccMNBZVQVVSUqqv165+OPP97lca699treBxsBwq7mnuUm95JKS+7GRILNmzeTn5/PJZdcwvjx49m9ezfz58+noKCA8ePHc9ttt7WWbalJ+/1+UlNTWbBgAUcffTRf+cpXKCoqAuBnP/sZ999/f2v5BQsWcNxxxzFmzBg+/PBDAKqrq/nmN79Jfn4+c+fOpaCgoMsa+lNPPcXEiROZMGECN910EwB+v59LL7209fUHHngAgPvuu4/8/HwmTZrEvHnzgn7OAtFlzV1ERuBMDTYUZ07ER1X11x2UPRb4CLhQVQOZ0qzbMpN8AJRUWZu7MT116+trWLurIqj7zB+ezM3njO/RtuvXr+fJJ5+koKAAgDvvvJP09HT8fj+zZs1i7ty55OfnH7RNeXk5M2bM4M477+T666/nscceY8GCBV/at6ry8ccf89prr3Hbbbfxzjvv8OCDDzJs2DBefPFFPvvsM6ZOndppfIWFhfzsZz9j6dKlpKSkcOqpp/LGG2+QlZVFSUkJq1atAqCsrAyAu+++mx07duDz+Vpf62+B1Nz9wE9UNR84AbhWRPIPLeROl3YXzqzwfSbeF028z0OJNcsYEzEOP/zw1sQO8OyzzzJ16lSmTp3KunXrWLt27Ze2iYuL48wzzwTgmGOOYfv27e3u+7zzzvtSmQ8++IALL7wQgKOPPprx4zv/UlqyZAknn3wymZmZeL1eLr74YhYvXswRRxzBhg0buO6661i4cCEpKSkAjB8/nnnz5vH000/3+Cak3uqy5q6qu3Em30VVK0VkHZANHHq2fwC8CBwb7CAPlZkYY8ndmF7oaQ27ryQkJLSub9q0iV//+td8/PHHpKamMm/evHb7e/t8vtZ1j8eD3+9vd98xMTFdlumpjIwMVq5cydtvv81DDz3Eiy++yKOPPsrChQt57733eO211/jv//5vVq5cicfjCeqxu9KtNncRyQOmcPAM7YhINjAHZ7b0PpeZ6LPkbkyEqqioICkpieTkZHbv3s3ChQuDfoxp06bx/PPPA7Bq1ap2fxm0dfzxx7No0SJKS0vx+/0899xzzJgxg+LiYlSV888/n9tuu43ly5fT1NREYWEhJ598MnfffTclJSXU1NQE/TN0JeDeMiKSiFMz/5GqHtpYdz9wo6o2d9bxXkTmA/MBcnNzux+tKzMxhh2l/X+yjDF9b+rUqeTn5zN27FhGjhzJtGnTgn6MH/zgB1x22WXk5+e3PlqaVNqTk5PD7bffzsyZM1FVzjnnHM4++2yWL1/OlVdeiaoiItx11134/X4uvvhiKisraW5u5oYbbiApKSnon6ErAc2hKiJe4A1goare287724CWrJ4J1ADzVfWVjvZZUFCgPZ2s46aXV7Fw9R6W/fy0Hm1vzGC0bt06xo0bF+owBgS/34/f7yc2NpZNmzZx+umns2nTJqKjB1bv8Pb+ZiKyTFULOtikVSC9ZQT4I7CuvcQOoKqj2pR/Anijs8TeW5mJMeyracDf1Ey0J+x6cxpjQqyqqopTTjkFv9+PqvK73/1uwCX23grk00wDLgVWiUhLR9CbgFwAVX2kj2LrUFaiD1XYV9PAkKTY/j68MSbMpaamsmzZslCH0acC6S3zAQeaXLqkqpf3JqBAZLbeyGTJ3Rhj2hOWbRqZSW5ytx4zxhjTrvBM7omW3I0xpjNhmtxbhiCw5G6MMe0Jy+SeGBNNTHSUjS9jTBiZNWvWl25Iuv/++7nmmms63S4xMRGAXbt2MXfu3HbLzJw5k666Vt9///0H3Ux01llnBWXcl1tuuYV77rmn1/sJtrBM7iLiDEFgI0MaEzYuuuginnvuuYNee+6557jooosC2n748OG88ELPxyM8NLm/9dZbpKam9nh/A11YJndwLqramO7GhI+5c+fy5ptvtk7MsX37dnbt2sX06dNb+51PnTqViRMn8uqrr35p++3btzNhwgQAamtrufDCCxk3bhxz5syhtvbA5D3XXHNN63DBN998MwAPPPAAu3btYtasWcyaNQuAvLw8SkpKALj33nuZMGECEyZMaB0uePv27YwbN47vfve7jB8/ntNPP/2g47RnxYoVnHDCCUyaNIk5c+awf//+1uO3DAHcMmDZe++91zpZyZQpU6isrOzxuW1P2Pbaz0r0sbOs55PHGjOovb0A9qwK7j6HTYQz7+zw7fT0dI477jjefvttZs+ezXPPPccFF1yAiBAbG8vLL79McnIyJSUlnHDCCZx77rkdziP68MMPEx8fz7p161i5cuVBQ/becccdpKen09TUxCmnnMLKlSu57rrruPfee1m0aBGZmZkH7WvZsmU8/vjjLFmyBFXl+OOPZ8aMGaSlpbFp0yaeffZZfv/733PBBRfw4osvdjo++2WXXcaDDz7IjBkz+MUvfsGtt97K/fffz5133sm2bduIiYlpbQq65557eOihh5g2bRpVVVXExga3W3f41txtZEhjwk7bppm2TTKqyk033cSkSZM49dRT2blzJ3v37u1wP4sXL25NspMmTWLSpEmt7z3//PNMnTqVKVOmsGbNmi4HBfvggw+YM2cOCQkJJCYmct555/H+++8DMGrUKCZPngx0PqwwOOPLl5WVMWPGDAC+/e1vs3jx4tYYL7nkEp566qnWO2GnTZvG9ddfzwMPPEBZWVnQ75AN25p7ZmIM+6obaG5WoqJsRndjuqWTGnZfmj17Nj/+8Y9Zvnw5NTU1HHPMMQA8/fTTFBcXs2zZMrxeL3l5ee0O89uVbdu2cc899/DJJ5+QlpbG5Zdf3qP9tGgZLhicIYO7apbpyJtvvsnixYt5/fXXueOOO1i1ahULFizg7LPP5q233mLatGksXLiQsWPH9jjWQ4Vxzd1HU7Oyv8Z6zBgTLhITE5k1axbf+c53DrqQWl5ezpAhQ/B6vSxatIgdO3Z0up+TTjqJZ555BoDVq1ezcuVKwBkuOCEhgZSUFPbu3cvbb7/duk1SUlK77drTp0/nlVdeoaamhurqal5++WWmT5/e7c+WkpJCWlpaa63/z3/+MzNmzKC5uZkvvviCWbNmcdddd1FeXk5VVRVbtmxh4sSJ3HjjjRx77LGsX7++28fsTPjW3FvvUm0gIzGmi9LGmIHioosuYs6cOQf1nLnkkks455xzmDhxIgUFBV3WYK+55hquuOIKxo0bx7hx41p/ARx99NFMmTKFsWPHMmLEiIOGC54/fz5nnHEGw4cPZ9GiRa2vT506lcsvv5zjjjsOgKuuuoopU6Z02gTTkT/96U9cffXV1NTUMHr0aB5//HGampqYN28e5eXlqCrXXXcdqamp/PznP2fRokVERUUxfvz41lmlgiWgIX/7Qm+G/AX4z9ZSLnz0Pzx91fFMOyKz6w2MGeRsyN/w05shf8O4WcaGIDDGmI6EbXLPcpN7sd3IZIwxXxK2yT05Lhqfx4YgMKY7QtUMa7qvt3+rsE3uIkKGTZRtTMBiY2MpLS21BB8GVJXS0tJe3dgUtr1lwG5kMqY7cnJyKCwspLi4ONShmADExsaSk5PT4+0DmUN1BPAkMBRQ4FFV/fUhZS4BbsSZsakSuEZVP+txVAHKTPTZ+DLGBMjr9TJq1KiuC5qIEEizjB/4iarmAycA14pI/iFltgEzVHUicDvwaHDDbJ8zMqS1uRtjzKG6TO6qultVl7vrlcA6IPuQMh+q6n736X+Anv+W6IbMpBhKq+tpbrY2RGOMaatbF1RFJA+YAizppNiVwNudvB80mYkxNDYp5bWN/XE4Y4wJGwFfUBWRROBF4EeqWtFBmVk4yf3EDt6fD8wHyM3N7Xawh2o73V5agq/X+zPGmEgRUM1dRLw4if1pVX2pgzKTgD8As1W1tL0yqvqoqhaoakFWVlZPY27VeiOTXVQ1xpiDdJncxRkt/4/AOlW9t4MyucBLwKWqujG4IXas7eBhxhhjDgikWWYacCmwSkRWuK/dBOQCqOojwC+ADOC37swp/kAGtumt1vFlbAgCY4w5SJfJXVU/wOm/3lmZq4CrghVUoFLjvHiixG5kMsaYQ4Tt8AMAUVFCpg1BYIwxXxLWyR1ahiCwNndjjGkrQpK71dyNMaatyEjudkHVGGMOEv7JPclHSVWDDWNqjDFthH1yz0qMoaGpmYo6f6hDMcaYASPsk7vNpWqMMV8WOcnd2t2NMaZV+Cf3pJbBw6w7pDHGtAj/5G7NMsYY8yVhn9zT4n1EiSV3Y4xpK+yTuydKSE+wG5mMMaatsE/u4E6UbXOpGmNMq4hI7llJVnM3xpi2IiK52/gyxhhzsAhJ7s6wvzYEgTHGOCIkucdQ19hMdUNTqEMxxpgBIZA5VEeIyCIRWSsia0Tkh+2UERF5QEQ2i8hKEZnaN+G2z+5SNcaYgwVSc/cDP1HVfOAE4FoRyT+kzJnAke5jPvBwUKPswoGJsi25G2MMBJDcVXW3qi531yuBdUD2IcVmA0+q4z9AqogcFvRoO5CZ2DIEgSV3Y4yBbra5i0geMAVYcshb2cAXbZ4X8uUvgD6T5TbLFNv4MsYYA3QjuYtIIvAi8CNVrejJwURkvogsFZGlxcXFPdlFu9ITfIhYm7sxxrQIKLmLiBcnsT+tqi+1U2QnMKLN8xz3tYOo6qOqWqCqBVlZWT2Jt13RnijS4n3WLGOMMa5AessI8Edgnare20Gx14DL3F4zJwDlqro7iHF2qaWvuzHGGIgOoMw04FJglYiscF+7CcgFUNVHgLeAs4DNQA1wRfBD7Zxzl6q1uRtjDASQ3FX1A0C6KKPAtcEKqicyE2P4rLAslCEYY8yAERF3qIJbc7cLqsYYA0RSck/yUd3QRK0NQWCMMRGU3G26PWOMaRUxyf3AjUyW3I0xJmKSe0vNvdja3Y0xJoKSe5KNL2OMMS0iJrlnJLQM+2t93Y0xJmKSuy86ipQ4r9XcjTGGCEruYEMQGGNMiwhL7jZRtjHGQKQl9yQbX8YYYyDCknuWDUFgjDFAhCX3zEQflfV+6hptCAJjzOAWYcndhiAwxhiI2ORu7e7GmMEtopJ7VlLLjUxWczfGDG4Rldwzk6xZxhhjILA5VB8TkSIRWd3B+yki8rqIfCYia0Sk36fYa5GRYOPLGGMMBFZzfwI4o5P3rwXWqurRwEzgf0XE1/vQui/W6yEpNtra3I0xg16XyV1VFwP7OisCJImIAIluWX9wwuu+rMQYG9PdGDPoBaPN/TfAOGAXsAr4oao2t1dQROaLyFIRWVpcXByEQ3+ZzaVqjDHBSe5fA1YAw4HJwG9EJLm9gqr6qKoWqGpBVlZWEA79ZZlJNniYMcYEI7lfAbykjs3ANmBsEPbbI87gYdbmbowZ3IKR3D8HTgEQkaHAGGBrEPbbI5mJMZTXNtLgb7dlyBhjBoXorgqIyLM4vWAyRaQQuBnwAqjqI8DtwBMisgoQ4EZVLemziLvQcpdqaXU9h6XEhSoMY4wJqS6Tu6pe1MX7u4DTgxZRL2Umun3dKxssuRtjBq2IukMV7C5VY4yBCEzuWW6zjPV1N8YMZhGX3G3YX2OMicDkHufzkODzUFJp3SGNMYNXxCV3gLzMBD79Yn+owzDGmJCJyOT+jcnZfPp5GZuLqkIdijHGhEREJvfZU4bjiRJeWFYY6lCMMSYkIjK5D0mKZdaYLF7+tJCmZg11OMYY0+8iMrkDzD0mh70V9Sze1DejTxpjzEAWscn95LFDSYv3WtOMMWZQitjk7ouOYvbkbP6+Zi9lNdYt0hgzuERscgc4vyCHhqZmXvtsV6hDMcaYfhXRyX388BTGHZZsTTPGmEEnopM7wPnH5LCysJwNeypDHYoxxvSbiE/u35iSjdcjvLDsi1CHYowx/Sb8knuTHz7/D2hg/dfTE3ycPHYIL3+6k8Ymm53JGDM4hF9y/+xZeOxrsHdNwJucf8wISqoaeG+D9Xk3xgwOXSZ3EXlMRIpEZHUnZWaKyAoRWSMi7wU3xEMceTogsOHtgDeZMSaLzEQff7WmGWPMIBFIzf0J4IyO3hSRVOC3wLmqOh44PzihdSBpKOQUwIY3A97E64lizpRs/rmuiFIb590YMwh0mdxVdTGwr5MiFwMvqernbvmiIMXWsTFnwq5PoSLw/utzjxmBv1l5dYX1eTfGRL5gtLkfBaSJyL9EZJmIXNZRQRGZLyJLRWRpcXEv2r/HnO0su9E0M2ZYEpNyUqzPuzFmUAhGco8GjgHOBr4G/FxEjmqvoKo+qqoFqlqQlZXV8yNmjYG0Ud1K7uAMJrZ2dwVrdpX3/NjGGBMGgpHcC4GFqlqtqiXAYuDoIOy3YyIw9mzY9h7UB35z0rlHD8fnibLauzEm4gUjub8KnCgi0SISDxwPrAvCfjs35kxoaoAt7wa8SWq8j9Pyh/Lqil00+K3PuzEmcgXSFfJZ4CNgjIgUisiVInK1iFwNoKrrgHeAlcDHwB9UtcNuk0Ez4gSIS4P1b3Vrs7kFOeyrbuDd9Xv7KDBjjAm96K4KqOpFAZT5FfCroEQUKE80HPk12LTQuWvV0+VHAWD6EZkMSYrhhWWFnDHhsD4O0hhjQiP87lBta+xZULsfvvhPwJtEe6I4b2oOizYU2wTaxpiIFd7J/fCTwePrdq+ZK08cRVJsNDe+uNLmWDXGRKTwTu4xSTBqBqx/M+CBxACykmL4xdfzWbZjP09+tL3PwjPGmFAJ7+QOTq+Z/dugeEO3NpszJZuZY7K4+50NfLGvpo+CM8aY0IiM5A7dGmsGQES4Y85EogR++tIqtBs1f2OMGejCP7knD4fhU7rd7g6QnRrHgrPG8cHmEv661G5sMsZEjvBP7uCMNVO4FCq733f9kuNyOW5UOre/uZa9FXV9EJwxxvS/CEnuZwIKG9/p9qZRUcJd35xEg7+Zn7+y2ppnjDERITKS+9DxkJoLG7p3t2qLUZkJXH/aUfxt7V7eXLU7yMEZY0z/i4zkLgJjzoKt/4KG6h7t4soTRzExO4WbX13D/uqG4MZnjDH9LDKSOzjJ3V8HWxb1aPNoTxR3z51EeW0jt72xNsjBGWNM/4qc5D7yqxCb0qNeMy3GHZbMf806gpc/3cmi9X0/oZQxxvSVyEnuHq8zefbGd6C5qce7uXbW4Rw5JJGbXl5FZV1jEAM0xpj+EznJHZxeMzUlUPhJj3cRE+3h7rmT2FNRxw1//Qx/k437bowJP5GV3I84FaK8zlgzvTAlN41ffD2fhWv28n9fWEmzDS5mjAkzkZXcY1Mg78Retbu3uGLaKH5y2lG89OlObn5tjfV/N8aElchK7uDMrVq6CUo29XpX3z/5CL43YzR//s8O7npngyV4Y0zYCGSavcdEpEhEOp06T0SOFRG/iMwNXng9cNQZznL9G73elYiw4IyxzDshl0fe28Jv/7Wl1/s0xpj+EEjN/QngjM4KiIgHuAv4WxBi6p3UEZD7VfjwQagu7fXuRITbzp3AeVOy+dXCDTz+721BCNIYY/pWl8ldVRcD+7oo9gPgRWBgdA4/+x6oK4eFNwVld1FRwt1zJ/G18UO59fW1PL/0i6Ds1xhj+kqv29xFJBuYAzwcQNn5IrJURJYWFxf39tAdGzoeTrweVj4Hm/4RlF1Ge6J44KIpTD8ykwUvruTNlTYGjTFm4ArGBdX7gRtVtcsO4ar6qKoWqGpBVlZWEA7diZNugMwx8MaPoD44E2HHRHt49NICjhmZxg+f+5R313d/iGFjjOkPwUjuBcBzIrIdmAv8VkS+EYT99k50DJz7IJQXwru3B223cT4Pf7z8WMYdlsz3/ryMP7y/1XrRGGMGnF4nd1Udpap5qpoHvAD8l6q+0uvIgiH3eDj2KljyO/ji46DtNjnWy1NXHs+sMUP45ZvruOpPS9lnI0kaYwaQQLpCPgt8BIwRkUIRuVJErhaRq/s+vCA49WZIzobXfgD+4CXglHgvv7v0GG45J5/3N5Vw1q/f5+NtXV13NsaY/iGhalIoKCjQpUuX9s/BNv4NnjkfZv4UZi4I+u5X7yzn+88s5/N9Nfz41KP4r1lH4ImSoB/HGGNEZJmqFnRVLvLuUG3PUafDxPNh8T1QtC7ou5+QncIb103nnKOH879/38ilf1xCkc3HaowJocGR3AHOuBNikpzmmV4MCdyRxJho7v/WZO7+5iSWf76fsx54n8Ub+7C7pzHGdGLwJPeETCfBF34Cn/yhTw4hIlxw7Ahe+/6JpCf4uOyxj/npS6soqarvk+MZY0xHBk9yB5h0ARxxGvzjVij7vM8Oc9TQJF699kS+M20Uf136BbN+9S8eXbyFen/wfzEYY0x7BldyF4Gv3+usv/5DaOq7mZbifB5+cU4+7/wIYPQYAAAV50lEQVToJI4dlc5/v7We0+9bzMI1e6xfvDGmzw2u5A6Qmgun3wZb3oU/ng4lm/v0cEcMSeSxy4/lT985Dp8niu/9eRkX/34Ja3dV9OlxjTGD2+BL7uDc2HTBk7BvK/xuOix7Avq4Nj3jqCze/uF0bp89nvV7Kjj7wff56UsrKa609nhjTPANjn7uHanYBS9fDdvegzFnw7kPOBde+1h5TSMPvLuJP324HV90FJd+ZSTzp48mIzGmz49tjAlvgfZzH9zJHaC5GZY8DP+4BeLSYPZv4chT++XQW4ur+PU/N/HaZ7uIjfY4Sf6k0WRakjfGdMCSe3ftWQ0vfReK1sJx34PTbgVvXL8cenNRFb9510nyMdEe5p2Qy/yTDicryZK8MeZgltx7orHOqcEveRiyxsJpt8HhJ4PH2y+H31JcxUPvbuaVFTvxRUcx7/iRzJ8xmiFJsf1yfGPMwGfJvTc2/wNe/T5U7ob4DBg/xxm+IOc4iOr7a9Bbi6v4zaLNvPLpTryeKM6cMIzZU7KZfkQm0Z7BeQ3cGOOw5N5b/nrY/E9Y9VfY8Db4ayElFyZ+00n0Q8f3eQjbSqr5w/tbeWPlbsprG8lM9PH1ScP5xpRsjs5JQcQGJzNmsLHkHkz1lbD+TSfRb1kE2gRDxsPki6DgSvDF9+3h/U38a0Mxr67YyT/WFdHgbyYvI57Zk7P5xpRsRmUm9OnxjTEDhyX3vlJVDGtfgZXPQ+HHkHQYzLoJjr4YPNF9fviKukbeWbWHV1bs5KOtpajCxOwUvjZ+KKePH8aRQxKtRm9MBLPk3h92fAR//7kzGFnWWDj1Vjjqa84wB/1gd3ktr3+2i7dX7+HTz8sAyMuI5/Txwzg9fyhTctNsXHljIowl9/6iCutecwYj27cFRk5zetnkdHnug2pvRR3/WLeXv63Zy4dbSmhsUjITfZw6biin5Q/lq4dnEufz9GtMxpjgC1pyF5HHgK8DRao6oZ33LwFuBASoBK5R1c+6OnDEJPcWTY2w/E/wrzuhuhjyvwGn/AIyDu/3UCrrGvnXhmL+tnYvi9YXUVXvx+eJ4thRaZx0ZBYnHZXF2GFJ1nxjTBgKZnI/CagCnuwguX8VWKeq+0XkTOAWVT2+qwNHXHJvUV8JH/4GPnwQmuqdbpQF34Hcr/Rbc81B4fib+HjbPhZvLOa9jcVs3FsFwJCkGKYfmcVJR2Uy/cgs0hN8/R6bMab7gtosIyJ5wBvtJfdDyqUBq1U1u6t9Rmxyb1G5Fz64D1Y8A/XlkDnGSfJHf8sZ5iBEdpfX8v6mEhZvLOaDzSWU1TQiAuOGJXPcqHQK8tI4Li+dIcl245QxA1GokvsNwFhVvaqD9+cD8wFyc3OP2bFjR5fHDnsN1bD6JVj2OOxcBtFxMOE8J9FnHxOS2nyLpmZl1c5yFm8sZsm2UpbvKKO20ZlQJDc9nmPz0jk2L42CvHQOz0qwZhxjBoB+T+4iMgv4LXCiqpZ2tc+Ir7m3Z/dnsPRxpxtlYzUMnQiTL4bRM53eNv1w92tnGpuaWburgk+27+OT7ftYun0/pdUNAKQn+JiYneI8cpzlYSmxlvCN6Wf9mtxFZBLwMnCmqm4MJMBBmdxb1FU4N0Qtexz2rHJei89wetqMOgnyTnSSfYgTp6qytaSapW6iX7WznE1FVTQ1O/9mMhN9TMhOYVJ2irPMSWVocowlfGP6UL8ldxHJBd4FLlPVDwMNcFAn97b274DtH8D292Hb+1BR6Lwen+kk+ZbHAEj2AHWNTazdXcHqneWsLCxn9c5yNu6txM33ZCXFtNbwJ7k1fGu/NyZ4gtlb5llgJpAJ7AVuBrwAqvqIiPwB+CbQ0oDuD+TAltzboQplO5wk35LwK3Y678VnOD1uRk6DkV+FYRMhamD0W69taGLt7nJWFZazcqeT8DcXVbUm/KHJTsLPH57C6MwERmbEk5eRQGq812r5xnST3cQUCVRh/zbY8aHz2P6Bk/wBYpIh9wQn0Q+fAqkjISWn34Yn7kp1vZ+1uytYVVjOqp3lrCwsY2tJ9UGzGSbHRpOXmcDIjATyMuIZmZHAEUMSOXJIIgkxfT+UgzHhyJJ7pCovdIY92PGBk/BL2lzikChIGu5MAt72kZIDcanOF0JMkvOIju33Zp66xiYK99ewvaSG7aXV7Cg9sCzcX9Na0wcYkR7HmKFJHDU0iTHDkjhySBKHD0kgJnpg/FoxJlQsuQ8WVUVQtA7KPofyL5xly6NiJ2hz+9tFRR9I9L4k50tg+JQDj8Ssfv0YDf5mCvfXsKmoio17Ktmwt5KNeyvZWlyN3836nihhZHo8I9LjyU2PZ0R6HLnp8eSkxZObEU9y7MD41WJMX7LkbpwhESp2QvlOqK9w7p5tXbZ51FU44+IUbwDcfw/JOTB88oFkP2yS0+7fz901G/zNbC+tZsMeJ9lvLqrii/01fLGvlvLaxoPKpsR53WQfR3ZqnLNMc5+nxVnyNxEh0ORuDZuRzOOFtDznEYj6Sqdr5q5PYedyZ7n+jQPvSxTEpUN8urvMgPg0ZxmX7vwK8MY5TT7eePDGOjdteds8fAngSwz42oAvOoqj3OaZQ5XXNvLFvhrnsb+Gz/fV8Pm+WjbureTd9UXU+w/+1ZIcG012WjzZqbEMTY5lWHIsQ1Oc5bAU57Xk2Gi7yGsigiV3c0BMknOBduRXD7xWW+bcfFW0FqpLoHYf1JRCzT7n4u6u5c56U333juXxOV8AvkQ34bd5eOOdCVB8iQfWve570THOhWaUFG0mRZuZoArxzRDXDMObwV+HNtRQW1NJdVUFddVVNNRW4q+rprm2hupKYbN/COsbh7BUD2ObDmOnZtKEhzivh6HJMQxNdr8AUmIZktTmeXIsQ5JjiPW2afv3N0Dxeuc87VnpLEs3O91ZU3IgdYSzTHGvf6TkQPLwAXPxu1O1+6G61Gm2i7bxh8KJNcuY3lN1hlloqIbGGvDXQWOt8/C7y8Y6567chhq3XPWBbRqq2qy32U9DjVOu2d+zuDw+9xdDgvsF4a77a6F0KzRUthZtFi8VcdkUeXMolGGUNsZQ1gD76qC22UMDXhrxUK9eGokmx1fF5OjPGcc2Rvh34MVpImr0xFOZOo6m9CNIaK4ktmYXUeWFUFNycGwS5Xx5Ic54qoh7gbvNMjoWhubD8KlO01j2VEgcEthnryt37qGoKXXGMkrIdL5svB3cc6DqNOHtXul8Qe1Z5ayXf+7G64H0UZBxJGS2PI5yHvHpB/bRUO00/dW1NP+VO+upIyHnmAD/cKYz1uZuIoe/wfkCaKxx5rYVcZKjROEkQ3e95fWWZqHOZsZSdYZmLt0MpVuc5b4tzvr+7c6xulAVlczW6MNZyyhWNOaypG4E23UoyoHrEiKQmRhDXjLkx1dyRGwZuZ5SDtNiEqUer0eIjgKvR/BGQXSU4BE3voYqJ8GWbDhwYbzlWki2m/ARJ96yHc6y5VG7v/2gfUlOok/IhIQspzmt/Asnmdfua4naGap62CQ4bBIkDIF9W52eWSWbnPPU1HBgn7GpzrK+0pmCsiMn/wym3zAgbsYLZ5bcjekNVeeCdFPDgYe//sBrMUlO80qbRNXY1Mz+6gZKqhoorqpnT3ktu8vr2F1Wx67W9VqqGzpJgDiJPt4XTWJMNGkJXg6La2aSZztjmjaT17CBYVXrSa45eNA9jfIiqbnuNZaRB661xGc6ib6mxPkyqy51ljUlTjNbTanza2DYJDjsaGc5dDzEJHYcYHOT82VSstlJ+Pu2ODX72GSnu23rMuVA99sP7oNVz8P482D2Q30+73Aks+RuzACkqlTU+dldXktZTSM1DX6q65uobWiiusFPTUMT1fXOsqKukf3VDeyrbqDUXda4XwzJVDE+ageK8HnzEPaQjs8bTUqcl5Q4L8mx7jLuwDI5ts377rJlPcHn6dsLyarw71/DP25xfg1c+Izz5Wi6zXrLGDMAiUhrUu2JusYm9rVJ+GU1DVTUNlLuPipq/a3ru8vrWL+nkoq6RirrOr9uESWQFOslKTaa5JZlnLNMifOSkeAjIzGG9AQfmYk+0hOc9YB7F4nAiT+CIePghSvh0Vnwracgt8t5fUwPWc3dmEGgqVmpqnMSf0VdyxdBY+vzyjo/FbXusq6RijbPy2sbqapv/8vB6xHSE3wk+KKJ8XqI80YR6/W4jwPrx+alMWeKW1Mv3gDPXujcbf31+2DKvH48E+HPau7GmFaeKCEl3ktKfO9/MZRU1Tu/HKqcXw+lVfXUNDZR39hEXWMztY1NlNc2Uuc+r6r388ySz0mN9zFrzBDIGgNX/RNeuAJevRb2rnUmle/sArjpNqu5G2P6VF1jE7N/82/21TSw8EcnHZivt8kPf/sZLHnYmbBmzFkHej5Fedx1z4Hn0TFOz5zYFGespFh3vKT27pr2N0BdmXMPRu1+pydQ7X6I8joXkBOHOo+4tK7vum5uci9Ku/d4+Gud3kvNzU7vIG12ymiTs5QoZ7/x6c4yLt25RyNI1zTsgqoxZsBYu6uCbzz0b2aNzeKReccc3E6//M/w5k+6fyMcAOL0zolNcbp51lc6ibyhKrDNW5O9m/BjU517BGpKD9ywV1tG67AcPeXxHXx396QL4Jhv92hX1ixjjBkw8ocn85PTj+J/3l7PC8sKOb9gxIE3p14KE+c6N7s1uzXhllpwy7qqc+9BXbmTbOvKnZp52/X6KifRx6V9+RGf7iTu5iao2us+iqBqj7vc69zEtXet80URnw4p7nhK8S1DbWQ4+/LGH/zrou2vjCiPc4zWXw37Dlm6vwDa3ifQRyy5G2P6xVXTR/PP9UXc+vpaThidwYj0Nn3dW8Ye6g+ZR/TPcUIstDMyG2MGDU+UcO8FRyPA9c+vaJ2L1/SNLpO7iDwmIkUisrqD90VEHhCRzSKyUkSmBj9MY0wkyEmL59bZ4/lk+35+t3hLqMOJaIHU3J8Azujk/TOBI93HfODh3odljIlUc6Zkc9bEYdz3942s3lke6nAiVpfJXVUXA/s6KTIbeFId/wFSReSwYAVojIksIsId35hIWryPH/9lBXWNnY+1Y3omGG3u2cAXbZ4Xuq99iYjMF5GlIrK0uLg4CIc2xoSjtAQfvzr/aDYVVXHXO+tDHU5E6tcLqqr6qKoWqGpBVlb/ztFpjBlYZhyVxbe/MpLH/72d9zdZZS/YgpHcdwJtOq2S475mjDGdWnDmOA7PSuCGv37GJ9v34W/qYEJ3023BSO6vAZe5vWZOAMpVdXcQ9muMiXBxPg/3f2sKlXV+zn/kI6bc/nf+6+ll/OWTz9ldXhvq8MJalzcxicizwEwgU0QKgZsBL4CqPgK8BZwFbAZqgCv6KlhjTOSZmJPCRwtO4d9bSnhvQzHvbSzmrVV7ABgzNImTjspkxlFDKMhLO3juWtMpG1vGGDOgqCob91bx3sYi3ttYzCfb9tPQ1IwIZKfGMSozgdGZCYzKTGBUViKjMxMYnhqHJ2pwTN9nA4cZYyJCTYOfj7aUsmpnOdtKqtlWUs3W4uqDxpj3RUeRmx7PkKQYMhNjyEj0kZkYQ2aij4yEA8+zkmLCvvZvA4cZYyJCvC+aU8YN5ZRxQ1tfU1VKqhrcZF/F1pJqdpTUUFxVz8rCMkqqGjqcYCQpJpqs5BiGJMWQlRTrLmNal2nxPtISfKTFe4nz9vH0g33IkrsxJuyICFluMj5uVHq7ZeoamyipqncnFamnpNKZuLy4sp6iyjqKK50vgqKKemo7uJHKFx1FWrzXSfjxPtISnPX0hIMfbV8bKL8MLLkbYyJSrNdDTlo8OWnxnZZTVaobmiiqcBL+/ppGymoa2iwb2FftrG/YU8n+mkb21zTQUYt2THQUXk8UnihpfUS3WUZFCRcfl8tV00f3wac+wJK7MWZQExESY6JJzEpkdFZiQNs0NSsVtY2UVjvJv7Sq5UvAmbDc36w0NSv+5maaWtcPLDMTY/r4U1lyN8aYbvNEidMu3zJl4ABk47kbY0wEsuRujDERyJK7McZEIEvuxhgTgSy5G2NMBLLkbowxEciSuzHGRCBL7sYYE4FCNiqkiBQDOzp4OxMo6cdweipc4oTwidXiDK5wiRPCJ9ZQxzlSVbucpzRkyb0zIrI0kCEtQy1c4oTwidXiDK5wiRPCJ9ZwidOaZYwxJgJZcjfGmAg0UJP7o6EOIEDhEieET6wWZ3CFS5wQPrGGRZwDss3dGGNM7wzUmrsxxphesORujDERaEAldxE5Q0Q2iMhmEVkQ6ng6IyLbRWSViKwQkaWhjqeFiDwmIkUisrrNa+ki8ncR2eQu00IZY4sOYr1FRHa653WFiJwV4hhHiMgiEVkrImtE5Ifu6wPunHYS60A7p7Ei8rGIfObGeav7+igRWeL+//+LiIR0JoxO4nxCRLa1OZ+TQxlnh1R1QDwAD7AFGA34gM+A/FDH1Um824HMUMfRTlwnAVOB1W1euxtY4K4vAO4KdZydxHoLcEOoY2sTz2HAVHc9CdgI5A/Ec9pJrAPtnAqQ6K57gSXACcDzwIXu648A1wzQOJ8A5ob6PHb1GEg19+OAzaq6VVUbgOeA2SGOKeyo6mJg3yEvzwb+5K7/CfhGvwbVgQ5iHVBUdbeqLnfXK4F1QDYD8Jx2EuuAoo4q96nXfShwMvCC+3rIz2kncYaFgZTcs4Ev2jwvZAD+w2xDgb+JyDIRmR/qYLowVFV3u+t7gKGhDCYA3xeRlW6zTcibO1qISB4wBacGN6DP6SGxwgA7pyLiEZEVQBHwd5xf7WWq6neLDIj//4fGqaot5/MO93zeJyJ9P9t1Dwyk5B5uTlTVqcCZwLUiclKoAwqEOr8xB3Lt42HgcGAysBv439CG4xCRROBF4EeqWtH2vYF2TtuJdcCdU1VtUtXJQA7Or/axIQ6pXYfGKSITgJ/ixHsskA7cGMIQOzSQkvtOYESb5znuawOSqu50l0XAyzj/QAeqvSJyGIC7LApxPB1S1b3uf6hm4PcMgPMqIl6cZPm0qr7kvjwgz2l7sQ7Ec9pCVcuARcBXgFQRiXbfGlD//9vEeYbb/KWqWg88zgA6n20NpOT+CXCke8XcB1wIvBbimNolIgkiktSyDpwOrO58q5B6Dfi2u/5t4NUQxtKploTpmkOIz6uICPBHYJ2q3tvmrQF3TjuKdQCe0ywRSXXX44DTcK4PLALmusVCfk47iHN9my91wbkuMCD/7w+oO1TdLlr34/SceUxV7whxSO0SkdE4tXWAaOCZgRKriDwLzMQZlnQvcDPwCk5PhFycYZYvUNWQX8jsINaZOM0HitMj6Xtt2rb7nYicCLwPrAKa3ZdvwmnLHlDntJNYL2JgndNJOBdMPTgVzOdV9Tb3/9VzOE0dnwLz3NrxQIvzXSALpzfNCuDqNhdeB4wBldyNMcYEx0BqljHGGBMkltyNMSYCWXI3xpgIZMndGGMikCV3Y4yJQJbcjTEmAllyN8aYCPT/AcxewVbsDNPzAAAAAElFTkSuQmCC\n"},"metadata":{}}]},{"metadata":{},"cell_type":"markdown","source":"# TEST 적용하기\n\n하나의 그림을 가장 비슷한 그림으로 예측 되는 3개로 label 예측해보기"},{"metadata":{"trusted":true},"cell_type":"code","source":"ttvlist=[]\nreader=pd.read_csv('../input/test_simplified.csv',index_col=['key_id'],chunksize=2048)\nfor chunk in tqdm(reader,total=55):\n    imagebag=bag.from_sequence(chunk.drawing.values).map(stroke_to_img)\n    testarray=np.array(imagebag.compute())\n    testarray=np.reshape(testarray,(testarray.shape[0],imheight,imwidth,1))\n    testpreds=model.predict(testarray,verbose=0)\n    ttvs=np.argsort(-testpreds)[:,0:3]\n    ttvlist.append(ttvs)\nttvarray=np.concatenate(ttvlist)\npred_df=pd.DataFrame({'first': ttvarray[:,0],'second':ttvarray[:,1],'third':ttvarray[:,2]})\npred_df=pred_df.replace(numstonames)\npred_df['words']=pred_df['first']+' '+pred_df['second']+' '+pred_df['third']\n\nsub=pd.read_csv('../input/sample_submission.csv',index_col=['key_id'])\nsub['word']=pred_df.words.values\nsub.to_csv('submission_summer.csv')","execution_count":16,"outputs":[{"output_type":"stream","text":"  0%|          | 0/55 [00:00<?, ?it/s]\n","name":"stderr"},{"output_type":"error","ename":"OSError","evalue":"[Errno 12] Cannot allocate memory","traceback":["\u001b[0;31m---------------------------------------------------------------------------\u001b[0m","\u001b[0;31mOSError\u001b[0m                                   Traceback (most recent call last)","\u001b[0;32m<ipython-input-16-7decc8e78f7f>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mchunk\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mtqdm\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mreader\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mtotal\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m55\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m     \u001b[0mimagebag\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mbag\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfrom_sequence\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mchunk\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdrawing\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvalues\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmap\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstroke_to_img\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m     \u001b[0mtestarray\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0marray\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mimagebag\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcompute\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      6\u001b[0m     \u001b[0mtestarray\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mreshape\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtestarray\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtestarray\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mshape\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mimheight\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mimwidth\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      7\u001b[0m     \u001b[0mtestpreds\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mmodel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpredict\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtestarray\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mverbose\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.6/site-packages/dask/base.py\u001b[0m in \u001b[0;36mcompute\u001b[0;34m(self, **kwargs)\u001b[0m\n\u001b[1;32m    154\u001b[0m         \u001b[0mdask\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbase\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcompute\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    155\u001b[0m         \"\"\"\n\u001b[0;32m--> 156\u001b[0;31m         \u001b[0;34m(\u001b[0m\u001b[0mresult\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcompute\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtraverse\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    157\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0mresult\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    158\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.6/site-packages/dask/base.py\u001b[0m in \u001b[0;36mcompute\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m    396\u001b[0m     \u001b[0mkeys\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__dask_keys__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mx\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcollections\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    397\u001b[0m     \u001b[0mpostcomputes\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__dask_postcompute__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mx\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mcollections\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 398\u001b[0;31m     \u001b[0mresults\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mschedule\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdsk\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkeys\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    399\u001b[0m     \u001b[0;32mreturn\u001b[0m \u001b[0mrepack\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mf\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mr\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mr\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mf\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0ma\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mzip\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mresults\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mpostcomputes\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    400\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.6/site-packages/dask/multiprocessing.py\u001b[0m in \u001b[0;36mget\u001b[0;34m(dsk, keys, num_workers, func_loads, func_dumps, optimize_graph, pool, **kwargs)\u001b[0m\n\u001b[1;32m    165\u001b[0m         \u001b[0mcontext\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mget_context\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    166\u001b[0m         pool = context.Pool(num_workers,\n\u001b[0;32m--> 167\u001b[0;31m                             initializer=initialize_worker_process)\n\u001b[0m\u001b[1;32m    168\u001b[0m         \u001b[0mcleanup\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    169\u001b[0m     \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.6/multiprocessing/context.py\u001b[0m in \u001b[0;36mPool\u001b[0;34m(self, processes, initializer, initargs, maxtasksperchild)\u001b[0m\n\u001b[1;32m    117\u001b[0m         \u001b[0;32mfrom\u001b[0m \u001b[0;34m.\u001b[0m\u001b[0mpool\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mPool\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    118\u001b[0m         return Pool(processes, initializer, initargs, maxtasksperchild,\n\u001b[0;32m--> 119\u001b[0;31m                     context=self.get_context())\n\u001b[0m\u001b[1;32m    120\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    121\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mRawValue\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtypecode_or_type\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.6/multiprocessing/pool.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, processes, initializer, initargs, maxtasksperchild, context)\u001b[0m\n\u001b[1;32m    172\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_processes\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mprocesses\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    173\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_pool\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 174\u001b[0;31m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_repopulate_pool\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    175\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    176\u001b[0m         self._worker_handler = threading.Thread(\n","\u001b[0;32m/opt/conda/lib/python3.6/multiprocessing/pool.py\u001b[0m in \u001b[0;36m_repopulate_pool\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    237\u001b[0m             \u001b[0mw\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mname\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mw\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mname\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mreplace\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'Process'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'PoolWorker'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    238\u001b[0m             \u001b[0mw\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdaemon\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 239\u001b[0;31m             \u001b[0mw\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstart\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    240\u001b[0m             \u001b[0mutil\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdebug\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m'added worker'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    241\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.6/multiprocessing/process.py\u001b[0m in \u001b[0;36mstart\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    103\u001b[0m                \u001b[0;34m'daemonic processes are not allowed to have children'\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    104\u001b[0m         \u001b[0m_cleanup\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 105\u001b[0;31m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_popen\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_Popen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    106\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_sentinel\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_popen\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msentinel\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    107\u001b[0m         \u001b[0;31m# Avoid a refcycle if the target function holds an indirect\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.6/multiprocessing/context.py\u001b[0m in \u001b[0;36m_Popen\u001b[0;34m(process_obj)\u001b[0m\n\u001b[1;32m    275\u001b[0m         \u001b[0;32mdef\u001b[0m \u001b[0m_Popen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mprocess_obj\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    276\u001b[0m             \u001b[0;32mfrom\u001b[0m \u001b[0;34m.\u001b[0m\u001b[0mpopen_fork\u001b[0m \u001b[0;32mimport\u001b[0m \u001b[0mPopen\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 277\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mPopen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mprocess_obj\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    278\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    279\u001b[0m     \u001b[0;32mclass\u001b[0m \u001b[0mSpawnProcess\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mprocess\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mBaseProcess\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.6/multiprocessing/popen_fork.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, process_obj)\u001b[0m\n\u001b[1;32m     17\u001b[0m         \u001b[0mutil\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_flush_std_streams\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     18\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mreturncode\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 19\u001b[0;31m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_launch\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mprocess_obj\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     20\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     21\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mduplicate_for_child\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfd\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;32m/opt/conda/lib/python3.6/multiprocessing/popen_fork.py\u001b[0m in \u001b[0;36m_launch\u001b[0;34m(self, process_obj)\u001b[0m\n\u001b[1;32m     64\u001b[0m         \u001b[0mcode\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     65\u001b[0m         \u001b[0mparent_r\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mchild_w\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mos\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpipe\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 66\u001b[0;31m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpid\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mos\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfork\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     67\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpid\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     68\u001b[0m             \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n","\u001b[0;31mOSError\u001b[0m: [Errno 12] Cannot allocate memory"]}]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.4","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat":4,"nbformat_minor":1}