{"cells":[{"metadata":{},"cell_type":"markdown","source":"![TF](https://www.gstatic.com/devrel-devsite/prod/ve312520032ba2ac0c4d23f7b46fc670cbbe051886a2d1f04563a5e4768ad9787/tensorflow/images/lockup.svg)"},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"import tensorflow as tf\nimport os, sys\ntf.__version__","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Install the TF Object Detection API\nImportant here that we install the same version of the API as the TensorFlow version."},{"metadata":{"trusted":true},"cell_type":"code","source":"%%capture\n!pip install ../input/tf-model-garden-official-models/tf_models_official-2.3.0-py2.py3-none-any.whl --user\n!pip install pycocotools","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Next we copy the training script from the source code."},{"metadata":{"trusted":true},"cell_type":"code","source":"!cp /kaggle/input/tf-model-garden-official-models/tf-models-official-2.3.0/tf-models-official-2.3.0/official/vision/detection/main.py .","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\"main.py\" will create the file \"params.yaml\" with the model configuration in the model directory we specify. We need to add the training and evaluation files (which were created in [this notebook](https://www.kaggle.com/mistag/data-create-tfrecords-of-vinbigdata-chest-x-rays)), and also override some of the default parameters. This is done by creating another .yaml file with our custom settings:"},{"metadata":{"trusted":true},"cell_type":"code","source":"%%writefile ./config.yaml\narchitecture:\n  num_classes: 14\ntrain:\n  train_file_pattern: \n    - \"/kaggle/input/data-create-tfrecords-of-vinbigdata-chest-x-rays/VinBig-0?0-of-020.tfrecord\"\n    - \"/kaggle/input/data-create-tfrecords-of-vinbigdata-chest-x-rays/VinBig-0?2-of-020.tfrecord\"\n    - \"/kaggle/input/data-create-tfrecords-of-vinbigdata-chest-x-rays/VinBig-0?3-of-020.tfrecord\"\n    - \"/kaggle/input/data-create-tfrecords-of-vinbigdata-chest-x-rays/VinBig-0?4-of-020.tfrecord\"\n    - \"/kaggle/input/data-create-tfrecords-of-vinbigdata-chest-x-rays/VinBig-0?5-of-020.tfrecord\"\n    - \"/kaggle/input/data-create-tfrecords-of-vinbigdata-chest-x-rays/VinBig-0?6-of-020.tfrecord\"\n    - \"/kaggle/input/data-create-tfrecords-of-vinbigdata-chest-x-rays/VinBig-0?7-of-020.tfrecord\"\n    - \"/kaggle/input/data-create-tfrecords-of-vinbigdata-chest-x-rays/VinBig-0?8-of-020.tfrecord\"\n  total_steps: 18000\n  batch_size: 12\n  input_sharding: true\n  iterations_per_loop: 1000\neval:\n  eval_file_pattern: \n    - \"/kaggle/input/data-create-tfrecords-of-vinbigdata-chest-x-rays/VinBig-0?1-of-020.tfrecord\"\n    - \"/kaggle/input/data-create-tfrecords-of-vinbigdata-chest-x-rays/VinBig-0?9-of-020.tfrecord\"\n  batch_size: 1\n  eval_samples: 879\n    \nretinanet_parser:\n  aug_rand_hflip: false\n  aug_scale_max: 1.0\n  aug_scale_min: 1.0","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"One important detail to be aware of is that the check-points created during training are about 260MByte each - and the maximum notebook diskspace use is 5GB. We could move the training from /kaggle/working to /tmp which has more space. Or we can increase the interval between checkpoints by setting the \"iterations_per_loop\" to desired interval. The default setting is 100 (steps). We increase it to 1000 in the .yaml file above. "},{"metadata":{},"cell_type":"markdown","source":"# Training\nRunning \"main.py\" below will perform training. The output is captured to file for parsing further down."},{"metadata":{"trusted":true},"cell_type":"code","source":"%%capture cap\n!python main.py \\\n  --strategy_type=one_device \\\n  --num_gpus=1 \\\n  --model_dir=./model1 \\\n  --mode=train \\\n  --model=retinanet \\\n  --config_file=\"./config.yaml\"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"with open('./train.log', 'w') as f:\n    f.write(cap.stdout)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Check the learning curves (by parsing the log file)."},{"metadata":{"trusted":true},"cell_type":"code","source":"%%capture\n!pip install --user parse","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from parse import *\nimport matplotlib.pyplot as plt\n%matplotlib inline","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"with open('./train.log', 'r') as f:\n    data=f.read()\nloss=[]\nfor r in findall(\"\\'model_loss\\': {:f}\", data):\n    loss.append(r[0])\ncls=[] \nfor r in findall(\"\\'cls_loss\\': {:f}\", data): # class loss\n    cls.append(r[0])\nstep=[]\nfor r in findall(\"Train Step: {:d}\", data):\n    step.append(r[0])\nplt.figure(figsize=(16, 8))\nplt.plot(step,loss[0::2])\nplt.plot(step,cls[0::2])\nplt.legend(['model loss', 'class loss'])\nplt.xlabel('Global step')\nplt.title('Learning curves');","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"That's it! The trained model in the \"model1\" directory can now be used for predictions, but that must be done in a notebook without internet enabled for submission...  \nAlso remember to use the same preprocessing (CLAHE) on the test images as the [TFRecords creation notebook](https://www.kaggle.com/mistag/data-create-tfrecords-of-vinbigdata-chest-x-rays). The CLAHE parameters used can be found in a .json file:"},{"metadata":{"trusted":true},"cell_type":"code","source":"import json \n\nwith open('../input/data-create-tfrecords-of-vinbigdata-chest-x-rays/dparams.json') as json_file:\n     dparams = json.load(json_file)\ndparams","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}