{
  "id": 159775,
  "title": "TPU Training Doubts",
  "url": "/competitions/prostate-cancer-grade-assessment/discussion/159775",
  "author_name": "Gaurav Yadav",
  "post_date": "2020-06-18T16:58:51.420000",
  "votes": 2,
  "comment_count": 0,
  "views": 0,
  "content": "<p>After experiencing TPU in <a href=\"https://www.kaggle.com/c/flower-classification-with-tpus\">Flower Classification</a>  I wanted to try using the TPU for this competition as well but faced quite a lot of problem as being Novice and limited Keras/TPU kernals  for reference.</p>\n\n<ol>\n<li>Reading of tiff file using Dataset API which is somehow solved by using this awesome <a href=\"https://www.kaggle.com/lopuhin/panda-2020-level-1-2\">dataset(panda-2020-level-1-2)</a>.</li>\n<li>I expected it to run similar to how I experienced in Flower Classification Comp but learning is too slow for this comp which I don't know why.</li>\n<li><a href=\"https://www.kaggle.com/iafoss/panda-concat-tile-pooling-starter-0-79-lb\">Tile Pooling Kernal</a> is difficult to implement in TPU so I switched to another approach people using and getting good result i.e creating a single image from tiles. But that also took some time due to python function restriction on TPU so after searching around able to do it using tensorflow function which can be seen in my <a href=\"https://www.kaggle.com/gaur128/panda-tiles-on-tpu/data?scriptVersionId=36690583\">kernal</a>.</li>\n<li>Still to my surprise my model is not learning anything.</li>\n<li>Then I tried custom training loop using <a href=\"https://www.kaggle.com/mgornergoogle/custom-training-loop-with-100-flowers-on-tpu\">kernal</a> as base but to no avail.</li>\n<li>I tried to use all possible approaches that I could understand from public kernals but to no avail.</li>\n<li>Notebook able to execute within an hour but it won't complete even after 3hours after commit.</li>\n<li>Dataset approach try to create the complete data at once instead of creating data for a batch like DateGenerator.</li>\n<li>Connection issue while creating dataset.</li>\n</ol>\n\n<p>I would be grateful if anyone can give some input on what exactly I'm doing wrong in my <a href=\"https://www.kaggle.com/gaur128/panda-tiles-on-tpu/data?scriptVersionId=36690583\">kernal</a> .\nAlso I tried using different loss function and metrics which you may not found in my commit but got no result.</p>\n\n<p>Thanks.</p>",
  "messages": [
    {
      "id": 892109,
      "postDate": "2020-06-18T16:58:51.420Z",
      "content": "<p>After experiencing TPU in <a href=\"https://www.kaggle.com/c/flower-classification-with-tpus\">Flower Classification</a>  I wanted to try using the TPU for this competition as well but faced quite a lot of problem as being Novice and limited Keras/TPU kernals  for reference.</p>\n\n<ol>\n<li>Reading of tiff file using Dataset API which is somehow solved by using this awesome <a href=\"https://www.kaggle.com/lopuhin/panda-2020-level-1-2\">dataset(panda-2020-level-1-2)</a>.</li>\n<li>I expected it to run similar to how I experienced in Flower Classification Comp but learning is too slow for this comp which I don't know why.</li>\n<li><a href=\"https://www.kaggle.com/iafoss/panda-concat-tile-pooling-starter-0-79-lb\">Tile Pooling Kernal</a> is difficult to implement in TPU so I switched to another approach people using and getting good result i.e creating a single image from tiles. But that also took some time due to python function restriction on TPU so after searching around able to do it using tensorflow function which can be seen in my <a href=\"https://www.kaggle.com/gaur128/panda-tiles-on-tpu/data?scriptVersionId=36690583\">kernal</a>.</li>\n<li>Still to my surprise my model is not learning anything.</li>\n<li>Then I tried custom training loop using <a href=\"https://www.kaggle.com/mgornergoogle/custom-training-loop-with-100-flowers-on-tpu\">kernal</a> as base but to no avail.</li>\n<li>I tried to use all possible approaches that I could understand from public kernals but to no avail.</li>\n<li>Notebook able to execute within an hour but it won't complete even after 3hours after commit.</li>\n<li>Dataset approach try to create the complete data at once instead of creating data for a batch like DateGenerator.</li>\n<li>Connection issue while creating dataset.</li>\n</ol>\n\n<p>I would be grateful if anyone can give some input on what exactly I'm doing wrong in my <a href=\"https://www.kaggle.com/gaur128/panda-tiles-on-tpu/data?scriptVersionId=36690583\">kernal</a> .\nAlso I tried using different loss function and metrics which you may not found in my commit but got no result.</p>\n\n<p>Thanks.</p>",
      "rawMarkdown": "After experiencing TPU in [Flower Classification](https://www.kaggle.com/c/flower-classification-with-tpus)  I wanted to try using the TPU for this competition as well but faced quite a lot of problem as being Novice and limited Keras/TPU kernals  for reference.\n\n1. Reading of tiff file using Dataset API which is somehow solved by using this awesome [dataset(panda-2020-level-1-2)](https://www.kaggle.com/lopuhin/panda-2020-level-1-2).\n2. I expected it to run similar to how I experienced in Flower Classification Comp but learning is too slow for this comp which I don't know why.\n3. [Tile Pooling Kernal](https://www.kaggle.com/iafoss/panda-concat-tile-pooling-starter-0-79-lb) is difficult to implement in TPU so I switched to another approach people using and getting good result i.e creating a single image from tiles. But that also took some time due to python function restriction on TPU so after searching around able to do it using tensorflow function which can be seen in my [kernal](https://www.kaggle.com/gaur128/panda-tiles-on-tpu/data?scriptVersionId=36690583).\n4. Still to my surprise my model is not learning anything.\n5. Then I tried custom training loop using [kernal](https://www.kaggle.com/mgornergoogle/custom-training-loop-with-100-flowers-on-tpu) as base but to no avail.\n6. I tried to use all possible approaches that I could understand from public kernals but to no avail.\n7. Notebook able to execute within an hour but it won't complete even after 3hours after commit.\n8. Dataset approach try to create the complete data at once instead of creating data for a batch like DateGenerator.\n9. Connection issue while creating dataset.\n\nI would be grateful if anyone can give some input on what exactly I'm doing wrong in my [kernal](https://www.kaggle.com/gaur128/panda-tiles-on-tpu/data?scriptVersionId=36690583) .\nAlso I tried using different loss function and metrics which you may not found in my commit but got no result.\n\nThanks.",
      "votes": 2
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "892109": "After experiencing TPU in [Flower Classification](https://www.kaggle.com/c/flower-classification-with-tpus)  I wanted to try using the TPU for this competition as well but faced quite a lot of problem as being Novice and limited Keras/TPU kernals  for reference.\n\n1. Reading of tiff file using Dataset API which is somehow solved by using this awesome [dataset(panda-2020-level-1-2)](https://www.kaggle.com/lopuhin/panda-2020-level-1-2).\n2. I expected it to run similar to how I experienced in Flower Classification Comp but learning is too slow for this comp which I don't know why.\n3. [Tile Pooling Kernal](https://www.kaggle.com/iafoss/panda-concat-tile-pooling-starter-0-79-lb) is difficult to implement in TPU so I switched to another approach people using and getting good result i.e creating a single image from tiles. But that also took some time due to python function restriction on TPU so after searching around able to do it using tensorflow function which can be seen in my [kernal](https://www.kaggle.com/gaur128/panda-tiles-on-tpu/data?scriptVersionId=36690583).\n4. Still to my surprise my model is not learning anything.\n5. Then I tried custom training loop using [kernal](https://www.kaggle.com/mgornergoogle/custom-training-loop-with-100-flowers-on-tpu) as base but to no avail.\n6. I tried to use all possible approaches that I could understand from public kernals but to no avail.\n7. Notebook able to execute within an hour but it won't complete even after 3hours after commit.\n8. Dataset approach try to create the complete data at once instead of creating data for a batch like DateGenerator.\n9. Connection issue while creating dataset.\n\nI would be grateful if anyone can give some input on what exactly I'm doing wrong in my [kernal](https://www.kaggle.com/gaur128/panda-tiles-on-tpu/data?scriptVersionId=36690583) .\nAlso I tried using different loss function and metrics which you may not found in my commit but got no result.\n\nThanks."
  }
}