{
  "id": 415352,
  "title": "pytorch-onnx-tflite: submission test note",
  "url": "/competitions/asl-fingerspelling/discussion/415352",
  "author_name": "canlion",
  "post_date": "2023-06-06T06:48:42.962000",
  "votes": 4,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Since the submission did not pass, the public score cannot be checked, but development can still proceed. </p>\n<p>It seems difficult to directly pinpoint the cause of the submission failure with torch-onnx-tflite. However, I still prefer using torch.</p>\n<ul>\n<li>default structure</li>\n</ul>\n<pre><code> (nn.Module):\n     ():\n        ().__init__()\n        self.preprocessor = preprocessor\n        self.model = SimpleModel()  \n\n        self.model.load_state_dict(net.state_dict())\n\n     ():\n        x = self.preprocessor(x)\n        x = torch.reshape(x, (, -, ))\n        x = self.model(x)[]\n         x\n</code></pre>\n<ul>\n<li>onnx opset: 11</li>\n</ul>\n<p>I'm not sure how to resolve the issues occurring during the preprocessing stage.</p>\n<h3>06/06</h3>\n<ul>\n<li>submission fails if it includes the <strong>permute</strong> operation</li>\n<li><ul>\n<li>I suspect that the two transposes may have been removed in ONNX because they were reverting the data back to its original form. I realize now that I made a mistake during the experiment. I need to retest it.</li></ul></li>\n<li><ul>\n<li>It appears that the variable name typo caused its removal during the ONNX conversion process.</li></ul></li>\n</ul>\n<pre><code>\n ():\n    length = data.size()\n    data = data.reshape(length, , -)\n     data.flatten(-)\n\n\n ():\n    length = data.size()\n    data = data.reshape(length, , -).permute(, , )\n     data.flatten(-)\n\n\n ():\n    length = data.size()\n    data = data.reshape(length, , -)\n    data = data.transpose(, )\n    data = data.transpose(, )\n     data.flatten(-)\n\n\n\n\n\n\n\n\n\n\n\n\n</code></pre>\n<h3>06/07</h3>\n<ul>\n<li>There is absolutely no progress. I have no idea what the problem is.</li>\n</ul>\n<pre><code>\n ():\n    length = data.size()\n    data = data.reshape(length, , -)\n    data = data.transpose(, )\n\n    data = torch.where(torch.isnan(data), torch.zeros_like(data), data)\n    data_mean = data.mean(dim=, keepdim=)\n    data_std = data.std(dim=, keepdim=)\n\n    data = (data - data_mean) / (data_std + )\n\n    data = data.transpose(, )\n     data.flatten(-)\n\n\n ():\n    length = data.size()\n    data = data.reshape(length, , -)\n    data = data.transpose(, )\n\n    data = torch.where(torch.isnan(data), torch.zeros_like(data), data)\n\n    data = data.transpose(, )\n     data.flatten(-)\n\n\n ():\n    length = data.size()\n    data = data.reshape(length, , -)\n    data = data.transpose(, )\n\n    data[torch.isnan(data)] = \n\n    data = data.transpose(, )\n     data.flatten(-)\n</code></pre>",
  "messages": [
    {
      "id": 2289441,
      "postDate": "2023-06-06T06:48:42.963Z",
      "content": "<p>Since the submission did not pass, the public score cannot be checked, but development can still proceed. </p>\n<p>It seems difficult to directly pinpoint the cause of the submission failure with torch-onnx-tflite. However, I still prefer using torch.</p>\n<ul>\n<li>default structure</li>\n</ul>\n<pre><code> (nn.Module):\n     ():\n        ().__init__()\n        self.preprocessor = preprocessor\n        self.model = SimpleModel()  \n\n        self.model.load_state_dict(net.state_dict())\n\n     ():\n        x = self.preprocessor(x)\n        x = torch.reshape(x, (, -, ))\n        x = self.model(x)[]\n         x\n</code></pre>\n<ul>\n<li>onnx opset: 11</li>\n</ul>\n<p>I'm not sure how to resolve the issues occurring during the preprocessing stage.</p>\n<h3>06/06</h3>\n<ul>\n<li>submission fails if it includes the <strong>permute</strong> operation</li>\n<li><ul>\n<li>I suspect that the two transposes may have been removed in ONNX because they were reverting the data back to its original form. I realize now that I made a mistake during the experiment. I need to retest it.</li></ul></li>\n<li><ul>\n<li>It appears that the variable name typo caused its removal during the ONNX conversion process.</li></ul></li>\n</ul>\n<pre><code>\n ():\n    length = data.size()\n    data = data.reshape(length, , -)\n     data.flatten(-)\n\n\n ():\n    length = data.size()\n    data = data.reshape(length, , -).permute(, , )\n     data.flatten(-)\n\n\n ():\n    length = data.size()\n    data = data.reshape(length, , -)\n    data = data.transpose(, )\n    data = data.transpose(, )\n     data.flatten(-)\n\n\n\n\n\n\n\n\n\n\n\n\n</code></pre>\n<h3>06/07</h3>\n<ul>\n<li>There is absolutely no progress. I have no idea what the problem is.</li>\n</ul>\n<pre><code>\n ():\n    length = data.size()\n    data = data.reshape(length, , -)\n    data = data.transpose(, )\n\n    data = torch.where(torch.isnan(data), torch.zeros_like(data), data)\n    data_mean = data.mean(dim=, keepdim=)\n    data_std = data.std(dim=, keepdim=)\n\n    data = (data - data_mean) / (data_std + )\n\n    data = data.transpose(, )\n     data.flatten(-)\n\n\n ():\n    length = data.size()\n    data = data.reshape(length, , -)\n    data = data.transpose(, )\n\n    data = torch.where(torch.isnan(data), torch.zeros_like(data), data)\n\n    data = data.transpose(, )\n     data.flatten(-)\n\n\n ():\n    length = data.size()\n    data = data.reshape(length, , -)\n    data = data.transpose(, )\n\n    data[torch.isnan(data)] = \n\n    data = data.transpose(, )\n     data.flatten(-)\n</code></pre>",
      "rawMarkdown": "Since the submission did not pass, the public score cannot be checked, but development can still proceed. \n\nIt seems difficult to directly pinpoint the cause of the submission failure with torch-onnx-tflite. However, I still prefer using torch.\n\n- default structure\n```python\nclass ExportModel(nn.Module):\n    def __init__(self, *args, **kwargs):\n        super().__init__()\n        self.preprocessor = preprocessor\n        self.model = SimpleModel()  # (Conv1D + BN + ReLU) x N\n        \n        self.model.load_state_dict(net.state_dict())\n    \n    def forward(self, x):\n        x = self.preprocessor(x)\n        x = torch.reshape(x, (1, -1, 108))\n        x = self.model(x)[0]\n        return x\n```\n- onnx opset: 11\n\nI'm not sure how to resolve the issues occurring during the preprocessing stage.\n\n### 06/06\n  - submission fails if it includes the **permute** operation\n  - ~~**transpose** operation work~~\n      - I suspect that the two transposes may have been removed in ONNX because they were reverting the data back to its original form. I realize now that I made a mistake during the experiment. I need to retest it.\n  - ~~It seems that the simple preprocessing works, excluding the permute operation.~~\n      - It appears that the variable name typo caused its removal during the ONNX conversion process.\n```python\n# PASS\ndef preprocessor(data):\n    length = data.size(0)\n    data = data.reshape(length, 3, -1)\n    return data.flatten(-2)\n\n# FAIL\ndef preprocessor(data):\n    length = data.size(0)\n    data = data.reshape(length, 3, -1).permute(0, 2, 1)\n    return data.flatten(-2)\n\n# PASS\ndef preprocessor(data):\n    length = data.size(0)\n    data = data.reshape(length, 3, -1)\n    data = data.transpose(1, 2)\n    data = data.transpose(1, 2)\n    return data.flatten(-2)\n\n# It appears that the variable name typo caused its removal during the ONNX conversion process.\n# def preprocessor(data):\n#     length = data.size(0)\n#     data = data.reshape(length, 3, -1)\n#     data = data.transpose(1, 2)\n# \n#     data_normalized = normalize(data)  # where, isnan, sum, square, sqrt\n#     # variable name typo\n# \n#     data = data.transpose(1, 2)\n#    return data.flatten(-2)\n```\n\n### 06/07\n  - There is absolutely no progress. I have no idea what the problem is.\n\n```python\n# FAIL\ndef preprocessor(data):\n    length = data.size(0)\n    data = data.reshape(length, 3, -1)\n    data = data.transpose(1, 2)\n    \n    data = torch.where(torch.isnan(data), torch.zeros_like(data), data)\n    data_mean = data.mean(dim=1, keepdim=True)\n    data_std = data.std(dim=1, keepdim=True)\n    \n    data = (data - data_mean) / (data_std + 1e-5)\n        \n    data = data.transpose(1, 2)\n    return data.flatten(-2)\n\n# FAIL\ndef preprocessor(data):\n    length = data.size(0)\n    data = data.reshape(length, 3, -1)\n    data = data.transpose(1, 2)\n    \n    data = torch.where(torch.isnan(data), torch.zeros_like(data), data)\n        \n    data = data.transpose(1, 2)\n    return data.flatten(-2)\n\n# FAIL\ndef preprocessor(data):\n    length = data.size(0)\n    data = data.reshape(length, 3, -1)\n    data = data.transpose(1, 2)\n    \n    data[torch.isnan(data)] = 0.\n        \n    data = data.transpose(1, 2)\n    return data.flatten(-2)\n```",
      "votes": 4
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "2289441": "Since the submission did not pass, the public score cannot be checked, but development can still proceed. \n\nIt seems difficult to directly pinpoint the cause of the submission failure with torch-onnx-tflite. However, I still prefer using torch.\n\n- default structure\n```python\nclass ExportModel(nn.Module):\n    def __init__(self, *args, **kwargs):\n        super().__init__()\n        self.preprocessor = preprocessor\n        self.model = SimpleModel()  # (Conv1D + BN + ReLU) x N\n        \n        self.model.load_state_dict(net.state_dict())\n    \n    def forward(self, x):\n        x = self.preprocessor(x)\n        x = torch.reshape(x, (1, -1, 108))\n        x = self.model(x)[0]\n        return x\n```\n- onnx opset: 11\n\nI'm not sure how to resolve the issues occurring during the preprocessing stage.\n\n### 06/06\n  - submission fails if it includes the **permute** operation\n  - ~~**transpose** operation work~~\n      - I suspect that the two transposes may have been removed in ONNX because they were reverting the data back to its original form. I realize now that I made a mistake during the experiment. I need to retest it.\n  - ~~It seems that the simple preprocessing works, excluding the permute operation.~~\n      - It appears that the variable name typo caused its removal during the ONNX conversion process.\n```python\n# PASS\ndef preprocessor(data):\n    length = data.size(0)\n    data = data.reshape(length, 3, -1)\n    return data.flatten(-2)\n\n# FAIL\ndef preprocessor(data):\n    length = data.size(0)\n    data = data.reshape(length, 3, -1).permute(0, 2, 1)\n    return data.flatten(-2)\n\n# PASS\ndef preprocessor(data):\n    length = data.size(0)\n    data = data.reshape(length, 3, -1)\n    data = data.transpose(1, 2)\n    data = data.transpose(1, 2)\n    return data.flatten(-2)\n\n# It appears that the variable name typo caused its removal during the ONNX conversion process.\n# def preprocessor(data):\n#     length = data.size(0)\n#     data = data.reshape(length, 3, -1)\n#     data = data.transpose(1, 2)\n# \n#     data_normalized = normalize(data)  # where, isnan, sum, square, sqrt\n#     # variable name typo\n# \n#     data = data.transpose(1, 2)\n#    return data.flatten(-2)\n```\n\n### 06/07\n  - There is absolutely no progress. I have no idea what the problem is.\n\n```python\n# FAIL\ndef preprocessor(data):\n    length = data.size(0)\n    data = data.reshape(length, 3, -1)\n    data = data.transpose(1, 2)\n    \n    data = torch.where(torch.isnan(data), torch.zeros_like(data), data)\n    data_mean = data.mean(dim=1, keepdim=True)\n    data_std = data.std(dim=1, keepdim=True)\n    \n    data = (data - data_mean) / (data_std + 1e-5)\n        \n    data = data.transpose(1, 2)\n    return data.flatten(-2)\n\n# FAIL\ndef preprocessor(data):\n    length = data.size(0)\n    data = data.reshape(length, 3, -1)\n    data = data.transpose(1, 2)\n    \n    data = torch.where(torch.isnan(data), torch.zeros_like(data), data)\n        \n    data = data.transpose(1, 2)\n    return data.flatten(-2)\n\n# FAIL\ndef preprocessor(data):\n    length = data.size(0)\n    data = data.reshape(length, 3, -1)\n    data = data.transpose(1, 2)\n    \n    data[torch.isnan(data)] = 0.\n        \n    data = data.transpose(1, 2)\n    return data.flatten(-2)\n```"
  }
}