{
  "id": 431531,
  "title": "submission format error question",
  "url": "/competitions/asl-fingerspelling/discussion/431531",
  "author_name": "arelius",
  "post_date": "2023-08-14T00:43:06.957000",
  "votes": 1,
  "comment_count": 7,
  "views": 0,
  "content": "<p>I am currently experiencing a scoring error when trying to use a model.tflite and inference_args.json that I load from my local machine that I then use in a notebook. Was wondering if I misunderstood how much actually needs to be in the notebook aside from the saving of the submission.zip (?)</p>\n<p>Here is the code:</p>\n<pre><code>\n! submission.  \n</code></pre>\n<p>Error appears as:</p>\n<blockquote>\n  <p>Submission Scoring Error<br>\n  Your notebook generated a submission file with incorrect format. Some examples causing this are: wrong number of rows or columns, empty values, an incorrect data type for a value, or invalid submission values from what is expected. See more debugging tips</p>\n</blockquote>\n<p>Here is what I changed my code to and now I have a 'Notebook Running' that has been doing so for a while</p>\n<pre><code> shutil\n\nshutil.copy(,)\nshutil.copy(,)\n\n! submission.   \n</code></pre>\n<p>any advice would help, thanks in advance.</p>",
  "messages": [
    {
      "id": 2389307,
      "postDate": "2023-08-14T01:05:54.820Z",
      "content": "<p>I'm getting the same Submission Scoring Error as you. I just now discovered I'm returning 0 rows, shape (0,59), in rare occurrences.  I'm  just now refreshing another model to see if this fixes it, I plan on returning a tensor with one character, shape (1,59) when this occurs.  Fingers crossed!!!  I'll let you know if it fixes it in an hour.  The things that I've done so far to avoid scoring errors are:</p>\n<ol>\n<li>putting this at the beginning of your preprocessing to remove blank inputs<br>\n x = tf.where(tf.math.is_nan(x), tf.zeros_like(x), x)<br>\n x= x[None]</li>\n<li>making sure that you're returning a float data type and not an integer.</li>\n<li>making sure you're returning the shape of (x, 59).  So removing the pad, sos, and eos tokens before returning.</li>\n<li>Stopped using tf.lite.OpsSet.SELECT_TF_OPS or any other additional supported ops when converting the tflite model, use only tflite runtime.  This meant that i had to bring in the multiheadattention source code and stop using this particular function, tf.linalg.band_part, within it, which was being used to create the causal attention mask.</li>\n</ol>",
      "rawMarkdown": "I'm getting the same Submission Scoring Error as you. I just now discovered I'm returning 0 rows, shape (0,59), in rare occurrences.  I'm  just now refreshing another model to see if this fixes it, I plan on returning a tensor with one character, shape (1,59) when this occurs.  Fingers crossed!!!  I'll let you know if it fixes it in an hour.  The things that I've done so far to avoid scoring errors are:\n1. putting this at the beginning of your preprocessing to remove blank inputs\n     x = tf.where(tf.math.is_nan(x), tf.zeros_like(x), x)\n     x= x[None]\n2. making sure that you're returning a float data type and not an integer.\n3. making sure you're returning the shape of (x, 59).  So removing the pad, sos, and eos tokens before returning.\n4. Stopped using tf.lite.OpsSet.SELECT_TF_OPS or any other additional supported ops when converting the tflite model, use only tflite runtime.  This meant that i had to bring in the multiheadattention source code and stop using this particular function, tf.linalg.band_part, within it, which was being used to create the causal attention mask.",
      "votes": 1,
      "replies": [
        {
          "id": 2389399,
          "postDate": "2023-08-14T03:19:37.880Z",
          "content": "<p>My issue was related to the naming of the files so my updated code worked. Best of luck with your situation and thanks for the 4 additional points to reference in case I hit this err again</p>",
          "rawMarkdown": "My issue was related to the naming of the files so my updated code worked. Best of luck with your situation and thanks for the 4 additional points to reference in case I hit this err again",
          "votes": 1,
          "replies": [
            {
              "id": 2389449,
              "postDate": "2023-08-14T04:20:01.360Z",
              "content": "<p>Dude!  I didn't realize I had to have the right file name for my model!  I followed your example above to rename my model.  I'd never gotten past 2 mins of scoring and now i got to 7 mins. So that's something!!! Thanks for asking the question.</p>",
              "rawMarkdown": "Dude!  I didn't realize I had to have the right file name for my model!  I followed your example above to rename my model.  I'd never gotten past 2 mins of scoring and now i got to 7 mins. So that's something!!! Thanks for asking the question."
            }
          ]
        },
        {
          "id": 2404639,
          "postDate": "2023-08-23T12:01:34.057Z",
          "content": "<p>hi, I'm wondering if returning (0, 59) is valid or not?</p>",
          "rawMarkdown": "hi, I'm wondering if returning (0, 59) is valid or not?"
        }
      ]
    },
    {
      "id": 2389300,
      "postDate": "2023-08-14T00:43:06.957Z",
      "content": "<p>I am currently experiencing a scoring error when trying to use a model.tflite and inference_args.json that I load from my local machine that I then use in a notebook. Was wondering if I misunderstood how much actually needs to be in the notebook aside from the saving of the submission.zip (?)</p>\n<p>Here is the code:</p>\n<pre><code>\n! submission.  \n</code></pre>\n<p>Error appears as:</p>\n<blockquote>\n  <p>Submission Scoring Error<br>\n  Your notebook generated a submission file with incorrect format. Some examples causing this are: wrong number of rows or columns, empty values, an incorrect data type for a value, or invalid submission values from what is expected. See more debugging tips</p>\n</blockquote>\n<p>Here is what I changed my code to and now I have a 'Notebook Running' that has been doing so for a while</p>\n<pre><code> shutil\n\nshutil.copy(,)\nshutil.copy(,)\n\n! submission.   \n</code></pre>\n<p>any advice would help, thanks in advance.</p>",
      "rawMarkdown": "I am currently experiencing a scoring error when trying to use a model.tflite and inference_args.json that I load from my local machine that I then use in a notebook. Was wondering if I misunderstood how much actually needs to be in the notebook aside from the saving of the submission.zip (?)\n\nHere is the code:\n```python\n# save sample submission zip to working and we should be set\n!zip submission.zip '/kaggle/input/sample-local-submission/output/tflite_models/tutorial_model.tflite' '/kaggle/input/sample-local-submission/output/inference_args/tutorial.json'\n```\n\nError appears as:\n>Submission Scoring Error\nYour notebook generated a submission file with incorrect format. Some examples causing this are: wrong number of rows or columns, empty values, an incorrect data type for a value, or invalid submission values from what is expected. See more debugging tips\n\nHere is what I changed my code to and now I have a 'Notebook Running' that has been doing so for a while\n```python\nimport shutil\n\nshutil.copy(\"/kaggle/input/sample-local-submission/output/tflite_models/tutorial_model.tflite\",\"./model.tflite\")\nshutil.copy(\"/kaggle/input/sample-local-submission/output/inference_args/tutorial.json\",\"./inference_args.json\")\n\n!zip submission.zip  './model.tflite' './inference_args.json'\n```\n\nany advice would help, thanks in advance.",
      "votes": 1
    },
    {
      "id": 2389305,
      "postDate": "2023-08-14T00:54:44.247Z",
      "content": "<p>For your non-notebook code, the error happens because of the file name.  Your notebook code should be fine.</p>",
      "rawMarkdown": "For your non-notebook code, the error happens because of the file name.  Your notebook code should be fine.\n",
      "votes": 2,
      "replies": [
        {
          "id": 2389397,
          "postDate": "2023-08-14T03:18:46.440Z",
          "content": "<p>You were totally correct, it just finished scoring after 3hrs</p>",
          "rawMarkdown": "You were totally correct, it just finished scoring after 3hrs"
        },
        {
          "id": 2389451,
          "postDate": "2023-08-14T04:20:50.173Z",
          "content": "<p>Thank you!  This allowed me to get past initialization in the scoring!  Now i need to figure out why it's breaking after 7 mins.</p>",
          "rawMarkdown": "Thank you!  This allowed me to get past initialization in the scoring!  Now i need to figure out why it's breaking after 7 mins."
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 2389307,
      "author_name": "Andy Atkinson",
      "author_url": "",
      "post_date": "2023-08-14T01:05:54.820000",
      "content": "<p>I'm getting the same Submission Scoring Error as you. I just now discovered I'm returning 0 rows, shape (0,59), in rare occurrences.  I'm  just now refreshing another model to see if this fixes it, I plan on returning a tensor with one character, shape (1,59) when this occurs.  Fingers crossed!!!  I'll let you know if it fixes it in an hour.  The things that I've done so far to avoid scoring errors are:</p>\n<ol>\n<li>putting this at the beginning of your preprocessing to remove blank inputs<br>\n x = tf.where(tf.math.is_nan(x), tf.zeros_like(x), x)<br>\n x= x[None]</li>\n<li>making sure that you're returning a float data type and not an integer.</li>\n<li>making sure you're returning the shape of (x, 59).  So removing the pad, sos, and eos tokens before returning.</li>\n<li>Stopped using tf.lite.OpsSet.SELECT_TF_OPS or any other additional supported ops when converting the tflite model, use only tflite runtime.  This meant that i had to bring in the multiheadattention source code and stop using this particular function, tf.linalg.band_part, within it, which was being used to create the causal attention mask.</li>\n</ol>",
      "votes": 1,
      "replies": [
        {
          "id": 2389399,
          "author_name": "arelius",
          "author_url": "",
          "post_date": "2023-08-14T03:19:37.880000",
          "content": "<p>My issue was related to the naming of the files so my updated code worked. Best of luck with your situation and thanks for the 4 additional points to reference in case I hit this err again</p>",
          "votes": 1,
          "replies": [
            {
              "id": 2389449,
              "author_name": "Andy Atkinson",
              "author_url": "",
              "post_date": "2023-08-14T04:20:01.360000",
              "content": "<p>Dude!  I didn't realize I had to have the right file name for my model!  I followed your example above to rename my model.  I'd never gotten past 2 mins of scoring and now i got to 7 mins. So that's something!!! Thanks for asking the question.</p>",
              "votes": 0,
              "replies": []
            }
          ]
        },
        {
          "id": 2404639,
          "author_name": "Đăng Nguyễn Hồng",
          "author_url": "",
          "post_date": "2023-08-23T12:01:34.057000",
          "content": "<p>hi, I'm wondering if returning (0, 59) is valid or not?</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 2389305,
      "author_name": "bliao",
      "author_url": "",
      "post_date": "2023-08-14T00:54:44.247000",
      "content": "<p>For your non-notebook code, the error happens because of the file name.  Your notebook code should be fine.</p>",
      "votes": 2,
      "replies": [
        {
          "id": 2389397,
          "author_name": "arelius",
          "author_url": "",
          "post_date": "2023-08-14T03:18:46.440000",
          "content": "<p>You were totally correct, it just finished scoring after 3hrs</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 2389451,
          "author_name": "Andy Atkinson",
          "author_url": "",
          "post_date": "2023-08-14T04:20:50.173000",
          "content": "<p>Thank you!  This allowed me to get past initialization in the scoring!  Now i need to figure out why it's breaking after 7 mins.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2389307": "I'm getting the same Submission Scoring Error as you. I just now discovered I'm returning 0 rows, shape (0,59), in rare occurrences.  I'm  just now refreshing another model to see if this fixes it, I plan on returning a tensor with one character, shape (1,59) when this occurs.  Fingers crossed!!!  I'll let you know if it fixes it in an hour.  The things that I've done so far to avoid scoring errors are:\n1. putting this at the beginning of your preprocessing to remove blank inputs\n     x = tf.where(tf.math.is_nan(x), tf.zeros_like(x), x)\n     x= x[None]\n2. making sure that you're returning a float data type and not an integer.\n3. making sure you're returning the shape of (x, 59).  So removing the pad, sos, and eos tokens before returning.\n4. Stopped using tf.lite.OpsSet.SELECT_TF_OPS or any other additional supported ops when converting the tflite model, use only tflite runtime.  This meant that i had to bring in the multiheadattention source code and stop using this particular function, tf.linalg.band_part, within it, which was being used to create the causal attention mask.",
    "2389300": "I am currently experiencing a scoring error when trying to use a model.tflite and inference_args.json that I load from my local machine that I then use in a notebook. Was wondering if I misunderstood how much actually needs to be in the notebook aside from the saving of the submission.zip (?)\n\nHere is the code:\n```python\n# save sample submission zip to working and we should be set\n!zip submission.zip '/kaggle/input/sample-local-submission/output/tflite_models/tutorial_model.tflite' '/kaggle/input/sample-local-submission/output/inference_args/tutorial.json'\n```\n\nError appears as:\n>Submission Scoring Error\nYour notebook generated a submission file with incorrect format. Some examples causing this are: wrong number of rows or columns, empty values, an incorrect data type for a value, or invalid submission values from what is expected. See more debugging tips\n\nHere is what I changed my code to and now I have a 'Notebook Running' that has been doing so for a while\n```python\nimport shutil\n\nshutil.copy(\"/kaggle/input/sample-local-submission/output/tflite_models/tutorial_model.tflite\",\"./model.tflite\")\nshutil.copy(\"/kaggle/input/sample-local-submission/output/inference_args/tutorial.json\",\"./inference_args.json\")\n\n!zip submission.zip  './model.tflite' './inference_args.json'\n```\n\nany advice would help, thanks in advance.",
    "2389305": "For your non-notebook code, the error happens because of the file name.  Your notebook code should be fine.\n"
  }
}