{
  "id": 411246,
  "title": "Persistent problems with making valid submissions",
  "url": "/competitions/asl-fingerspelling/discussion/411246",
  "author_name": "Wondering Alice",
  "post_date": "2023-05-18T10:18:38.972000",
  "votes": 11,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Dear Organisers ( <a href=\"https://www.kaggle.com/thadstarner\" target=\"_blank\">@thadstarner</a> , <a href=\"https://www.kaggle.com/sohier\" target=\"_blank\">@sohier</a> ),</p>\n<p>We are very enthousiastic about this new competition. And I assume you want us to contribute to the progress in the field and focus on models and data problems.</p>\n<p>However, more than a week after the start of the competition, no-one has managed to make a successful submission, not even high-scoring participants from the previous competition (who are familiar with the data format and making TfLite submissions), and not even with a dummy model that makes random or constant predictions.</p>\n<p>This is very frustrating and a waste of time. Also, clearly, this has nothing to do with the time and memory constraints that are part of the challenge in this competition, but rather with a lack of information with respect to the assumptions that are made by the scoring code. For this reason I would like to insist that the organisers share code that can make a dummy submission (e.g. constant or random predictions) and clarify the requirements to the model output format in order to make a successful submission.</p>",
  "messages": [
    {
      "id": 2264314,
      "postDate": "2023-05-18T10:18:38.973Z",
      "content": "<p>Dear Organisers ( <a href=\"https://www.kaggle.com/thadstarner\" target=\"_blank\">@thadstarner</a> , <a href=\"https://www.kaggle.com/sohier\" target=\"_blank\">@sohier</a> ),</p>\n<p>We are very enthousiastic about this new competition. And I assume you want us to contribute to the progress in the field and focus on models and data problems.</p>\n<p>However, more than a week after the start of the competition, no-one has managed to make a successful submission, not even high-scoring participants from the previous competition (who are familiar with the data format and making TfLite submissions), and not even with a dummy model that makes random or constant predictions.</p>\n<p>This is very frustrating and a waste of time. Also, clearly, this has nothing to do with the time and memory constraints that are part of the challenge in this competition, but rather with a lack of information with respect to the assumptions that are made by the scoring code. For this reason I would like to insist that the organisers share code that can make a dummy submission (e.g. constant or random predictions) and clarify the requirements to the model output format in order to make a successful submission.</p>",
      "rawMarkdown": "Dear Organisers ( @thadstarner , @sohier ),\n\nWe are very enthousiastic about this new competition. And I assume you want us to contribute to the progress in the field and focus on models and data problems.\n\nHowever, more than a week after the start of the competition, no-one has managed to make a successful submission, not even high-scoring participants from the previous competition (who are familiar with the data format and making TfLite submissions), and not even with a dummy model that makes random or constant predictions.\n\nThis is very frustrating and a waste of time. Also, clearly, this has nothing to do with the time and memory constraints that are part of the challenge in this competition, but rather with a lack of information with respect to the assumptions that are made by the scoring code. For this reason I would like to insist that the organisers share code that can make a dummy submission (e.g. constant or random predictions) and clarify the requirements to the model output format in order to make a successful submission.",
      "votes": 11
    },
    {
      "id": 2278769,
      "postDate": "2023-05-29T00:34:56.247Z",
      "content": "<p>Also struggling with this… Managed to make a model which works great using the provided evaluation code, but can't get it to work on submission.</p>",
      "rawMarkdown": "Also struggling with this... Managed to make a model which works great using the provided evaluation code, but can't get it to work on submission."
    },
    {
      "id": 2264630,
      "postDate": "2023-05-18T15:34:24.417Z",
      "content": "<p>last competition took as input a numpy array, seems from their eval description, this time the input is a dataframe?</p>\n<p>def load_relevant_data_subset(pq_path):<br>\n    return pd.read_parquet(pq_path, columns=selected_columns)</p>",
      "rawMarkdown": "last competition took as input a numpy array, seems from their eval description, this time the input is a dataframe?\n\ndef load_relevant_data_subset(pq_path):\n    return pd.read_parquet(pq_path, columns=selected_columns)"
    }
  ],
  "comments": [
    {
      "id": 2278769,
      "author_name": "anokas",
      "author_url": "",
      "post_date": "2023-05-29T00:34:56.247000",
      "content": "<p>Also struggling with this… Managed to make a model which works great using the provided evaluation code, but can't get it to work on submission.</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2264630,
      "author_name": "Jon",
      "author_url": "",
      "post_date": "2023-05-18T15:34:24.417000",
      "content": "<p>last competition took as input a numpy array, seems from their eval description, this time the input is a dataframe?</p>\n<p>def load_relevant_data_subset(pq_path):<br>\n    return pd.read_parquet(pq_path, columns=selected_columns)</p>",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2264314": "Dear Organisers ( @thadstarner , @sohier ),\n\nWe are very enthousiastic about this new competition. And I assume you want us to contribute to the progress in the field and focus on models and data problems.\n\nHowever, more than a week after the start of the competition, no-one has managed to make a successful submission, not even high-scoring participants from the previous competition (who are familiar with the data format and making TfLite submissions), and not even with a dummy model that makes random or constant predictions.\n\nThis is very frustrating and a waste of time. Also, clearly, this has nothing to do with the time and memory constraints that are part of the challenge in this competition, but rather with a lack of information with respect to the assumptions that are made by the scoring code. For this reason I would like to insist that the organisers share code that can make a dummy submission (e.g. constant or random predictions) and clarify the requirements to the model output format in order to make a successful submission.",
    "2278769": "Also struggling with this... Managed to make a model which works great using the provided evaluation code, but can't get it to work on submission.",
    "2264630": "last competition took as input a numpy array, seems from their eval description, this time the input is a dataframe?\n\ndef load_relevant_data_subset(pq_path):\n    return pd.read_parquet(pq_path, columns=selected_columns)"
  }
}