{
  "id": 603071,
  "title": "Does model size matter in evaluation?",
  "url": "/competitions/grand-xray-slam-division-a/discussion/603071",
  "author_name": "Duong Nguyen",
  "post_date": "2025-08-31T15:35:32.111000",
  "votes": 1,
  "comment_count": 12,
  "views": 0,
  "content": "<p>Hi everyone,</p>\n<p>Are there any constraints/penalties on bigger models, or is it purely based on predictive performance (AUC)?<br>\nThanks in advance!</p>",
  "messages": [
    {
      "id": 3280154,
      "postDate": "2025-09-02T08:37:24.543Z",
      "content": "<p>Thanks for the question!<br>\nFor this competition, evaluation is purely based on predictive performance (AUC) — there are no penalties related to model size.</p>\n<p>However, please keep in mind Kaggle’s standard runtime and resource limits for notebooks (GPU/TPU session lengths, memory). As long as your solution runs within those limits, you’re free to use larger or smaller models.</p>",
      "rawMarkdown": "Thanks for the question!\nFor this competition, evaluation is purely based on predictive performance (AUC) — there are no penalties related to model size.\n\nHowever, please keep in mind Kaggle’s standard runtime and resource limits for notebooks (GPU/TPU session lengths, memory). As long as your solution runs within those limits, you’re free to use larger or smaller models.",
      "votes": 1
    },
    {
      "id": 3279284,
      "postDate": "2025-08-31T15:35:32.110Z",
      "content": "<p>Hi everyone,</p>\n<p>Are there any constraints/penalties on bigger models, or is it purely based on predictive performance (AUC)?<br>\nThanks in advance!</p>",
      "rawMarkdown": "Hi everyone,\n\nAre there any constraints/penalties on bigger models, or is it purely based on predictive performance (AUC)?\nThanks in advance!",
      "votes": 1
    },
    {
      "id": 3279306,
      "postDate": "2025-08-31T15:58:53.777Z",
      "content": "<p>Same question here. Usually there are training time limitations such as &lt;= 9h GPU to avoid a model trains for 30h straight but there is none in this competition, kinda odd tbh.</p>",
      "rawMarkdown": "Same question here. Usually there are training time limitations such as <= 9h GPU to avoid a model trains for 30h straight but there is none in this competition, kinda odd tbh.",
      "replies": [
        {
          "id": 3281510,
          "postDate": "2025-09-04T13:11:13.273Z",
          "content": "<p>Yes you’re allowed to use external GPUs (e.g., Google Colab) for experimentation and development its just an option. The main restriction is simply that the final model must be trained and submitted within Kaggle Notebooks. Using external resources to iterate faster, test architectures, or tune hyperparameters is fine, as long as you retrain your final submission on Kaggle.</p>",
          "rawMarkdown": "Yes you’re allowed to use external GPUs (e.g., Google Colab) for experimentation and development its just an option. The main restriction is simply that the final model must be trained and submitted within Kaggle Notebooks. Using external resources to iterate faster, test architectures, or tune hyperparameters is fine, as long as you retrain your final submission on Kaggle.",
          "replies": [
            {
              "id": 3281527,
              "postDate": "2025-09-04T13:27:58.300Z",
              "content": "<p>Could I ask what the time limitations are for training and inference?</p>",
              "rawMarkdown": "Could I ask what the time limitations are for training and inference?"
            },
            {
              "id": 3281917,
              "postDate": "2025-09-05T08:24:06.707Z",
              "content": "<p>For this competition, the main runtime limits are Kaggle’s standard notebook constraints: up to 12 hours per GPU/TPU session and up to 9 hours per CPU-only session. Inference must also complete within these limits. There are no additional competition-specific runtime restrictions.</p>",
              "rawMarkdown": "For this competition, the main runtime limits are Kaggle’s standard notebook constraints: up to 12 hours per GPU/TPU session and up to 9 hours per CPU-only session. Inference must also complete within these limits. There are no additional competition-specific runtime restrictions."
            },
            {
              "id": 3282095,
              "postDate": "2025-09-05T16:29:35.653Z",
              "content": "<p>The runtime limit is only for inference notebook na? I mean can we train multiple models as long as we make sure the inference notebook runs within the time limit</p>",
              "rawMarkdown": "The runtime limit is only for inference notebook na? I mean can we train multiple models as long as we make sure the inference notebook runs within the time limit\n"
            },
            {
              "id": 3282380,
              "postDate": "2025-09-06T08:09:37.943Z",
              "content": "<p>The runtime limits apply to all Kaggle notebooks, whether for training or inference. You can of course train multiple models during development, but for your final submission the inference notebook must:  </p>\n<ul>\n<li>Stay within Kaggle’s session limits (e.g., 12h for GPU/TPU, 9h for CPU).  </li>\n<li>Be fully reproducible (no pre-trained external weights).  </li>\n</ul>\n<p>So yes, you can train multiple models and ensembles as long as the submitted notebook trains (or loads your saved models) and completes inference within the standard limits.</p>",
              "rawMarkdown": "The runtime limits apply to all Kaggle notebooks, whether for training or inference. You can of course train multiple models during development, but for your final submission the inference notebook must:  \n- Stay within Kaggle’s session limits (e.g., 12h for GPU/TPU, 9h for CPU).  \n- Be fully reproducible (no pre-trained external weights).  \n\nSo yes, you can train multiple models and ensembles as long as the submitted notebook trains (or loads your saved models) and completes inference within the standard limits."
            },
            {
              "id": 3283209,
              "postDate": "2025-09-07T12:25:09.450Z",
              "content": "<p>Hi <a href=\"https://www.kaggle.com/guntasdhanjal\" target=\"_blank\">@guntasdhanjal</a>  Let say I trained a model which take 20 hours to complete  but inference time take only 15 minutes to generate submission file . Is this the valid consideration ?</p>",
              "rawMarkdown": "Hi @guntasdhanjal  Let say I trained a model which take 20 hours to complete  but inference time take only 15 minutes to generate submission file . Is this the valid consideration ?",
              "votes": 1,
              "isDeleted": true
            },
            {
              "id": 3284397,
              "postDate": "2025-09-08T08:06:23.360Z",
              "content": "<p>Yep, that’s fine, as long as your final submission notebook runs fully within Kaggle’s limits and is reproducible. You can train longer outside for experiments, but the submitted model must be trained or loaded from Kaggle.</p>",
              "rawMarkdown": "Yep, that’s fine, as long as your final submission notebook runs fully within Kaggle’s limits and is reproducible. You can train longer outside for experiments, but the submitted model must be trained or loaded from Kaggle."
            },
            {
              "id": 3298252,
              "postDate": "2025-10-05T01:39:12.727Z",
              "content": "<p><a href=\"https://www.kaggle.com/guntasdhanjal\" target=\"_blank\">@guntasdhanjal</a> - So e.g. if we train and save models across multiple Kaggle notebooks, and load them for use in a final inference notebook, it's fine if the inference notebook runs within the 12 hour limit etc.?</p>",
              "rawMarkdown": "@guntasdhanjal - So e.g. if we train and save models across multiple Kaggle notebooks, and load them for use in a final inference notebook, it's fine if the inference notebook runs within the 12 hour limit etc.?"
            },
            {
              "id": 3298732,
              "postDate": "2025-10-06T08:05:57.970Z",
              "content": "<p>Exactly <a href=\"https://www.kaggle.com/optimistix\" target=\"_blank\">@optimistix</a> — that’s perfectly fine.<br>\nYou can train and save models across multiple Kaggle notebooks during development.</p>",
              "rawMarkdown": "Exactly @optimistix — that’s perfectly fine.\nYou can train and save models across multiple Kaggle notebooks during development.",
              "votes": 1
            },
            {
              "id": 3299357,
              "postDate": "2025-10-07T21:02:58.143Z",
              "content": "<p>Thanks for confirming.</p>",
              "rawMarkdown": "Thanks for confirming."
            }
          ]
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 3280154,
      "author_name": "Guntas Dhanjal",
      "author_url": "",
      "post_date": "2025-09-02T08:37:24.543000",
      "content": "<p>Thanks for the question!<br>\nFor this competition, evaluation is purely based on predictive performance (AUC) — there are no penalties related to model size.</p>\n<p>However, please keep in mind Kaggle’s standard runtime and resource limits for notebooks (GPU/TPU session lengths, memory). As long as your solution runs within those limits, you’re free to use larger or smaller models.</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 3279306,
      "author_name": "Hiếu Lê Ngọc",
      "author_url": "",
      "post_date": "2025-08-31T15:58:53.777000",
      "content": "<p>Same question here. Usually there are training time limitations such as &lt;= 9h GPU to avoid a model trains for 30h straight but there is none in this competition, kinda odd tbh.</p>",
      "votes": 0,
      "replies": [
        {
          "id": 3281510,
          "author_name": "Guntas Dhanjal",
          "author_url": "",
          "post_date": "2025-09-04T13:11:13.273000",
          "content": "<p>Yes you’re allowed to use external GPUs (e.g., Google Colab) for experimentation and development its just an option. The main restriction is simply that the final model must be trained and submitted within Kaggle Notebooks. Using external resources to iterate faster, test architectures, or tune hyperparameters is fine, as long as you retrain your final submission on Kaggle.</p>",
          "votes": 0,
          "replies": [
            {
              "id": 3281527,
              "author_name": "AnnieGo",
              "author_url": "",
              "post_date": "2025-09-04T13:27:58.300000",
              "content": "<p>Could I ask what the time limitations are for training and inference?</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 3281917,
              "author_name": "Guntas Dhanjal",
              "author_url": "",
              "post_date": "2025-09-05T08:24:06.707000",
              "content": "<p>For this competition, the main runtime limits are Kaggle’s standard notebook constraints: up to 12 hours per GPU/TPU session and up to 9 hours per CPU-only session. Inference must also complete within these limits. There are no additional competition-specific runtime restrictions.</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 3282095,
              "author_name": "adarsh1403",
              "author_url": "",
              "post_date": "2025-09-05T16:29:35.653000",
              "content": "<p>The runtime limit is only for inference notebook na? I mean can we train multiple models as long as we make sure the inference notebook runs within the time limit</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 3282380,
              "author_name": "Guntas Dhanjal",
              "author_url": "",
              "post_date": "2025-09-06T08:09:37.943000",
              "content": "<p>The runtime limits apply to all Kaggle notebooks, whether for training or inference. You can of course train multiple models during development, but for your final submission the inference notebook must:  </p>\n<ul>\n<li>Stay within Kaggle’s session limits (e.g., 12h for GPU/TPU, 9h for CPU).  </li>\n<li>Be fully reproducible (no pre-trained external weights).  </li>\n</ul>\n<p>So yes, you can train multiple models and ensembles as long as the submitted notebook trains (or loads your saved models) and completes inference within the standard limits.</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 3283209,
              "author_name": "",
              "author_url": "",
              "post_date": "2025-09-07T12:25:09.450000",
              "content": "<p>Hi <a href=\"https://www.kaggle.com/guntasdhanjal\" target=\"_blank\">@guntasdhanjal</a>  Let say I trained a model which take 20 hours to complete  but inference time take only 15 minutes to generate submission file . Is this the valid consideration ?</p>",
              "votes": 1,
              "replies": []
            },
            {
              "id": 3284397,
              "author_name": "Guntas Dhanjal",
              "author_url": "",
              "post_date": "2025-09-08T08:06:23.360000",
              "content": "<p>Yep, that’s fine, as long as your final submission notebook runs fully within Kaggle’s limits and is reproducible. You can train longer outside for experiments, but the submitted model must be trained or loaded from Kaggle.</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 3298252,
              "author_name": "Optimistix",
              "author_url": "",
              "post_date": "2025-10-05T01:39:12.727000",
              "content": "<p><a href=\"https://www.kaggle.com/guntasdhanjal\" target=\"_blank\">@guntasdhanjal</a> - So e.g. if we train and save models across multiple Kaggle notebooks, and load them for use in a final inference notebook, it's fine if the inference notebook runs within the 12 hour limit etc.?</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 3298732,
              "author_name": "Guntas Dhanjal",
              "author_url": "",
              "post_date": "2025-10-06T08:05:57.970000",
              "content": "<p>Exactly <a href=\"https://www.kaggle.com/optimistix\" target=\"_blank\">@optimistix</a> — that’s perfectly fine.<br>\nYou can train and save models across multiple Kaggle notebooks during development.</p>",
              "votes": 1,
              "replies": []
            },
            {
              "id": 3299357,
              "author_name": "Optimistix",
              "author_url": "",
              "post_date": "2025-10-07T21:02:58.143000",
              "content": "<p>Thanks for confirming.</p>",
              "votes": 0,
              "replies": []
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "3280154": "Thanks for the question!\nFor this competition, evaluation is purely based on predictive performance (AUC) — there are no penalties related to model size.\n\nHowever, please keep in mind Kaggle’s standard runtime and resource limits for notebooks (GPU/TPU session lengths, memory). As long as your solution runs within those limits, you’re free to use larger or smaller models.",
    "3279284": "Hi everyone,\n\nAre there any constraints/penalties on bigger models, or is it purely based on predictive performance (AUC)?\nThanks in advance!",
    "3279306": "Same question here. Usually there are training time limitations such as <= 9h GPU to avoid a model trains for 30h straight but there is none in this competition, kinda odd tbh."
  }
}