In other Kaggle competitions, we can detect whether the context is competition submission or notebook run by checking the size of the test set, thereby saving time and GPU hours from the 30hr weekly limit. However, it seems that the publicly available test.csv and sample_submission.csv is the same as the hidden ones.
\nThis makes GPU-based approaches (such as neural networks) submissions take up the 30hr weekly limit very quickly.
\nDoes anyone know how to detect the context in notebook code using other methods? Thanks!
\n@sohier @inversion @mylesoneill @jerrylin96
", "messages": [ { "id": 2787486, "postDate": "2024-05-01T18:40:50.363Z", "content": "There is an environment variable that will tell you if it is a submission run
\n os\nos.getenv()\nyou can check this and make it do different behavior based on if it is a submission run or not
", "rawMarkdown": "There is an environment variable that will tell you if it is a submission run\n\n```python\nimport os\nos.getenv('KAGGLE_IS_COMPETITION_RERUN')\n```\n\nyou can check this and make it do different behavior based on if it is a submission run or not", "votes": 3, "replies": [ { "id": 2787950, "postDate": "2024-05-02T03:02:33.040Z", "content": "Thanks for the information! However, this does not seem to work.
\n polars pl\n pandas pd\n os\n shutil\n\n os.getenv():\n assert , \n:\n shutil.copy(, )\nMy notebook and submission is able to run without errors in both competition submission and notebook run.
", "rawMarkdown": "Thanks for the information! However, this does not seem to work.\n\n```\nimport polars as pl\nimport pandas as pd\nimport os\nimport shutil\n\nif os.getenv(\"KAGGLE_IS_COMPETITION_RERUN\"):\n assert False, \"Temp error!\"\nelse:\n shutil.copy(\"/kaggle/input/leap-atmospheric-physics-ai-climsim/sample_submission.csv\", \"submission.csv\")\n```\n\nMy notebook and submission is able to run without errors in both competition submission and notebook run.", "votes": 1, "replies": [ { "id": 2788008, "postDate": "2024-05-02T04:09:58.753Z", "content": "Since this isnt a code competition dont you just run the notebook once on our publicly available test data and then submit the csv? It isnt run a second time in the background on hidden data. We have all the test data
", "rawMarkdown": "Since this isnt a code competition dont you just run the notebook once on our publicly available test data and then submit the csv? It isnt run a second time in the background on hidden data. We have all the test data", "votes": 1 }, { "id": 2788026, "postDate": "2024-05-02T04:22:15.133Z", "content": "Ah I see😂. I have only participated in code competitions before. Didn't see the upload .csv option in this competition. Thanks so much for the information!
\nDoes this mean we can predict using our own computer and its enough to upload the submission.csv file without relying on Kaggle notebooks?
", "rawMarkdown": "Ah I see😂. I have only participated in code competitions before. Didn't see the upload .csv option in this competition. Thanks so much for the information!\n\nDoes this mean we can predict using our own computer and its enough to upload the submission.csv file without relying on Kaggle notebooks?" }, { "id": 2788050, "postDate": "2024-05-02T04:46:43.077Z", "content": "Yep exactly. You can run inference anywhere and then just upload the file. I havent seen any non-code competitions in a while. Seemed to become the standard. Not sure why they didnt on this one
", "rawMarkdown": "Yep exactly. You can run inference anywhere and then just upload the file. I havent seen any non-code competitions in a while. Seemed to become the standard. Not sure why they didnt on this one", "votes": 1 }, { "id": 2789974, "postDate": "2024-05-03T00:05:40.633Z", "content": "You don't see a lot of non code because it's problematic with speech/LLM/CV due to the possibility for hand labeling beating models, impossible with time series future prediction (obviously) and not very wise with small test set that the host want to generalize to (instead of overfitting). For the rest of the competitions, it's not so rare…BELKA is also non-code, as was also Ribonanza. Especially for competitions with large data set, if it's code then it puts limits on ensembles size and model complexity, then the focus shifts from 'best possible solution' (with reasonable compute) to 'best solution that I can run on a notebook in 9 hours'+it's puts huge strain on Kaggle resources for conpute-heavy competitions (9*5*7=315(!) GPU weekly 'test quota' which is more than ten times the regular quota…)
", "rawMarkdown": "You don't see a lot of non code because it's problematic with speech/LLM/CV due to the possibility for hand labeling beating models, impossible with time series future prediction (obviously) and not very wise with small test set that the host want to generalize to (instead of overfitting). For the rest of the competitions, it's not so rare...BELKA is also non-code, as was also Ribonanza. Especially for competitions with large data set, if it's code then it puts limits on ensembles size and model complexity, then the focus shifts from 'best possible solution' (with reasonable compute) to 'best solution that I can run on a notebook in 9 hours'+it's puts huge strain on Kaggle resources for conpute-heavy competitions (9\\*5\\*7=315(!) GPU weekly 'test quota' which is more than ten times the regular quota...)", "votes": 2 } ] } ] }, { "id": 2787455, "postDate": "2024-05-01T18:18:15.093Z", "content": "In other Kaggle competitions, we can detect whether the context is competition submission or notebook run by checking the size of the test set, thereby saving time and GPU hours from the 30hr weekly limit. However, it seems that the publicly available test.csv and sample_submission.csv is the same as the hidden ones.
\nThis makes GPU-based approaches (such as neural networks) submissions take up the 30hr weekly limit very quickly.
\nDoes anyone know how to detect the context in notebook code using other methods? Thanks!
\n@sohier @inversion @mylesoneill @jerrylin96
", "rawMarkdown": "In other Kaggle competitions, we can detect whether the context is competition submission or notebook run by checking the size of the test set, thereby saving time and GPU hours from the 30hr weekly limit. However, it seems that the publicly available test.csv and sample_submission.csv is the same as the hidden ones.\n\nThis makes GPU-based approaches (such as neural networks) submissions take up the 30hr weekly limit very quickly.\n\nDoes anyone know how to detect the context in notebook code using other methods? Thanks!\n\n@sohier @inversion @mylesoneill @jerrylin96 ", "votes": 1 } ], "comments": [ { "id": 2787486, "author_name": "ryches", "author_url": "", "post_date": "2024-05-01T18:40:50.363000", "content": "There is an environment variable that will tell you if it is a submission run
\n os\nos.getenv()\nyou can check this and make it do different behavior based on if it is a submission run or not
", "votes": 3, "replies": [ { "id": 2787950, "author_name": "Chau YH", "author_url": "", "post_date": "2024-05-02T03:02:33.040000", "content": "Thanks for the information! However, this does not seem to work.
\n polars pl\n pandas pd\n os\n shutil\n\n os.getenv():\n assert , \n:\n shutil.copy(, )\nMy notebook and submission is able to run without errors in both competition submission and notebook run.
", "votes": 1, "replies": [ { "id": 2788008, "author_name": "ryches", "author_url": "", "post_date": "2024-05-02T04:09:58.753000", "content": "Since this isnt a code competition dont you just run the notebook once on our publicly available test data and then submit the csv? It isnt run a second time in the background on hidden data. We have all the test data
", "votes": 1, "replies": [] }, { "id": 2788026, "author_name": "Chau YH", "author_url": "", "post_date": "2024-05-02T04:22:15.133000", "content": "Ah I see😂. I have only participated in code competitions before. Didn't see the upload .csv option in this competition. Thanks so much for the information!
\nDoes this mean we can predict using our own computer and its enough to upload the submission.csv file without relying on Kaggle notebooks?
", "votes": 0, "replies": [] }, { "id": 2788050, "author_name": "ryches", "author_url": "", "post_date": "2024-05-02T04:46:43.077000", "content": "Yep exactly. You can run inference anywhere and then just upload the file. I havent seen any non-code competitions in a while. Seemed to become the standard. Not sure why they didnt on this one
", "votes": 1, "replies": [] }, { "id": 2789974, "author_name": "greySnow", "author_url": "", "post_date": "2024-05-03T00:05:40.633000", "content": "You don't see a lot of non code because it's problematic with speech/LLM/CV due to the possibility for hand labeling beating models, impossible with time series future prediction (obviously) and not very wise with small test set that the host want to generalize to (instead of overfitting). For the rest of the competitions, it's not so rare…BELKA is also non-code, as was also Ribonanza. Especially for competitions with large data set, if it's code then it puts limits on ensembles size and model complexity, then the focus shifts from 'best possible solution' (with reasonable compute) to 'best solution that I can run on a notebook in 9 hours'+it's puts huge strain on Kaggle resources for conpute-heavy competitions (9*5*7=315(!) GPU weekly 'test quota' which is more than ten times the regular quota…)
", "votes": 2, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2787486": "There is an environment variable that will tell you if it is a submission run\n\n```python\nimport os\nos.getenv('KAGGLE_IS_COMPETITION_RERUN')\n```\n\nyou can check this and make it do different behavior based on if it is a submission run or not", "2787455": "In other Kaggle competitions, we can detect whether the context is competition submission or notebook run by checking the size of the test set, thereby saving time and GPU hours from the 30hr weekly limit. However, it seems that the publicly available test.csv and sample_submission.csv is the same as the hidden ones.\n\nThis makes GPU-based approaches (such as neural networks) submissions take up the 30hr weekly limit very quickly.\n\nDoes anyone know how to detect the context in notebook code using other methods? Thanks!\n\n@sohier @inversion @mylesoneill @jerrylin96 " } }