This is my first competition that is not a code competition and hence competitors are not obligated to use kaggle notebooks. So, is that going to be significant disadvantage for those who use kaggle notebook since others can use far better computers with larger cpu, gpu, and memory or is it not like I described and I can compete using kaggle notebooks with its limited memory and gpu
", "messages": [ { "id": 2786995, "postDate": "2024-05-01T14:35:29.120Z", "content": "All you need is the HDF5 version of the data which allows for direct assess of the data without loading it all into memory (https://www.kaggle.com/datasets/taichiuemura/leap-climsim-hdf5-training-data/).
\nYou can check this discussion for more details.
\nhttps://www.kaggle.com/competitions/leap-atmospheric-physics-ai-climsim/discussion/497304
'Since other can use far better computers'
\nYou have access to TPU.
\nLearn to utilize the available resources. If you use TPU, its usually the opposite of the first sentence. Good luck.
Is TPU considered better than GPU?
", "rawMarkdown": "Is TPU considered better than GPU?" } ] }, { "id": 2786274, "postDate": "2024-05-01T08:46:34.670Z", "content": "Don't want to spoil my secret stash of gpu's but right now I'm only running on a single 4090 system rented for peanuts off of vast.ai. 70 cents an hour for a 4090 and enough storage to easily handle all the data is a steal
", "rawMarkdown": "Don't want to spoil my secret stash of gpu's but right now I'm only running on a single 4090 system rented for peanuts off of vast.ai. 70 cents an hour for a 4090 and enough storage to easily handle all the data is a steal", "votes": 4, "replies": [ { "id": 2786702, "postDate": "2024-05-01T12:19:01.140Z", "content": "Great job 👏
", "rawMarkdown": "Great job 👏" } ] }, { "id": 2761141, "postDate": "2024-04-19T17:13:16.650Z", "content": "This is my first competition that is not a code competition and hence competitors are not obligated to use kaggle notebooks. So, is that going to be significant disadvantage for those who use kaggle notebook since others can use far better computers with larger cpu, gpu, and memory or is it not like I described and I can compete using kaggle notebooks with its limited memory and gpu
", "rawMarkdown": "This is my first competition that is not a code competition and hence competitors are not obligated to use kaggle notebooks. So, is that going to be significant disadvantage for those who use kaggle notebook since others can use far better computers with larger cpu, gpu, and memory or is it not like I described and I can compete using kaggle notebooks with its limited memory and gpu ", "votes": 4 }, { "id": 2762762, "postDate": "2024-04-20T04:44:24.383Z", "content": "IMO in the most deep learning competitions you can win gold medal if you have knowledge of TPU (must be familiar with TF/jax) or you have a 24Gb GPU available with some exceptions e.g. recent Stable Diffusion competition was compute hungry, so don't discourage yourself!
", "rawMarkdown": "IMO in the most deep learning competitions you can win gold medal if you have knowledge of TPU (must be familiar with TF/jax) or you have a 24Gb GPU available with some exceptions e.g. recent Stable Diffusion competition was compute hungry, so don't discourage yourself!", "votes": 2, "replies": [ { "id": 2762992, "postDate": "2024-04-20T07:45:36.917Z", "content": "All you need is RTX 3090
", "rawMarkdown": "All you need is RTX 3090", "votes": 2, "replies": [ { "id": 2783569, "postDate": "2024-04-29T19:57:44.603Z", "content": "How could a 3090 help? Training speed? Because 16GB(P100) to 24GB of VRAM doesn't make a big difference when it comes to loading the dataset.
", "rawMarkdown": "How could a 3090 help? Training speed? Because 16GB(P100) to 24GB of VRAM doesn't make a big difference when it comes to loading the dataset.", "votes": -1 } ] } ] }, { "id": 2761157, "postDate": "2024-04-19T17:18:37.420Z", "content": "😀 @aymanallawi this seems so!
\nWe have a few more competitions like this too, kindly find them as below-
You can use for free github with 20 parallel jobs each with 4 cores
\n", "rawMarkdown": "You can use for free github with 20 parallel jobs each with 4 cores\n\nhttps://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories", "votes": 1 } ], "comments": [ { "id": 2786995, "author_name": "HungryLearner", "author_url": "", "post_date": "2024-05-01T14:35:29.120000", "content": "All you need is the HDF5 version of the data which allows for direct assess of the data without loading it all into memory (https://www.kaggle.com/datasets/taichiuemura/leap-climsim-hdf5-training-data/).
\nYou can check this discussion for more details.
\nhttps://www.kaggle.com/competitions/leap-atmospheric-physics-ai-climsim/discussion/497304
'Since other can use far better computers'
\nYou have access to TPU.
\nLearn to utilize the available resources. If you use TPU, its usually the opposite of the first sentence. Good luck.
Is TPU considered better than GPU?
", "votes": 0, "replies": [] } ] }, { "id": 2786274, "author_name": "ryches", "author_url": "", "post_date": "2024-05-01T08:46:34.670000", "content": "Don't want to spoil my secret stash of gpu's but right now I'm only running on a single 4090 system rented for peanuts off of vast.ai. 70 cents an hour for a 4090 and enough storage to easily handle all the data is a steal
", "votes": 4, "replies": [ { "id": 2786702, "author_name": "Ayman Allawi", "author_url": "", "post_date": "2024-05-01T12:19:01.140000", "content": "Great job 👏
", "votes": 0, "replies": [] } ] }, { "id": 2762762, "author_name": "slime", "author_url": "", "post_date": "2024-04-20T04:44:24.383000", "content": "IMO in the most deep learning competitions you can win gold medal if you have knowledge of TPU (must be familiar with TF/jax) or you have a 24Gb GPU available with some exceptions e.g. recent Stable Diffusion competition was compute hungry, so don't discourage yourself!
", "votes": 2, "replies": [ { "id": 2762992, "author_name": "Gunes Evitan", "author_url": "", "post_date": "2024-04-20T07:45:36.917000", "content": "All you need is RTX 3090
", "votes": 2, "replies": [ { "id": 2783569, "author_name": "Fernando Melo", "author_url": "", "post_date": "2024-04-29T19:57:44.603000", "content": "How could a 3090 help? Training speed? Because 16GB(P100) to 24GB of VRAM doesn't make a big difference when it comes to loading the dataset.
", "votes": -1, "replies": [] } ] } ] }, { "id": 2761157, "author_name": "Ravi Ramakrishnan", "author_url": "", "post_date": "2024-04-19T17:18:37.420000", "content": "😀 @aymanallawi this seems so!
\nWe have a few more competitions like this too, kindly find them as below-
You can use for free github with 20 parallel jobs each with 4 cores
\n", "votes": 1, "replies": [] } ], "raw_markdown_by_id": { "2786995": "All you need is the HDF5 version of the data which allows for direct assess of the data without loading it all into memory (https://www.kaggle.com/datasets/taichiuemura/leap-climsim-hdf5-training-data/). \n\nYou can check this discussion for more details.\nhttps://www.kaggle.com/competitions/leap-atmospheric-physics-ai-climsim/discussion/497304\n", "2761590": "'Since other can use far better computers' \nYou have access to TPU. \nLearn to utilize the available resources. If you use TPU, its usually the opposite of the first sentence. Good luck.", "2786274": "Don't want to spoil my secret stash of gpu's but right now I'm only running on a single 4090 system rented for peanuts off of vast.ai. 70 cents an hour for a 4090 and enough storage to easily handle all the data is a steal", "2761141": "This is my first competition that is not a code competition and hence competitors are not obligated to use kaggle notebooks. So, is that going to be significant disadvantage for those who use kaggle notebook since others can use far better computers with larger cpu, gpu, and memory or is it not like I described and I can compete using kaggle notebooks with its limited memory and gpu ", "2762762": "IMO in the most deep learning competitions you can win gold medal if you have knowledge of TPU (must be familiar with TF/jax) or you have a 24Gb GPU available with some exceptions e.g. recent Stable Diffusion competition was compute hungry, so don't discourage yourself!", "2761157": "😀 @aymanallawi this seems so!\nWe have a few more competitions like this too, kindly find them as below-\n1. [Stanford Ribonanza RNA Folding](https://www.kaggle.com/competitions/stanford-ribonanza-rna-folding)\n2. [Leash Bio - Predict New Medicines with BELKA](https://www.kaggle.com/competitions/leash-BELKA)", "2765711": "You can use for free github with 20 parallel jobs each with 4 cores\n\nhttps://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories" } }