I've been tuning my model for this competition and realized that its taking multiple hours just to finish training 1 set of hyperparameters and the majority is using the same model taken from the previous winner. There's so much energy wasted from redundancy.
\nThe environment is dying from everyone individually tuning their models. What is the fix for this?
", "messages": [ { "id": 2388257, "postDate": "2023-08-13T09:08:04.903Z", "content": "I've been tuning my model for this competition and realized that its taking multiple hours just to finish training 1 set of hyperparameters and the majority is using the same model taken from the previous winner. There's so much energy wasted from redundancy.
\nThe environment is dying from everyone individually tuning their models. What is the fix for this?
", "rawMarkdown": "I've been tuning my model for this competition and realized that its taking multiple hours just to finish training 1 set of hyperparameters and the majority is using the same model taken from the previous winner. There's so much energy wasted from redundancy.\n\nThe environment is dying from everyone individually tuning their models. What is the fix for this?", "votes": 6 }, { "id": 2390743, "postDate": "2023-08-14T17:50:44.833Z", "content": "Hello,
\nGreat topic!
\nHere are relevant links:
\n\n(Note the NeurIPS 2023 workshop \"Tackling Climate Change with Machine Learning\" and the corresponding mentor-mentee matching program with due date coming up on Aug 18, 2023)
\nI also started some thoughts on this concern back in June:
\nhttps://www.kaggle.com/code/vcolliym/climate-research/notebook#Conclusions
Perhaps we should be reminded of recycling practices: if everyone does a small part, the impact could be big.
\nHere are more inspirations:
\nhttps://beachmetro.com/2023/07/15/beacher-markus-pukonen-completes-motor-free-circumnavigation-of-the-globe-eight-years-after-paddling-out-into-lake-ontario/
It's hard to find a way around this. I think it's pretty morally valuable to share good notebooks and training strategies etc. at least early in competitions, as you raise the quality of solutions the competition organizers get, for problems that are often important to society, while probably slightly lowering compute used by the average participant (about this second point I'm pretty uncertain though).
", "rawMarkdown": "It's hard to find a way around this. I think it's pretty morally valuable to share good notebooks and training strategies etc. at least early in competitions, as you raise the quality of solutions the competition organizers get, for problems that are often important to society, while probably slightly lowering compute used by the average participant (about this second point I'm pretty uncertain though).", "votes": 1 } ], "comments": [ { "id": 2390743, "author_name": "Vcol", "author_url": "", "post_date": "2023-08-14T17:50:44.833000", "content": "Hello,
\nGreat topic!
\nHere are relevant links:
\n\n(Note the NeurIPS 2023 workshop \"Tackling Climate Change with Machine Learning\" and the corresponding mentor-mentee matching program with due date coming up on Aug 18, 2023)
\nI also started some thoughts on this concern back in June:
\nhttps://www.kaggle.com/code/vcolliym/climate-research/notebook#Conclusions
Perhaps we should be reminded of recycling practices: if everyone does a small part, the impact could be big.
\nHere are more inspirations:
\nhttps://beachmetro.com/2023/07/15/beacher-markus-pukonen-completes-motor-free-circumnavigation-of-the-globe-eight-years-after-paddling-out-into-lake-ontario/
It's hard to find a way around this. I think it's pretty morally valuable to share good notebooks and training strategies etc. at least early in competitions, as you raise the quality of solutions the competition organizers get, for problems that are often important to society, while probably slightly lowering compute used by the average participant (about this second point I'm pretty uncertain though).
", "votes": 1, "replies": [] } ], "raw_markdown_by_id": { "2388257": "I've been tuning my model for this competition and realized that its taking multiple hours just to finish training 1 set of hyperparameters and the majority is using the same model taken from the previous winner. There's so much energy wasted from redundancy.\n\nThe environment is dying from everyone individually tuning their models. What is the fix for this?", "2390743": "Hello,\n\nGreat topic!\n\nHere are relevant links:\n- https://www.climatechange.ai/\n- https://greenai.cloud/\n\n(Note the NeurIPS 2023 workshop \"Tackling Climate Change with Machine Learning\" and the corresponding **mentor-mentee matching program** with due date coming up on **Aug 18, 2023**) \n\nI also started some thoughts on this concern back in June:\nhttps://www.kaggle.com/code/vcolliym/climate-research/notebook#Conclusions\n\nPerhaps we should be reminded of recycling practices: if everyone does a small part, the impact could be big. \n\nHere are more inspirations:\nhttps://beachmetro.com/2023/07/15/beacher-markus-pukonen-completes-motor-free-circumnavigation-of-the-globe-eight-years-after-paddling-out-into-lake-ontario/", "2389062": "It's hard to find a way around this. I think it's pretty morally valuable to share good notebooks and training strategies etc. at least early in competitions, as you raise the quality of solutions the competition organizers get, for problems that are often important to society, while probably slightly lowering compute used by the average participant (about this second point I'm pretty uncertain though)." } }