{
  "id": 529358,
  "title": "Which regression algorithm is more suitable for this match?",
  "url": "/competitions/ariel-data-challenge-2024/discussion/529358",
  "author_name": "E-Max AI",
  "post_date": "2024-08-20T09:49:43.314000",
  "votes": 6,
  "comment_count": 2,
  "views": 0,
  "content": "<p>I experimented with a variety of regression algorithms to compare their performance differences and find the model that best suits this competition. The results of the experiment are as follows:<br>\nRidge and Linear Regression is the best.<br>\nI used MultiOutputRegressor to wrap a single regressor to make it suitable for multi-target regression tasks, but the efficiency is somewhat low. It takes about 7 hours to compute the results on my 13900K.</p>\n<p>Ridge<br>\n% R2 score: 0.9945<br>\n% Root mean squared error: 0.0001241<br>\n% Estimated competition score: 0.379</p>\n<p>Linear Regression<br>\n% R2 score: 0.995<br>\n% Root mean squared error: 0.000124<br>\n% Estimated competition score: 0.379</p>\n<p>k-Nearest Neighbors<br>\n% R2 score: 0.938<br>\n% Root mean squared error: 0.000428<br>\n% Estimated competition score: 0.203</p>\n<p>Decision Tree<br>\n% R2 score: 0.943<br>\n% Root mean squared error: 0.000412<br>\n% Estimated competition score: 0.209</p>\n<p>Random Forest<br>\n% R2 score: 0.976<br>\n% Root mean squared error: 0.000267<br>\n% Estimated competition score: 0.272</p>\n<p>SGD<br>\n% R2 score: -341251262544492403707458665757358839300096.000<br>\n% Root mean squared error: 1007245294093768960.000000<br>\n% Estimated competition score: 0.000</p>\n<p>XGBoost<br>\n% R2 score: 0.976<br>\n% Root mean squared error: 0.000266<br>\n% Estimated competition score: 0.273</p>\n<p>AdaBoost<br>\n% R2 score: 0.974<br>\n% Root mean squared error: 0.000277<br>\n% Estimated competition score: 0.266</p>\n<p>ExtreTrees<br>\n% R2 score: 0.980<br>\n% Root mean squared error: 0.000242<br>\n% Estimated competition score: 0.286</p>\n<p>Notebook link: <a href=\"https://www.kaggle.com/code/royalacecat/comparison-of-different-regression-algorithm\" target=\"_blank\">https://www.kaggle.com/code/royalacecat/comparison-of-different-regression-algorithm</a></p>",
  "messages": [
    {
      "id": 2964879,
      "postDate": "2024-08-20T09:49:43.313Z",
      "content": "<p>I experimented with a variety of regression algorithms to compare their performance differences and find the model that best suits this competition. The results of the experiment are as follows:<br>\nRidge and Linear Regression is the best.<br>\nI used MultiOutputRegressor to wrap a single regressor to make it suitable for multi-target regression tasks, but the efficiency is somewhat low. It takes about 7 hours to compute the results on my 13900K.</p>\n<p>Ridge<br>\n% R2 score: 0.9945<br>\n% Root mean squared error: 0.0001241<br>\n% Estimated competition score: 0.379</p>\n<p>Linear Regression<br>\n% R2 score: 0.995<br>\n% Root mean squared error: 0.000124<br>\n% Estimated competition score: 0.379</p>\n<p>k-Nearest Neighbors<br>\n% R2 score: 0.938<br>\n% Root mean squared error: 0.000428<br>\n% Estimated competition score: 0.203</p>\n<p>Decision Tree<br>\n% R2 score: 0.943<br>\n% Root mean squared error: 0.000412<br>\n% Estimated competition score: 0.209</p>\n<p>Random Forest<br>\n% R2 score: 0.976<br>\n% Root mean squared error: 0.000267<br>\n% Estimated competition score: 0.272</p>\n<p>SGD<br>\n% R2 score: -341251262544492403707458665757358839300096.000<br>\n% Root mean squared error: 1007245294093768960.000000<br>\n% Estimated competition score: 0.000</p>\n<p>XGBoost<br>\n% R2 score: 0.976<br>\n% Root mean squared error: 0.000266<br>\n% Estimated competition score: 0.273</p>\n<p>AdaBoost<br>\n% R2 score: 0.974<br>\n% Root mean squared error: 0.000277<br>\n% Estimated competition score: 0.266</p>\n<p>ExtreTrees<br>\n% R2 score: 0.980<br>\n% Root mean squared error: 0.000242<br>\n% Estimated competition score: 0.286</p>\n<p>Notebook link: <a href=\"https://www.kaggle.com/code/royalacecat/comparison-of-different-regression-algorithm\" target=\"_blank\">https://www.kaggle.com/code/royalacecat/comparison-of-different-regression-algorithm</a></p>",
      "rawMarkdown": "I experimented with a variety of regression algorithms to compare their performance differences and find the model that best suits this competition. The results of the experiment are as follows:\nRidge and Linear Regression is the best.\nI used MultiOutputRegressor to wrap a single regressor to make it suitable for multi-target regression tasks, but the efficiency is somewhat low. It takes about 7 hours to compute the results on my 13900K.\n\nRidge\n% R2 score: 0.9945\n% Root mean squared error: 0.0001241\n% Estimated competition score: 0.379\n\nLinear Regression\n% R2 score: 0.995\n% Root mean squared error: 0.000124\n% Estimated competition score: 0.379\n\nk-Nearest Neighbors\n% R2 score: 0.938\n% Root mean squared error: 0.000428\n% Estimated competition score: 0.203\n\nDecision Tree\n% R2 score: 0.943\n% Root mean squared error: 0.000412\n% Estimated competition score: 0.209\n\nRandom Forest\n% R2 score: 0.976\n% Root mean squared error: 0.000267\n% Estimated competition score: 0.272\n\nSGD\n% R2 score: -341251262544492403707458665757358839300096.000\n% Root mean squared error: 1007245294093768960.000000\n% Estimated competition score: 0.000\n\nXGBoost\n% R2 score: 0.976\n% Root mean squared error: 0.000266\n% Estimated competition score: 0.273\n\nAdaBoost\n% R2 score: 0.974\n% Root mean squared error: 0.000277\n% Estimated competition score: 0.266\n\nExtreTrees\n% R2 score: 0.980\n% Root mean squared error: 0.000242\n% Estimated competition score: 0.286\n\nNotebook link: https://www.kaggle.com/code/royalacecat/comparison-of-different-regression-algorithm",
      "votes": 6
    },
    {
      "id": 2965078,
      "postDate": "2024-08-20T14:22:01.180Z",
      "content": "<p>Did you have tried with ElasticNet? I wonder if it will works.</p>",
      "rawMarkdown": "Did you have tried with ElasticNet? I wonder if it will works.",
      "replies": [
        {
          "id": 2965690,
          "postDate": "2024-08-21T07:25:51.933Z",
          "content": "<p>from sklearn.linear_model import ElasticNetfrom sklearn.linear_model import ElasticNet<br>\nmodel = ElasticNet(alpha=1e-12, l1_ratio=1e-12)<br>\noof_pred = cross_val_predict(model, train, train_labels)<br>\nprint(f\"# R2 score: {r2_score(train_labels, oof_pred):.4f}\")<br>\nsigma_pred = mean_squared_error(train_labels, oof_pred, squared=False)<br>\nprint(f\"# Root mean squared error: {sigma_pred:.7f}\")<br>\noof_df = postprocessing(oof_pred, train_adc_info.index, sigma_pred)<br>\ngll_score = competition_score(train_labels.copy().reset_index(),<br>\n                                oof_df.copy().reset_index(),<br>\n                                naive_mean=train_labels.values.mean(),<br>\n                                naive_sigma=train_labels.values.std(),<br>\n                                sigma_true=0.000003)<br>\nprint(f\"# Estimated competition score: {gll_score:.3f}\")</p>\n<h1>R2 score: 0.9193</h1>\n<h1>Root mean squared error: 0.0004900</h1>\n<h1>Estimated competition score: 0.185</h1>",
          "rawMarkdown": "from sklearn.linear_model import ElasticNetfrom sklearn.linear_model import ElasticNet\nmodel = ElasticNet(alpha=1e-12, l1_ratio=1e-12)\noof_pred = cross_val_predict(model, train, train_labels)\nprint(f\"# R2 score: {r2_score(train_labels, oof_pred):.4f}\")\nsigma_pred = mean_squared_error(train_labels, oof_pred, squared=False)\nprint(f\"# Root mean squared error: {sigma_pred:.7f}\")\noof_df = postprocessing(oof_pred, train_adc_info.index, sigma_pred)\ngll_score = competition_score(train_labels.copy().reset_index(),\n                                oof_df.copy().reset_index(),\n                                naive_mean=train_labels.values.mean(),\n                                naive_sigma=train_labels.values.std(),\n                                sigma_true=0.000003)\nprint(f\"# Estimated competition score: {gll_score:.3f}\")\n\n\n# R2 score: 0.9193\n# Root mean squared error: 0.0004900\n# Estimated competition score: 0.185"
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 2965078,
      "author_name": "Thopmann",
      "author_url": "",
      "post_date": "2024-08-20T14:22:01.180000",
      "content": "<p>Did you have tried with ElasticNet? I wonder if it will works.</p>",
      "votes": 0,
      "replies": [
        {
          "id": 2965690,
          "author_name": "E-Max AI",
          "author_url": "",
          "post_date": "2024-08-21T07:25:51.933000",
          "content": "<p>from sklearn.linear_model import ElasticNetfrom sklearn.linear_model import ElasticNet<br>\nmodel = ElasticNet(alpha=1e-12, l1_ratio=1e-12)<br>\noof_pred = cross_val_predict(model, train, train_labels)<br>\nprint(f\"# R2 score: {r2_score(train_labels, oof_pred):.4f}\")<br>\nsigma_pred = mean_squared_error(train_labels, oof_pred, squared=False)<br>\nprint(f\"# Root mean squared error: {sigma_pred:.7f}\")<br>\noof_df = postprocessing(oof_pred, train_adc_info.index, sigma_pred)<br>\ngll_score = competition_score(train_labels.copy().reset_index(),<br>\n                                oof_df.copy().reset_index(),<br>\n                                naive_mean=train_labels.values.mean(),<br>\n                                naive_sigma=train_labels.values.std(),<br>\n                                sigma_true=0.000003)<br>\nprint(f\"# Estimated competition score: {gll_score:.3f}\")</p>\n<h1>R2 score: 0.9193</h1>\n<h1>Root mean squared error: 0.0004900</h1>\n<h1>Estimated competition score: 0.185</h1>",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2964879": "I experimented with a variety of regression algorithms to compare their performance differences and find the model that best suits this competition. The results of the experiment are as follows:\nRidge and Linear Regression is the best.\nI used MultiOutputRegressor to wrap a single regressor to make it suitable for multi-target regression tasks, but the efficiency is somewhat low. It takes about 7 hours to compute the results on my 13900K.\n\nRidge\n% R2 score: 0.9945\n% Root mean squared error: 0.0001241\n% Estimated competition score: 0.379\n\nLinear Regression\n% R2 score: 0.995\n% Root mean squared error: 0.000124\n% Estimated competition score: 0.379\n\nk-Nearest Neighbors\n% R2 score: 0.938\n% Root mean squared error: 0.000428\n% Estimated competition score: 0.203\n\nDecision Tree\n% R2 score: 0.943\n% Root mean squared error: 0.000412\n% Estimated competition score: 0.209\n\nRandom Forest\n% R2 score: 0.976\n% Root mean squared error: 0.000267\n% Estimated competition score: 0.272\n\nSGD\n% R2 score: -341251262544492403707458665757358839300096.000\n% Root mean squared error: 1007245294093768960.000000\n% Estimated competition score: 0.000\n\nXGBoost\n% R2 score: 0.976\n% Root mean squared error: 0.000266\n% Estimated competition score: 0.273\n\nAdaBoost\n% R2 score: 0.974\n% Root mean squared error: 0.000277\n% Estimated competition score: 0.266\n\nExtreTrees\n% R2 score: 0.980\n% Root mean squared error: 0.000242\n% Estimated competition score: 0.286\n\nNotebook link: https://www.kaggle.com/code/royalacecat/comparison-of-different-regression-algorithm",
    "2965078": "Did you have tried with ElasticNet? I wonder if it will works."
  }
}