{"metadata":{"kernelspec":{"display_name":"Python 3 (ipykernel)","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.12.7"}},"nbformat_minor":4,"nbformat":4,"cells":[{"id":"b240f923-566a-46fa-bad0-3be802d7722e","cell_type":"markdown","source":"<img src=\"https://cms-img.coverfox.com/health-insurance-premiums-3d-illustration-668960257-1200x628.jpg\" height = 400>\n\n[image Source]('https://www.coverfox.com/general-insurance/articles/how-to-pay-tata-aig-general-insurance-premium-online/')","metadata":{}},{"id":"e3343396-e5f6-4c88-b1d1-a01fe450fc30","cell_type":"markdown","source":"# 1. What is this project about ? \nThe objective of this project is to develop a machine learning model capable of predicting the Premium Amount based on customer, policy, and insurance-related features. The dataset used for this analysis was obtained from the Kaggle competition `Regression with an Insurance Dataset`, part of the Playground Series - Season 4, Episode 12. The competition and dataset are available at: [Playground Series - Season 4, Episode 12](https://www.kaggle.com/competitions/playground-series-s4e12/data)\n","metadata":{}},{"id":"3b220ba9-5cbd-4056-9d3f-061a90e303d7","cell_type":"code","source":"import pandas as pd\nimport numpy as np\nimport seaborn as sns\nimport matplotlib.pyplot as plt\n\nfrom sklearn.model_selection import train_test_split\nfrom sklearn.compose import ColumnTransformer\nfrom sklearn.pipeline import Pipeline\nfrom sklearn.impute import SimpleImputer\nfrom sklearn.preprocessing import OneHotEncoder, StandardScaler\n\nfrom sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score\n\npd.set_option('display.max_columns', None)","metadata":{},"outputs":[],"execution_count":null},{"id":"a533425e-e942-462b-948b-c68df12a1cdd","cell_type":"code","source":"sns.set_theme(\n    style=\"whitegrid\",  \n    font=\"sans-serif\",\n    font_scale=1.1\n)\ncustom_colors = [\"#4E79A7\", \"#FFCD73\",\"#F3BFB3\" ,\"#E15759\", \"#B4C9C7\", \"#76B7B2\",\"#c9a86c\"]\nsns.set_palette(custom_colors)\nsns.palplot(custom_colors)\nplt.show()","metadata":{},"outputs":[],"execution_count":null},{"id":"cda87258-5c97-4ab0-95bd-88b29fbcb193","cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python Docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load\n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the read-only \"../input/\" directory\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))\n\n# You can write up to 20GB to the current directory (/kaggle/working/) that gets preserved as output when you create a version using \"Save & Run All\" \n# You can also write temporary files to /kaggle/temp/, but they won't be saved outside of the current session\n\n# Use the kagglehub client library to attach Kaggle resources like competitions, datasets, and models to your session\n# Learn more about kagglehub: https://github.com/Kaggle/kagglehub/blob/main/README.md\n\nimport kagglehub\n# kagglehub.dataset_download('<owner>/<dataset-slug>')","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"a87ee028-b7a9-48dc-bcd8-bf118d7a06db","cell_type":"code","source":"#/kaggle/input/competitions/playground-series-s4e12/sample_submission.csv\n#/kaggle/input/competitions/playground-series-s4e12/train.csv\n#/kaggle/input/competitions/playground-series-s4e12/test.csv\n# load the datasets\ntrain = pd.read_csv('/kaggle/input/competitions/playground-series-s4e12/train.csv')\ntest_df = pd.read_csv('/kaggle/input/competitions/playground-series-s4e12/test.csv')\nsubmission = pd.read_csv('/kaggle/input/competitions/playground-series-s4e12/sample_submission.csv')","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"b4b3dc92-82ac-47aa-bb73-525e620df54d","cell_type":"code","source":"#train = pd.read_csv('train.csv')\n#test_df = pd.read_csv('test.csv')\n#submission = pd.read_csv('sample_submission.csv')\ntrain.head(3)","metadata":{},"outputs":[],"execution_count":null},{"id":"b6b3ba6d-570e-4ae9-837a-1feed4d52cd8","cell_type":"markdown","source":"# 2. Exploratory Data Analysis(EDA)","metadata":{}},{"id":"05177209-955d-4c75-985b-98f7aedf8b3d","cell_type":"code","source":"train.info()","metadata":{},"outputs":[],"execution_count":null},{"id":"9c882530-064d-4c91-bf79-7f7fbd3ff0b3","cell_type":"code","source":"# check for duplicated rows\nprint(\"\\033[4m Duplicated Row counts by Columns \\033[0m\" )\ntrain[train.duplicated()].sum()","metadata":{},"outputs":[],"execution_count":null},{"id":"d67d17b1-b621-4189-b999-cf1355cef82f","cell_type":"code","source":"train.describe(exclude='object')","metadata":{},"outputs":[],"execution_count":null},{"id":"5302d22f-63a6-41fe-9d71-6a18fd2db97e","cell_type":"code","source":"train.describe(include='object')","metadata":{},"outputs":[],"execution_count":null},{"id":"66c071f6-470e-4f81-9f10-259248de4041","cell_type":"code","source":"for col in train.select_dtypes(include='object').columns:\n    print(train[col].value_counts(),'\\n')","metadata":{},"outputs":[],"execution_count":null},{"id":"21336769-7302-4d40-a550-50bc7dbc4a87","cell_type":"markdown","source":"### 2.1. Target Column Distribution","metadata":{}},{"id":"3c75e418-114e-4038-a276-e6a7729aea43","cell_type":"code","source":"import seaborn as sns\n\nplt.figure(figsize=(6,4))\n\nsns.histplot(train['Premium Amount'],\n             bins=50,\n             kde=True,\n             color=custom_colors[0],\n             alpha =0.8,\n             line_kws={\n                         'linewidth': 2 }).lines[0].set_color('navy')\n\n\nplt.title(\"Premium Amount Distribution\")\nplt.show()","metadata":{},"outputs":[],"execution_count":null},{"id":"870fadd7-45b1-4e27-908f-d677d1e0022e","cell_type":"code","source":"plt.figure(figsize=(6,4))\nsns.boxplot(x=train['Premium Amount'])","metadata":{},"outputs":[],"execution_count":null},{"id":"1d9f678c-84c7-4a88-adb6-7c74f3b504ef","cell_type":"code","source":"# checking if the target distribution can be normalized by log transformation, since there is a right skeweness.\nplt.figure(figsize=(6,4))\nsns.histplot(np.log1p(train['Premium Amount']))\nplt.title(\"Premium Amount log Distribution\")\nplt.show()\nprint('The distribution is not close to normal after the log transform.\\n So the log transform did not really \"normalize\" the target')","metadata":{},"outputs":[],"execution_count":null},{"id":"af53ccdf-68b5-499c-82f3-16beb5a252b3","cell_type":"markdown","source":"### 2.2. Null Value Counts and Percentage Visualization","metadata":{}},{"id":"1d5c3285-9392-4003-b595-325a37d5c123","cell_type":"code","source":"missing_data_count = (train.isnull().sum()).sort_values(ascending=False)\nmissing_percentage= (train.isnull().mean().sort_values(ascending=False)* 100 )\nNull_Value_df = pd.DataFrame( {'Null_counts':missing_data_count,\n                                   'missing_percentage': missing_percentage })\nNull_Value_df","metadata":{},"outputs":[],"execution_count":null},{"id":"9a61375c-2f2d-4968-87e9-6f0e11ec9219","cell_type":"code","source":"\"\"\"<5% → mode/median often acceptable\n   5–20% → distribution-preserving methods preferred\n   >20% → investigate why values are missing before imputing \"\"\"\n\nmissing_percent = (train.isnull().mean().sort_values(ascending=False)* 100 )\n\nplt.figure(figsize=(4, 6))\nsns.heatmap( missing_percent.to_frame(), annot=True, fmt=\".1f\", cmap=\"Blues\")\nplt.title(\"Percentage of Missing Values\")\nplt.show()","metadata":{},"outputs":[],"execution_count":null},{"id":"843e605b-b312-4338-ae94-5eb3b2cc6217","cell_type":"markdown","source":"### 2.3. Numeric Column Correlations","metadata":{}},{"id":"83e2ad7d-c3ce-4a7e-911d-2dbe4d887eaa","cell_type":"code","source":"# correlations between the numeric columns except the 'id' column\ncorr = train.loc[:, train.columns != 'id'].corr(numeric_only=True)\ncorr","metadata":{},"outputs":[],"execution_count":null},{"id":"d47a9d32-e4df-4b97-8af1-a890e2af6fe4","cell_type":"code","source":"# target column correlation with other numeric columns as absoluate value\ncorr['Premium Amount'].abs().sort_values(ascending = False)","metadata":{},"outputs":[],"execution_count":null},{"id":"6bc4f6d9-02a5-4b00-96e8-88561e043893","cell_type":"code","source":"# lets see the correlations between the numerical columns\nprint()\nprint(\"\\033[ mCorrelation between the numerical columns and target column Premium_Amount  \\033[0m\")\nprint()\n# Setting up the matplotlib figure size \nplt.figure(figsize=(6,6))\n\n# Creating the heatmap\nsns.heatmap(corr, annot=True, fmt=\".2f\", cmap='Blues', square=True, cbar_kws={\"shrink\": .8})\n\n# Adding the title and labels\nplt.title('Correlation Heatmap')\nplt.show()","metadata":{},"outputs":[],"execution_count":null},{"id":"d30f775d-1a30-4daf-a1b2-bea81606bea4","cell_type":"code","source":"plt.figure(figsize=(7,6))\nplt.hexbin(\n    train['Age'],\n    train['Premium Amount'],\n    gridsize=50,\n    cmap='viridis'\n)\nplt.colorbar(label='Count')\nplt.xlabel('Age')\nplt.ylabel('Premium Amount')\nplt.title('Age Premium Amaount Correlation')\nplt.show()","metadata":{},"outputs":[],"execution_count":null},{"id":"e4011913-6199-4995-917a-15c2523d8bda","cell_type":"markdown","source":"# 3. Preprocesses and And Models","metadata":{}},{"id":"7f640af0-d10e-414c-bb9b-10fdc7b6a585","cell_type":"markdown","source":"## 3.1. XGBOOST and Pipelines","metadata":{}},{"id":"d6441b6c-7294-4f0b-8032-2832d90b2a7f","cell_type":"markdown","source":"### 3.1.1. Feature Extraction from Policy Start Date ","metadata":{}},{"id":"f965f4b5-3322-4106-9fab-5624fac3c678","cell_type":"code","source":"# data conversion from category to date\ntrain['Policy Start Date'] = pd.to_datetime(train['Policy Start Date'])\nreference_day = pd.Timestamp('2026-06-06')\n# create another column to show duration \ntrain['Days_Since'] = (reference_day - train['Policy Start Date']).dt.days\ntrain['Days_Since']\n","metadata":{},"outputs":[],"execution_count":null},{"id":"ecdf14ce-818b-4c1f-bcb3-4380b84e92f7","cell_type":"markdown","source":"### 3.1.2 Data Imputation","metadata":{}},{"id":"c4b52660-e99e-4543-9fb0-e9d4be905203","cell_type":"code","source":"# create numeric and categorical lists to use simple imputer\nnumeric_cols = train.select_dtypes(exclude=['object']).columns.drop(['id','Policy Start Date','Premium Amount'])\nprint(f'numeric columns : ',numeric_cols ,'\\n' )\ncategorical_cols = train.select_dtypes(include=['object']).columns\nprint(f'categorical columns : ',categorical_cols )","metadata":{},"outputs":[],"execution_count":null},{"id":"1a727e8f-b601-49a7-83f3-376d6db8c9b2","cell_type":"code","source":"num_pipeline = Pipeline([\n    ('imputer', SimpleImputer(strategy='median', add_indicator =True)),\n    (\"scaler\", StandardScaler())\n     ])\ncat_pipeline = Pipeline([\n    ('imputer', SimpleImputer(strategy='most_frequent',add_indicator =True)),\n    ('encoder', OneHotEncoder(handle_unknown='ignore'))\n])","metadata":{},"outputs":[],"execution_count":null},{"id":"88edd875-abad-4191-bbcb-a95631a86bd6","cell_type":"code","source":"preprocessor = ColumnTransformer([\n                                    ('num', num_pipeline, numeric_cols),\n                                    ('cat', cat_pipeline, categorical_cols)\n                                ])","metadata":{},"outputs":[],"execution_count":null},{"id":"a013d386-f3eb-4979-9e3e-8d7f6d87a0c3","cell_type":"code","source":"import xgboost as xgb\n\nmodel_xgb = Pipeline([\n                        ('preprocess', preprocessor),\n                        ('regressor', xgb.XGBRegressor( objective='reg:squarederror',\n                                                        n_estimators=100,  \n                                                        random_state=42))\n                    ])\n                    ","metadata":{},"outputs":[],"execution_count":null},{"id":"67d68edc-5c8a-4117-8d33-e26813f2d66a","cell_type":"markdown","source":"### 3.1.3 Split the Data Before Imputation","metadata":{}},{"id":"b3e384d4-83db-44c5-9940-5f2caf48d9a2","cell_type":"code","source":"from sklearn.model_selection import train_test_split\n\nX =train.drop(['Premium Amount','id','Policy Start Date'], axis =1)\ny = train['Premium Amount']\n\nX_train, X_test,y_train,y_test = train_test_split(X,y,test_size= 0.2,random_state=42)","metadata":{},"outputs":[],"execution_count":null},{"id":"59d04075-573a-4789-bfa9-c833007966eb","cell_type":"markdown","source":"### 3.1.4. Train the Model","metadata":{}},{"id":"f69d2a42-e6cd-46d0-9719-29b050681077","cell_type":"code","source":"model_xgb.fit(X_train,y_train)","metadata":{},"outputs":[],"execution_count":null},{"id":"ed35c2f7-853b-4357-8ac9-683bab6eed85","cell_type":"markdown","source":"### 3.1.5. Prediction of Target Feature and Model Evaluation","metadata":{}},{"id":"50ab1ce1-6b44-4c89-a010-ef692e347a6e","cell_type":"code","source":"y_pred_xgb = model_xgb.predict(X_test)","metadata":{},"outputs":[],"execution_count":null},{"id":"37b518ea-9469-46c7-985f-baf5d0649706","cell_type":"code","source":"mae = mean_absolute_error(y_test, y_pred_xgb)\nrmse = mean_squared_error(y_test, y_pred_xgb) ** 0.5\nr2 = r2_score(y_test, y_pred_xgb)\n\nprint(f\"MAE : {mae:.4f}\")\nprint(f\"\\033[42m RMSE: {rmse:.4f} \\033[0m\")\nprint(f\"R²  : {r2:.4f}\")","metadata":{},"outputs":[],"execution_count":null},{"id":"542a409d-89c7-4aae-b3b5-e1ca5ddab2ab","cell_type":"code","source":"# calculate the feature importance with permutation_importance because model is in the pipeline\nfrom sklearn.inspection import permutation_importance\n\nresult = permutation_importance(\n    model_xgb,\n    X_test,\n    y_test,\n    n_repeats=5,\n    random_state=42\n)","metadata":{},"outputs":[],"execution_count":null},{"id":"e1d80a55-940a-44b6-8b38-166e3d76abda","cell_type":"code","source":"# visualize the feature importances\nimportance = pd.DataFrame({\n    \"feature\": X_test.columns,\n    \"importance\": result.importances_mean\n}).sort_values(by=\"importance\", ascending=False)\n\nprint(importance.head(20))","metadata":{},"outputs":[],"execution_count":null},{"id":"16343a62-0b8e-41f6-a708-be2ced8195e4","cell_type":"markdown","source":"## 3.2. Feature Selection and New XGBOOST Model","metadata":{}},{"id":"280043bf-d9b5-4cd3-bdcf-fd6bce43faaf","cell_type":"markdown","source":"### 3.2.1 Feature Selection","metadata":{}},{"id":"4080151b-3621-4264-aced-3b94f82b1834","cell_type":"code","source":"# create numeric and categorical lists to use simple imputer\nnumeric_cols_1 = train[['Annual Income','Credit Score','Previous Claims','Health Score','Days_Since']].columns\nprint(f'numeric columns : ',numeric_cols_1 ,'\\n' )\ncategorical_cols_1 = train[['Customer Feedback']].columns\nprint(f'categorical columns : ',categorical_cols_1 )","metadata":{},"outputs":[],"execution_count":null},{"id":"d73533e4-c3b9-463b-b77b-a0d73810f942","cell_type":"markdown","source":"### 3.2.2. New Pipeline after Feature Selection","metadata":{}},{"id":"c99f9b34-8eff-4a26-aaf7-4366d98c1d64","cell_type":"code","source":"# prepare new pipeline because we have less columns from the original\nnum_pipeline_1 = Pipeline([\n    ('imputer', SimpleImputer(strategy='median', add_indicator =True)),\n    (\"scaler\", StandardScaler())\n     ])\ncat_pipeline_1 = Pipeline([\n    ('imputer', SimpleImputer(strategy='most_frequent',add_indicator =True)),\n    ('encoder', OneHotEncoder(handle_unknown='ignore'))\n])","metadata":{},"outputs":[],"execution_count":null},{"id":"7696ee11-ad1c-4fad-b099-cd29e4348882","cell_type":"code","source":"# collect pipelines under the column transformers\npreprocessor = ColumnTransformer([\n                                    ('num', num_pipeline_1, numeric_cols_1),\n                                    ('cat', cat_pipeline_1, categorical_cols_1)\n                                ])","metadata":{},"outputs":[],"execution_count":null},{"id":"846fa00d-e87d-4fb4-ab06-6856857413f6","cell_type":"markdown","source":"### 3.2.3. Collecting Pipelines and Regressor XGBoost into Pipeline","metadata":{}},{"id":"53eeafe6-e0e2-47ce-8f97-011cb2021d5d","cell_type":"code","source":"# combine the regressor and the preprocessor under a new pipeline\nimport xgboost as xgb\n\nmodel_xgb_1 = Pipeline([\n                        ('preprocess', preprocessor),\n                        ('regressor', xgb.XGBRegressor( objective='reg:squarederror',\n                                                        n_estimators=100,  \n                                                        random_state=42))\n                    ])","metadata":{},"outputs":[],"execution_count":null},{"id":"eedbda13-4117-4252-9f7a-49995cc02d04","cell_type":"markdown","source":"### 3.2.4. Splitting data and Training the Model","metadata":{}},{"id":"e6c16282-648e-403b-b065-c4a9f1504a88","cell_type":"code","source":"# split the new dataset\nX_1 = train[['Annual Income','Credit Score','Previous Claims','Health Score','Customer Feedback','Days_Since']]\ny = train['Premium Amount']\n\nX_1_train, X_1_test,y_train,y_test = train_test_split(X_1,y,test_size= 0.2,random_state=42)","metadata":{},"outputs":[],"execution_count":null},{"id":"1d5ab8c4-197c-428d-9dd2-c15e1b668f1a","cell_type":"code","source":"# train the model \nmodel_xgb_1.fit(X_1_train,y_train)","metadata":{},"outputs":[],"execution_count":null},{"id":"381fc214-4d02-4305-9a2e-8690475b2802","cell_type":"markdown","source":"### 3.2.5. Predicting and Evaluating the Model","metadata":{}},{"id":"34a5c678-28d9-40c0-acb2-cd2eeb9d2de8","cell_type":"code","source":"# predict the target y column values for the test data\ny_pred_xgb_1 = model_xgb_1.predict(X_1_test)","metadata":{},"outputs":[],"execution_count":null},{"id":"4d96dba4-d8ae-4540-aa41-2e678f2e49f8","cell_type":"code","source":"# evaluate the model\nmae = mean_absolute_error(y_test, y_pred_xgb_1)\nrmse = mean_squared_error(y_test, y_pred_xgb_1) ** 0.5\nr2 = r2_score(y_test, y_pred_xgb)\n\nprint(f\"MAE : {mae:.4f}\")\nprint(f\"\\033[42m RMSE: {rmse:.4f} \\033[0m\")\nprint(f\"R²  : {r2:.4f}\")","metadata":{},"outputs":[],"execution_count":null},{"id":"84d3151b-4d47-4dfe-bb38-3c110eacb870","cell_type":"markdown","source":"## 3.3  CatBoost with Miceforest Imputation","metadata":{}},{"id":"ba13263c-3791-48c9-a42f-d7ae980eb821","cell_type":"code","source":"# create copy of the original dataframe to work \ntrain_ctb =train.copy()","metadata":{},"outputs":[],"execution_count":null},{"id":"d686c812-d6af-425a-9461-56a9a94abb43","cell_type":"markdown","source":"### 3.3.1 Create Missing Indicator\nSimpleImputer has add_indicator parameter to signal the model the value was missing but filled with imputation. Miceforest does not have such a parameter and creating one may help model to learn the pattern.","metadata":{}},{"id":"144e7842-fd28-49cc-a1c8-d3f2c53522bf","cell_type":"code","source":"\"\"\" sometimes too many missing records also indicate a pattern, \nand models can use this pattern to create better prediction.\"\"\"\n# name the columns with more than 1.5 percent missing value\ncol_with_many_nan = Null_Value_df[Null_Value_df['missing_percentage'] > 1.5].index\ncol_with_many_nan\n\n# create missing indicator columns to catch the pattern\nfor col in col_with_many_nan:\n    train_ctb[f'{col}_missing'] = train_ctb[col].isna().astype(int)","metadata":{},"outputs":[],"execution_count":null},{"id":"158f62fa-e3d5-4d1b-b700-d61f2217ead7","cell_type":"markdown","source":"### 3.3.2 Feature Extraction and Type Casting","metadata":{}},{"id":"e55e1ef8-ec2f-46a9-bac2-0f66d72c6615","cell_type":"code","source":"# data conversion from object to date\ntrain_ctb['Policy Start Date'] = pd.to_datetime(train_ctb['Policy Start Date'])\nreference_day = pd.Timestamp('2026-06-06')\n# create another column to show duration \ntrain_ctb['Days_Since'] = (reference_day - train_ctb['Policy Start Date']).dt.days\n\n# create year,month columns from Policy_Year\ntrain_ctb['Policy_Year'] = train_ctb['Policy Start Date'].dt.year\n\ntrain_ctb['Policy_Month'] = train_ctb['Policy Start Date'].dt.month\n\ntrain_ctb.drop('Policy Start Date', axis=1, inplace=True) # drop the date column since we got parts of it as new column\n","metadata":{},"outputs":[],"execution_count":null},{"id":"87907713-6bdb-4e07-ba8e-e7cc5abab080","cell_type":"code","source":"# unfortunatelly MICE does not like object datatype and want it to be categorical or numeric data type.\nfor col in train_ctb.select_dtypes(include='object').columns:\n    train_ctb[col] = train_ctb[col].str.strip().str.lower().astype('category')","metadata":{},"outputs":[],"execution_count":null},{"id":"88fcb971-7043-4a5f-aea2-195f6d8f1889","cell_type":"code","source":"# create numeric and categorical lists \nnumeric_cols = train_ctb.select_dtypes(exclude=['object']).columns.drop(['id','Premium Amount'])\nprint(f'numeric columns : ',numeric_cols ,'\\n' )\ncategorical_cols = train_ctb.select_dtypes(include=['category']).columns\nprint(f'categorical columns : ',categorical_cols )","metadata":{},"outputs":[],"execution_count":null},{"id":"b275dee4-830c-4275-beab-7c6f2cf46001","cell_type":"markdown","source":"### 3.3.3. Imputation with Miceforest","metadata":{}},{"id":"7c29bae7-64ce-4810-8364-575104e2925f","cell_type":"code","source":"!pip install miceforest","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-06-15T05:55:09.610083Z","iopub.execute_input":"2026-06-15T05:55:09.610467Z","iopub.status.idle":"2026-06-15T05:55:15.520346Z","shell.execute_reply.started":"2026-06-15T05:55:09.610433Z","shell.execute_reply":"2026-06-15T05:55:15.519311Z"}},"outputs":[],"execution_count":null},{"id":"cf98e43e-eb1b-40da-8278-70bcd0402a69","cell_type":"code","source":"import miceforest as mf\nkernel = mf.ImputationKernel(data=train_ctb, save_all_iterations_data=False, random_state=42)\n    \nkernel.mice(2)\ncompleted_train = kernel.complete_data()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-06-15T05:55:15.522067Z","iopub.execute_input":"2026-06-15T05:55:15.522333Z","iopub.status.idle":"2026-06-15T05:55:20.380595Z","shell.execute_reply.started":"2026-06-15T05:55:15.522305Z","shell.execute_reply":"2026-06-15T05:55:20.379542Z"}},"outputs":[],"execution_count":null},{"id":"74b81e23-d8aa-4c65-b5fb-86be556a9d42","cell_type":"markdown","source":"### 3.3.4. Split the Data","metadata":{}},{"id":"180a0d41-dca2-4078-8aed-bfbfbb7a5499","cell_type":"code","source":"from sklearn.model_selection import train_test_split\nX =completed_train.drop(['Premium Amount','id'], axis =1)\ny =train_ctb['Premium Amount']\n\nX_train, X_test,y_train,y_test = train_test_split(X,y ,test_size= 0.2,random_state=42)","metadata":{},"outputs":[],"execution_count":null},{"id":"176fc10e-a65b-4eb6-b67b-44d1189f203b","cell_type":"markdown","source":"### 3.3.5. Create and Train the Model","metadata":{}},{"id":"f808861d-e932-4bf5-ba64-116662cd02f2","cell_type":"code","source":"from catboost import CatBoostRegressor\ncat_features = list(categorical_cols)\nmodel_ctb = CatBoostRegressor(\n    iterations=500,\n    learning_rate=0.01,\n    depth=5,\n    loss_function='RMSE',\n    random_state=42,\n    verbose=100\n)\n\nmodel_ctb.fit( X_train ,y_train ,cat_features = cat_features)","metadata":{},"outputs":[],"execution_count":null},{"id":"a5a12572-f57a-477e-bef0-2c8e2a975f8e","cell_type":"markdown","source":"### 3.3.4. Predict and Evaluate the Model","metadata":{}},{"id":"0950ee08-326c-4570-9fef-f1ac615b0ef6","cell_type":"code","source":"from sklearn.metrics import mean_squared_error, r2_score\ny_predict_ctb = model_ctb.predict(X_test)\n# evaluating model performance with mse,rmse, and r2\nmse_ctb = mean_squared_error(y_test, y_predict_ctb)\nprint(\"Mean Squared Error:\", mse_ctb,\"\\n\")\n\nrmse_ctb = np.sqrt(mse_ctb)\nprint(f\"\\033[42m Root Mean Squared Error: {rmse_ctb :.4f} \\033[0m\\n\")\n\nr2_ctb= r2_score(y_test, y_predict_ctb)\nprint(\"R-squared:\", r2_ctb)","metadata":{},"outputs":[],"execution_count":null},{"id":"b3fdf614-4610-49d9-947f-f617b0a9fd59","cell_type":"code","source":"from sklearn.inspection import permutation_importance\n\nresult = permutation_importance(model_ctb, X_test, y_test, n_repeats=5, random_state=42 )\n  ","metadata":{},"outputs":[],"execution_count":null},{"id":"3bb014c6-fd18-4d90-9d2d-1268aea8e138","cell_type":"code","source":"# visualize the feature importances\nimportance = pd.DataFrame({\n    \"feature\": X_test.columns,\n    \"importance\": result.importances_mean\n}).sort_values(by=\"importance\", ascending=False)\n\nprint(importance.head(20))","metadata":{},"outputs":[],"execution_count":null},{"id":"52f33167-a434-4bac-a978-efa1c50f757f","cell_type":"markdown","source":"## 4. Running the Best Model on Test Data","metadata":{}},{"id":"dbe54c0d-03c0-4a12-a3c4-25df9d027938","cell_type":"code","source":"test_df.info()","metadata":{},"outputs":[],"execution_count":null},{"id":"bb84a0d2-ff50-4adc-86ae-4f7f92fb8d81","cell_type":"code","source":"# check for duplicated rows\nprint(\"\\033[4m Duplicated Row counts by Columns \\033[0m\" )\ntest_df[test_df.duplicated()].sum()","metadata":{},"outputs":[],"execution_count":null},{"id":"125a8f6a-b7fb-4faf-99fc-03ad22aea5e0","cell_type":"code","source":"missing_data_count = (test_df.isnull().sum()).sort_values(ascending=False)\nmissing_percentage= (test_df.isnull().mean().sort_values(ascending=False)* 100 )\nTest_Null_df = pd.DataFrame( {'Null_counts':missing_data_count,\n                                   'missing_percentage': missing_percentage })\nTest_Null_df","metadata":{},"outputs":[],"execution_count":null},{"id":"df071a4c-02f9-4714-8aad-e46b2cd2d783","cell_type":"code","source":"# data conversion from category to date\ntest_df['Policy Start Date'] = pd.to_datetime(test_df['Policy Start Date'])\nreference_day = pd.Timestamp('2026-06-06')\n# create another column to show duration \ntest_df['Days_Since'] = (reference_day - test_df['Policy Start Date']).dt.days","metadata":{},"outputs":[],"execution_count":null},{"id":"d623392e-222d-4bc3-981a-ecf7240c5869","cell_type":"code","source":"# drop the columns we wont use in prediction\ntest = test_df.drop(['id','Policy Start Date'], axis =1)","metadata":{},"outputs":[],"execution_count":null},{"id":"8adf6d80-74d9-47e0-b4ce-298815f7479e","cell_type":"code","source":"test_df['Premium Amount'] = model_xgb.predict(test)","metadata":{},"outputs":[],"execution_count":null},{"id":"27914741-bc4d-4ba6-81a5-c51ed74fae59","cell_type":"code","source":"submission['Premium Amount'] = test_df['Premium Amount']","metadata":{},"outputs":[],"execution_count":null},{"id":"260139e2-4b33-4486-ad00-f1ea145791a5","cell_type":"code","source":"submission.head(3)","metadata":{},"outputs":[],"execution_count":null},{"id":"9eb5c383-d317-4025-abb3-ccb4e0076ff9","cell_type":"code","source":"#submission.to_csv(\"submission.csv\",index=False)","metadata":{},"outputs":[],"execution_count":null},{"id":"3e6c97be-1284-4e1f-8745-f8b2b2cf8dc2","cell_type":"code","source":"import seaborn as sns\n\nplt.figure(figsize=(6,4))\n\nsns.histplot(train['Premium Amount'],\n             bins=50,\n             kde=True,\n             color=custom_colors[0],\n             alpha =0.8,\n             line_kws={\n                         'linewidth': 2 }).lines[0].set_color('navy')\n\n\nplt.title(\"Train Data-set Premium Amount Distribution\")\nplt.show()\n\nprint()\n\nplt.figure(figsize=(6,4))\nsns.histplot(test_df['Premium Amount'],\n             bins=50,\n             kde=True,\n             color=custom_colors[5],\n             alpha =0.8,\n             line_kws={\n                         'linewidth': 2 }).lines[0].set_color('navy')\n\n\nplt.title(\"Test Data-set Premium Amount Distribution\")\nplt.show()\n","metadata":{},"outputs":[],"execution_count":null},{"id":"2d7daf63-9da1-4315-98dc-993cabd0ebce","cell_type":"markdown","source":"## 5. Conclusion\nIn this project, I used two machine learning algorithms to predict the target variable. The dataset contained a significant amount of missing data, particularly in the Previous Claims and Occupation features, where approximately 30% of the values were missing.To address this issue, I applied two different imputation techniques. For the first model, XGBRegressor, I used SimpleImputer from scikit-learn. For the second model, CatBoostRegressor, I used MICE (Multiple Imputation by Chained Equations) implemented through the miceforest library.\n\nInterestingly, XGBRegressor achieved better performance than CatBoostRegressor, despite CatBoost's ability to handle categorical features natively. As a result, I selected the XGBRegressor model to generate predictions for the test dataset.\n\nTo streamline preprocessing and model training, I implemented a Pipeline for the XGBRegressor workflow. A pipeline was not used for CatBoostRegressor because the algorithm is specifically designed to work directly with categorical features.\n\nAdditionally, I incorporated missing value indicators into both models. This decision was based on recommendations from an article I encountered during the development of the project, which suggested that missingness itself can contain valuable predictive information.\n","metadata":{}},{"id":"76bd19c7-0586-4339-a855-1c72632c209f","cell_type":"markdown","source":"## Sources:\n**1. Data Imputation :**  https://medium.com/@tarangds/a-comprehensive-guide-to-data-imputation-techniques-strategies-and-best-practices-152a10fee543 <br>\n**2. Missing indicator :** https://medium.com/@mohsinsahb53/missing-indicator-7d179cd37446   <br>\n**3. Outlier Detection :** https://www.geeksforgeeks.org/data-analysis/detect-and-remove-the-outliers-using-python/  <br>\n**4. CatBoost :**  https://www.tutorialspoint.com/catboost/catboost-data-preprocessing.htm","metadata":{}},{"id":"ef4a8ef2-0a8a-4c07-a435-2154be57816a","cell_type":"code","source":"","metadata":{},"outputs":[],"execution_count":null}]}