{"metadata":{"kernelspec":{"name":"ir","display_name":"R","language":"R"},"language_info":{"name":"R","codemirror_mode":"r","pygments_lexer":"r","mimetype":"text/x-r-source","file_extension":".r","version":"4.4.0"},"kaggle":{"accelerator":"none","dataSources":[{"sourceId":84896,"databundleVersionId":10305135,"sourceType":"competition"}],"dockerImageVersionId":30749,"isInternetEnabled":true,"language":"r","sourceType":"notebook","isGpuEnabled":false}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# **Regression with an Insurance Dataset**\n\n**Kaggle Competition**\n\n**By Alexander Thompson**\n\n**This will be done using the R Programming Language.**\n\n## **Introduction** \nThis project focuses on building a regression model to predict insurance premium amounts using a dataset from the Kaggle competition, *Regression with an Insurance Dataset*. The dataset contains demographic, financial, and behavioral information about individuals, with the goal of predicting the target variable, premium_amount.\n\nThe analysis involves data preprocessing steps such as handling missing values, encoding categorical variables, and aligning features between training and test datasets. An XGBoost model was used for prediction, with hyperparameter tuning and feature engineering to improve performance. This document outlines the end-to-end workflow, including data cleaning, exploratory data analysis (EDA), model training, and evaluation.","metadata":{}},{"cell_type":"markdown","source":"## Loading the packages","metadata":{}},{"cell_type":"code","source":"library(data.table)\nlibrary(janitor)\nlibrary(caret)\nlibrary(xgboost)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:32:36.356761Z","iopub.execute_input":"2024-12-30T00:32:36.396156Z","iopub.status.idle":"2024-12-30T00:32:40.045295Z","shell.execute_reply":"2024-12-30T00:32:40.043197Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Loading the Data","metadata":{}},{"cell_type":"code","source":"train <- fread(\"/kaggle/input/playground-series-s4e12/train.csv\")\ntest <- fread(\"/kaggle/input/playground-series-s4e12/test.csv\")","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:32:40.048492Z","iopub.execute_input":"2024-12-30T00:32:40.050696Z","iopub.status.idle":"2024-12-30T00:32:46.506221Z","shell.execute_reply":"2024-12-30T00:32:46.504194Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Exploratory Data Analysis (EDA)","metadata":{}},{"cell_type":"code","source":"summary(train)\nsummary(test)\nstr(train)\nstr(test)\nView(train)\nView(test)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:32:46.511133Z","iopub.execute_input":"2024-12-30T00:32:46.513415Z","iopub.status.idle":"2024-12-30T00:32:49.520918Z","shell.execute_reply":"2024-12-30T00:32:49.517969Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Cleaning Column Names","metadata":{}},{"cell_type":"code","source":"train <- janitor::clean_names(train)\ntest <- janitor::clean_names(test)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:32:49.525343Z","iopub.execute_input":"2024-12-30T00:32:49.527233Z","iopub.status.idle":"2024-12-30T00:32:49.664107Z","shell.execute_reply":"2024-12-30T00:32:49.662154Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Handling Null Values","metadata":{}},{"cell_type":"code","source":"colSums(is.na(train))\ncolSums(is.na(test))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:32:49.668396Z","iopub.execute_input":"2024-12-30T00:32:49.670684Z","iopub.status.idle":"2024-12-30T00:32:50.308425Z","shell.execute_reply":"2024-12-30T00:32:50.30619Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"train[is.na(age), age := median(train$age, na.rm = TRUE)]\ntrain[is.na(annual_income), annual_income := median(train$annual_income, na.rm = TRUE)]\ntrain[is.na(number_of_dependents), number_of_dependents := median(train$number_of_dependents, na.rm = TRUE)]\ntrain[is.na(health_score), health_score := median(train$health_score, na.rm = TRUE)]\ntrain[is.na(previous_claims), previous_claims := median(train$previous_claims, na.rm = TRUE)]\ntrain[is.na(vehicle_age), vehicle_age := median(train$vehicle_age, na.rm = TRUE)]\ntrain[is.na(credit_score), credit_score := median(train$credit_score, na.rm = TRUE)]\ntrain[is.na(insurance_duration), insurance_duration := median(train$insurance_duration, na.rm = TRUE)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:32:50.311757Z","iopub.execute_input":"2024-12-30T00:32:50.313237Z","iopub.status.idle":"2024-12-30T00:32:50.831777Z","shell.execute_reply":"2024-12-30T00:32:50.829967Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"test[is.na(age), age := median(test$age, na.rm = TRUE)]\ntest[is.na(annual_income), annual_income := median(test$annual_income, na.rm = TRUE)]\ntest[is.na(number_of_dependents), number_of_dependents := median(test$number_of_dependents, na.rm = TRUE)]\ntest[is.na(health_score), health_score := median(test$health_score, na.rm = TRUE)]\ntest[is.na(previous_claims), previous_claims := median(test$previous_claims, na.rm = TRUE)]\ntest[is.na(vehicle_age), vehicle_age := median(test$vehicle_age, na.rm = TRUE)]\ntest[is.na(credit_score), credit_score := median(test$credit_score, na.rm = TRUE)]\ntest[is.na(insurance_duration), insurance_duration := median(test$insurance_duration, na.rm = TRUE)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:32:50.834406Z","iopub.execute_input":"2024-12-30T00:32:50.835835Z","iopub.status.idle":"2024-12-30T00:32:51.220005Z","shell.execute_reply":"2024-12-30T00:32:51.217453Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Checking Column Consistency","metadata":{}},{"cell_type":"code","source":"setdiff(names(train), names(test))\nsetdiff(names(test), names(train))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:32:51.223497Z","iopub.execute_input":"2024-12-30T00:32:51.225926Z","iopub.status.idle":"2024-12-30T00:32:51.248387Z","shell.execute_reply":"2024-12-30T00:32:51.246379Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Ensuring Column Consistency","metadata":{}},{"cell_type":"code","source":"missing_vars <- setdiff(names(train), names(test))\nfor (var in missing_vars) {\n  test[[var]] <- 0\n}","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:32:51.251387Z","iopub.execute_input":"2024-12-30T00:32:51.253322Z","iopub.status.idle":"2024-12-30T00:32:51.27406Z","shell.execute_reply":"2024-12-30T00:32:51.272204Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Creating a DummyVars Object","metadata":{}},{"cell_type":"code","source":"dummies <- dummyVars(\" ~ .\", data = train[, !names(train) %in% \"premium_amount\", with = FALSE])","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:32:51.276707Z","iopub.execute_input":"2024-12-30T00:32:51.278108Z","iopub.status.idle":"2024-12-30T00:32:52.608859Z","shell.execute_reply":"2024-12-30T00:32:52.606961Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Applying Encoding","metadata":{}},{"cell_type":"code","source":"train_encoded <- as.data.frame(predict(dummies, newdata = train))\ntest_encoded <- as.data.frame(predict(dummies, newdata = test))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:32:52.611659Z","iopub.execute_input":"2024-12-30T00:32:52.613181Z","iopub.status.idle":"2024-12-30T00:33:29.131597Z","shell.execute_reply":"2024-12-30T00:33:29.129385Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Spliting Train into Train/Validation","metadata":{}},{"cell_type":"code","source":"set.seed(123)\ntrainIndex <- createDataPartition(train$premium_amount, p = 0.8, list = FALSE)\ntrain_data <- train[trainIndex, ]\nval_data <- train[-trainIndex, ]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:33:29.134397Z","iopub.execute_input":"2024-12-30T00:33:29.135945Z","iopub.status.idle":"2024-12-30T00:33:30.283725Z","shell.execute_reply":"2024-12-30T00:33:30.281821Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Encoding the Validation Data","metadata":{}},{"cell_type":"code","source":"val_encoded <- as.data.frame(predict(dummies, newdata = val_data))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:33:30.288088Z","iopub.execute_input":"2024-12-30T00:33:30.289709Z","iopub.status.idle":"2024-12-30T00:33:34.92766Z","shell.execute_reply":"2024-12-30T00:33:34.925612Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Encoding the Train and Test Datasets","metadata":{}},{"cell_type":"code","source":"train_data_encoded <- as.data.frame(predict(dummies, newdata = train_data))\ntest_encoded <- as.data.frame(predict(dummies, newdata = test))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:33:34.931316Z","iopub.execute_input":"2024-12-30T00:33:34.933351Z","iopub.status.idle":"2024-12-30T00:34:05.978329Z","shell.execute_reply":"2024-12-30T00:34:05.976286Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Aligning Columns in test_encoded with train_data_encoded","metadata":{}},{"cell_type":"code","source":"missing_cols <- setdiff(colnames(train_data_encoded), colnames(test_encoded))\nfor (col in missing_cols) {\n  test_encoded[[col]] <- 0\n}","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:34:05.981591Z","iopub.execute_input":"2024-12-30T00:34:05.983562Z","iopub.status.idle":"2024-12-30T00:34:06.003222Z","shell.execute_reply":"2024-12-30T00:34:06.001356Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"extra_cols <- setdiff(colnames(test_encoded), colnames(train_data_encoded))\ntest_encoded <- test_encoded[, !(colnames(test_encoded) %in% extra_cols)]\ntest_encoded <- test_encoded[, colnames(train_data_encoded)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:34:06.007339Z","iopub.execute_input":"2024-12-30T00:34:06.009326Z","iopub.status.idle":"2024-12-30T00:34:06.027701Z","shell.execute_reply":"2024-12-30T00:34:06.025239Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Converting to Numerical Matrices","metadata":{}},{"cell_type":"code","source":"train_matrix <- xgb.DMatrix(data = as.matrix(train_data_encoded[, -ncol(train_data_encoded)]), label = train_data$premium_amount)\ntest_matrix <- as.matrix(test_encoded)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:34:06.030763Z","iopub.execute_input":"2024-12-30T00:34:06.032254Z","iopub.status.idle":"2024-12-30T00:34:08.046277Z","shell.execute_reply":"2024-12-30T00:34:08.044404Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## The Model","metadata":{}},{"cell_type":"code","source":"params <- list(objective = \"reg:squarederror\", eval_metric = \"rmse\")\nxgb_model <- xgboost(params = params, data = train_matrix, nrounds = 100)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:34:08.050458Z","iopub.execute_input":"2024-12-30T00:34:08.05197Z","iopub.status.idle":"2024-12-30T00:36:57.967073Z","shell.execute_reply":"2024-12-30T00:36:57.963771Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Features in train data used to train the Model","metadata":{}},{"cell_type":"code","source":"trained_features <- colnames(as.matrix(train_data_encoded[, -ncol(train_data_encoded)]))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:36:57.971365Z","iopub.execute_input":"2024-12-30T00:36:57.974025Z","iopub.status.idle":"2024-12-30T00:36:58.390207Z","shell.execute_reply":"2024-12-30T00:36:58.388398Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Features in the test data","metadata":{}},{"cell_type":"code","source":"test_features <- colnames(test_encoded)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:36:58.394274Z","iopub.execute_input":"2024-12-30T00:36:58.395855Z","iopub.status.idle":"2024-12-30T00:36:58.41177Z","shell.execute_reply":"2024-12-30T00:36:58.409606Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Identifying any mismatches","metadata":{}},{"cell_type":"code","source":"setdiff(trained_features, test_features)\nsetdiff(test_features, trained_features)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:36:58.417335Z","iopub.execute_input":"2024-12-30T00:36:58.419129Z","iopub.status.idle":"2024-12-30T00:36:58.441305Z","shell.execute_reply":"2024-12-30T00:36:58.43945Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Adding missing Columns to test_encoded","metadata":{}},{"cell_type":"code","source":"missing_cols <- setdiff(trained_features, test_features)\nfor (col in missing_cols) {\n  test_encoded[[\"property_typeHouse\"]] <- 0  \n}","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:36:58.443982Z","iopub.execute_input":"2024-12-30T00:36:58.445398Z","iopub.status.idle":"2024-12-30T00:36:58.462641Z","shell.execute_reply":"2024-12-30T00:36:58.460814Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Removing extra Columns from test_encoded","metadata":{}},{"cell_type":"code","source":"extra_cols <- setdiff(test_features, trained_features)\ntest_encoded <- test_encoded[, !(colnames(test_encoded) %in% extra_cols)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:36:58.46523Z","iopub.execute_input":"2024-12-30T00:36:58.466606Z","iopub.status.idle":"2024-12-30T00:36:58.47991Z","shell.execute_reply":"2024-12-30T00:36:58.478143Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Reordering the Columns","metadata":{}},{"cell_type":"code","source":"test_encoded <- test_encoded[, trained_features]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:36:58.482538Z","iopub.execute_input":"2024-12-30T00:36:58.483957Z","iopub.status.idle":"2024-12-30T00:36:58.495538Z","shell.execute_reply":"2024-12-30T00:36:58.493723Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Verifying the Alignment","metadata":{}},{"cell_type":"code","source":"all.equal(colnames(as.matrix(train_data_encoded[, -ncol(train_data_encoded)])), colnames(as.matrix(test_encoded)))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:36:58.498913Z","iopub.execute_input":"2024-12-30T00:36:58.500432Z","iopub.status.idle":"2024-12-30T00:36:59.272603Z","shell.execute_reply":"2024-12-30T00:36:59.270343Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Converting to Matrix and Predicting","metadata":{}},{"cell_type":"code","source":"test_matrix <- as.matrix(test_encoded)\ntest_predictions <- predict(xgb_model, test_matrix)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:36:59.276617Z","iopub.execute_input":"2024-12-30T00:36:59.27907Z","iopub.status.idle":"2024-12-30T00:37:01.228097Z","shell.execute_reply":"2024-12-30T00:37:01.225907Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Saving the Submission","metadata":{}},{"cell_type":"code","source":"submission <- data.frame(Id = test$id, Premium_Amount = test_predictions)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:37:01.23136Z","iopub.execute_input":"2024-12-30T00:37:01.233231Z","iopub.status.idle":"2024-12-30T00:37:01.247429Z","shell.execute_reply":"2024-12-30T00:37:01.245556Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Exporting the Submission","metadata":{}},{"cell_type":"code","source":"write.csv(submission, \"/kaggle/working/submission.csv\", row.names = FALSE)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2024-12-30T00:37:01.250033Z","iopub.execute_input":"2024-12-30T00:37:01.251392Z","iopub.status.idle":"2024-12-30T00:37:03.258135Z","shell.execute_reply":"2024-12-30T00:37:03.256332Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"When Submiting the submission file on Kaggle, had to load file on Excel and change the Column name for Premium_Amount to Premium Amount.","metadata":{}}]}