{"metadata":{"kaggle":{"accelerator":"none","dataSources":[{"sourceId":70367,"databundleVersionId":9188054,"sourceType":"competition"},{"sourceId":191155259,"sourceType":"kernelVersion"}],"dockerImageVersionId":30746,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":false},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.10.13"},"papermill":{"default_parameters":{},"duration":1087.424982,"end_time":"2024-08-03T13:00:18.164826","environment_variables":{},"exception":null,"input_path":"__notebook__.ipynb","output_path":"__notebook__.ipynb","parameters":{},"start_time":"2024-08-03T12:42:10.739844","version":"2.5.0"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Ariel Data Challenge 2024: Introductory model: inference\n\nIn this notebook, we compute test predictions using the model saved in [ADC24 Intro training](https://www.kaggle.com/code/ambrosm/adc24-intro-training).\n\n\n\nThis image has been taken from [last year's competition](https://www.ariel-datachallenge.space/ML/documentation/about). It shows how a planet transits in front of its star and how this transit maps to the lightcurve (a dip in the brightness of the star). This dip is directly proportional to the ratio of the areas of the planet and star. It's this ratio (the \"transit depth\") that we are modeling in the present notebook.\n\nThe present notebook is simple:\n- It reads the pre- and postprocessing code, which is the same as the code used for training.\n- It reads the test data.\n- It reads the saved model.\n- It executes the prediction pipeline and saves the submission file.\n\nThe real work was done in the [training notebook](https://www.kaggle.com/code/ambrosm/adc24-intro-training)!","metadata":{"_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","papermill":{"duration":0.009657,"end_time":"2024-08-03T12:42:14.074227","exception":false,"start_time":"2024-08-03T12:42:14.06457","status":"completed"},"tags":[]}},{"cell_type":"code","source":"import pandas as pd\nimport polars as pl\nimport matplotlib.pyplot as plt\nimport numpy as np\nimport scipy.stats\nfrom tqdm import tqdm\nimport pickle\n\nfrom sklearn.linear_model import Ridge\n","metadata":{"_kg_hide-input":true,"papermill":{"duration":3.023083,"end_time":"2024-08-03T12:42:17.107326","exception":false,"start_time":"2024-08-03T12:42:14.084243","status":"completed"},"tags":[],"execution":{"iopub.status.busy":"2024-08-14T20:01:16.687169Z","iopub.execute_input":"2024-08-14T20:01:16.688346Z","iopub.status.idle":"2024-08-14T20:01:17.757356Z","shell.execute_reply.started":"2024-08-14T20:01:16.688255Z","shell.execute_reply":"2024-08-14T20:01:17.756343Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"directory = \"/kaggle/input/adc24-intro-training/\"\n\nexec(open(directory + 'f_read_and_preprocess.py', 'r').read())\nexec(open(directory + 'a_read_and_preprocess.py', 'r').read())\nexec(open(directory + 'feature_engineering.py', 'r').read())\nexec(open(directory + 'postprocessing.py', 'r').read())\n","metadata":{"papermill":{"duration":1067.258124,"end_time":"2024-08-03T13:00:08.197814","exception":false,"start_time":"2024-08-03T12:42:20.93969","status":"completed"},"tags":[],"_kg_hide-input":false,"execution":{"iopub.status.busy":"2024-08-14T20:01:17.75903Z","iopub.execute_input":"2024-08-14T20:01:17.759467Z","iopub.status.idle":"2024-08-14T20:01:17.769989Z","shell.execute_reply.started":"2024-08-14T20:01:17.75944Z","shell.execute_reply":"2024-08-14T20:01:17.768997Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"People have been asking how to choose a good value for sigma_pred. As explained in [Understanding the competition metric](https://www.kaggle.com/competitions/ariel-data-challenge-2024/discussion/528114), with sigma_pred we indicate what root mean squared error (rmse) we expect for our test predictions.\n\nThe training data cover planets of only two stars (stars 0 and 1), but the test data include planets of other stars.\n\nThis leads to the following recipe:\n- For known stars (stars 0 and 1), we expect the test rmse to be equal to our cross-validation rmse, i.e. we predict the out-of-fold rmse of our model (0.000293 as shown in the training notebook).\n- For unknown stars, the prediction error can only be higher. We thus predict a higher value (0.001 in this notebook).","metadata":{}},{"cell_type":"code","source":"# Load the data\nwavelengths = pd.read_csv('/kaggle/input/ariel-data-challenge-2024/wavelengths.csv')\ntest_adc_info = pd.read_csv('/kaggle/input/ariel-data-challenge-2024/test_adc_info.csv',\n index_col='planet_id')\nf_raw_test = f_read_and_preprocess('test', test_adc_info, test_adc_info.index)\na_raw_test = a_read_and_preprocess('test', test_adc_info, test_adc_info.index)\ntest = feature_engineering(f_raw_test, a_raw_test)\n\n# Load the model\nwith open(directory + 'model.pickle', 'rb') as f:\n model = pickle.load(f)\nwith open(directory + 'sigma_pred.pickle', 'rb') as f:\n sigma_pred = pickle.load(f)\n \n# Predict\ntest_pred = model.predict(test)\n\n# Package into submission file\nsub_df = postprocessing(test_pred,\n test_adc_info.index,\n sigma_pred=np.tile(np.where(test_adc_info[['star']] <= 1, sigma_pred, 0.001), (1, 283)))\ndisplay(sub_df)\nsub_df.to_csv('submission.csv')\n#!head submission.csv","metadata":{"execution":{"iopub.status.busy":"2024-08-14T20:01:17.771285Z","iopub.execute_input":"2024-08-14T20:01:17.771669Z","iopub.status.idle":"2024-08-14T20:01:19.961621Z","shell.execute_reply.started":"2024-08-14T20:01:17.771633Z","shell.execute_reply":"2024-08-14T20:01:19.96064Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}