{"cells":[{"metadata":{},"cell_type":"markdown","source":"In this notebook, we will build a simple LGBM model where we will use 6 simple image stats features which are inspired from [this](https://www.kaggle.com/iafoss/panda-16x128x128-tiles) great kernel. The purpose of this kernel is just to provide others a pipeline/structure how to properly submit in this competition. As we see a lot of discussions around submission problem that kagglers are facing in this competition.","execution_count":null},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-output":true},"cell_type":"code","source":"\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n\nimport os\nfor dirname, _, filenames in os.walk('/kaggle/input'):\n    for filename in filenames:\n        print(os.path.join(dirname, filename))","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"train = pd.read_csv(\"/kaggle/input/prostate-cancer-grade-assessment/train.csv\")\ntest = pd.read_csv(\"/kaggle/input/prostate-cancer-grade-assessment/test.csv\")\ntrain.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sz = 128\nN=16","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import os\nimport cv2\nimport skimage.io\nfrom tqdm.notebook import tqdm\nimport zipfile\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport gc\n\nfrom sklearn.metrics import cohen_kappa_score , confusion_matrix\nimport lightgbm as lgb\nfrom sklearn.model_selection import train_test_split\n\nimport warnings\nwarnings.filterwarnings(\"ignore\")\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's create the feature engineering function which would create those 6 image stats features and the indicator features from data provider","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def feature_engineering(data = train , dir_name = \"train_images\"):\n    r_mean = []\n    g_mean=[]\n    b_mean = []\n    r_sd = []\n    g_sd = []\n    b_sd = []\n    \n    for i in data['image_id'].values:\n        img = skimage.io.MultiImage(os.path.join(f\"/kaggle/input/prostate-cancer-grade-assessment/{dir_name}\"+\"/\"+str(i)+\".tiff\"))[2]\n    #print(img.shape)\n        shape = img.shape\n        pad0 = (sz-shape[0]%sz)%sz  #### horizontal padding\n        pad1 = (sz-shape[1]%sz)%sz  #### vartical padding\n        img = np.pad(img,[[pad0//2,pad0-pad0//2],[pad1//2,pad1-pad1//2],[0,0]],constant_values=255)\n        img = img.reshape(img.shape[0]//sz,sz,img.shape[1]//sz,sz,3)\n        img = img.transpose(0,2,1,3,4)\n        img = img.reshape(-1,sz,sz,3)\n        if len(img) < N:\n            img = np.pad(img,[[0,N-len(img)],[0,0],[0,0],[0,0]],constant_values=255)\n \n        idxs = np.argsort(img.reshape(img.shape[0],-1).sum(-1))[:N]\n        img = img[idxs]\n        img = (img/255.0).reshape(-1,3)\n        #print(img.mean(0)[0])\n        r_mean.append(img.mean(0)[0])\n        g_mean.append(img.mean(0)[1])\n        b_mean.append(img.mean(0)[2])\n    \n        r_sd.append(img.std(0)[0])\n        g_sd.append(img.std(0)[1])\n        b_sd.append(img.std(0)[2])\n        \n        del img\n        gc.collect()\n    \n    data['r_mean'] = r_mean\n    data['g_mean'] = g_mean\n    data['b_mean'] = b_mean\n\n    data['r_sd'] = r_sd\n    data['g_sd'] = g_sd\n    data['b_sd'] = b_sd\n    \n    data['data_prov_ind'] = np.where(data['data_provider'] == \"radboud\" , 1 , 0)\n    \n    return data","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train = feature_engineering(data = train , dir_name = \"train_images\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"train.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's quickly check the object sizes of the local files that we have generated in the process as we need to be careful about memory optimization throughout this competition.","execution_count":null},{"metadata":{"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"#https://stackoverflow.com/questions/24455615/python-how-to-display-size-of-all-variables\nfrom __future__ import print_function  \nimport sys\n\nlocal_vars = list(locals().items())\nfor var, obj in local_vars:\n    if not var.startswith('_'):\n        print(var, sys.getsizeof(obj))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Our LGBM features are the seven following ones","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"features = [\"data_prov_ind\" , 'r_mean', 'g_mean', 'b_mean', 'r_sd', 'g_sd', 'b_sd']","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"QWK: Our competition metric ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def quadratic_weighted_kappa(y_hat, y):\n    return cohen_kappa_score(y_hat, y, weights='quadratic')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def QWK(preds, dtrain):\n    labels = dtrain.get_label()\n    preds = np.rint(preds)\n    score = quadratic_weighted_kappa(preds, labels)\n    return (\"QWK\", score, True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"y = train[\"isup_grade\"]\ntrain = train[features]\nX_train, X_test, y_train, y_test = train_test_split(train, y, test_size=0.2, random_state=0)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now Let's quickly build the LGBM model.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"train_dataset = lgb.Dataset(X_train, y_train)\nvalid_dataset = lgb.Dataset(X_test, y_test)\n\nparams = {\n            \"objective\": 'regression',\n            \"metric\": 'rmse',\n            \"seed\": 0,\n            \"learning_rate\": 0.01,\n            \"boosting\": \"gbdt\",\n            }\n        \nmodel = lgb.train(\n            params=params,\n            num_boost_round=10000,\n            early_stopping_rounds=100,\n            train_set=train_dataset,\n            valid_sets=[train_dataset, valid_dataset],\n            verbose_eval=100,\n            feval=QWK)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"preds = model.predict(X_test, num_iteration=model.best_iteration)\npreds = np.rint(preds)\npreds = np.clip(preds, 0 , 5)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"model.best_iteration","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(\"our validation score is\" , quadratic_weighted_kappa(preds, y_test))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's also look at the confusion matrix for the validation set","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"print(confusion_matrix(preds,y_test))","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"This does not look good, but fair enough as we are mostly interested in what is coming next, that is submitting using this model","execution_count":null},{"metadata":{"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"from __future__ import print_function  \nimport sys\n\nlocal_vars = list(locals().items())\nfor var, obj in local_vars:\n    if not var.startswith('_'):\n        print(var, sys.getsizeof(obj))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"del X_train ,X_test ,y_train ,y_test \ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"del train_dataset, valid_dataset , train , filenames\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now, let's create our inference function ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def inference (da = test , dir_path = \"test_images\"):\n    if os.path.exists(f'../input/prostate-cancer-grade-assessment/{dir_path}'):\n        print('run inference')\n        \n        preds = model.predict(da[features], num_iteration=500)\n        preds = np.rint(preds)\n        preds = np.clip(preds, 0 ,5)\n        da['isup_grade'] = preds.astype(int)\n        cols = [\"image_id\" , \"isup_grade\"]\n        da = da[cols]\n        \n    return da","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"In the following section, we will check if the inference function is working on the training set or not, keep this part just to mnake sure that the pipeline is ready enough to handle the unseen test data as well, since, we have already deleted the train data, let's start afresh by reading the train data again.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"train = pd.read_csv(\"/kaggle/input/prostate-cancer-grade-assessment/train.csv\")","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sub = inference(da = feature_engineering(data = train.head(10) , dir_name = \"train_images\") , dir_path = \"train_images\")\nsub['isup_grade'] = sub['isup_grade'].astype(int)\nsub.to_csv('submission.csv', index=False)\nsub.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now, in this final section, we will try to submit in this competition.\nWhen, we are committing our kernel, note that, the code would not be able to access the test data, so your model submission would not be prepared. Probably, because hosts did not want us to receive any information from the test data. However, when we submit, then your kernel would access the test data. Now, that is why we need to prepare this *if-else statement*, so that when we are just committing, sample submission file would be generated in the output and when we are submitting, the intended file would be submitted. Finally, keep the internet off in your inference kernels.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"if os.path.exists(f'../input/prostate-cancer-grade-assessment/test_images'):\n    print(\"still can not access the test file ?\")\n    sub = inference(da = feature_engineering(data = test , dir_name = \"test_images\") , dir_path = \"test_images\")\n    sub['isup_grade'] = sub['isup_grade'].astype(int)\n    sub.to_csv('submission.csv', index=False)\n    \nelse:\n    sub = pd.read_csv(\"/kaggle/input/prostate-cancer-grade-assessment/sample_submission.csv\")\n    sub.to_csv('submission.csv', index=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Final note, I know the model is substandard, but the objective was to keep everyone on the same page regarding how to submit in this competition. \nHappy Kaggling ","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}