{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# 🌟2 Class Filter🌟\nOriginal version: https://www.kaggle.com/awsaf49/vinbigdata-2-class-filter\n\nPreviously I have trained `YOLOv5` using `14` class data. As it creates `FP` we can tackle that just simply using a `2 class filter`. Here I'll be using 2 class model (`AUC`:`0.98`) prediction to filter out the `FP` predictions. I used `EfficientNetB6` to generate these predictions.\nIt should increase the score as `FP` would be reduced significantly\n* [14 class train](https://www.kaggle.com/awsaf49/vinbigdata-cxr-ad-yolov5-14-class-train)\n* [14 class infer](https://www.kaggle.com/awsaf49/vinbigdata-cxr-ad-yolov5-14-class-infer)","metadata":{}},{"cell_type":"markdown","source":"# Version\n\n* `v12`: vfnet_r101_fold4_v3_epoch18\n* `v11`: vfnet_r101_fold3_v1_epoch25\n* `v10`: vfnet_r101_fold2_v4_epoch18\n* `v9`: vfnet_r101_fold1_v4_epoch18\n* `v8`: vfnet_r101_fold0_v3_epoch4\n* `v7`: vfnet_r101_8020_v1_epoch18\n* `v6`: yolov5x_fold4_finetune_768_tta conf_0.01\n* `v5`: yolov5x_fold3_finetune_768_tta conf_0.01\n* `v4`: yolov5x_fold2_finetune_768_tta conf_0.01\n* `v3`: yolov5x_fold1_finetune_768_tta conf_0.01\n* `v2`: yolov5x_fold0_finetune_768_tta conf_0.01","metadata":{}},{"cell_type":"markdown","source":"# Loading Package","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport numpy as np\nfrom glob import glob\nimport shutil","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Threshold For `2 Class Filter`\n**NB**: The threshold was chosen arbitarily","metadata":{}},{"cell_type":"code","source":"thr = 0.08","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Loading csv","metadata":{}},{"cell_type":"code","source":"# yolov5x_fold4_finetune768\npred_14cls = pd.read_csv('../input/vinbigdata-final-models-infer/vfnet_r101_fold4_v3_epoch18_submission.csv')\npred_2cls = pd.read_csv('../input/vinbigdata-2class-prediction/2-cls test pred.csv')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pred_14cls.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pred_2cls.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"pred = pd.merge(pred_14cls, pred_2cls, on = 'image_id', how = 'left')\npred.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Before 2 Class Filter Number of `No Finding`","metadata":{}},{"cell_type":"code","source":"pred['PredictionString'].value_counts().iloc[[0]]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# 2 Class Filter","metadata":{}},{"cell_type":"code","source":"def filter_2cls(row, thr=thr):\n    if row['target']<thr:\n        row['PredictionString'] = '14 1 0 0 1 1'\n    return row","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sub = pred.apply(filter_2cls, axis=1)\nsub.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# After 2 Class Filter Number of `No Finding`","metadata":{}},{"cell_type":"code","source":"sub['PredictionString'].value_counts().iloc[[0]]","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"As we can see from above that applying `2 class filter` Number of `'No Finding'`increases significanly. **[614->2010]**","metadata":{}},{"cell_type":"code","source":"sub[['image_id', 'PredictionString']].to_csv('vfnet_r101_fold4_v3_epoch18_2cls_filter_submission.csv',index = False)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Result\nAs we can see applying the `2 class filter` improves the result significantly, from `0.154` to `0.201`. But bear in mind that choosing the `thershold` could be a bit `tricky`.","metadata":{}}]}