{"cells":[{"metadata":{},"cell_type":"markdown","source":"\n\n\n\n
\n\n# VinBigData 2-class classifier complete pipeline\n\nThis competition is object detaction task to find a class and location of thoracic abnormalities from chest x-ray image (radiographs).\n\nHowever, it is mentioned that training 2 class classifier to understand which is the normal image is important to get high score.\n\n本竞赛的目的是通过胸部x线片发现胸部异常的类别和位置。\n但是，要想获得高分，训练2个分类器了解哪一个是正常图像是很重要的。\n - Kernel: [VinBigData 🌟2 Class Filter🌟](https://www.kaggle.com/awsaf49/vinbigdata-2-class-filter)\n - Discussion: [[LB0.155] baseline solution](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/208837)\n\nHere, I will introduce complete **EDA, Training (with 5-fold cross validation) and Prediction pipeline** for training 2-class classifier.\n\n在这里，我将介绍完整的EDA、训练(使用5折交叉验证)和训练2类分类器的预测管道。\n\nYou can learn the usage of following tools to accelerate deep learning tasks in computer vision!\n - [pytorch](https://github.com/pytorch/pytorch): Deep learning framework, it's popular among researchers for its flexible usage. no need to explain detail!\n - [albumentations](https://github.com/albumentations-team/albumentations): Image augmentation library, developed by famous kagglers!\n - [timm](https://github.com/rwightman/pytorch-image-models): pytorch-image-models, it provides a lot of popular SoTA CNN models with pretrained weights.\n - [pytorch ignite](https://github.com/pytorch/ignite): Traning/Evaluation abstraction framework on top of pytorch.\n - [pytorch pfn extras](https://github.com/pfnet/pytorch-pfn-extras): It is used to add more feature-rich functionality on Ignite.\n 在这里，我将介绍完整的EDA，训练（具有5倍交叉验证）和预测管道，用于训练2类分类器。\n\n您可以学习以下工具的用法，以加速计算机视觉中的深度学习任务！\n\npytorch：深度学习框架，由于其灵活的用法而在研究人员中很受欢迎。 无需解释细节！\n\nalbumentations：图像增强库，由著名的kagglers开发！\n\ntimm：pytorch-image-models，它提供了许多流行的具有预训练权重的SoTA CNN模型。\n\npytorch ignite：在pytorch之上的Traning / Evaluation抽象框架。\n\npytorch pfn extras：用于在Ignite上添加更多功能丰富的功能。"},{"metadata":{},"cell_type":"markdown","source":"# Table of Contents\n\n** [Dataset preparation](#dataset)**
\n** [Installation](#installation)**
\n** [EDA: distribution between normal & abnormal class](#eda)**
\n** [Image visualizaion & augmentation with albumentations](#aug)**
\n** [Defining CNN models](#model)**
\n** [Training utils](#trainutil)**
\n** [Training scripts](#trainscript)**
\n** [Prediction on validation & test dataset](#prediction)**
\n** [Next step](#nextstep)**
"},{"metadata":{},"cell_type":"markdown","source":"\n# Dataset preparation\n\nPreprocessing x-ray image format (dicom) into normal png image format is already done by @xhlulu in the below discussion:\n - [Multiple preprocessed datasets: 256/512/1024px, PNG and JPG, modified and original ratio](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/207955).\n\nHere I will just use the dataset [VinBigData Chest X-ray Resized PNG (256x256)](https://www.kaggle.com/xhlulu/vinbigdata-chest-xray-resized-png-256x256) to skip the preprocessing and focus on modeling part. Please upvote the dataset as well!"},{"metadata":{},"cell_type":"markdown","source":"数据准备\n\n@xhlulu在下面的讨论中已经将x射线图像格式（dicom）预处理为普通png图像格式：\n\n多个预处理数据集：256/512 / 1024px，PNG和JPG，修改后的比例和原始比例。\n\n在这里，我将仅使用数据集VinBigData胸部X射线调整大小的PNG（256x256）来跳过预处理并专注于建模部分。"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"import gc\nimport os\nfrom pathlib import Path\nimport random\nimport sys\n\nfrom tqdm.notebook import tqdm\nimport numpy as np\nimport pandas as pd\nimport scipy as sp\n\n\nimport matplotlib.pyplot as plt\nimport seaborn as sns\n\nfrom IPython.core.display import display, HTML\n\n# --- plotly ---\nfrom plotly import tools, subplots\nimport plotly.offline as py\npy.init_notebook_mode(connected=True)\nimport plotly.graph_objs as go\nimport plotly.express as px\nimport plotly.figure_factory as ff\nimport plotly.io as pio\npio.templates.default = \"plotly_dark\"\n\n# --- models ---\nfrom sklearn import preprocessing\nfrom sklearn.model_selection import KFold\nimport lightgbm as lgb\nimport xgboost as xgb\nimport catboost as cb\nimport torch\n\n# --- setup ---\npd.set_option('max_columns', 50)\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n# Installation\n\ndetectron2 is not pre-installed in this kaggle docker, so let's install it. \nWe can follow [installation instruction](https://github.com/facebookresearch/detectron2/blob/master/INSTALL.md), we need to know CUDA and pytorch version to install correct `detectron2`.\n\ndetectron2没有预先安装在这个kaggle docker中，所以让我们来安装它。我们可以按照安装说明，我们需要知道CUDA和pytorch版本来安装正确的detectron2。"},{"metadata":{"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"!pip install detectron2 -f \\\n https://dl.fbaipublicfiles.com/detectron2/wheels/cu102/torch1.7/index.html\n!pip install pytorch-pfn-extras timm","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"This `Flags` class summarizes all the configuratoin available during the training.\n\nAs I will show later, you can change various hyperparameters to experiment improving your models!"},{"metadata":{},"cell_type":"markdown","source":"这个Flags类涵盖了训练期间所有可用的配置。\n正如我将在后面展示的，您可以更改各种超参数来试验改进您的模型！"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"from typing import Any\nimport yaml\n\ndef save_yaml(filepath: str, content: Any, width: int = 120):\n with open(filepath, \"w\") as f:\n yaml.dump(content, f, width=width)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"from dataclasses import dataclass, field\nfrom typing import Dict, Any, Tuple, Union, List\n\n\n@dataclass\nclass Flags:\n # General\n debug: bool = True\n outdir: str = \"results/det\"\n device: str = \"cuda:0\"\n\n # Data config\n imgdir_name: str = \"vinbigdata-chest-xray-resized-png-256x256\"\n seed: int = 111\n target_fold: int = 0 # 0~4\n # Model config\n model_name: str = \"efficientnet_b6\"\n # Training config\n epoch: int = 20\n batchsize: int = 8\n valid_batchsize: int = 16\n num_workers: int = 4\n snapshot_freq: int = 5\n ema_decay: float = 0.999 # negative value is to inactivate ema.\n scheduler_type: str = \"\"\n scheduler_kwargs: Dict[str, Any] = field(default_factory=lambda: {})\n scheduler_trigger: List[Union[int, str]] = field(default_factory=lambda: [1, \"iteration\"])\n\n def update(self, param_dict: Dict) -> \"Flags\":\n # Overwrite by `param_dict`\n for key, value in param_dict.items():\n if not hasattr(self, key):\n raise ValueError(f\"[ERROR] Unexpected key for flag = {key}\")\n setattr(self, key, value)\n return self\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"flags_dict = {\n \"debug\": False, # Change to True for fast debug run!\n \"outdir\": \"results/tmp_debug\",\n # Data\n \"imgdir_name\": \"vinbigdata-chest-xray-resized-png-256x256\",\n # Model\n \"model_name\": \"efficientnet_b6\",\n # Training\n \"num_workers\": 4,\n \"epoch\": 15,\n \"batchsize\": 8,\n \"scheduler_type\": \"CosineAnnealingWarmRestarts\",\n \"scheduler_kwargs\": {\"T_0\": 28125}, # 15000 * 15 epoch // (batchsize=8)\n \"scheduler_trigger\": [1, \"iteration\"]\n}","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"import dataclasses\n\n# args = parse()\nprint(\"torch\", torch.__version__)\nflags = Flags().update(flags_dict)\nprint(\"flags\", flags)\ndebug = flags.debug\noutdir = Path(flags.outdir)\nos.makedirs(str(outdir), exist_ok=True)\nflags_dict = dataclasses.asdict(flags)\nsave_yaml(str(outdir / \"flags.yaml\"), flags_dict)\n\n# --- Read data ---\ninputdir = Path(\"/kaggle/input\")\ndatadir = inputdir / \"vinbigdata-chest-xray-abnormalities-detection\"\nimgdir = inputdir / flags.imgdir_name\n\n# Read in the data CSV files\ntrain = pd.read_csv(datadir / \"train.csv\")\n# sample_submission = pd.read_csv(datadir / 'sample_submission.csv')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n# EDA: distribution between normal & abnormal class\n\nAt first, let's check how many normal class exist in the training data.\nIt is classified as \"class_name = No finding\" and \"class_id = 14\".\n\nHowever you need to be careful that 3 radiologists annotated for each image, so you can find 3 annotations as you can see below."},{"metadata":{},"cell_type":"markdown","source":"正态和异常类之间的分布首先，我们检查一下训练数据中存在多少正态类。它被分类为“class_name=no finding”和“class_id = 14”。然而，你需要注意的是，3名放射科医生为每一张图像做了注释，所以你可以找到3个注释，如下所示。"},{"metadata":{"trusted":true},"cell_type":"code","source":"train.query(\"image_id == '50a418190bc3fb1ef1633bf9678929b3'\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"So the question arises, is there an image that the 3 radiologists' opinions differ?\n\nLet's check number of \"No finding\" annotations for each image, if the opinions are in complete agreement the number of \"No finding\" annotations should be **0 -> Abnormal(all radiologists does not think this is normal)\" or \"1 -> Normal(all radiologists think this is normal)\"**."},{"metadata":{},"cell_type":"markdown","source":"那么问题来了，这三位放射学家是否有不同的看法?\n让我们检查每个图像的“无发现”注释的数量，如果意见完全一致，“无发现”注释的数量应该是“0 ->异常(所有放射科医生认为这是正常的)”或“1 ->正常(所有放射科医生认为这是正常的)”。"},{"metadata":{"trusted":true},"cell_type":"code","source":"is_normal_df = train.groupby(\"image_id\")[\"class_id\"].agg(lambda s: (s == 14).sum()).reset_index().rename({\"class_id\": \"num_normal_annotations\"}, axis=1)\nis_normal_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We could confirm that **always 3 radiologists opinions match** for normal - abnormal diagnosis.\n\n[Note] I noticed that it does not apply for the other classes. i.e., 3 radiologists opinions sometimes do not match for the other class of thoracic abnormalities."},{"metadata":{},"cell_type":"markdown","source":"我们可以确认，对于正常-异常诊断，总是有3位放射科医生的意见匹配。\n\n[注意]我注意到它不适用于其他类的病。 即，3位放射科医生的意见有时与另一类胸腔异常不符。\n也就是说，对于有病没病，三个医生的意见是一致的。\n但是具体是什么病，病区是哪块，意见不一致。"},{"metadata":{"trusted":true},"cell_type":"code","source":"# 每张图片中“找不到”注释的数量\nnum_normal_anno_counts = is_normal_df[\"num_normal_annotations\"].value_counts()\nnum_normal_anno_counts.plot(kind=\"bar\")\nplt.title(\"The number of 'No finding' annotations in each image\")","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"num_normal_anno_counts_df = num_normal_anno_counts.reset_index()\nnum_normal_anno_counts_df[\"name\"] = num_normal_anno_counts_df[\"index\"].map({0: \"Abnormal\", 3: \"Normal\"})\nnum_normal_anno_counts_df","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"So almost 70% of the data is actually \"Normal\" X-ray images.\n\nOnly 30% of the images need thoracic abnormality location detection."},{"metadata":{},"cell_type":"markdown","source":"因此，几乎70%的数据实际上是“正常的”x射线图像。只有30%的图像需要胸部异常定位。"},{"metadata":{"trusted":true},"cell_type":"code","source":"px.pie(num_normal_anno_counts_df, values=\"num_normal_annotations\", names=\"name\", title=\"Normal/Abnormal ratio\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n# Image visualizaion & augmentation with albumentations\n\nWhen you train CNN models, image augmentation is important to avoid model to overfit.
\nI'll show examples to use Albumentations to run image augmentation very easily.
\nAt first, I will define pytorch Dataset class for this competition, which can be also used later in the training."},{"metadata":{},"cell_type":"markdown","source":"当你训练CNN模型时，图像增强对于避免模型过拟合是很重要的。\n我将展示使用Albumentations非常容易地运行图像增强的示例。\n 首先，我将为这个竞赛定义pytorch Dataset类，它也可以在稍后的训练中使用。"},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"import pickle\nfrom pathlib import Path\nfrom typing import Optional\n\nimport cv2\nimport numpy as np\nimport pandas as pd\nfrom detectron2.structures import BoxMode\nfrom tqdm import tqdm\n\n\ndef get_vinbigdata_dicts(\n imgdir: Path,\n train_df: pd.DataFrame,\n train_data_type: str = \"original\",\n use_cache: bool = True,\n debug: bool = True,\n target_indices: Optional[np.ndarray] = None,\n):\n debug_str = f\"_debug{int(debug)}\"\n train_data_type_str = f\"_{train_data_type}\"\n cache_path = Path(\".\") / f\"dataset_dicts_cache{train_data_type_str}{debug_str}.pkl\"\n if not use_cache or not cache_path.exists():\n print(\"Creating data...\")\n train_meta = pd.read_csv(imgdir / \"train_meta.csv\")\n if debug:\n train_meta = train_meta.iloc[:500] # For debug....\n\n # Load 1 image to get image size.\n image_id = train_meta.loc[0, \"image_id\"]\n image_path = str(imgdir / \"train\" / f\"{image_id}.png\")\n image = cv2.imread(image_path)\n resized_height, resized_width, ch = image.shape\n print(f\"image shape: {image.shape}\")\n\n dataset_dicts = []\n for index, train_meta_row in tqdm(train_meta.iterrows(), total=len(train_meta)):\n record = {}\n\n image_id, height, width = train_meta_row.values\n filename = str(imgdir / \"train\" / f\"{image_id}.png\")\n record[\"file_name\"] = filename\n record[\"image_id\"] = image_id\n record[\"height\"] = resized_height\n record[\"width\"] = resized_width\n objs = []\n for index2, row in train_df.query(\"image_id == @image_id\").iterrows():\n # print(row)\n # print(row[\"class_name\"])\n # class_name = row[\"class_name\"]\n class_id = row[\"class_id\"]\n if class_id == 14:\n # It is \"No finding\"\n # This annotator does not find anything, skip.\n pass\n else:\n # bbox_original = [int(row[\"x_min\"]), int(row[\"y_min\"]), int(row[\"x_max\"]), int(row[\"y_max\"])]\n h_ratio = resized_height / height\n w_ratio = resized_width / width\n bbox_resized = [\n int(row[\"x_min\"]) * w_ratio,\n int(row[\"y_min\"]) * h_ratio,\n int(row[\"x_max\"]) * w_ratio,\n int(row[\"y_max\"]) * h_ratio,\n ]\n obj = {\n \"bbox\": bbox_resized,\n \"bbox_mode\": BoxMode.XYXY_ABS,\n \"category_id\": class_id,\n }\n objs.append(obj)\n record[\"annotations\"] = objs\n dataset_dicts.append(record)\n with open(cache_path, mode=\"wb\") as f:\n pickle.dump(dataset_dicts, f)\n\n print(f\"Load from cache {cache_path}\")\n with open(cache_path, mode=\"rb\") as f:\n dataset_dicts = pickle.load(f)\n if target_indices is not None:\n dataset_dicts = [dataset_dicts[i] for i in target_indices]\n return dataset_dicts\n\n\ndef get_vinbigdata_dicts_test(\n imgdir: Path, test_meta: pd.DataFrame, use_cache: bool = True, debug: bool = True,\n):\n debug_str = f\"_debug{int(debug)}\"\n cache_path = Path(\".\") / f\"dataset_dicts_cache_test{debug_str}.pkl\"\n if not use_cache or not cache_path.exists():\n print(\"Creating data...\")\n # test_meta = pd.read_csv(imgdir / \"test_meta.csv\")\n if debug:\n test_meta = test_meta.iloc[:500] # For debug....\n\n # Load 1 image to get image size.\n image_id = test_meta.loc[0, \"image_id\"]\n image_path = str(imgdir / \"test\" / f\"{image_id}.png\")\n image = cv2.imread(image_path)\n resized_height, resized_width, ch = image.shape\n print(f\"image shape: {image.shape}\")\n\n dataset_dicts = []\n for index, test_meta_row in tqdm(test_meta.iterrows(), total=len(test_meta)):\n record = {}\n\n image_id, height, width = test_meta_row.values\n filename = str(imgdir / \"test\" / f\"{image_id}.png\")\n record[\"file_name\"] = filename\n # record[\"image_id\"] = index\n record[\"image_id\"] = image_id\n record[\"height\"] = resized_height\n record[\"width\"] = resized_width\n # objs = []\n # record[\"annotations\"] = objs\n dataset_dicts.append(record)\n with open(cache_path, mode=\"wb\") as f:\n pickle.dump(dataset_dicts, f)\n\n print(f\"Load from cache {cache_path}\")\n with open(cache_path, mode=\"rb\") as f:\n dataset_dicts = pickle.load(f)\n return dataset_dicts\n","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"\"\"\"\nReferenced `chainer.dataset.DatasetMixin` to work with pytorch Dataset.\n\"\"\"\nimport numpy\nimport six\nimport torch\nfrom torch.utils.data.dataset import Dataset\n\n\nclass DatasetMixin(Dataset):\n\n def __init__(self, transform=None):\n self.transform = transform\n\n def __getitem__(self, index):\n \"\"\"Returns an example or a sequence of examples.\"\"\"\n if torch.is_tensor(index):\n index = index.tolist()\n if isinstance(index, slice):\n current, stop, step = index.indices(len(self))\n return [self.get_example_wrapper(i) for i in\n six.moves.range(current, stop, step)]\n elif isinstance(index, list) or isinstance(index, numpy.ndarray):\n return [self.get_example_wrapper(i) for i in index]\n else:\n return self.get_example_wrapper(index)\n\n def __len__(self):\n \"\"\"Returns the number of data points.\"\"\"\n raise NotImplementedError\n\n def get_example_wrapper(self, i):\n \"\"\"Wrapper of `get_example`, to apply `transform` if necessary\"\"\"\n example = self.get_example(i)\n if self.transform:\n example = self.transform(example)\n return example\n\n def get_example(self, i):\n \"\"\"Returns the i-th example.\n\n Implementations should override it. It should raise :class:`IndexError`\n if the index is invalid.\n\n Args:\n i (int): The index of the example.\n\n Returns:\n The i-th example.\n\n \"\"\"\n raise NotImplementedError\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import cv2\nimport numpy as np\n\n\nclass VinbigdataTwoClassDataset(DatasetMixin):\n def __init__(self, dataset_dicts, image_transform=None, transform=None, train: bool = True):\n super(VinbigdataTwoClassDataset, self).__init__(transform=transform)\n self.dataset_dicts = dataset_dicts\n self.image_transform = image_transform\n self.train = train\n\n def get_example(self, i):\n d = self.dataset_dicts[i]\n filename = d[\"file_name\"]\n\n img = cv2.imread(filename)\n if self.image_transform:\n img = self.image_transform(img)\n img = np.transpose(img, (2, 0, 1)).astype(np.float32)\n if self.train:\n label = int(len(d[\"annotations\"]) > 0) # 0 normal, 1 abnormal\n return img, label\n else:\n # Only return img\n return img,\n\n def __len__(self):\n return len(self.dataset_dicts)\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now creating the dataset is just easy as following:"},{"metadata":{"_kg_hide-output":true,"trusted":true},"cell_type":"code","source":"dataset_dicts = get_vinbigdata_dicts(imgdir, train, debug=debug)\ndataset = VinbigdataTwoClassDataset(dataset_dicts)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"You can access each image and its label (0=Normal, 1=Abnormal) by just access `dataset` with index."},{"metadata":{},"cell_type":"markdown","source":"您可以访问每个图像及其标签(0=正常，1=不正常)仅通过访问具有索引的数据集。"},{"metadata":{"trusted":true},"cell_type":"code","source":"index = 0\nimg, label = dataset[index]\nplt.imshow(img.transpose((1, 2, 0)) / 255.)\nplt.title(f\"{index}-th image: label {label}\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"To run augmentation on this image, I will define `Transform` class which is applied each time the data is accessed.\n\nYou can refer [albumentations](https://github.com/albumentations-team/albumentations) page, that various kinds of augmentation is already implemented and can be used very easily!"},{"metadata":{},"cell_type":"markdown","source":"为了在图像上运行增强，我将定义Transform类，每次访问数据时应用它。\n你可以参考albumentations页面，各种扩展已经实现，可以很容易地使用!"},{"metadata":{"trusted":true},"cell_type":"code","source":"import albumentations as A\n\n\nclass Transform:\n def __init__(\n self, hflip_prob: float = 0.5, ssr_prob: float = 0.5, random_bc_prob: float = 0.5\n ):\n self.transform = A.Compose(\n [\n A.HorizontalFlip(p=hflip_prob),\n A.ShiftScaleRotate(\n shift_limit=0.0625, scale_limit=0.1, rotate_limit=10, p=ssr_prob\n ),\n A.RandomBrightnessContrast(p=random_bc_prob),\n ]\n )\n\n def __call__(self, image):\n image = self.transform(image=image)[\"image\"]\n return image\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"To use augmentation, you can just define dataset with the `Transform` function."},{"metadata":{"trusted":true},"cell_type":"code","source":"aug_dataset = VinbigdataTwoClassDataset(dataset_dicts, image_transform=Transform())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's visualize, looks good.
\nYou can see each image looks different (rotated, brightness is different etc...) even if it is generated from the same image :)"},{"metadata":{},"cell_type":"markdown","source":"让我们可视化一下，看起来不错。\n\n您可以看到每个图像看起来都不同（旋转，亮度不同等），即使它是从同一图像生成的也是如此：）"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"index = 0\n\nn_images = 4\n\nfig, axes = plt.subplots(1, n_images, figsize=(16, 5))\nfor i in range(n_images):\n # Each time the data is accessed, the result is different due to random augmentation!\n img, label = aug_dataset[index]\n ax = axes[i]\n ax.imshow(img.transpose((1, 2, 0)) / 255.)\n ax.set_title(f\"{index}-th image: label {label}\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n# Defining CNN models\n\nRecently, several libraries of CNN-collection are available on public.\n\nI will use `timm` this time. You don't need to impelment deep CNN models by yourself, you can just re-use latest research results without hustle.
\nYou can focus on more about looking data and try experiment now."},{"metadata":{},"cell_type":"markdown","source":"定义CNN模型\n\n最近，CNN集合的几个库公开可用。\n\n这次我将使用timm。 您无需自己推动深层的CNN模型，您可以轻松使用最新的研究结果。\n\n您可以集中精力查看数据并立即尝试进行实验。"},{"metadata":{"trusted":true},"cell_type":"code","source":"import timm\n\n\ndef build_predictor(model_name: str):\n return timm.create_model(model_name, pretrained=True, num_classes=2, in_chans=3)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"import torch\n\n\ndef accuracy(y: torch.Tensor, t: torch.Tensor) -> torch.Tensor:\n \"\"\"Computes multi-class classification accuracy\"\"\"\n assert y.shape[:-1] == t.shape, f\"y {y.shape}, t {t.shape} is inconsistent.\"\n pred_label = torch.max(y.detach(), dim=-1)[1]\n count = t.nelement()\n correct = (pred_label == t).sum().float()\n acc = correct / count\n return acc","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"import torch\nimport torch.nn.functional as F\nfrom torch import nn\nimport pytorch_pfn_extras as ppe\n\n\nclass Classifier(nn.Module):\n \"\"\"two class classfication\"\"\"\n\n def __init__(self, predictor, lossfun=F.cross_entropy):\n super().__init__()\n self.predictor = predictor\n self.lossfun = lossfun\n self.prefix = \"\"\n\n def forward(self, image, targets):\n outputs = self.predictor(image)\n loss = self.lossfun(outputs, targets)\n metrics = {\n f\"{self.prefix}loss\": loss.item(),\n f\"{self.prefix}acc\": accuracy(outputs, targets).item()\n }\n ppe.reporting.report(metrics, self)\n return loss, metrics\n\n def predict(self, data_loader):\n pred = self.predict_proba(data_loader)\n label = torch.argmax(pred, dim=1)\n return label\n\n def predict_proba(self, data_loader):\n device: torch.device = next(self.parameters()).device\n y_list = []\n self.eval()\n with torch.no_grad():\n for batch in data_loader:\n if isinstance(batch, (tuple, list)):\n # Assumes first argument is \"image\"\n batch = batch[0].to(device)\n else:\n batch = batch.to(device)\n y = self.predictor(batch)\n y = torch.softmax(y, dim=-1)\n y_list.append(y)\n pred = torch.cat(y_list)\n return pred","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"What kind of models are supported in the `timm` library?"},{"metadata":{},"cell_type":"markdown","source":"timm库支持哪些模型？"},{"metadata":{"trusted":true},"cell_type":"code","source":"supported_models = timm.list_models()\nprint(f\"{len(supported_models)} models are supported in timm.\")\nprint(supported_models)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Wow more than 300 models are supported!
\nIt of course includes **resnet** related models, **efficientnet**, etc.
\nYou may wonder which model should be used?
\nI will go with `resnet18` as a baseline at first, and try using more deeper/latest models in the experiment."},{"metadata":{},"cell_type":"markdown","source":"哇，支持300多种型号！\n\n当然，它包括与Resnet相关的模型，efficiencynet等。\n\n您可能想知道应该使用哪种模型？\n\n首先，我将以resnet18为基准，并尝试在实验中使用更深入/最新的模型。"},{"metadata":{},"cell_type":"markdown","source":"\n# Training utils\n\nHere are training util methods. You can just copy these to use in other projects."},{"metadata":{},"cell_type":"markdown","source":"这里是训练util方法。您可以将这些复制到其他项目中使用。"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"\"\"\"\nFrom https://github.com/pfnet-research/kaggle-lyft-motion-prediction-4th-place-solution\n\"\"\"\nfrom logging import getLogger\n\nfrom torch import nn\n\n\nclass EMA(object):\n \"\"\"Exponential moving average of model parameters.\n\n Ref\n - https://github.com/tensorflow/addons/blob/v0.10.0/tensorflow_addons/optimizers/moving_average.py#L26-L103\n - https://anmoljoshi.com/Pytorch-Dicussions/\n\n Args:\n model (nn.Module): Model with parameters whose EMA will be kept.\n decay (float): Decay rate for exponential moving average.\n strict (bool): Apply strict check for `assign` & `resume`.\n use_dynamic_decay (bool): Dynamically change decay rate. If `True`, small decay rate is\n used at the beginning of training to move moving average faster.\n \"\"\" # NOQA\n\n def __init__(\n self,\n model: nn.Module,\n decay: float,\n strict: bool = True,\n use_dynamic_decay: bool = True,\n ):\n self.decay = decay\n self.model = model\n self.strict = strict\n self.use_dynamic_decay = use_dynamic_decay\n self.logger = getLogger(__name__)\n self.n_step = 0\n\n self.shadow = {}\n self.original = {}\n\n # Flag to manage which parameter is assigned.\n # When `False`, original model's parameter is used.\n # When `True` (`assign` method is called), `shadow` parameter (ema param) is used.\n self._assigned = False\n\n # Register model parameters\n for name, param in model.named_parameters():\n if param.requires_grad:\n self.shadow[name] = param.data.clone()\n\n def step(self):\n self.n_step += 1\n if self.use_dynamic_decay:\n _n_step = float(self.n_step)\n decay = min(self.decay, (1.0 + _n_step) / (10.0 + _n_step))\n else:\n decay = self.decay\n\n for name, param in self.model.named_parameters():\n if param.requires_grad:\n assert name in self.shadow\n new_average = (1.0 - decay) * param.data + decay * self.shadow[name]\n self.shadow[name] = new_average.clone()\n\n # alias\n __call__ = step\n\n def assign(self):\n \"\"\"Assign exponential moving average of parameter values to the respective parameters.\"\"\"\n if self._assigned:\n if self.strict:\n raise ValueError(\"[ERROR] `assign` is called again before `resume`.\")\n else:\n self.logger.warning(\n \"`assign` is called again before `resume`.\"\n \"shadow parameter is already assigned, skip.\"\n )\n return\n\n for name, param in self.model.named_parameters():\n if param.requires_grad:\n assert name in self.shadow\n self.original[name] = param.data.clone()\n param.data = self.shadow[name]\n self._assigned = True\n\n def resume(self):\n \"\"\"Restore original parameters to a model.\n\n That is, put back the values that were in each parameter at the last call to `assign`.\n \"\"\"\n if not self._assigned:\n if self.strict:\n raise ValueError(\"[ERROR] `resume` is called before `assign`.\")\n else:\n self.logger.warning(\"`resume` is called before `assign`, skip.\")\n return\n\n for name, param in self.model.named_parameters():\n if param.requires_grad:\n assert name in self.shadow\n param.data = self.original[name]\n self._assigned = False\n","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"\"\"\"\nFrom https://github.com/pfnet-research/kaggle-lyft-motion-prediction-4th-place-solution\n\"\"\"\nfrom typing import Mapping, Any\n\nfrom torch import optim\n\nfrom pytorch_pfn_extras.training.extension import Extension, PRIORITY_READER\nfrom pytorch_pfn_extras.training.manager import ExtensionsManager\n\n\nclass LRScheduler(Extension):\n \"\"\"A thin wrapper to resume the lr_scheduler\"\"\"\n\n trigger = 1, 'iteration'\n priority = PRIORITY_READER\n name = None\n\n def __init__(self, optimizer: optim.Optimizer, scheduler_type: str, scheduler_kwargs: Mapping[str, Any]) -> None:\n super().__init__()\n self.scheduler = getattr(optim.lr_scheduler, scheduler_type)(optimizer, **scheduler_kwargs)\n\n def __call__(self, manager: ExtensionsManager) -> None:\n self.scheduler.step()\n\n def state_dict(self) -> None:\n return self.scheduler.state_dict()\n\n def load_state_dict(self, to_load) -> None:\n self.scheduler.load_state_dict(to_load)\n","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"from ignite.engine import Engine\n\n\ndef create_trainer(model, optimizer, device) -> Engine:\n model.to(device)\n\n def update_fn(engine, batch):\n model.train()\n optimizer.zero_grad()\n loss, metrics = model(*[elem.to(device) for elem in batch])\n loss.backward()\n optimizer.step()\n return metrics\n trainer = Engine(update_fn)\n return trainer\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n# Training scripts"},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"import dataclasses\nimport os\nimport sys\nfrom pathlib import Path\n\nimport numpy as np\nimport pandas as pd\nimport pytorch_pfn_extras.training.extensions as E\nimport torch\nfrom ignite.engine import Events\nfrom pytorch_pfn_extras.training import IgniteExtensionsManager\nfrom sklearn.model_selection import StratifiedKFold\nfrom torch import nn, optim\nfrom torch.utils.data.dataloader import DataLoader","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Preparing data by 5-fold cross validation\n\nWhen we have few data, running stable evaluation is very important. \nWe can use cross validation to reduce validation error standard deviation.\n\nHere, I will use **[`StratifiedKFold`](https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.StratifiedKFold.html)** to keep the balance between normal/abnormal ratio same for the train & validation dataset.\n\nAccording to [this discussion](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/208837#1139712), using multi label stratified kfold https://github.com/trent-b/iterative-stratification may be more stable."},{"metadata":{},"cell_type":"markdown","source":"通过5倍交叉验证准备数据\n\n当我们的数据很少时，进行稳定的评估非常重要。 我们可以使用交叉验证来减少验证错误的标准偏差。\n\n在这里，我将使用StratifiedKFold来保持训练和验证数据集的正常/异常比率之间的平衡相同。\n\n根据此讨论，使用多标签分层kfold https://github.com/trent-b/iterative-stratification可能更稳定。"},{"metadata":{"trusted":true},"cell_type":"code","source":"skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=flags.seed)\n# skf.get_n_splits(None, None)\ny = np.array([int(len(d[\"annotations\"]) > 0) for d in dataset_dicts])\nsplit_inds = list(skf.split(dataset_dicts, y))\ntrain_inds, valid_inds = split_inds[flags.target_fold] # Choose which fold to train, 0th fold selected this time.\ntrain_dataset = VinbigdataTwoClassDataset(\n [dataset_dicts[i] for i in train_inds], image_transform=Transform()\n)\nvalid_dataset = VinbigdataTwoClassDataset([dataset_dicts[i] for i in valid_inds])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Write training code\n\npytorch-ignite & pytorch-pfn-extras are used here.\n\n - [pytorch/ignite](https://github.com/pytorch/ignite): It provides abstraction for writing training loop.\n - [pfnet/pytorch-pfn-extras](https://github.com/pfnet/pytorch-pfn-extras): It provides several \"extensions\" useful for training. Useful for **logging, printing, evaluating, saving the model, scheduling the learning rate** during training.\n \n**[Note] Why training abstraction library is used?**\n\nYou may feel understanding training abstraction code below is a bit unintuitive compared to writing \"raw\" training loop.
\nThe advantage of abstracting the code is that we can re-use implemented handler class for other training, other competition.
\nYou don't need to write code for saving models, logging training loss/metric, show progressbar etc.\nThese are done by provided util classes in `pytorch-pfn-extras` library!\n\nYou may refer my other kernel in previous competition too:\n - [Bengali: SEResNeXt training with pytorch](https://www.kaggle.com/corochann/bengali-seresnext-training-with-pytorch)\n - [Lyft: Training with multi-mode confidence](https://www.kaggle.com/corochann/lyft-training-with-multi-mode-confidence)"},{"metadata":{},"cell_type":"markdown","source":"编写培训代码\n\n此处使用pytorch-ignite和pytorch-pfn-extras。\n\npytorch / ignite：为编写训练循环提供抽象。\n\npfnet / pytorch-pfn-extras：它提供了一些对培训有用的“扩展”。 对于记录，打印，评估，保存模型，安排训练期间的学习率很有用。\n\n[注意]为什么要使用训练抽象库？\n\n与编写“原始”训练循环相比，您可能会觉得理解下面的训练抽象代码有点不直观。\n\n抽象代码的优点是我们可以将实现的处理程序类重新用于其他培训和其他比赛。\n\n您无需编写代码来保存模型，记录训练损失/指标，显示进度条等。这些操作由pytorch-pfn-extras库中提供的util类完成！\n\n\n\n您也可以在以前的比赛中引用我的其他内核：\n\n孟加拉语：使用pytorch进行SEResNeXt培训\n\nLyft：充满信心地进行训练"},{"metadata":{"trusted":true},"cell_type":"code","source":"# 训练集装载器\ntrain_loader = DataLoader(\n train_dataset,\n batch_size=flags.batchsize,\n num_workers=flags.num_workers,\n shuffle=True,\n pin_memory=True,\n)\n# 验证集装载器\nvalid_loader = DataLoader(\n valid_dataset,\n batch_size=flags.valid_batchsize,\n num_workers=flags.num_workers,\n shuffle=False,\n pin_memory=True,\n)\n\ndevice = torch.device(flags.device)\n\npredictor = build_predictor(model_name=flags.model_name)\nclassifier = Classifier(predictor)\nmodel = classifier\noptimizer = optim.Adam(model.parameters(), lr=1e-3)\n\n# Train setup\ntrainer = create_trainer(model, optimizer, device)\n\nema = EMA(predictor, decay=flags.ema_decay)\n\ndef eval_func(*batch):\n loss, metrics = model(*[elem.to(device) for elem in batch])\n # HACKING: report ema value with prefix.\n if flags.ema_decay > 0:\n classifier.prefix = \"ema_\"\n ema.assign()\n loss, metrics = model(*[elem.to(device) for elem in batch])\n ema.resume()\n classifier.prefix = \"\"\n\nvalid_evaluator = E.Evaluator(\n valid_loader, model, progress_bar=False, eval_func=eval_func, device=device\n)\n\n# log_trigger = (10 if debug else 1000, \"iteration\")\nlog_trigger = (1, \"epoch\")\nlog_report = E.LogReport(trigger=log_trigger)\nextensions = [\n log_report,\n E.ProgressBarNotebook(update_interval=10 if debug else 100), # Show progress bar during training\n E.PrintReportNotebook(), # Show \"log\" on jupyter notebook \n # E.ProgressBar(update_interval=10 if debug else 100), # Show progress bar during training\n # E.PrintReport(), # Print \"log\" to terminal\n E.FailOnNonNumber(), # Stop training when nan is detected.\n]\nepoch = flags.epoch\nmodels = {\"main\": model}\noptimizers = {\"main\": optimizer}\nmanager = IgniteExtensionsManager(\n trainer, models, optimizers, epoch, extensions=extensions, out_dir=str(outdir),\n)\n# Run evaluation for valid dataset in each epoch.\nmanager.extend(valid_evaluator)\n\n# Save predictor.pt every epoch\nmanager.extend(\n E.snapshot_object(predictor, \"predictor.pt\"), trigger=(flags.snapshot_freq, \"epoch\")\n)\n# Check & Save best validation predictor.pt every epoch\n# manager.extend(E.snapshot_object(predictor, \"best_predictor.pt\"),\n# trigger=MinValueTrigger(\"validation/module/nll\",\n# trigger=(flags.snapshot_freq, \"iteration\")))\n\n# --- lr scheduler ---\nif flags.scheduler_type != \"\":\n scheduler_type = flags.scheduler_type\n print(f\"using {scheduler_type} scheduler with kwargs {flags.scheduler_kwargs}\")\n manager.extend(\n LRScheduler(optimizer, scheduler_type, flags.scheduler_kwargs),\n trigger=flags.scheduler_trigger,\n )\n\nmanager.extend(E.observe_lr(optimizer=optimizer), trigger=log_trigger)\n\nif flags.ema_decay > 0:\n # Exponential moving average\n manager.extend(lambda manager: ema(), trigger=(1, \"iteration\"))\n\n def save_ema_model(manager):\n ema.assign()\n torch.save(predictor.state_dict(), outdir / \"predictor_ema.pt\")\n ema.resume()\n\n manager.extend(save_ema_model, trigger=(flags.snapshot_freq, \"epoch\"))\n\n_ = trainer.run(train_loader, max_epochs=epoch)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"So what is happening in above training abstraction? Let's understand what each extension did.\n\n**Extensions** - Each role:\n - **`ProgressBar` (`ProgressBarNotebook`)**: Shows training progress in formatted style.\n - **`LogReport`**: Logging metrics reported by `ppe.reporter.report` (see `LyftMultiRegressor` for reporting point) method and save to **log** file. It automatically collects reported value in each iteration and saves the \"mean\" of reported value for regular frequency (for example every 1 epoch).\n - **`PrintReport` (`PrintReportNotebook`)**: Prints the value which `LogReport` collected in formatted style.\n - **`Evaluator`**: Evaluate on validation dataset.\n - **`snapshot_object`**: Saves the object. Here the `model` is saved in regular interval `flags.snapshot_freq`. Even you quit training using Ctrl+C without finishing all the epoch, the intermediate trained model is saved and you can use it for inference.\n - **`LRScheduler`**: You can insert learning rate scheduling with this extension, together with the regular interval call specified by `trigger`. Here cosine annealing is applied (configured by Flags) by calling `scheduler.step()` every iteration.\n - **`observe_lr`**: `LogReport` will check optimizer's learning rate using this extension. So you can follow how the learning rate changed through the training.\n\n\nSuch many functionalities can be \"added\" easily using extensions!"},{"metadata":{},"cell_type":"markdown","source":"那么，以上训练抽象发生了什么？ 让我们了解每个扩展的功能。\n\n扩展-每个角色：\n\nProgressBar（ProgressBarNotebook）：以格式化的样式显示训练进度。\n\nLogReport：记录由ppe.reporter.report报告的度量标准（有关报告点，请参见LyftMultiRegressor），并保存到日志文件中。 它会在每次迭代中自动收集报告值，并以常规频率（例如，每1个周期）保存报告值的“平均值”。\n\nPrintReport（PrintReportNotebook）：以格式化的样式打印LogReport收集的值。\n\nEvaluator：评估验证数据集。\n\nsnapshot_object：保存对象。 在这里，模型以规则的时间间隔flags.snapshot_freq保存。 即使您在没有完成所有纪元的情况下使用Ctrl + C退出了训练，中间训练的模型也会被保存，您可以将其用于推理。\n\nLRScheduler：您可以插入带有此扩展名的学习率计划，以及触发器指定的常规间隔调用。 这里，通过每次迭代调用scheduler.step（）来应用余弦退火（由Flags配置）。\n\nwatch_lr：LogReport将使用此扩展名检查优化器的学习率。 因此，您可以通过培训了解学习率的变化。\n\n使用扩展可以轻松地“添加”这么多的功能！Evaluator"},{"metadata":{},"cell_type":"markdown","source":"Also **Exponential Moving Average of model weights** is calculated by `EMA` class during training, together with showing its validation loss. We can usually obtrain more stable models with EMA."},{"metadata":{},"cell_type":"markdown","source":"此外，EMA类在训练过程中计算模型权值的指数移动平均，并显示其有效性损失。我们通常可以用EMA得到更稳定的模型。"},{"metadata":{},"cell_type":"markdown","source":"You can obtrain training history results really easily by just accessing `LogReport` class, which is useful for managing a lot of experiments during kaggle competitions."},{"metadata":{},"cell_type":"markdown","source":"通过访问LogReport类，您可以很容易地获得训练历史结果，这对于管理kaggle比赛期间的大量实验非常有用。"},{"metadata":{"trusted":true},"cell_type":"code","source":"torch.save(predictor.state_dict(), outdir / \"predictor_last.pt\")\ndf = log_report.to_dataframe()\ndf.to_csv(outdir / \"log.csv\", index=False)\ndf","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"\n# Prediction on validation & test dataset"},{"metadata":{"trusted":true},"cell_type":"code","source":"# --- Prediction ---\n# 作预测\nprint(\"Training done! Start prediction...\")\n# valid data\n# 对验证集数据做预测\nvalid_pred = classifier.predict_proba(valid_loader).cpu().numpy()\nvalid_pred_df = pd.DataFrame({\n \"image_id\": [dataset_dicts[i][\"image_id\"] for i in valid_inds],\n \"class0\": valid_pred[:, 0],\n \"class1\": valid_pred[:, 1]\n})\nvalid_pred_df.to_csv(outdir/\"valid_pred.csv\", index=False)\n\n# test data\n# 读取测试数据\ntest_meta = pd.read_csv(inputdir / \"vinbigdata-testmeta\" / \"test_meta.csv\")\ndataset_dicts_test = get_vinbigdata_dicts_test(imgdir, test_meta, debug=debug)\ntest_dataset = VinbigdataTwoClassDataset(dataset_dicts_test, train=False)\ntest_loader = DataLoader(\n test_dataset,\n batch_size=flags.valid_batchsize,\n num_workers=flags.num_workers,\n shuffle=False,\n pin_memory=True,\n)\n\n# 对测试集数据做预测\ntest_pred = classifier.predict_proba(test_loader).cpu().numpy()\ntest_pred_df = pd.DataFrame({\n \"image_id\": [d[\"image_id\"] for d in dataset_dicts_test],\n \"class0\": test_pred[:, 0],\n \"class1\": test_pred[:, 1]\n})\ntest_pred_df.to_csv(outdir/\"test_pred.csv\", index=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"valid_loader.sampler","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"classifier.predict_proba(valid_loader)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"is_normal_df","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"eqw","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# --- Test dataset prediction result ---\ntest_pred_df","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"binary_clf=test_pred_df[['image_id','class0']]\n\nnew_col = ['image_id', 'target']\nbinary_clf.columns = new_col\nbinary_clf\nbinary_clf.to_csv(outdir / \"2-cls test pred.csv\", index=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# 画出训练集和验证集的概率分布\nsns.distplot(valid_pred_df[\"class0\"].values, color='green', label='valid pred')\nsns.distplot(test_pred_df[\"class0\"].values, color='orange', label='test pred')\nplt.title(\"Prediction results histogram\")\nplt.xlim([0., 1.])\nplt.legend()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.metrics import roc_curve, auc ###计算roc和auc\nfrom sklearn import cross_validation\n\neqw=pd.merge(is_normal_df,valid_pred_df,on='image_id')\n\ntrue_class=eqw['num_normal_annotations'].tolist()\ntrue_class=np.array([0 if value == 0 else 1 for value in true_class])\n\npred_class=np.array(eqw['class0'].tolist())\n\n# Compute ROC curve and ROC area for each class\nfpr,tpr,threshold = roc_curve(true_class, pred_class) ###计算真正率和假正率\nroc_auc = auc(fpr,tpr) ###计算auc的值\n\nplt.figure()\nlw = 2\nplt.figure(figsize=(10,10))\nplt.plot(fpr, tpr, color='darkorange',\n lw=lw, label='ROC curve (area = %0.2f)' % roc_auc) ###假正率为横坐标，真正率为纵坐标做曲线\nplt.plot([0, 1], [0, 1], color='navy', lw=lw, linestyle='--')\nplt.xlim([0.0, 1.0])\nplt.ylim([0.0, 1.05])\nplt.xlabel('False Positive Rate')\nplt.ylabel('True Positive Rate')\nplt.title('Receiver operating characteristic example')\nplt.legend(loc=\"lower right\")\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"That's all!\n\nIf this kernel helps you, please upvote to keep me motivated 😁
Thanks!
"},{"metadata":{},"cell_type":"markdown","source":"\n# Next step\n\nI explained EDA - Training - Prediction pipeline for 2-class image classification in this kernel.
\nYou can try changing training configurations by just changing `Flags` (`flags_dict`) configuration.\n\n我解释了该内核中的EDA、2类图像分类的训练-预测管道。\n\n您可以尝试仅通过更改Flags（flags_dict）配置来更改训练配置。\n\n\n\nFor example, you can change these paramters:\n\n例如，您可以更改以下参数：\n\n - **Data**\n - `imgdir_name`: You can use different preprocessed image introduced in [Multiple preprocessed datasets: 256/512/1024px, PNG and JPG, modified and original ratio](https://www.kaggle.com/c/vinbigdata-chest-xray-abnormalities-detection/discussion/207955) by @xhlulu.\n - **Model**\n - `model_name`: You can try various kinds of models `timm` library support, by just changing model_name.\n - **Training**\n - `epoch`, `batch_size`, `scheduler_type` etc: Try changing these hyperparamters, to see the difference!\n - Augmentation: Please modify `Transform` class to add your augmentation, it's easy to support more augmentations with `albumentations` library.\n\n\nMy basic strategy is as follows:\n - Check training loss/training accuracy: If it is almost same with validation loss/accuracy and it is not accurate enough, model's representation power may be not enough, or data augmentation is too strong. You can try more deeper models, decrease data augmentation or using more rich data (high-resolution image).\n - Check training loss/validation loss difference: If validation loss is very high compared to training loss, it is a sign of overfitting. Try using smaller models, increase data augmentation or apply regularization (dropout etc).\n \n 我的基本策略如下：\n\n检查训练损失/训练准确性：如果与验证损失/准确性几乎相同并且不够准确，则模型的表示能力可能不够，或者数据增强太强。 您可以尝试更深入的模型，减少数据扩充或使用更丰富的数据（高分辨率图像）。\n\n检查训练损失/验证损失差异：如果验证损失与训练损失相比非常高，则表明过拟合。 尝试使用较小的模型，增加数据扩充或应用正则化（dropout等）。"},{"metadata":{},"cell_type":"markdown","source":"# Next to read\n\n[📸VinBigData detectron2 train](https://www.kaggle.com/corochann/vinbigdata-detectron2-train) kernel explains how to run object detection training, using `detectron2` library.\n\n[📸VinBigData detectron2 prediction](https://www.kaggle.com/corochann/vinbigdata-detectron2-prediction) kernel explains how to use trained model for the prediction and submisssion for this competition."},{"metadata":{"trusted":true},"cell_type":"markdown","source":"这两个Notebook，解释了如何去训练，如何去预测"},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}