{"cells":[{"metadata":{},"cell_type":"markdown","source":"...in progress..<br>\n<br>(This notebook may be helpful for you if you have just started in this competition and are looking to understand the data).<br>\n<br>\nI use this notebook to (1) learn about the data and explore my hypotheses, and (2) gather information that I find maybe helpful for me and other participants. Below is summary of the information I've found:\n- Dr. Ian Pan (@vaillant) gives a very helpful explanation about Pulmonary Embolism [here](https://www.kaggle.com/c/rsna-str-pulmonary-embolism-detection/discussion/182376).\n- Data structure:\n    - Each instance (represented by a unique `StudyInstanceUID`/`SeriesInstanceUID`) has various number of images (each represented by a unique `SOPInstanceUID`).\n    - Each `StudyInstanceUID` is associated with a unique `SeriesInstanceUID`.\n- **What are we predicting?** We are predicting PE diagnosis **at the instance-level**, not image-level.\n    - This means that an instance with PE symptoms (`negative_exam_for_pe`=0) can still ahve images with no presence of PE (`pe_present_on_image`=0).\n    - For each instance, we need to predict:\n        - PE is present (`negative_exam_for_pe`, or NOT-`negative_exam_for_pe` to be precise) or not, or indeterminate (`indeterminate`). In `train.csv`, `indeterminate` is associated with either `qa_motion`=1 or `qa_contrast`=1 or both.\n        - PE, if present, is on the left side (`leftsided_pe`=1), right side (`rightsided_pe`=1) or central (`central_pe`=1). Based on the data these are **NOT** mutually exclusive (confirmed by Dr. Ian Pan in the discussion above).\n        - PE, if present, is chronic (`chronic_pe`=1), both acute and chronic (`acute_and_chronic_pe`=1) or only acute (implicit category). These are mutually exclusive.\n        - Whether the right ventricle is smaller than the left ventricle (`rv_lv_ratio_lt_1`=1) or not (`rv_lv_ratio_gte_1`=1). These are mutually exclusive.\n        \n        "},{"metadata":{},"cell_type":"markdown","source":"Feel free to comment if you have questions about anything or have information you would like to add to this list and share with other participants."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"!conda install -c conda-forge gdcm -y\n\nimport os\nfrom matplotlib import pyplot as plt, style\nimport seaborn as sns\nimport numpy as np\nfrom mpl_toolkits.mplot3d.art3d import Poly3DCollection\nimport pandas as pd\nimport glob\nimport pydicom\nimport re\nimport scipy\nfrom skimage import measure\nfrom skimage.morphology import disk, closing, opening\nfrom tqdm import tqdm","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"ROOT = '../input/rsna-str-pulmonary-embolism-detection/'\nTRAIN_IMG = glob.glob(ROOT + 'train/*')\ntrain = pd.read_csv(ROOT + 'train.csv')\ntest = pd.read_csv(ROOT + 'test.csv')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def add_dcmpath(df, data='train', root=ROOT):\n    df['dcm_path'] = root + data + '/' + df.StudyInstanceUID + '/' + df.SeriesInstanceUID\nadd_dcmpath(train)\nadd_dcmpath(test)\ntrain.head(2)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# I. Datasets' structures\n## 1. `StudyInstanceUID`, `SeriesInstanceUID`, `SOPInstanceUID`"},{"metadata":{"trusted":true},"cell_type":"code","source":"train_id_agg = train.iloc[:, :3].groupby(\"StudyInstanceUID\").nunique()\ntrain_id_agg.agg(['max', 'min'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test_id_agg = test.iloc[:, :3].groupby(\"SeriesInstanceUID\").nunique()\ntest_id_agg.agg(['max', 'min'])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(train.StudyInstanceUID.nunique())\nprint(train.SeriesInstanceUID.nunique())\nprint(test.StudyInstanceUID.nunique())\nprint(test.SeriesInstanceUID.nunique())","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"In both train and test set, each `StudyInstanceUID` is associated with a unique\t`SeriesInstanceUID`\tand with various number of `SOPInstanceUID` (or images). Therefore we can remove `SeriesInstanceUID`."},{"metadata":{"trusted":true},"cell_type":"code","source":"qa = ['qa_motion', 'qa_contrast', 'indeterminate']\ndef counting(cols):\n    return train.groupby(cols).size().reset_index(name='count')\ncounting(qa)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"In the train set, `indeterminate` can be determined by `qa_motion` and `qa_contrast`."},{"metadata":{},"cell_type":"markdown","source":"## 2. PE Conclusion"},{"metadata":{"trusted":true},"cell_type":"code","source":"conclusion = ['pe_present_on_image', 'negative_exam_for_pe', 'indeterminate']\ncounting(conclusion)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"An instance with PE can still have images without PE presence. However if an instance PE diagnosis is indeterminate then every image is regarded as no PE present."},{"metadata":{},"cell_type":"markdown","source":"## 3. Left Ventricle/Right Ventricle ratio"},{"metadata":{"trusted":true},"cell_type":"code","source":"rv_lv = ['rv_lv_ratio_gte_1', 'rv_lv_ratio_lt_1']\ncounting(rv_lv)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"These are mutually exclusive."},{"metadata":{},"cell_type":"markdown","source":"## 4. Position of PE"},{"metadata":{"trusted":true},"cell_type":"code","source":"positional = ['leftsided_pe', 'rightsided_pe', 'central_pe']\ncounting(positional)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"`leftsided_pe`, `rightsided_pe` and `central_pe` are not mutually exclusive, as mentioned in [this explanation](https://www.kaggle.com/c/rsna-str-pulmonary-embolism-detection/discussion/182376) by Dr. Ian Pan."},{"metadata":{},"cell_type":"markdown","source":"## 5. Acute/Chronic"},{"metadata":{"trusted":true},"cell_type":"code","source":"acute_chronic = ['chronic_pe', 'acute_and_chronic_pe']\ncounting(acute_chronic + ['negative_exam_for_pe'])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Acute and Chronic are not mutually exclusive characteristics."},{"metadata":{},"cell_type":"markdown","source":"Sections from here downwards are based on this [awesome notebook](https://www.kaggle.com/allunia/pulmonary-dicom-preprocessing) by @allunia.\n# II. Convert to Hounsfield units\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"def load_scans(path): \n    slices = [pydicom.dcmread(path + '/' + file) for file in os.listdir(path)]\n    slices.sort(key = lambda x: float(x.ImagePositionPatient[2])) # sort by the z position of the image\n    return slices","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"example_scans = load_scans(train.dcm_path[0])\nplt.style.use('default')\nfig, ax = plt.subplots(1, 2, figsize=(13,5))\nn = 15\nfor i in range(n):\n    scan = example_scans[i]\n    image = scan.pixel_array.flatten()\n    # Transforming image data to Hounsfield unit for comparability\n    # since images from different CT system can have different measurements\n    rescaled_image = image * scan.RescaleSlope + scan.RescaleIntercept \n    \n    sns.distplot(image, ax=ax[0])\n    sns.distplot(rescaled_image, ax=ax[1])\n\nax[0].set_title(f'Raw image data distributions for {n} examples')\nax[1].set_title(f'Hounsfield unit distribution for {n} examples')\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Convert to Hounsfield unit\ndef to_hu(dicoms):\n    images = np.stack([image.pixel_array for image in dicoms])\n    images = images.astype(np.int16)\n    \n#     # Convert outside pixels (of circular images) to air's Hu value\n#     images[images <= -1000] = 0\n    \n    # Convert to HU\n    for i in range(len(images)):\n        intercept = dicoms[i].RescaleIntercept\n        slope = dicoms[i].RescaleSlope\n        if slope != 1:\n            images[i] = slope * images.astype(np.float64)\n            images[i] = images[i].astype(np.in16)\n        images[i] += np.int16(intercept)\n    # Convert outside pixels (of circular images) to air's Hu value\n    images[images <= -1000] = -1000\n    return images.astype(np.int16)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"hu_scans = to_hu(example_scans)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots(2, 2, figsize=(7,7))\nax[0, 0].set_title('Original CT Scan')\nax[0, 0].axis('off')\nax[0, 0].imshow(example_scans[0].pixel_array, cmap='gray')\nax[0, 1].set_title('Original pixel array\\' distribution')\nsns.distplot(example_scans[0].pixel_array.flatten(), ax=ax[0, 1])\n\nax[1, 0].set_title('CT Scan in Hu')\nax[1, 0].axis('off')\nax[1, 0].imshow(hu_scans[0], cmap='gray')\nax[1, 1].set_title('Hu values distribution')\nsns.distplot(hu_scans[0].flatten(), ax=ax[1, 1])\n\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# III. Voxel size & PixelSpacing\n### Pixel spacing & Slice thickness"},{"metadata":{"trusted":true},"cell_type":"code","source":"N = 200\ndef get_window_value(window):\n    if type(window) == pydicom.multival.MultiValue:\n        return np.int(window[0])\n    else:\n        return np.int(window)\n\npatient_id = []\nrow_values= []\ncol_values = []\npixelspacing_r = []\npixelspacing_c = []\nslice_thickness = []\npatient_pth = []\nwindow_widths = []\nwindow_centers = []\n\nnp.random.seed(713)\nids = np.random.randint(0, len(train), N)\npatients = train.loc[ids, 'SeriesInstanceUID']\nfor patient in tqdm(patients):\n    path = train[train.SeriesInstanceUID == patient].dcm_path.values[0]\n    patient_pth.append(path)\n    dcom_name = os.listdir(path)[0]\n    dcom_file = pydicom.dcmread(path + '/' + dcom_name)\n    \n    row_values.append(dcom_file.Rows)\n    col_values.append(dcom_file.Columns)\n    \n    pixelspacing_r.append(dcom_file.PixelSpacing[0])\n    pixelspacing_c.append(dcom_file.PixelSpacing[1])\n    \n    window_widths.append(get_window_value(dcom_file.WindowWidth))\n    window_centers.append(get_window_value(dcom_file.WindowCenter))\n    \n    slice_thickness.append(float(dcom_file.SliceThickness))\n\nexample_prop = pd.DataFrame()\nexample_prop['patients'] = patients\nexample_prop['row_values'] = row_values\nexample_prop['col_values'] =  col_values\nexample_prop['area'] = example_prop.row_values * example_prop.col_values\nexample_prop['pixelspacing_r'] = pixelspacing_r\nexample_prop['pixelspacing_c'] = pixelspacing_c\nexample_prop['slice_thickness'] = slice_thickness\nexample_prop['patient_pth'] = patient_pth\nexample_prop['window_width'] = window_widths\nexample_prop['window_center'] = window_centers\nexample_prop.head()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots(2, 2, figsize=(8, 6))\nsns.distplot(example_prop.pixelspacing_r, ax=ax[0, 0])\nax[0, 0].set_title('Pixel-spacing distribution\\nin row-direction', size=10)\nax[0, 0].set_xlabel('mm')\n\nsns.distplot(example_prop.pixelspacing_c, ax=ax[0, 1])\nax[0, 1].set_title('Pixel-spacing distribution\\nin column-direction', size=10)\nax[0, 1].set_xlabel('mm')\n\nreso = example_prop.groupby([\"row_values\", \"col_values\"]).size()\nreso = reso.reset_index(name='counts')\nsns.scatterplot(x=reso.col_values, y=reso.row_values, s=reso.counts, ax=ax[1, 0])\nax[1, 0].set_title('Pixel sizes of images\\' rows and columns', size=10)\nax[1, 0].set_xlabel('Column')\nax[1, 0].set_ylabel('Row')\n\nsns.distplot(example_prop.slice_thickness, ax=ax[1, 1], kde=False)\nax[1, 1].set_title('Slice thickness distribution', size=10)\nax[1, 1].set_xlabel('mm')\n\nplt.tight_layout()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## Physical distance & volume covered by a CT-scan"},{"metadata":{"trusted":true},"cell_type":"code","source":"example_prop['phys_distance_r'] = example_prop.pixelspacing_r * example_prop.row_values\nexample_prop['phys_distance_c'] = example_prop.pixelspacing_c * example_prop.col_values\nexample_prop['phys_area'] = example_prop.phys_distance_r * example_prop.phys_distance_c\nexample_prop['phys_vol'] = example_prop.phys_area * example_prop.slice_thickness\n\nfig, ax = plt.subplots(1, 2, figsize=(9,4))\nsns.distplot(example_prop.phys_area/1e2, ax=ax[0])\nax[0].set_title('Distribution of physical distance\\ncovered by a CT-slice', size=10)\nax[0].set_xlabel('cm^2')\n\nsns.distplot(example_prop.phys_vol/1e3, ax=ax[1])\nax[1].set_title('Distribution of physical volume\\ncovered by a CT-slice', size=10)\nax[1].set_xlabel('cm^3')\n\nplt.tight_layout()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"### CT-slices with biggest vs. smallest physical area/region\n#### Physical area"},{"metadata":{"trusted":true},"cell_type":"code","source":"import pydicom","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def custom_window(pixel_array, center, width):\n    pixel_array = np.array(pixel_array.copy())\n    lower_bound = center - width/2\n    upper_bound = center + width/2\n    pixel_array[pixel_array < lower_bound] = lower_bound\n    pixel_array[pixel_array > upper_bound] = upper_bound\n    return pixel_array\n\ndef plot_biggest_smallest(plot_area=True, center=-50, width=300): # if False then plot volume\n    if plot_area:\n        series = example_prop.phys_area\n    else:\n        series = example_prop.phys_vol\n        \n    biggest = example_prop[series == series.max()]\n    biggest = biggest.patient_pth.values[0]\n    biggest = pydicom.dcmread(biggest + '/' + os.listdir(biggest)[0])\n    \n\n    smallest = example_prop[series == series.min()]\n    smallest = smallest.patient_pth.values[0]\n    smallest = pydicom.dcmread(smallest + '/' + os.listdir(smallest)[0])\n    \n    biggest, smallest = to_hu([biggest, smallest])\n    biggest_window = custom_window(biggest, center, width)\n    smallest_window = custom_window(smallest, center, width)\n\n    fig, ax = plt.subplots(2, 2, figsize=(7,7))\n    title = 'area' if plot_area else 'volume'\n    ax[0, 0].imshow(biggest_window, cmap='gray')\n    ax[0, 0].set_title(f'CT-scan with biggest physical {title}', size=10)\n    ax[0, 0].axis('off')\n    \n    sns.distplot(biggest.flatten(), ax=ax[1,0])\n\n    ax[0, 1].imshow(smallest_window, cmap='gray')\n    ax[0, 1].set_title(f'CT-scan with smallest physical {title}', size=10)\n    ax[0, 1].axis('off')\n    \n    sns.distplot(smallest.flatten(), ax=ax[1,1])\n    \n    fig.suptitle('Image and distribution of Hu')\n    plt.tight_layout()\n    plt.show()\nplot_biggest_smallest()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"The scan with bigger area has lots of unnecessary space that can be cropped out."},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_biggest_smallest(False, 0, 300)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## 3D Reconstruction"},{"metadata":{"trusted":true},"cell_type":"code","source":"example_dcm = load_scans(example_prop.patient_pth.values[0])\nexample_imgs = to_hu(example_dcm)\n\nplt.figure(figsize=(5,5))\nplt.title('Distribution of pixels\\' Hu')\nsns.distplot(example_imgs.flatten(), norm_hist=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def plot_3d(image, threshold=-300, color='navy'):\n    # Position the scan upright, \n    # so the head of the patient would be at the top facing the camera\n    p = image.transpose(2,1,0)\n    \n    verts, faces, _, _ = measure.marching_cubes_lewiner(p, threshold)\n    \n    fig = plt.figure(figsize=(8,8))\n    ax = fig.add_subplot(1,1,1, projection='3d')\n    \n    # Fancy indexing: `verts[faces]` to generate a collection of triangles\n    mesh = Poly3DCollection(verts[faces], alpha=0.2)\n    mesh.set_facecolor(color)\n    ax.add_collection3d(mesh)\n    \n    ax.set_xlim(0, p.shape[0])\n    ax.set_ylim(0, p.shape[1])\n    ax.set_zlim(0, p.shape[2])\n    \n    plt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Resampling\nWhat is the purpose of resampling?<br>\n<br>\nA scan may have pixel spacing of [2.5, 0.5, 0.5], meaning each slice is 2.5mm apart and each pixel in a slice is 0.5mm-long. However, the spacing may vary from one scan to another. (**but what are the problems**, and **what are the benefits after converting scans to the same spacing?**)"},{"metadata":{"trusted":true},"cell_type":"code","source":"def resample(image, scan, new_spacing=[1,1,1]):\n    # Current spacing\n    spacing = np.array([scan[0].SliceThickness] + list(scan[0].PixelSpacing), dtype=np.float32)\n    \n    resize_factor = spacing / new_spacing\n    new_real_shape = image.shape * resize_factor\n    new_shape = new_real_shape.round()\n    real_resize_factor = new_shape / image.shape\n    new_spacing = spacing / real_resize_factor\n    \n    image = scipy.ndimage.zoom(image, real_resize_factor, mode='nearest')\n    \n    return image, new_spacing","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"new_example_imgs, new_imgs_spacing = resample(example_imgs, example_dcm)\nprint(example_imgs.shape)\nprint(new_example_imgs.shape)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Tissue segmentation"},{"metadata":{"trusted":true},"cell_type":"code","source":"def segment_lung_mask(images, threshold=-320, selem=disk(4)):\n    segmented = np.zeros(images.shape)\n    \n    for i, image in enumerate(images):\n        # Separate lung and air from the rest: lung/air=1, others=2\n        binary = np.array(image > threshold, dtype=np.int8) + 1\n        \n        # Segment using connected component analysis\n        labeling = measure.label(binary)\n        # Convert all edge-labels (labels of pixels on the furthest left/right/top/bottom)\n        # to others. These edge-labels should all be non-lung\n        bg_labelings = np.unique(\n            [labeling[0,:], labeling[-1,:], labeling[:,0], labeling[:,-1]]\n        )\n        for bg in bg_labelings:\n            binary[labeling == bg] = 2 # now lung=1, non-lung=2\n            \n        # Revert: lung=1, non-lung=0\n        binary -= 1\n        binary = 1 - binary\n        \n        # Remove air-pocket inside lung\n        if selem is not None:\n            binary = closing(binary, selem)\n        \n        segmented[i] = binary * image\n    \n    return segmented","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"segmented_example_imgs = segment_lung_mask(example_imgs)\nex_idx = 90\nc = -500\nw = 255\n\nfig, ax = plt.subplots(1,2, figsize=(9,4))\n\nax[0].imshow(custom_window(example_imgs[ex_idx], c, w),  cmap='gray')\nax[0].axis('off')\nax[0].set_title('Original image')\n\nax[1].imshow(custom_window(segmented_example_imgs[ex_idx], c, w), cmap='gray')\nax[1].axis('off')\nax[1].set_title('Segmented image')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"segmented_example_imgs.shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"study_columns = ['StudyInstanceUID', 'SeriesInstanceUID'\n                 , 'negative_exam_for_pe', 'rv_lv_ratio_gte_1', 'rv_lv_ratio_lt_1'\n                 ,'leftsided_pe', 'chronic_pe', 'true_filling_defect_not_pe'\n                 ,'rightsided_pe', 'acute_and_chronic_pe', 'central_pe', 'indeterminate']\ndiagnosis = ['negative_exam_for_pe', 'indeterminate', 'positive']\nrv_lv_ratio = ['rv_lv_ratio_gte_1', 'rv_lv_ratio_lt_1']\nleftsided_pe = ['leftsided_pe']\nrightsided_pe = ['rightsided_pe']\ncentral_pe = ['central_pe']\nchronic_acute = ['chronic_pe', 'acute_pe', 'acute_and_chronic_pe']\n\ntrain_study_level = train[study_columns + ['SOPInstanceUID']]\ntrain_study_level = train_study_level.groupby(study_columns).agg('count').reset_index()\ntrain_study_level.columns = study_columns + ['scan_count']\ntrain_study_level['positive'] = np.where((train_study_level.negative_exam_for_pe == 0)\n                                         & (train_study_level.indeterminate == 0), 1, 0)\ntrain_study_level['acute_pe'] = np.where((train_study_level.chronic_pe == 0)\n                                         & (train_study_level.positive == 1)\n                                         & (train_study_level.acute_and_chronic_pe == 0), 1, 0)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"study_count = len(train_study_level)\nlabel_group = [diagnosis, rv_lv_ratio, leftsided_pe\n               , rightsided_pe, central_pe, chronic_acute]\nfig, ax = plt.subplots(len(label_group), 1, figsize=(5, 20))\nmin_bin = train_study_level.scan_count.min()\nmax_bin = train_study_level.scan_count.max()\nplt.setp(ax, xlim=(min_bin, max_bin))\nfor i in range(len(label_group)):\n    labels = label_group[i]\n    for label in labels:\n        count = sum(train_study_level[label])\n        pct = round(count / study_count * 100, 2)\n        print(label, f': {count}/{study_count} ({pct}%)')\n    bins = np.linspace(min_bin, max_bin, 20)\n    for label in labels:\n        train_study_level[train_study_level[label]==1].hist(column='scan_count'\n                                                            , grid=False, bins=bins\n                                                            , ax=ax[i], label=label\n                                                            , alpha=0.5\n                                                            , density=True)\n    ax[i].legend()\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"!pip install iterative-stratification\nfrom iterstrat.ml_stratifiers import MultilabelStratifiedKFold\n\ncolumns = [c for cols in label_group for c in cols]\nkfolds = pd.DataFrame(columns = ['Fold', 'Size'] + columns)\nMSKF = MultilabelStratifiedKFold(n_splits=15)\nf = 0\nfor _, test_idx in MSKF.split(X=np.zeros(len(train_study_level)), y=train_study_level[columns]):\n    fold = train_study_level.iloc[test_idx, :]\n    size = len(fold)\n    row = [f, size]\n    for c in columns:\n        row.append(round(fold[c].sum()/size * 100,2))\n    kfolds.loc[f] = row\n    f += 1\n\nkfolds","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"plot_3d(segmented_example_imgs, threshold=-600)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}