{"cells":[{"metadata":{},"cell_type":"markdown","source":"References: https://www.kaggle.com/xhlulu/panda-resize-and-save-train-data","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Pipeline Step 1 -  Pre-Processing:\n\nStep 1 Part A - Downsizing:\nEach of our training examples are huge, around 25,000 x 15,000 px. There is a lot of empty space in these training examples, and GPU usage is limited here on Kaggle. It will be useful to downsize the training images. We can then obtain several labeled tiles per image and train our model to output a Gleason grade per tile.","execution_count":null},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# let us now import some useful libraries\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport os # file directories\nimport openslide # accessing large images\nimport matplotlib.pyplot as plt # plotting figures\nfrom PIL import Image # open and display images\nimport cv2 #computer vision library\nfrom tqdm.notebook import tqdm # progress bar, tqdm shorthand for progress in Arabic \nimport skimage.io #image processing\nfrom skimage.transform import resize, rescale","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"# setting the main directory and loading the train CSV file\nMAIN_DIR = '../input/prostate-cancer-grade-assessment'\ntrain = pd.read_csv(os.path.join(MAIN_DIR, 'train.csv')).set_index('image_id')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# setting the directory where the images and masks are located\ndata_dir = os.path.join(MAIN_DIR, 'train_images/')\nmask_dir = os.path.join(MAIN_DIR, 'train_label_masks/')\nmask_files = os.listdir(mask_dir)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# set the path for the first image\nimg_id = train.index[0]\npath = data_dir + img_id + '.tiff'","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Check the time it takes to open the image with two methods\n%time biopsy = openslide.OpenSlide(path)\n%time biopsy_a = skimage.io.MultiImage(path)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Check the time it takes to resize an image and compare quality\n# note theses tiff files are multi-level images, there are three levels of differing quality and hence size. We select the lowest quality level for resizing.\n%timeit img_a = biopsy.get_thumbnail(size=(512, 512))\n%timeit img_b = resize(biopsy_a[-1], (512, 512))\n%timeit img_c = cv2.resize(biopsy_a[-1], (512, 512))\n%timeit img_d = Image.fromarray(biopsy_a[-1]).resize((512, 512))\n\n\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We see that skimage is quickest to load an image and cv2 is fastest for resizing. We now check to ensure there is no difference in quality by resizing method.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"biopsy = openslide.OpenSlide(path)\nbiopsy_a = skimage.io.MultiImage(path)\nimg_0 = biopsy.get_thumbnail(size=(512, 512))\nimg_1 = resize(biopsy_a[-1], (512, 512))\nimg_2 = cv2.resize(biopsy_a[-1], (512, 512))\nimg_3 = Image.fromarray(biopsy_a[-1]).resize((512, 512))\n\nfig, axs = plt.subplots(2, 2, figsize=(15, 15))\nfig.suptitle('Ensuring no variance in image quality by resize method')\naxs[0,0].imshow(img_0)\naxs[0,1].imshow(img_1)\naxs[1,0].imshow(img_2)\naxs[1,1].imshow(img_3)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Since there appears to be no major difference across resizing methods we will go with the fastest cv2.resize. There are several interpolation methods for how cv2 resizes images let us quickly check these methods.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"interpolations = [cv2.INTER_NEAREST, cv2.INTER_LINEAR, cv2.INTER_AREA, cv2.INTER_CUBIC, cv2.INTER_LANCZOS4]\nfig, axs = plt.subplots(3, 2, figsize=(15,15))\nfig.suptitle('Ensuring no variance in image quality by interpolation method')\nfor i in range(0,5):\n    axs[0,0].imshow(cv2.resize(biopsy_a[-1], (512, 512), interpolation = cv2.INTER_NEAREST))\n    axs[0,1].imshow(cv2.resize(biopsy_a[-1], (512, 512), interpolation = cv2.INTER_LINEAR))\n    axs[1,0].imshow(cv2.resize(biopsy_a[-1], (512, 512), interpolation = cv2.INTER_AREA))\n    axs[1,1].imshow(cv2.resize(biopsy_a[-1], (512, 512), interpolation = cv2.INTER_CUBIC))\n    axs[2,0].imshow(cv2.resize(biopsy_a[-1], (512, 512), interpolation = cv2.INTER_LANCZOS4))\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"There is no glaring deviation in quality so we can continue with default interpolation method. Let us now check times for saving our resized image.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"%timeit Image.fromarray(img_2).save(img_id + '.png')\n%timeit cv2.imwrite(img_id+'.png', img_2)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"cv2 is also the fastest method for saving. Let us now load our masks.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"mask = skimage.io.MultiImage(mask_dir + mask_files[1])\nimg = skimage.io.MultiImage(data_dir + mask_files[1].replace(\"_mask\", \"\"))\n# check the shapes of lowest resolution layer\nmask[-1].shape, img[-1].shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# we set our save directory\nsave_dir = \"/kaggle/train_images/\"\nos.makedirs(save_dir, exist_ok=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# we resize and save all our images, and use tqdm to give our progress\nfor img_id in tqdm(train.index):\n    load_path = data_dir + img_id + '.tiff'\n    save_path = save_dir + img_id + '.png'\n    \n    biopsy = skimage.io.MultiImage(load_path)\n    img = cv2.resize(biopsy[-1], (512, 512))\n    cv2.imwrite(save_path, img)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# same for masks\nsave_mask_dir = '/kaggle/train_label_masks/'\nos.makedirs(save_mask_dir, exist_ok=True)\n\nfor mask_file in tqdm(mask_files):\n    load_path = mask_dir + mask_file\n    save_path = save_mask_dir + mask_file.replace('.tiff', '.png')\n    \n    mask = skimage.io.MultiImage(load_path)\n    img = cv2.resize(mask[-1], (512, 512))\n    cv2.imwrite(save_path, img)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}