{"cells":[{"metadata":{},"cell_type":"markdown","source":"References: https://www.kaggle.com/xhlulu/panda-resize-and-save-train-data","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Pipeline Step 1 -  Pre-Processing:\n\nStep 1 Part A - Downsizing:\nEach of our training examples are huge, around 25,000 x 15,000 px. There is a lot of empty space in these training examples, and GPU usage is limited here on Kaggle. It will be useful to downsize the training images. We can then obtain several labeled tiles per image and train our model to output a Gleason grade per tile.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"! pip install imutils\nfrom imutils import resize as imut_resize","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"# let us now import some useful libraries\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport os # file directories\nimport openslide # accessing large images\nimport matplotlib.pyplot as plt # plotting figures\nfrom PIL import Image # open and display images\nimport cv2 #computer vision library\nfrom tqdm.notebook import tqdm # progress bar, tqdm shorthand for progress in Arabic \nimport skimage.io #image processing\nfrom skimage.transform import resize, rescale","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"# setting the main directory and loading the train CSV file\nMAIN_DIR = '../input/prostate-cancer-grade-assessment'\ntrain = pd.read_csv(os.path.join(MAIN_DIR, 'train.csv')).set_index('image_id')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# setting the directory where the images and masks are located\ndata_dir = os.path.join(MAIN_DIR, 'train_images/')\nmask_dir = os.path.join(MAIN_DIR, 'train_label_masks/')\nmask_files = os.listdir(mask_dir)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# set the path for the test image\n# we chose this image as it is not square and we want to make sure resizing does not distort non square images.\nimg_id = train.index[1106]\npath = data_dir + img_id + '.tiff'","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Check the time it takes to open the image with two methods\n%time biopsy = openslide.OpenSlide(path)\n%time biopsy_a = skimage.io.MultiImage(path)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Check the time it takes to resize an image and compare quality\n# note theses tiff files are multi-level images, there are three levels of differing quality and hence size. We select the lowest quality level for resizing.\n%timeit img_a = biopsy.get_thumbnail(size=(512, 512))\n%timeit img_b = resize(biopsy_a[-1], (512, 512))\n%timeit img_c = cv2.resize(biopsy_a[-1], (512, 512))\n%timeit img_d = Image.fromarray(biopsy_a[-1]).resize((512, 512))\n%timeit img_e = img_4 = imut_resize(biopsy_a[-1], width=512)\n\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We see that skimage is quickest to load an image and cv2 is fastest for resizing. We now check to ensure there is no difference in quality by resizing method.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"biopsy = openslide.OpenSlide(path)\nbiopsy_a = skimage.io.MultiImage(path)\nimg_0 = biopsy.get_thumbnail(size=(512, 512))\nimg_1 = resize(biopsy_a[-1], (512, 512))\nimg_2 = cv2.resize(biopsy_a[-1], (512, 512))\nimg_3 = Image.fromarray(biopsy_a[-1]).resize((512, 512))\nimg_4 = imut_resize(biopsy_a[-1], width=512)\n\nfig, axs = plt.subplots(3, 2, figsize=(15, 15))\nfig.suptitle('Ensuring no variance in image quality by resize method')\naxs[0,0].imshow(img_0)\naxs[0,1].imshow(img_1)\naxs[1,0].imshow(img_2)\naxs[1,1].imshow(img_3)\naxs[2,0].imshow(img_4)\nplt.show()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We see above the distortion some of the resize methods have upon our training slides. \n\nAlthough out imutils resize method is not the fastest, it it the quickest method which maintains aspect ratio and it is not extremely slow. We will use this method upon our training examples and masks.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"interpolations = [cv2.INTER_NEAREST, cv2.INTER_LINEAR, cv2.INTER_AREA, cv2.INTER_CUBIC, cv2.INTER_LANCZOS4]\nfig, axs = plt.subplots(3, 2, figsize=(15,15))\nfig.suptitle('Ensuring no variance in image quality by interpolation method')\nfor i in range(0,5):\n    axs[0,0].imshow(imut_resize(biopsy_a[-1], width=512, inter = 0))\n    axs[0,1].imshow(imut_resize(biopsy_a[-1], width=512, inter = 1))\n    axs[1,0].imshow(imut_resize(biopsy_a[-1], width=512, inter = 2))\n    axs[1,1].imshow(imut_resize(biopsy_a[-1], width=512, inter = 3))\n    axs[2,0].imshow(imut_resize(biopsy_a[-1], width=512, inter = 4))\nplt.show()\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Nearest Neighbor interpolation method seems to be best for definition. Let us now check times for saving our resized image.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"%timeit Image.fromarray(img_2).save(img_id + '.png')\n%timeit cv2.imwrite(img_id+'.png', img_2)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"cv2 is also the fastest method for saving. Let us now load our masks.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"mask = skimage.io.MultiImage(mask_dir + mask_files[1])\nimg = skimage.io.MultiImage(data_dir + mask_files[1].replace(\"_mask\", \"\"))\n# check the shapes of lowest resolution layer\nmask[-1].shape, img[-1].shape","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# we set our save directory\nsave_dir = \"../output/kaggle/train_images/\"\nos.makedirs(save_dir, exist_ok=True)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# we resize and save all our images, and use tqdm to give our progress\nfor img_id in tqdm(train.index):\n    load_path = data_dir + img_id + '.tiff'\n    save_path = save_dir + img_id + '.png'\n    \n    biopsy = skimage.io.MultiImage(load_path)\n    img = imut_resize(biopsy[-1], width=512, inter = 0)\n    cv2.imwrite(save_path, img)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# same for masks\nsave_mask_dir = '../output/kaggle/train_label_masks/'\nos.makedirs(save_mask_dir, exist_ok=True)\n\nfor mask_file in tqdm(mask_files):\n    load_path = mask_dir + mask_file\n    save_path = save_mask_dir + mask_file.replace('.tiff', '.png')\n    \n    mask = skimage.io.MultiImage(load_path)\n    img = imut_resize(mask[-1], width=512, inter = 0)\n    cv2.imwrite(save_path, img)","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}