Increasing the amount and diversity of data using scikit-image in Python

Making Sense of Big Data

Deep learning techniques have found great success in computer vision tasks, namely image recognition, image segmentation, object detection, etc. Such deep learning techniques are heavily dependent on big data to avoid overfitting. So what do you do when you have limited data? You go for Data Augmentation. Data Augmentation techniques enhance training datasets’ size and quality such that better Deep Learning models can be built using them. In this article, we shall look at some of the standard image augmentation techniques and some popular libraries that help achieve these augmentations.

Types of Augmentations Techniques

Augmentations can be broadly classified as, i.e., Offline and Online Augmentations based on when they are performed in the pipeline.

Commonly used Augmentation Techniques.

Before jumping into the augmentation libraries, let’s first have a look at some of the image augmentations technique:

Flipping means flipping the image horizontally or vertically.
Rotation means to rotate the image by a given angle in the clockwise or anticlockwise direction.
Cropping involves ** sampling** a section of the image randomly.
Altering Brightness involves increasing or decreasing the brightness of the image.
Scaling – Images can be scaled outward or inward. When scaled outward, the image size increases while the image size decreases when scaled inwards.
Adding Noise means adding Gaussian noise to the existing images.

Exploring the dataset: Images as Arrays

Unlike humans, computers perceive images as data points. An image is nothing but a standard Numpy array containing pixels of data points. More the number of pixels in an image, the better is its resolution.

Keeping this very fact in mind, let’s explore the dataset we shall be working in this article. The dataset belongs to the Plant Pathology 2020 – FGVC7 hosted by Kaggle in March 2020 and consists of manually captured 3,651 high-quality, real-life symptom images of multiple apple foliar diseases.

Given a photo of an apple leaf, can you accurately assess its health? This competition will challenge you to distinguish between leaves which are healthy, those which are infected with apple rust, those that have apple scab, and those with more than one disease.

We’ll only use the images from the training dataset for this article.

Importing necessary libraries and the dataset

import numpy as np
import pandas as pd
from skimage.io import imshow, imread, imsave
import matplotlib.pyplot as plt
%matplotlib inline


# Defining data path
Image_Data_Path = "./plant-pathology-2020-fgvc7/images/"

train_data = pd.read_csv("./plant-pathology-2020-fgvc7/train.csv")

# Loading the training images 
def load_image(image_id):
    file_path = image_id + ".jpg"
    image = imread(Image_Data_Path + file_path)
    return image

train_images = train_data["image_id"][:50].apply(load_image)

After having uploaded the dataset, let’s look at some of the images in the training set

A look at some of the images in the training set

# plotting multiple images using subplots
fig,ax = plt.subplots(nrows=2,ncols=3,figsize=(30,16))
for col in range(3):
    for row in range(2):
        ax[row,col].imshow(train_images.loc[train_images.index[row*3+col]])
        #ax[row,col].set_title(image_titles[i])    
        ax[row,col].set_xticks([])
        ax[row,col].set_yticks([])

You can think of pixels to be tiny blocks of information arranged in the form of a 2 D grid, and the depth of a pixel refers to the color information present in it. Let’s look at an arbitrary image in the dataset.

image = train_images[15]
imshow(image)
print(image.shape)

Colored images are represented as a combination of Red, Blue, and Green, and all the other colors can be achieved by mixing these primary colors in the correct proportions.

# red filter [R,G,B]
red_filter = [1,0,0]
# blue filter
blue_filter = [0,0,1]
# green filter
green_filter = [0,1,0]

# matplotlib code to display
fig,ax = plt.subplots(nrows=1,ncols=3,figsize=(30,16))
ax[0].imshow(image*red_filter)
ax[0].set_title("Red Filter",fontweight="bold", size=30)
ax[1].imshow(image*blue_filter)
ax[1].set_title("BLue Filter",fontweight="bold", size=30)
ax[2].imshow(image*green_filter)
ax[2].set_title("Green Filter",fontweight="bold", size=30);

A grayscale image consists of 8 bits per pixel. This means it can have 256 different shades where 0 pixels will represent black color while 255 denotes white. For example, the image below shows a grayscale image represented in the form of an array. A grayscale image has only 1 channel where the channel represents a dimension.

# import color sub-module
from skimage import color

# converting image to grayscale
grayscale_image = color.rgb2gray(image)
grayscale_image.shape
imshow(grayscale_image)

Popular Image Augmentation packages

There are a lot of image augmentations packages. However, in this series, we shall cover some of the popular ones like:

skimage
OpenCV
imgaug
Albumentations
Keras(ImageDataGenerator class)

We shall use the above five libraries to demonstrate the basic augmentation techniques with code examples. Each library is pretty vast and deserves an article of its own. Hence, I’ll dedicate an article to each of the libraries starting with the well-known scikit-image library.

Data Augmentation using scikit image

Scikit-Image or skimage is an open-source Python package that works with numpy arrays. It is a collection of algorithms for Image Processing. The code is maintained by a team of collaborators and is completely community-driven while maintaining the quality of the code and promoting research-based education.

Installation

scikit-image can be installed as follows:

pip install scikit-image

or

conda install -c conda-forge scikit-image

Importing the library and necessary modules

Let’s load the requisite modules that we shall use for the various image manipulation operations. The skimage library consists of various modules containing functions like rotate, rescale, resize, etc, which are used for the manipulation of images.

from skimage.io import imshow, imread, imsave
from skimage.transform import rotate,rescale, resize, 
from skimage import color,data
from skimage.exposure import adjust_gamma
from skimage.util import random_noise

Image Flipping

Flipping can be both horizontally as well as vertically. The function used to flip an image,, or technically an array is [numpy.flip()](https://numpy.org/doc/stable/reference/generated/numpy.flip.html). There are two variations of this function. While the fliplr() function is used to flip any array in the left or right direction, the flipud() function flips an array in the up/down direction. This implies fliplr() flips the order of columns of pixels in the matrix while flipu() reverses the order of rows of pixels in the matrix.

#Horizontally flipped
hflipped_image= np.fliplr(image) #fliplr reverses the order of columns of pixels in matrix

#Vertically flipped
vflipped_image= np.flipud(image) #flipud reverses the order of rows of pixels in matrix

fig,ax = plt.subplots(nrows=1,ncols=3,figsize=(30,16))
ax[0].imshow(image)
ax[0].set_title("Original Image", size=30)
ax[1].imshow(hflipped_image)
ax[1].set_title("Horizontally flipped", size=30)
ax[2].imshow(vflipped_image)
ax[2].set_title("Vertically flipped", size=30);

Image Rotation

Similarly, we can rotate the image in the clockwise direction or an anti-clockwise direction. The rotation is made possible by transform.rotate(); a function that rotates an image by a certain angle around its center. The user can specify any rotation angle of choice.

# clockwise rotation
rot_clockwise_image = rotate(image, angle=45)

# Anticlockwise rotation
rot_anticlockwise_image = rotate(image, angle=-45)

fig,ax = plt.subplots(nrows=1,ncols=3,figsize=(30,16))
ax[0].imshow(image)
ax[0].set_title("Original Image", size=30)
ax[1].imshow(rot_clockwise_image)
ax[1].set_title("+45 degree Rotation", size=30)
ax[2].imshow(rot_anticlockwise_image)
ax[2].set_title("-45 degree rotation", size=30);

Image Cropping

For cropping an image, you can define a utility function – randRange(); to generate random float values in the desired range. You can then pass these generated range values into another function called randomCrop(), which takes the image as input and crops it in the center from a desired margin from the borders

P.S: The code below has been adapted from this Kaggle kernel.

# source: https://www.kaggle.com/safavieh/image-augmentation-using-skimage
import random
import pylab as pl 
def randRange(a, b):
    '''
    a utility function to generate random float values in desired    range
    '''
    return pl.rand() * (b - a) + a

def randomCrop(im):
    '''
    croping the image in the center from a random margin from the borders
    '''
    margin = 1/3.5
    start = [int(randRange(0, im.shape[0] * margin)),
             int(randRange(0, im.shape[1] * margin))]
    end = [int(randRange(im.shape[0] * (1-margin), im.shape[0])), 
           int(randRange(im.shape[1] * (1-margin), im.shape[1]))]
    cropped_image = (im[start[0]:end[0], start[1]:end[1]])
    return cropped_image


fig,ax = plt.subplots(nrows=1,ncols=2,figsize=(20,12))
ax[0].imshow(image)
ax[0].set_title("Original Image", size=20)
ax[1].imshow(randomCrop(image))
ax[1].set_title("Cropped", size=20)

Brightness Manipulation

The exposure module is used for brightness manipulation. The function adjust_gammaperforms Gamma Correction on the input image. The higher the value of gamma, the darker is the resultant image.

image_bright = adjust_gamma(image, gamma=0.5,gain=1)
image_dark = adjust_gamma(image, gamma=2,gain=1)

fig,ax = plt.subplots(nrows=1,ncols=3,figsize=(20,12))
ax[0].imshow(image)
ax[0].set_title("Original Image", size=20)
ax[1].imshow(image_bright)
ax[1].set_title("Brightened Image", size=20)
ax[2].imshow(image_dark)
ax[2].set_title("Darkened Image", size=20)

Image Resizing & Scaling

The transform.resize()function transforms a given image to match a specific size. This function can either interpolate to up-size or down-size an image. Make sure to enable anti-aliasing should when down-sizing images to avoid aliasing artifacts. Anti–aliasing is the smoothing of jagged edges in digital images.

image_resized = resize(image, (image.shape[0] // 2, image.shape[1] // 2),anti_aliasing=True)

fig, ax = plt.subplots(nrows=1, ncols=2,figsize=(30,16))
ax[0].imshow(image)
ax[0].set_title("Original Image", size=20)
ax[1].imshow(image_resized)
ax[1].set_title("Resized image",size=20)

The resize() function also serves the same purpose except that it allows specifying an output image shape instead of a scaling factor.

Adding Noise

The random_noise() is a function from the skimage’s util’s module. It is used to add random noise of various types to an image. Noise is a kind of imperfection that is added to images to introduce some kind of variability.

noisy_image= random_noise(image)

fig, ax = plt.subplots(nrows=1, ncols=2,figsize=(30,16))
ax[0].imshow(image)
ax[0].set_title("Original Image", size=20)
ax[1].imshow(noisy_image)
ax[1].set_title("Image after adding noise",size=20)

Conclusion & Next Steps

Data augmentation has been shown to produce promising ways to increase the accuracy of classification tasks. It is especially important in situations of limited data availability. Python provides a robust library in the form of a scikit-image with many algorithms for image augmentation. It is available free of charge and free of restriction, having an active community behind it. Have a look at the documentation to learn more about the library and its use cases. In the next few articles, we will discuss other commonly used augmentation libraries. Stay tuned!