The world’s leading publication for data science, AI, and ML professionals.

Facebook Just Launched the Coolest Augmentation Library: Augly

Image Augmentation with Augly

Left: Original Image, Right: After augmentation with Augly (Image Created by Author)
Left: Original Image, Right: After augmentation with Augly (Image Created by Author)

Facebook recently released [1] an augmentation library, Augly, that combines several modalities (audio, image, video, and text) under the same roof. Data augmentation is quite a common technique for increasing both the size and the diversity of labeled training data which also helps to build robust models. Here I focus only on few of the augmentation functions I tested on an image but, this library can be used for text and audio too.

Augly is more dedicated towards transformations that happen in social media platforms, including Facebook. So apart from the usual crop, flip, other augmentation functions include very realistic transformations that real people do to image/video sharing platforms, like overlay image, emoji, text etc. One of the main important applications according to the Facebook press release is –

detecting exact copies or near duplicates of a particular piece of content. The same piece of misinformation, for example, can appear repeatedly in slightly different forms, such as as an image modified with a few pixels cropped, or augmented with a filter or new text overlaid. By augmenting AI models with AugLy data, they can learn to spot when someone is uploading content that is known to be infringing, such as a song or video.

Augly was developed to stop people from evading the automatic detection system by transforming (read ‘augmentation’) the data and, the same library was also used to evaluate deepfake detection models.


Image Augmentations with Augly:

Below are few working examples that I used to test some of the augmentation functions for images in Augly. To try this in Colab, first, we need to install Libmagic and Augly

!pip install augly
!apt-get install libmagic-dev
!pip install Python-magic
import augly.image as imaugs

Now, we are ready to use Augly but before that, we need to load an example image. Let’s do that

from skimage import data
camera = data.astronaut()

Since Augly works with the PIL format, let’s transform this –

from PIL import Image
camera_pil = Image.fromarray(camera)
camera_pil.show()
Fig. 1: Original Image loaded using Skimage. [This image is available here and has no known copyright restrictions.]
Fig. 1: Original Image loaded using Skimage. [This image is available here and has no known copyright restrictions.]

So, this is our original image. By the way, she’s Eileen Collins and was selected as an astronaut in 1992 and first piloted the space shuttle STS-63 in 1995.

First let’s get started with some relatively simple augmentation functions, like, changing perspective

aug_perspective = imaugs.PerspectiveTransform(sigma=20.0)
camera_pil_perspective = aug_perspective(camera_pil)
camera_pil_perspective.show()
Perspective Transform of Fig. 1 (Image created by Author)
Perspective Transform of Fig. 1 (Image created by Author)

Here sigma refers to the standard deviation of the distribution of destination coordinates and, higher the sigma value, more intense the transformation is.

We can also randomly change the aspect ratio as below –

aug_aspect = imaugs.RandomAspectRatio()
camera_pil_aspect = aug_aspect(camera_pil)
camera_pil_aspect.show()
Change Aspect Ratio randomly on Fig. 1 (Image created by Author)
Change Aspect Ratio randomly on Fig. 1 (Image created by Author)

We can also augment the image by adding some saturation –

aug_saturation = imaugs.Saturation(factor=5.0)
camera_pil_sat = aug_saturation(camera_pil)
camera_pil_sat.show()
Increasing saturation of original image in Fig. 1 (Image created by Author)
Increasing saturation of original image in Fig. 1 (Image created by Author)

Now let’s move towards some of the unique transformations that can be added using Augly. First I tried overlaying stripes and the transformed image is as below –

Overlay stripes on the original fig. 1 (Image created by Author).
Overlay stripes on the original fig. 1 (Image created by Author).

Code used for the above transformation is –

camera_pil_overlay_stripes = imaugs.overlay_stripes(camera_pil, line_type='dashed', line_opacity=0.5, line_color=(120, 0, 200), line_angle=25.0)
camera_pil_overlay_stripes.show()

What about overlaying texts ? That’s possible too –

Overlay meme text on original Fig. 1 (Image created by Author)
Overlay meme text on original Fig. 1 (Image created by Author)

We can also add emojis –

camera_pil_overlay_emoji = imaugs.overlay_emoji(camera_pil)
camera_pil_overlay_emoji.show()
Overlay emoji on original Fig. 1 (Image created by Author).
Overlay emoji on original Fig. 1 (Image created by Author).

We can also transform the image to make it look like a screenshot on Instagram –

Overlay Instagram Screen-Shot on original Fig. 1 (Image created by Author).
Overlay Instagram Screen-Shot on original Fig. 1 (Image created by Author).

Code block for this is as below –

import os
import augly.utils as utils
aug_overlay = imaugs.OverlayOntoScreenshot(template_filepath=os.path.join(utils.SCREENSHOT_TEMPLATES_DIR, "mobile.png"))
camera_pil_overlay = aug_overlay(camera_pil)
camera_pil_overlay.show()

By changing the overlay image mobile.png to web.png we can get another transformation –

Overlay Screen-Shot of Instagram web-page on original Fig. 1 (Image created by Author).
Overlay Screen-Shot of Instagram web-page on original Fig. 1 (Image created by Author).

I personally think, these kind of transformations are so cool and, as mentioned in the press-release by Facebook, that they are geared towards understanding human interactions with social media. I am sure people interested in speech and language processing will also find this library very useful.

Thanks to Facebook for making this library open source and to Joanna Bitton & Zoe Papkipos, who I believe are the main brains behind this project.

You can find all the codes used here in my notebook which I put in the references. Stay strong and cheers !!

P.S: One of the Co-Authors of Augly library Joanna B. left a very kind comment on this post. So, thanks to her ! As I mentioned in the post here is her GitHub.


References:

[1] Facebook Press-Release of Augly

[2] Augly : GitHub

[3] Link to My Notebook for Codes Used Here.


Related Articles