Facebook recently released [1] an augmentation library, Augly, that combines several modalities (audio, image, video, and text) under the same roof. Data augmentation is quite a common technique for increasing both the size and the diversity of labeled training data which also helps to build robust models. Here I focus only on few of the augmentation functions I tested on an image but, this library can be used for text and audio too.
Augly is more dedicated towards transformations that happen in social media platforms, including Facebook. So apart from the usual crop, flip, other augmentation functions include very realistic transformations that real people do to image/video sharing platforms, like overlay image, emoji, text etc. One of the main important applications according to the Facebook press release is –
detecting exact copies or near duplicates of a particular piece of content. The same piece of misinformation, for example, can appear repeatedly in slightly different forms, such as as an image modified with a few pixels cropped, or augmented with a filter or new text overlaid. By augmenting AI models with AugLy data, they can learn to spot when someone is uploading content that is known to be infringing, such as a song or video.
Augly was developed to stop people from evading the automatic detection system by transforming (read ‘augmentation’) the data and, the same library was also used to evaluate deepfake detection models.
Image Augmentations with Augly:
Below are few working examples that I used to test some of the augmentation functions for images in Augly. To try this in Colab, first, we need to install Libmagic and Augly
!pip install augly
!apt-get install libmagic-dev
!pip install Python-magic
import augly.image as imaugs
Now, we are ready to use Augly but before that, we need to load an example image. Let’s do that
from skimage import data
camera = data.astronaut()
Since Augly works with the PIL format, let’s transform this –
from PIL import Image
camera_pil = Image.fromarray(camera)
camera_pil.show()
![Fig. 1: Original Image loaded using Skimage. [This image is available here and has no known copyright restrictions.]](https://towardsdatascience.com/wp-content/uploads/2021/06/1Zw2UV4cQoqK1lSBCMIJcpQ.png)
So, this is our original image. By the way, she’s Eileen Collins and was selected as an astronaut in 1992 and first piloted the space shuttle STS-63 in 1995.
First let’s get started with some relatively simple augmentation functions, like, changing perspective
aug_perspective = imaugs.PerspectiveTransform(sigma=20.0)
camera_pil_perspective = aug_perspective(camera_pil)
camera_pil_perspective.show()

Here sigma
refers to the standard deviation of the distribution of destination coordinates and, higher the sigma
value, more intense the transformation is.
We can also randomly change the aspect ratio as below –
aug_aspect = imaugs.RandomAspectRatio()
camera_pil_aspect = aug_aspect(camera_pil)
camera_pil_aspect.show()

We can also augment the image by adding some saturation –
aug_saturation = imaugs.Saturation(factor=5.0)
camera_pil_sat = aug_saturation(camera_pil)
camera_pil_sat.show()

Now let’s move towards some of the unique transformations that can be added using Augly. First I tried overlaying stripes and the transformed image is as below –

Code used for the above transformation is –
camera_pil_overlay_stripes = imaugs.overlay_stripes(camera_pil, line_type='dashed', line_opacity=0.5, line_color=(120, 0, 200), line_angle=25.0)
camera_pil_overlay_stripes.show()
What about overlaying texts ? That’s possible too –

We can also add emojis –
camera_pil_overlay_emoji = imaugs.overlay_emoji(camera_pil)
camera_pil_overlay_emoji.show()

We can also transform the image to make it look like a screenshot on Instagram –

Code block for this is as below –
import os
import augly.utils as utils
aug_overlay = imaugs.OverlayOntoScreenshot(template_filepath=os.path.join(utils.SCREENSHOT_TEMPLATES_DIR, "mobile.png"))
camera_pil_overlay = aug_overlay(camera_pil)
camera_pil_overlay.show()
By changing the overlay image mobile.png
to web.png
we can get another transformation –

I personally think, these kind of transformations are so cool and, as mentioned in the press-release by Facebook, that they are geared towards understanding human interactions with social media. I am sure people interested in speech and language processing will also find this library very useful.
Thanks to Facebook for making this library open source and to Joanna Bitton & Zoe Papkipos, who I believe are the main brains behind this project.
You can find all the codes used here in my notebook which I put in the references. Stay strong and cheers !!
P.S: One of the Co-Authors of Augly library Joanna B. left a very kind comment on this post. So, thanks to her ! As I mentioned in the post here is her GitHub.
References:
[1] Facebook Press-Release of Augly
[2] Augly : GitHub
[3] Link to My Notebook for Codes Used Here.