TensorFlow and Image Augmentation
Yes, I know. You have already heard about image augmentation a lot. But guess what! I would like to surprise you anyway. Let’s dive in!
Augmentation is a well known technique for preventing overfitting of deep neural networks. If you do it well, you can multiply your dataset without the high cost of collecting and annotating new data. There are many libraries for this purpose in Python, but I would like to talk mainly about TensorFlow.
Quick Overview
For the best performance, you should consider using tf.data API. From the first glance, it seems very nice. Simple to use, effective. A bit later on, you start wondering about image augmentations. How do I fit them in?
If you are used to the old and great libraries like imgaug, albumentations or even THE OpenCV, you need a help of tf.py_function. But what about some purely TensorFlow solutions? Some which could be compiled and run as a graph?
tf.image
This is a first place where you would probably look. At least I did. There are some pretty useful functions, for example:
- Image Adjustment — brightness, contrast, saturation, JPEG quality etc.
- Cropping.
- Flipping, Rotating and Transposing —flip left/right, up/down, rotate 90 degrees. Unfortunately not the full rotation.
tfa.image
Image package from TensorFlow Addons is another package you should regularly check. Not only for augmentations, there are additional layers, losses, optimizer and so on. But now back on the track. What is there for us? Mainly rotate, shear and general translate and transform.
Image Detection?
Unfortunately, none of those two takes any interest in image detection. Yes, you can flip the image. But what about the selected bounding boxes?! I guess you do not want to loose them!
Our Library — tf-image
If you are eager to check the library yourself, please visit our GitHub repository. All is open source. We have only just began so we welcome any feedback on what you like (do not like) or what you need. If you find it useful, we would greatly appreciate your code contribution!
Different Workflows
This blog post is meant to be an introduction of the library, not the full manual or description of every single function. Therefore I will limit myself to two examples only.
All in One
To make augmentation as simple as possible, there is the random_augmentations function. You provide image, augmentation setup and optionally bounding boxes. The rest is on the function itself. In addition, the augmentations are performed in a random order to make the process even more powerful.
Cool, but limiting in some ways. Mainly, this setup is too simple. There are many parameters for each of the options. In the future we will come with a more advanced setup. But for now, you can easily create your own augmentation function using all the rest of our library.
Using the Core Functions
Using core function is easy as well. I have demonstrated below one example in order to present you the random_function (random_function_bboxes). If you do not want to apply your augmentation every time, this is a place to look.
Conclusion
That was all for today. Thank you if you have gotten this far. Please, comment, criticize, share, send merge request!
I would like to thank Michal Lukac for working with me on this project, Ximilar for giving me the opportunity and a positive approach towards open-sourcing some pieces of our work.
