Images Generation with Neural Style Transfer and Tensorflow
Creation of unique images using machine learning algorithms
3 Minutes Machine Learning
3 minutes machine learning is a series of tutorials and informative videos related to the world of machine learning. You can find the complete videos on YouTube. The repository contains all the files shown in the videos. This series is in an experimental phase and is developed totally free of charge, for fun and culture! Any feedback is welcome.
Why read this article?
This article will see how to create unique and original images by exploiting an interesting machine learning algorithm: neural style transfer. We’ll see how to upload already trained neural network models to speed up the process, and how to upload your own photos to test the algorithm and generate your own images.
Here’s a preview of what our final result will look like:
Neural Style Transfer
Neural Style Transfer is an optimisation technique. Its implementation involves the use of two images: one representing the main content (content image) and one representing the style (style reference image) representative of the image we wish to obtain. The algorithm mixes these images, taking the content image and applying the style reference image to it. This is done by optimizing the final image to match the content statistics of the content image and the style reference image's style statistics. The implementation of the algorithm involves the use of convolutional networks.
For more information, please refer to the original article or this interesting tutorial by Tensorflow.
The Code
The code requires the import of several libraries and the definition of some environment variables. Matplotlib is used for image plotting, Tensorflow and Keras for surrogate model management and image preprocessing, the os system library for defining some system variables, NumPy as a basic tool for numerical analysis. We import some tools directly from Google Colab to facilitate the import of images directly from your PC.
All that remains is to import the images on which we want to perform the analysis. Remember that the first one will be the base image, which will assign the main theme to the final result (content image). The second one will assign the style (style reference image). This code is very useful because it allows us, in a few lines, to upload image files directly from our local storage space, while also creating an interactive interface with the user in the case of code sharing (as in this case). We note that, as written, this code allows several images to be imported simultaneously, but in this case, we only need two images: only the last one will be stored. Images are automatically uploaded and saved in your workspace.
Let us define some useful functions:
tensor_to_image allows a TensorFlow Tensor to be converted into an image file, representable as such. load_img loads an image from the desired path. imshow allows us to visualize an image, paying attention to the size of the input file.
Let’s display the images:
Let’s create our Neural Style Transfer model:
To exploit the potential of Tensorflow Hub, all that is needed is:
- load the pre-trained model from Tensorflow Hub
- generate the new stylized image passing our two images to the model as constant Tensorflow tensors
- convert the new image to an image
- let’s visualize the result!
Let’s add some Python commands to automatically name and download the new image:
Conclusions
I hope you enjoy the content of this article. I remind you that you can find the source code here and the complete repository on GitHub and YouTube.
Until next time,
Marco
