The world’s leading publication for data science, AI, and ML professionals.

Hybrid Quantum Neural Network for reduced MNIST Data

Solving Fashion MNIST using Pennylane and Quantum Variational Circuits in less than 200 lines.

Logo picture for Quantum Fashion MNIST. Edited from Photo by Matthew Henry on Unsplash
Logo picture for Quantum Fashion MNIST. Edited from Photo by Matthew Henry on Unsplash

Quantum Machine Learning(QML) has been shown to be effective in machine learning problems with low input dimensionality. The Cartpole enviroment has been solved by Jerbi[1] and Owen[2]. Quantum convolutions have been used to solve a reduced version of the Mnist digit dataset[3].

Another solution based on dimensionality reduction algorithms has been proposed[4], but in this post, we will focus only on a Variational Quantum Circuit(VQC) based implementations. We will develop a Quantum Neural Network to classify the MNIST Fashion Dataset with a sligthly reduced input using Pennylane.

Introduction

The problem we will try to solve using QML is the classification of the MNIST Dataset using a VQC. Due to the time it takes to solve the whole problem, we will be using a dimension reduced dataset. The dimension we will try to fit in our VQC will be 64, instead of the original 784.

We will be working with Python code runned on a Colab machine. On this Colab, we will need to install Pennylane as a quantum machine learning library and we will use the alredy installed Tensorflow as an interface.

Data preparation

The first step towards or goal will be to prepare the libraries and load our beloved MNIST data.

After doing the setup, we will define the parameters we will be using for the model.

The filtering of the data is done by selecting the indexes of the categories that are in our list of interest. This piece of code is really ad hoc and I’m sure there is a better implementation.

Preprocessing with autoencoder

The model chosen to reduce the dimensions of the data is a really straightforward autoencoder model based on the Tensorflow’s own tutorials.[5]

We will be using a latent dimension size relative to the number of qubits. The number of layers and the number of neurons in each layer is up to the user.

Encoding the data is as simple as passing it to the encoder. Then, we will need to replace our outputs to a categorical value.

Quantum Neural Networks (QNN)

We will now prepare the quantum network to classify our fashion data. First, we will give a short explanation on VQC.

A Variational Quantum Circuit is a quantum circuit that depends on a number of free parameters which we can adjust manually or with learning algorithms to approximate a certain function. It can be viewed as a mathematical approximation model with learnable parameters. They work similarly to Neural Networks and their weights.

The Circuit

To work with quantum computers or simulators, we will need to import our libraries and the layers we will be working with.

The two main layers we will work with are:

  • MottonenStatePreparation: Deals with amplitude state preparation. Amplitude state preparation involves fixing a quantum state to a given vector with dimension equal to 2 to the power of the number of qubits.
  • StronglyEntanglingLayers: Quantum neural network gate which rotates each qubit and then entangles them all with CNOT’s in a circular way. This layers are similar to a dense layer between neurons, but they behave in a quantum way.

Note: The reason we are not using AmplitudeEmbedding in this circuit is due to a known issue in Pennylane, although when running an actual quantum computer, this algorithm would apply.

The model will use strongly entangled layers to approximate a function that minimizes a certain loss function. Besides, we will be using a technique called "Data re-uploading" which consists on sampling the input data several times into the network. This techniques is known to improve the circuit results.[6]

The QNN model we will be using consist of the quantum circuit defined above, followed by a dense layer to prepare the output.

Once the model is compiled, we will be ready to train the model with the data we selected before. Since the training can take up to 3 hours, but does not improve much after the first epoch, we will limit our training to only that first epoch. Further epochs may improve the system, but this is left as an exercise for the reader. 😉

Using this parameters, we get a categorical accuracy of 95%.

If we actually tried to use the same 6 qubits and the 10 classes, the training proccess would take much longer and the accuracy would fall down to a modest 70%. Although this isn’t as impresive as the previus 95% with the 4 labels, it shows that these models do indeed have the ability to solve real problems without too much classical processing.

Let’s see some examples of the 10 labels QNN! Warning, it didn’t do great in all of them.

Example 1: Pair of troursers

Trouser image of the MNIST Dataset
Trouser image of the MNIST Dataset
Previous picture category (left) and picture prediction (right)
Previous picture category (left) and picture prediction (right)

The QNN can clearly detect that the picture is a trouser.

Example 2: Failing example

Shirt image of the MNIST Dataset
Shirt image of the MNIST Dataset
Previous picture category (left) and picture prediction (right)
Previous picture category (left) and picture prediction (right)

This example seems hard for our network, as it is printing 5 possible categories. Something interesting emerges when looking at the categories predicted: the labels detected were all the shirt type labels, while all shoe and trousers ones where discarded!

Example 3: Shoes are weird

Shoe image of the MNIST Dataset
Shoe image of the MNIST Dataset
Previous picture category (left) and picture prediction (right)
Previous picture category (left) and picture prediction (right)

Some shoes do have better results than others, but overall, they tend to cluster in the shoe labels.

Conclusions

As we have just seen, Quantum Neural Networks are much closer to real life than we think, with some practical cases already implemented in introductory environments and datasets.

In this post, we have proven that Fashion MNIST data can be classified using a QNN with good results on slightly reduced dimensional data. We have achived a 95% accuracy with 4 labels on a 64 dimensional data and a 70% accuracy with the 10 labels. With this as a first step, we can start imagining a near future where systems with only 10 qubits will be able to classify the whole dataset without reduced data!

References

[1] S. Jerbi, Variational Quantum Policies for Reinforcement Learning (2021), ArXiV [2] O. Lockwood and M. Si, Reinforcement Learning with Quantum Variational Circuits (2020), ArXiV [3] E. Farhi and H. Neven, Classification with Quantum Neural Networks on Near Term Processors (2018), ArXiV [4] I. Kerenidis and A. Luongo, Classification of MNIST dataset with Quantum Slow Feature Analysis (2020), ArXiV [5] TensorFlow Core, Introduction to Autoencoders (2021), Tensorflow Core [6] A. Perez, A. Cervera, E. Gil, José L., Data re-uploading for a universal quantum classifier (2020), ArXiV


Related Articles