The world’s leading publication for data science, AI, and ML professionals.

When crypto meets ML

Applying a mean filter over an encrypted image (with complete Colab notebook)

Photo by Jason Blackeye on Unsplash
Photo by Jason Blackeye on Unsplash

In this post we will see how homomorphic encryption can be used to perform a mean filter over an encrypted image [1][2]. Why would anyone do that? Well, as long as you do the processing locally, encryption doesn’t make sense, but if you want to use an online service, are you sure you want to give your personal images to a remote web server? If your answer is no, keep reading this post.

Why mean filter?

A mean filter is used to create a blur effect. In some situations, this effect helps eliminate the noise from an image. There are many web services that allow you to apply a mean filter over an image [3][4]. All of these services work according to the following steps:

  1. You upload the image on the web server
  2. The image is process on the server (the mean filter is applied)
  3. You download the processed image returned by the server

The main problem of this approach is privacy: you give your personal image to a remote server you know nothing about! To solve this problem we add two more steps to the above procedure:

  1. You encrypt the image using your public key
  2. You upload the encrypted image on the web server
  3. The server processes the encrypted image
  4. You download the encrypted result returned by the server
  5. Decrypt what you downloaded

All the time the image is on the server it is encrypted using your public key. In order for the server to be able to decrypt it, it needs your private key, which only you know 😎

Behind the scene

To encrypt the image, we will used Paillier cryptosystem [5]. The scheme is homomorphic with respect to addition and constant multiplication. What does this mean for the Paillier scheme? Homomorphic with respect to addition means that, if c1 is the ciphertext of m1 and c2 the ciphertext of m2 then c1*c2 is the ciphertext of _m1+m2._Homomorphic with respect to constant multiplication means that, if c is the ciphertext of m and k is a constant known by everyone then c^k is the encryption of c*k. Since division is just multiplication we can also make ho_momorphic d_ivision with a constant. If you want to read more about Paillier crypto scheme check this beautiful explanation. For this post, we will not need all the details of the scheme, this being already implemented in phe library [6].

The main idea

To apply a mean filter of dimension kxk we simply replace each current pixel with the mean of all pixels from a kxk square that has the current pixel in the center. So, the core of a mean filter consists of a sum followed by a division at a constant value (, known to everyone). This sounds like a perfect application for Paillier encryption 😁
If we encrypt the image using Paillier scheme, we can calculate the sums directly over the encrypted pixels. Then if we divide the encrypted sum to a constant (which is the size of the filter) we will get the encrypted mean of the pixels.

The server will never need to know the original image to apply the mean filter.

If you are looking for a complete implementation with further explanations, here, I’ve prepared a Colab notebook for you ready to run on your browser 😉

References

[1] https://medium.com/privacy-preserving-natural-language-processing/homomorphic-encryption-for-beginners-a-practical-guide-part-1-b8f26d03a98a

[2] https://homepages.inf.ed.ac.uk/rbf/HIPR2/mean.htm

[3] https://www10.lunapic.com/editor/?action=blur

[4] https://pinetools.com/blur-image

[5] https://medium.com/coinmonks/paillier-shines-a-light-on-a-new-world-of-computing-15c5aceed3ab

[6] https://pypi.org/project/phe/


Related Articles