How to Check Profile Picture Uploads Automatically
…using existing AI APIs
Introduction
A profile picture upload functionality is widely used in a lot of applications. It allows the user to customize his profile and allows easy identification of them. In some use cases it might be helpful to show some warnings or restrict the user to upload a picture which is inappropriate. It might be the case that it’s not allowed/useful to upload a picture which is really blurry or on which headgear is worn. To implement a feature like this it would be really cumbersome to check the images manually. This is where face recognition could help to improve the process of uploading a profile picture.
I will use the Google Cloud Vision API to implement this feature. Google provides a lot of Machine Learning APIs which can be used easily. The costs to use such endpoints might vary in the future, so please look it up yourself if you are interested. Right now you have 1000 requests free for the beginning, so it’s for free to try it out.
Setup Google Cloud Vision API
At first you should create yourself an account for the Google Cloud.
Afterwards you can login into the Google Cloud Console.
In the Google Cloud Console you have to perform certain tasks to successfully make requests to the API.
In order to archive this you can also use the Quickstart Setup.
The necessary steps are:
- Create a new project
- Activate the Google Cloud Vision API
- Enable billing (no worries, there is a free plan)
But it really needs to be enabled, otherwise it won’t work. - Setup authentication
Feel free to use the recommended ways from Google using service accounts and generate private and public keys. For me it was much easier to generate an API key and just use this in my requests. But keep in mind that it might be less secure than the other methods. Depending on your use case it can be enough to secure the API key for certain IP-addresses. So if you just want to try out the API an API key should be enough.
Testing the Face Recognition Request Results
In order to test the face recognition API I did some requests manually. I tested it with different pictures to see how the results vary. The request should look like so:
The API key is provided in the request URL itself. The request method is POST because we want to send something to the endpoint. The request body has to follow this given format, because you can also send multiple images in one request. But we will not use this functionality here.
You also have to define which features of the API should be used. For this use case it is the “FACE_DETECTION” feature. The maxResults value is unnecessary here because we are only sending one image.
For the following image I would expect to get the information that headwear is worn.
When successfully calling the face recognition API you get a result that looks like the one shown below. I erased some data from it to keep it simple. For example you get a lot of data with the location of the nose or the eyes, but we will not need this information for now.
So as you can see we get some information about emotions (joy, sorrow, anger, surprise) and image properties (under exposed, blurred and headwear). The value shows how likely the different parameters are for the given picture. In this example all parameters are very unlikely except the headwearLikelihood, which is “LIKELY”. For this result it could make sense to display some warning if it’s not optimal to wear headwear for this profile picture upload.
The Likelihood ratings can have 6 different values: UNKNOWN, VERY_UNLIKELY, UNLIKELY, POSSIBLE, LIKELY, VERY_LIKELY
Setup a basic React JS Project
To actually use the API you can use everything what can do a HTTP request. It can be a Web Application or a mobile Application for example. I created a basic Web Application using React-JS.
You can go ahead and have a look at the GitHub Repository where my implementation is provided. Feel free to use it to test the API yourself. (Don’t forget to change the API key).
To create a react project yourself I suggest to follow the steps from the official React documentation. Additionally I installed Material UI as an UI framework so that the application looks good out of the box.
Implementation of the use case
To create an example use case I started with implementing 3 components which will work together to create the ProfilePictureUploader component.
The component consists of:
- ImagePreview
- ImageWarning
- ImageSelection
Preview
The ImagePreview is basically an <img/> tag that waits for image information. If the information is updated it will rerender with the new image inside. I did some restrictions on the height and added some additional styling to make it look better.
Warnings
The ImageWarning component consumes the result of the API call and decides to show warnings or not. The Component also gets the information if the API call is currently pending to display a loading indicator. In this example I decided to display warnings if the anger, blurred, or headwear parameter is not “VERY_UNLIKELY”. If there is a warning a little icon and a text is generated to inform the user. You can implement other logics suiting your needs for sure.
Input and Submit
The ImageSelection Component consists of a File input and a submit button. The file input is used to get the image from the user and the submit button is currently without functionality, it’s only there to show how you could proceed with the profile image. For example on hitting submit it could be uploaded to your server.
Profile Picture Uploader
The three components are used in one wrapper component called ProfilePictureUploader which provides the API request itself and defines how the other three components are arranged.
Please have a look at the detectFaces function. The function is performing the request to the Google Cloud Vision API. As you can see there is a required request body to provide in order to do face detection. With this request body you can do a POST request to
https://vision.googleapis.com/v1/images:annotate?key=${APIkey}
The APIkey is the APIkey you get from the Google Cloud console. Here I just wrote it directly into the code, which is generally not the best approach but it’s okay to try it out at the beginning. After the request was successful the response is given to the ImageWarning component to determine if warnings should be displayed.
Result
The API itself was really reliable and fast in my tests. It always took about 1 second to get a result which is relatively fast for doing such a complex task.
Generally spoken I think it is a good way to improve the profile picture upload with a functionality like this because it can help to reduce the amount of bad or inappropriate pictures in your application. Anyway you should decide whether you want to completely restrict the user from uploading such content, or if you just want to display warning like I did. As always it highly depends on your specific use case.
