Automatically Locking & Unlocking Ubuntu with Computer Vision Using a Human Face!!!

I have implemented an auto face unlock technique using OpenCV and face recognition library.

Ubuntu Face

Before playing with the code, let us see the libraries that need to be installed on your Ubuntu machine.

Requirements

1. python 3.7
2. OpenCV 4.1.0
3. NumPy
4. face-recognition
5. sudo apt-get install gnome-screensaver
6. sudo apt-get install xdotool

Below are the three python files that are just enough to achieve this!!!

1. face_generate.py

2. face_train.py

3. face_unlock.py

Demo video

1. face_generate.py

Generate face

As a first step, we have to generate training images to train the models. I have created a python file to generate our face image. When you execute this file, it will ask your name in the command line then the system will create a folder with the given name.

number=0;
frame_count=0
detector = dlib.get_frontal_face_detector()
print("enter the person name")
name = input()
folder_name="dataset/"+name
if os.path.exists(folder_name):
    print ("Folder exist")
else:
 os.mkdir(folder_name)

After that, we need to use the OpenCV library to read our face via webcam and store it in the respective folder, before saving to that folder we should resize our image.

image = imutils.resize(image, width=500)
  gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
   # detect faces in the grayscale image
  rects = detector(gray, 1)
  # loop over the face detections
  for (i, rect) in enumerate(rects):
   # determine the facial landmarks for the face region, then   
   (x, y, w, h) = face_utils.rect_to_bb(rect)
   #print rect.dtype
   cro=image[y: y + h, x: x + w]
out_image = cv2.resize(cro,(108,108))
   
   fram= os.path.join(folder_name+"/",str(number)+ "." + "jpg")
   number+=1  
   cv2.imwrite(fram,out_image)  
   cv2.rectangle(image, (x, y), (x + w, y + h), (0, 255, 0), 2)   
  frame_count+=1

This will take 50 face images to train the model, you can change the number of image counts as per your requirements.

2. face_train.py

This file will read every face image that is generated by face_genrate.py file. Upon completion, we need to pass them into the load_image_file method to get the face location of the images.

face = face_recognition.load_image_file("dataset/" + person + "/" + person_img)
# location is in css order - top, right, bottom, left
height, width, _ = face.shape
face_location = (0, width, height, 0)

Before training our face model, we should encode our images, so I used the face_encoding method. Also, we should store our names in one array to train with the k-nearest neighbor algorithm.

face_enc = face_recognition.face_encodings(face, known_face_locations=[face_location])
face_enc = np.array(face_enc)
face_enc = face_enc.flatten()
        
# Add face encoding for current image with corresponding label (name) to the training data
encodings.append(face_enc)
names.append(person)

Sci-kit learn library is popular for the machine learning algorithm. We have used the KNeighborsClassifier algorithm which is provided by sci-kit learn library.

knn_clf = neighbors.KNeighborsClassifier(n_neighbors=n_neighbors,         algorithm=knn_algo, weights='distance')
knn_clf.fit(encodings,names)

Finally, We should save our model file using the pickle library.

# Save the trained KNN classifier
if model_save_path is not None:
    with open(model_save_path, 'wb') as f:
        pickle.dump(knn_clf, f)
return knn_clf

3. face_unlock.py

This python file will lock & unlock the ubuntu system based on the trained image. Let’s first capture our face via webcam using the OpenCV library. After using the read method, we can read face images as frames.

camera = cv2.VideoCapture(0)
(grabbed, image1) = camera.read()

We must detect the face location from the image, hence I’ve used the face_location method. After getting the face location we should pass into the face_encodings method to encode face image to process the next step.

image = image1[:, :, ::-1]
X_face_locations = face_recognition.face_locations(image)
# If no faces are found in the image, return an empty result.
if len(X_face_locations) != 0:
  
# Find encodings for faces in the test iamge
faces_encodings = face_recognition.face_encodings(image,   known_face_locations=X_face_locations)

Before predicting, knn model should be used to find the best matches for the given face. Below is the prediction method to verify the face. It will return the name and top, left, right & bottom variable.

closest_distances = knn_clf.kneighbors(faces_encodings,   n_neighbors=1)
are_matches = [closest_distances[0][i][0] <= 0.4 for i in range(len(X_face_locations))]
predictions = [(pred, loc) if rec else ("unknown", loc) for pred, loc, rec in zip(knn_clf.predict(faces_encodings), X_face_locations, are_matches)]

Use gnome shell command to Lock and Unlock the ubuntu system.

if name not in "unknown":
    os.popen('gnome-screensaver-command -d && xdotool key Return')
else
    os.popen('gnome-screensaver-command -a')

If the user’s face is unknown or not detected, the system will remain in a locked state.

Conclusion

That’s it, we have successfully tested the auto-lock & unlock technique of ubuntu using a human face with the help of three python files listed above. You can get the complete source code here

if you need any help or assistance please connect with me LinkedIn and Twitter.