Facial Tracking + Recognition with OpenCV and face_recognition python libraries

Hey everyone, appreciate the support over the past few days, have grown to 7.5k followers in the last 4 days!

With that being said, I’m gonna break down how to set up facial tracking and recognition with OpenCV and face_recognition in python.

The very firs thing I recommend that you do is to check out this open source repo by Driveline Baseball on biomechanics/computer vision here:

Specifically go to the Computer Vision folder. there is a tutorial.md file that is super helpful and it’s what I used to get into it.

This is the code I used:

import face_recognition  # Library for face recognition
import cv2  # OpenCV library
import numpy as np  # Library for numerical operations
import os  # Library for operating system dependent functionalityprint("Initializing...")

# Path to the video file to be processed
video_path = 'test.mp4'
# Create a VideoCapture object
cap = cv2.VideoCapture(video_path)
# Get the width and height of frames in the video
frame_width = int(cap.get(3))
frame_height = int(cap.get(4))

# Define the codec and create a VideoWriter object
fourcc = cv2.VideoWriter_fourcc(*'mp4v')
out = cv2.VideoWriter('output.mp4',
                      fourcc, 30.0, (frame_width, frame_height))

print("Loading training images...")
# Initialize lists to store face encodings and names
known_face_encodings = []
known_face_names = []

# Load training images from the 'training' directory
if os.path.exists('C:\\Open Source\\facial recognition\\training/'):
    for filename in os.listdir('C:\\Open Source\\facial recognition\\training/'):
        if filename.endswith('.jpg'):
            print(f"Processing {filename}...")
            image_path = os.path.join('C:\\Open Source\\facial recognition\\training/', filename)
            # Load an image file to find face locations
            image = face_recognition.load_image_file(image_path)
            # Get face encodings for any faces in the uploaded image
            encodings = face_recognition.face_encodings(image)
            if len(encodings) > 0:
                face_encoding = encodings[0]
                known_face_encodings.append(face_encoding)
                known_face_names.append("Carson S.")
            else:
                print(f"No faces found in {filename}")

print(f"Loaded {len(known_face_encodings)} face encodings.")

# Initialize lists to store face locations, encodings, and names
face_locations = []
face_encodings = []
face_names = []
# Initialize variable to control frame processing
process_this_frame = True

print("Starting video processing...")

# Loop over frames from the video file stream
while cap.isOpened():
    # Read the next frame from the file
    ret, frame = cap.read()
    # If the frame was not grabbed, then we have reached the end of the stream
    if not ret:
        print("Reached end of video.")
        break

    # Only process every other frame of video to save time
    if process_this_frame:
        print("Processing frame...")
        # Resize frame of video for faster face recognition processing
        small_frame = cv2.resize(frame, (0, 0), fx=0.5, fy=0.5)
        small_frame = frame  # using original frame for now

        # Find all the faces and face encodings in the current frame of video
        face_locations = face_recognition.face_locations(small_frame)
        print(f"Found {len(face_locations)} face(s) in this frame.")

        face_encodings = face_recognition.face_encodings(
            small_frame, face_locations)

        face_names = []
        for face_encoding in face_encodings:
            # See if the face is a match for the known face(s)
            matches = face_recognition.compare_faces(
                known_face_encodings, face_encoding, tolerance=0.6)
            name = "Unknown"

            # Use the known face with the smallest distance to the new face
            face_distances = face_recognition.face_distance(
                known_face_encodings, face_encoding)
            best_match_index = np.argmin(face_distances)
            if matches[best_match_index]:
                name = known_face_names[best_match_index]

            face_names.append(name)
        print(f"Identified faces: {face_names}")

    # Switch to not process the next frame
    process_this_frame = not process_this_frame

    # Display the results
    for (top, right, bottom, left), name in zip(face_locations, face_names):
        # Draw a box around the face
        cv2.rectangle(frame, (left, top), (right, bottom), (0, 0, 255), 2)
        # Draw a label with a name below the face
        cv2.rectangle(frame, (left, bottom - 35),
                      (right, bottom), (0, 0, 255), cv2.FILLED)
        font = cv2.FONT_HERSHEY_DUPLEX
        cv2.putText(frame, name, (left + 6, bottom - 6),
                    font, 1.0, (255, 255, 255), 1)

    # Write the resulting image to the output video file
    out.write(frame)

print("Releasing video objects...")
# Release the file pointers
out.release()
cap.release()

Quick note: to install facial recognition you’re gonna need to install VS build tools because face_recognition needs something to handle C++ Code.

You will need a folder called “Training” or whatever you want in that directory. that folder will contain a selfie, and side pics from both sides, and maybe just a normal photo of you. This is how the software will Identify you. You will then make sure that it labels the name you want it to label for that person.

Note: if you want to do this with multiple people you need to set up a loop for the training image loading + the known faces to go through all of those images and folders. I would recommend making the foloders the name of the person so you can make the “known face names” dynamic with the folder being ran through.

Then in the video processing section, its gonna go through every frame of your video. it’ll identify the faces with

and mark their locations in the frame.

This code:

# Display the results
    for (top, right, bottom, left), name in zip(face_locations, face_names):
        # Draw a box around the face
        cv2.rectangle(frame, (left, top), (right, bottom), (0, 0, 255), 2)
        # Draw a label with a name below the face
        cv2.rectangle(frame, (left, bottom - 35),
                      (right, bottom), (0, 0, 255), cv2.FILLED)
        font = cv2.FONT_HERSHEY_DUPLEX
        cv2.putText(frame, name, (left + 6, bottom - 6),
                    font, 1.0, (255, 255, 255), 1)

Will go and display a box + name attached to that face.

So that’s pretty much how the code works. Again, check out that driveline repo EVEN if you’re not a baseball person. that’s how I learned how to do this and this script is pretty much from there.

Be on the lookout for more cool stuff :)