Balthazar Rouberol

In this article, we’ll present several face detection tools that could solve the following problem:
we have a collection of ~11000 images, and we would like to know how many of them contain (human) faces.

We’ll go through 3 different tools: the face.com API, OpenCV,
and ccv.

Note that face.com has been acquired by Facebook since I first drafted this article, and the API is no longer open to developers.

The problem

We have a collection of 11050 images coming from Instagram and Twitpic.
These images contain pretty much anything: coffee mugs, hipster shoes, but also self portraits, group photos, etc.
We would like to automatically filter out the pictures with no faces, with the highest possible accuracy.

To be able to compare the detectors accuracy, we’ll manually select a test set of 200 pictures, 150 of them
containing faces, as we would not be able to check the accuracy of the results related to our entire image collection.

Face.com

face.com is an API I used on other A.I. projects which involved face detection.
I know by experience it gives pretty good results, and it would be the first tool I would think of for this kind of problem.
However, the public API will soon close its doors to developpers, due to its recent acquisition by Facebook. Darn.

Our first move is to send all 200 images to face.com, and analyze the results
(I will not go over the technical details in this article. If you’re interested in having
a look at the code, drop me an email).

Results

96% (144/150) of the images containing faces were detected by face.com and there was no false-postitive (pictures containing
no faces but still getting detected).

I had a look at the 6 pictures that weren’t detected, to identify a possible root cause: the faces were either cropped,
rotated in a weird angle or both.

An example of undetected faces.

OpenCV

OpenCV is an open-source computer-vision library. In this experiment, I use
OpenCV v2.3.1 for Unix.

The face detection engine: Haar Cascade Classifier

OpenCV object detection is based on trained statistical models, capable of detecting certain features.
The face detection process is based on a model called « Haar Cascade Classifier ».
A very good introduction on the Haar Cascade Classifier is given in this blogpost.

It is important to note that the Haar Cascade Classifier comes with different cascade files: each one being a trained classifier
capable of detecting a particular object (profile face, frontal face, mouth, eyes, etc)

We’ll consider the following classifiers (normally located in /usr/local/share/OpenCV/haarcascades on a linux system):

• haarcascade_frontalface_alt.xml
• haarcascade_frontalface_alt_tree.xml
• haarcascade_frontalface_alt2.xml
• haarcascade_frontalface_default.xml
• haarcascade_profileface.xml

Results

I’ll use an adapted version of the code written by Ian Ozsvald in the A.I CookBook, provided by Ian himself.
This code takes an image path as argument and returns a face features list, which length indicates how many faces were detected in the argument picture.

We are going to run each classifier on the 200 test images, and then compare the results.

• haarcascade_frontalface_alt.xml → 45% (67/150) detected images
• haarcascade_frontalface_alt_tree.xml → 29% (44/150) detected images
• haarcascade_frontalface_alt2.xml → 45% (68/150) detected images
• haarcascade_frontalface_default.xml → 55% (82/150) detected images
• haarcascade_profileface.xml → 15% (23/150) detected images

NB: any false-positive detection was excluded from these results. There were 8% (4/50) false-positive detections, picked up by the
default.xml, alt.xml and alt2.xml classifiers.

Cumulating the classifiers

Another approach to maximise the positive results would be to cumulate the classifiers: if an image was detected by any classifier,
we then consider that it is a positive match.
For example, a person standing in profile would only be detected with the haarcascade_profileface.xml classifier.

Using this approach, we are able to detect 93 images, meaning an accuracy of 62%.

ccv

ccv is a pretty new project, providing several computer vision algorithms. Among them, a « detection algorithm for rigid object (face etc.), an object detection algorithm for somewhat difficult object (pedestrian, cat etc.) », while focusing on high performance and fast runtime.
We thus test the rigid object detector on our images, as shown in the documentation.

Results

66% (99/150) of the images containing faces were detected by face.com and there were 6 false-postitive.

Conclusions

It seems that face.com has not been acquired for nothing: their API gave the best results among the three tools we tested. However, as it is no longer available, ccv seems to be the best choice: it is faster than OpenCV and slightly more accurate (+4% detection with our data).

It could be nice to test whether detection ratio could be improved using image pre-processing (for example) rather than by training the detector: as our data is extremely heterogeneous, I do not believe that training will improve our results. It might even worsen them, as the detector would then get too « focused » on details specific to our training data.