Comparison of face detection tools (face.com, openCV, libccv)
In this article, we’ll present several face detection tools that could solve the following problem:
we have a collection of ~11000 images, and we would like to know how many of them contain (human) faces.
Note that face.com has been acquired by Facebook since I first drafted this article, and the API is no longer open to developers.
We have a collection of 11050 images coming from Instagram and Twitpic.
These images contain pretty much anything: coffee mugs, hipster shoes, but also self portraits, group photos, etc.
We would like to automatically filter out the pictures with no faces, with the highest possible accuracy.
To be able to compare the detectors accuracy, we’ll manually select a test set of 200 pictures, 150 of them
containing faces, as we would not be able to check the accuracy of the results related to our entire image collection.
face.com is an API I used on other A.I. projects which involved face detection.
I know by experience it gives pretty good results, and it would be the first tool I would think of for this kind of problem.
However, the public API will soon close its doors to developpers, due to its recent acquisition by Facebook. Darn.
Our first move is to send all 200 images to face.com, and analyze the results
(I will not go over the technical details in this article. If you’re interested in having
a look at the code, drop me an email).
96% (144/150) of the images containing faces were detected by face.com and there was no false-postitive (pictures containing
no faces but still getting detected).
I had a look at the 6 pictures that weren’t detected, to identify a possible root cause: the faces were either cropped,
rotated in a weird angle or both.
An example of undetected faces.
The face detection engine: Haar Cascade Classifier
OpenCV object detection is based on trained statistical models, capable of detecting certain features.
The face detection process is based on a model called « Haar Cascade Classifier ».
A very good introduction on the Haar Cascade Classifier is given in this blogpost.
It is important to note that the Haar Cascade Classifier comes with different cascade files: each one being a trained classifier
capable of detecting a particular object (profile face, frontal face, mouth, eyes, etc)
We’ll consider the following classifiers (normally located in
/usr/local/share/OpenCV/haarcascades on a linux system):
I’ll use an adapted version of the code written by Ian Ozsvald in the A.I CookBook, provided by Ian himself.
This code takes an image path as argument and returns a face features list, which length indicates how many faces were detected in the argument picture.
We are going to run each classifier on the 200 test images, and then compare the results.
haarcascade_frontalface_alt.xml→ 45% (67/150) detected images
haarcascade_frontalface_alt_tree.xml→ 29% (44/150) detected images
haarcascade_frontalface_alt2.xml→ 45% (68/150) detected images
haarcascade_frontalface_default.xml→ 55% (82/150) detected images
haarcascade_profileface.xml→ 15% (23/150) detected images
NB: any false-positive detection was excluded from these results. There were 8% (4/50) false-positive detections, picked up by the
Cumulating the classifiers
Another approach to maximise the positive results would be to cumulate the classifiers: if an image was detected by any classifier,
we then consider that it is a positive match.
For example, a person standing in profile would only be detected with the
Using this approach, we are able to detect 93 images, meaning an accuracy of 62%.
ccv is a pretty new project, providing several computer vision algorithms. Among them, a « detection algorithm for rigid object (face etc.), an object detection algorithm for somewhat difficult object (pedestrian, cat etc.) », while focusing on high performance and fast runtime.
We thus test the rigid object detector on our images, as shown in the documentation.
66% (99/150) of the images containing faces were detected by face.com and there were 6 false-postitive.
It seems that face.com has not been acquired for nothing: their API gave the best results among the three tools we tested. However, as it is no longer available, ccv seems to be the best choice: it is faster than OpenCV and slightly more accurate (+4% detection with our data).
It could be nice to test whether detection ratio could be improved using image pre-processing (for example) rather than by training the detector: as our data is extremely heterogeneous, I do not believe that training will improve our results. It might even worsen them, as the detector would then get too « focused » on details specific to our training data.