Most similar face recognition in Python

Question

How can I use Python and OpenCV to find facial similarity?

I've successfully used OpenCV and Python to extract faces from multiple photographs using Haar Cascades.

I now have a directory of images, all of which are faces of different people.

What I'd like to do is take a sample image, and then see which face it most looks like.

I've tried using pyssim:

pyssim needle.jpg "haystack/*"

But, unfortunately, it's looking a image similarity (colours etc) rather than facial features.

To reiterate - I'm quite happy detecting faces, what I'd like to be able to do is compare them.