How do you go about teaching a computer what is attractive and what is not?
This is a very difficult question I have been thinking about recently.
Do you create a duck face detector and subtract points? Is it a series of features we are looking for (specific hair color, eye color, skin smoothness symmetry).
Would this data come out in some sort of statistical analysis?
I decided to research this further using Eigen Faces and SVMs.
For those of you that don't know about Eigen Faces[1], they are a decomposition of a set of images into eigenvalues (weights) and eigenvectors (eigenfaces). The amazing thing about these is that given a large enough training set, any image of a face can be reconstructed by multiplying a set of weights (eigenvalues) with the eigenvectors (eigenfaces).
Top 20 Eigen Faces from our data set
Average image from our data set
The attractiveness of the average of all images is greater than the average of the attractiveness of all images.
We are currenlty using a similar model for detecting attractiveness. Currently we are accurate ~64% of the time; however, that is just using the L2 Norm. The hope is that using an SVM classifier and the weights for all the faces, we will be able to determine the important eigenfaces and their weights for attractiveness and then create a classification system.
If you review the heat-mapped eigenfaces in the first figure, you will notice specific expressions and features that are highlighted. We have 2027 of these eigenfaces and each face can be seen as a weighted combination of these. Our hope is to find the most attractive and unattractive features in the eigenfaces.
I hypothesize that somewhere in their is an eigenface that corresponds to duckface. Think of the utilization of such a thing. Anytime someone uploads a duckface on facebook, it could warn them that they should stop doing that.
Below are our current statistics of the mean and standard deviation of the weight vectors for each level of attractiveness (1 - 5 with 5 being the most attractive). We have also included a close view of the first 50 weights and a histogram of the weights.
Histogram
It seems there are some very interesting differences in the mean, standard deviation, and histogram based on the level of attractiveness. However, this may be because we have a different amount of training examples for each level of attractiveness (attractiveness is semi-gaussian).
It will be interesting what more tinkering will yield. We can only hope to find the elusive eigenface corresponding to duck face.
References:
[1] Turk, Matthew A and Pentland, Alex P. Face recognition using eigenfaces.Computer Vision and Pattern Recognition, 1991. Proceedings {CVPR'91.}, {IEEE} Computer Society Conference on 1991