Classification of Faces in Man and Machine
We attempt to shed light on the algorithms used by humans to classify images of human faces according to their gender. For this, a novel methodology combining human psychophysics and machine learning is introduced. We proceed as follows. In a first stage, we apply Principal Component Analysis(PCA) on the pixel information of the face stimuli. We then obtain a dataset composed of these PCA-eigenvectors combined with the subjects’ gender estimates of the corresponding stimuli. In a second stage we model the gender classification process on this dataset using a separating hyperplane (SH) between both classes. This SH is computed using algorithms from machine learning, namely the Support Vector Machine (SVM), the Relevance Vector Machine, the Prototype classifier and the Kmeans classifier. The classification behavior of humans and machines is then analyzed in three steps. First the classification errors of humans and machines are compared for the various classifiers, and we also assess how well machines can recreate the subjects’ internal decision boundary by studying the training errors of the machines. Second we study the correlations between the rank-order of the subjects’ responses to each stimulus—the gender estimate with its reaction time and confidence rating—and the rank-order of the distance of these stimuli to the SH. Finally, we attempt to compare the metric of the representations used by humans and machines for classification by relating the subjects’ gender estimate of each stimulus and the distance of this stimulus to the SH. While we show that the classification error alone is not a sufficient selection criterion between the different algorithms humans might use to classify face stimuli, the distance of these stimuli to the SH is shown to capture essentials of the internal decision space of humans. Furthermore, algorithms such as the prototype classifier using stimuli in the center of the classes are shown to be less adapted to model human classification behavior than algorithms such as the SVM based upon stimuli close to the boundary between the classes.