Literature review

We've set up a Mendeley group to organize our literature review. Unfortunately, it's not permitted to share pdfs in public Mendeley groups (limiting its usefulness). So we set up a parallel private group from which the public group will be occasionally updated.

"Relative Contributions of Internal and External Features to Face Recognition" by Jarudi and Sinha: This is a psychometrics paper, in which the subject's ability to recognize a celebrity's face is measured across variations in the presentation of the face. There are four conditions: 1) The eyes, nose, and mouth have been cropped and are presented individually, 2) Only the interior of the face is visible, 3) The entire head is visible, but the face has been blurred, 4) The entire head is visible.

Performance in each of these conditions is measured as a function of the amount of synthetic blur added to the image. A result: performance with external features is robust to blurring; this isn't surprising, but it is useful to have quantified as it helps justify the use of external features when the data is blurry surveillance video. Another result: performance with whole faces is better than performance with external features plus performance with internal features. I haven't yet decided what this means.

Also, in the related works mentions the idea of a hierarchy over facial features (which features are the most useful for discrimination), citing Fraser and citing Haig. Apparently for unfamiliar faces, the shape of the head is the most informative, while for familiar faces, what's inside the face is most informative.

"Are External Face Features Useful for Automatic Face Classification?" by Lapedriza and Masip: A computer vision paper in which external facial cues are used to determine gender. More precisely, their system uses three rectangular regions around the face (they assume the face shot is frontal and aligned): a block to the left of the face (ears and side hair), a block above the face (forehead and top hair), and a block to the right of the face. This mostly captures hair style, which (obviously) predicts gender. They have a heuristic translation-invariant nonnegative matrix factorization (NMF) scheme which uses normalized cross correlation to determine where to place features, which is fairly cool, though also crying out for rigor. The weights they get when doing the NMF reconstruction become the descriptor for an image, which they feed to an ML black box.

This paper seemed more of a proof-of-concept than an attempt to build the best system possible; they start knowing the faces are aligned, and use that information to black out the interior of the face, surely losing something in the process. Still, we could cite this in the vein of "stuff you can do with just external features".