Literature review (part 2)

"The contribution of external features to face recognition" by Lapedriza et al.: Point is that external features are useful for classification. They use the ARFace database (front-facing images, some with occlusion). External features are calculated using Building Blocks (BB) trained on outside-the-face rectangles (forehead, left side, right side, head).

BB is an image reconstruction / re-representation technique in which a region is reconstructed using a set of fragments. Each fragment is a small image patch, and the "correct location" of the patch is determined by convolving it with the image to be reconstructed and finding the best match. Once a set of patches have been placed, their weights are adjusted using nonnegative matrix factorization to approximately reproduce the pixels in the target image.

Once the BB fragments are selected, a new face is characterized as a vector of the normalized convolutions of the BBs with the query image. The inside-the-face region is represented as raw pixels. In the combined condition, external features are concatenated with internal features and dimension is reduced using nonparametric discriminant analysis (NDA), followed by nearest neighbor classification. In the internal-only condition, only the raw pixel values from inside the face are used, with dimensionality reduction via either PCA or NDA.

The authors find including the external information boosts performance. The conclude external information is useful, even when internal information is available.

"Face Verification using External Features" by Lapedriza et al.: Most recent of this set of papers reviewed so far. They basically do the same thing, though the application this time is face verification. The verification application seems to be motivated by a new technique they use called the Local Boosted Discriminant Projections (LBDP) Learning Algorithm, which is a dimensionality reduction technique which aims to minimize intra-class distance while maximizing inter-class distance, and works only with two classes.

Pipeline: External features are reconstructed with Building Blocks, and the BB reconstruction weights are used to represent the external features. This is in contrast to the previous paper, where the convolutions of the BB fragments with the original image were the representation. Internal features are represented as raw pixels. LBPD is used to reduce dimensionality.

Experiments contrast external features only versus internal features only, using the FRGC database and ARFace. They found external features outperform internal features in conditions of occlusion and variations in illumination, and otherwise internal features win.