← Back

Phys 1379 · Machine Learning in the Sciences · demo

Eigenfaces

A photograph of a face is 4,096 numbers, but faces differ from one another in far fewer ways than that. Principal component analysis finds those ways, and a few dozen numbers then rebuild a recognisable face.

Course demo · 7 November 2026

1A face is a vector

The Olivetti collection holds 400 grey-scale photographs of 40 people, ten of each, taken in the early 1990s at AT&T Laboratories Cambridge. Each is 64 × 64 = 4,096 pixels. Writing the pixel brightnesses in one long list turns each photograph into a vector \(x\) with 4,096 components, a single point in a 4,096-dimensional space.

Most points in that space are television static. Faces occupy a tiny region of it: every face has two eyes above a nose above a mouth, lit from roughly the front. That suggests describing a face not by its pixels but by how it differs from a typical face, along a small number of well-chosen directions.

2Principal components

Start from the average face \(\bar x\), the pixel-by-pixel mean of all 400 photographs. Each face differs from it by \(x_n - \bar x\). The covariance matrix records how the pixels vary together across the collection:

\[ \begin{equation} C = \frac{1}{N-1} \sum_{n=1}^{N} (x_n - \bar x)(x_n - \bar x)^{\mathsf T} . \tag{1}\label{eq:cov} \end{equation} \]

It is a 4,096 × 4,096 symmetric matrix, so it has 4,096 orthogonal eigenvectors \(e_k\) with eigenvalues \(\lambda_k \ge 0\):

\[ \begin{equation} C\, e_k = \lambda_k\, e_k , \qquad \lambda_1 \ge \lambda_2 \ge \cdots . \tag{2}\label{eq:eig} \end{equation} \]

The eigenvalue \(\lambda_k\) is the variance of the collection along the direction \(e_k\). The first eigenvector is the single direction in which faces differ most; the second is the direction of largest variance at right angles to the first, and so on. These eigenvectors are the principal components. Each one is itself a 4,096-pixel image, an eigenface. With only 400 photographs, at most 399 eigenvalues are non-zero. In practice the eigenvectors are found from the singular value decomposition of the 400 × 4,096 data matrix rather than by building \(C\).

3Rebuilding a face

Because the eigenvectors are orthonormal, any face can be written as the average face plus a sum over eigenfaces, with coefficients found by projecting:

\[ \begin{equation} x \approx \bar x + \sum_{k=1}^{K} c_k\, e_k , \qquad c_k = e_k \cdot (x - \bar x) . \tag{3}\label{eq:rebuild} \end{equation} \]

With all 399 components the sum is exact. Keeping only the first \(K\) gives the best possible approximation by \(K\) numbers, in the sense of the smallest squared error over the collection. The fraction of the variation it captures is \(\sum_{k \le K} \lambda_k / \sum_k \lambda_k\), shown in Table 1.

Table 1. Fraction of the variation among the 400 faces captured by the first \(K\) principal components.
\(K\)125102050100200
captured24%38%54%66%76%87%94%98%

Fifty numbers in place of 4,096 keep 87% of the variation, and the face is easily recognised. This is lossy compression, and the coefficients \(c_k\) are a compact description that can be compared between photographs: the basis of early face recognition.

4Try it

All computed in your browser from the average face and the first 100 eigenfaces of the Olivetti collection (0.9 MB).

Loading the eigenfaces…

Rebuild a face

original
rebuilt
Components \(K\)

The eigenfaces

The first twelve principal components. Grey is zero, light is positive, dark is negative. The early ones are mostly about lighting and the overall shape of the head; detail such as glasses, eyebrows and mouths comes in later.

Make a face

Set the first six coefficients \(c_k\), in units of their spread \(\sigma_k = \sqrt{\lambda_k}\) over the collection, and the face is \(\bar x + \sum_k c_k e_k\). The course script does the same with typed numbers.

your face

Things to try

5PCA as a linear autoencoder

Equation \eqref{eq:rebuild} is an encoder and a decoder. The encoder turns 4,096 pixels into \(K\) numbers, \(c = E^{\mathsf T}(x - \bar x)\), where the columns of \(E\) are the first \(K\) eigenfaces; the decoder turns them back, \(\hat x = \bar x + E c\). Of all encoders and decoders that are linear maps, this pair has the smallest squared reconstruction error. An autoencoder replaces both linear maps with neural networks, which can follow curved families of images that no flat subspace fits, at the price of a training run in place of one matrix decomposition.

6Code and references

← Back to the course page