*Sandipan Dey, **29 April 2017*

The following face modeling problem was discussed in one of the lectures in the *edX course: ColumbiaX: CSMM.102x Machine Learning*. For this experiment the *CBCL Face Database* from the *MIT Center For Biological and Computation Learning* is used. This dataset contains *6977* face images to be trained and *24045* images to be tested with svm (each image is of size 21×21 pixels), the images in these two files are combined and then the first *10000* images are used for modeling the faces with NMF, Kmeans/VQ and SVD, after that the models are compared.

The following figure shows the first *625* images from the dataset.

## k-means (Modeling the images with VQ)

The following figures show the averages of full faces learnt by *K-means* with different numbers *(k=25,64,100)* of clusters respectively. As can be seen some of the cluster centroids represent good quality average images, where for some others the average image is very noisy and can not be identified as face at all.

**k=64**

**k=100**

## SVD (Modeling images with first *k orthonormal* vectors *V*)

SVD Finds the singular value decomposition of the image matrix. As can be seen from the below figure with k=100 images representing the top *k* singular vectors (from right to left column-wise), the results not interpretable because of negative values and orthogonality constraint.

**k=100**

## Using NMF with Squared error objective

The following figures show the iterative *coordinate-ascent* algorithm that is going to be used.

NMF learns a “parts-based” representation. Each column captures something interpretable. This is a result of the nonnegativity constraint. The following figures show the *W* matrix learnt by squared-error objective NMF, for different values of *k=25, 64, 100* and the k columns of *W* are shown as images.