Modeling Face Images with Nonnegative Matrix Factorization (NMF), Kmeans with Vector Quantization (VQ) and Singular Value Decompostion (SVD) in R

The following face modeling problem was discussed in one of the lectures in the edX course: ColumbiaX: CSMM.102x Machine Learning. For this experiment the CBCL Face Database from the MIT Center For Biological and Computation Learning is used. This dataset contains 6977 face images to be trained and 24045 images to be tested with svm (each image is of size 21×21 pixels), the images in these two files are combined and then the first 10000 images are used for modeling the faces with NMF, Kmeans/VQ and SVD, after that the models are compared.

The following figure shows the first 625 images from the dataset.


k-means (Modeling the images with VQ)

The following figures show the averages of full faces learnt by K-means with different numbers (k=25,64,100) of clusters respectively. As can be seen some of the cluster centroids represent good quality average images, where for some others the average image is very noisy and can not be identified as face at all.





SVD (Modeling images with first k orthonormal vectors V)

SVD Finds the singular value decomposition of the image matrix. As can be seen from the below figure with k=100 images representing the top k singular vectors (from right to left column-wise), the results not interpretable because of negative values and orthogonality constraint.



Using NMF with Squared error objective

The following figures show the iterative coordinate-ascent algorithm that is going to be used.



NMF learns a “parts-based” representation. Each column captures something interpretable. This is a result of the nonnegativity constraint. The following figures show the W matrix learnt by squared-error objective NMF, for different values of k=25, 64, 100 and the k columns of W are shown as images.



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s