Using Expectation Maximization Algorithm for the Gaussian Mixture Models to detect outliers in R

In this article, the Expectation Maximization technique will be used to find the outliers in images.

  1. We shall use the following apples and oranges image for the outlier detection.
  2. The colors R,G,B will form the variables for this image data.
  3. Two initial 3-dimensional Gaussian models the first one with red (255,0,0) and the second one with green (0,255,0) mean vectors along with random covariance matrices will be used as initial models.
  4. Then iteratively the E-step and the M-step are used till convergence is obtained (no change in estimated parameter values).
  5. First the E-step is used to compute the (normalized) probability of assignment of each datapoint (pixel) to both the models.
  6. Then the M-step is used to obtain the MLE for the parameters mean and covariance matrix for each of the models as weighted average contribution of the datapoints.
  7. The EM algorithm for GMM is shown as below:gmmalgo.png
  8. The convergence of the GMM EM with iterations as well as change in the estimated parameters are shown in the below animation (with the contours of the two Gaussian Models changing over iterations)
  9. The following figure shows that how the change in estimated mean decreases with iteration, leading the EM algorithm to converge.


10. Finally after the algorithm convergence, the pixels assigned to one of the Gaussian models were marked as black pixels. The below figure shows that it identified the orange among the apples as a different cluster.


11. The following animation shows how the image changes per iteration of the EM algorithm:


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s