Testing Bayesian Concepts in R: using the Gaussian Conjugate Priors to compute the Posterior Distribution

  • In this article, the Gaussian Conjugate Priors will be used to compute the Posterior distribution for some online dataset (1D and 2D) following Gaussian Distribution.
  • First, a N(μ0,σ0) prior is chosen to model the unknown mean μ variable of the data, while assuming the data variance σ2 as fixed.
  • Then a few i.i.d. samples are drawn from a Gaussian distribution the prior belief about the mean μ is updated, X_iN(μ,σ2) with prior μN(μ0,σ0).
  • The posterior probability distribution is also a Gaussian distribution as shown in the figure below from the videos of professor Herbert Lee from the coursera course Bayesian Statistics Concepts by UCSC.

    math.png

  • Then the recursive Bayesian updates and the prior and posterior hyper-parameters and the means are updated as and when a new datapoint is received. Also, the frequentist’s MLE and 95% confidence interval are computed, along with the Bayesian 95% credible interval.
  • The following animation shows the results of simulation of 50 such data samples, starting with the prior N(0,10).

    gauss1D

  • The left bottom plot visualizes the histogram of the data generated.
  • Every time a new datapoint is received, the prior belief is updated.
  • The right bottom table represents the summary statistics. Prior and Posterior means (of the arrival rate) respectively correspond to the previous and updated beliefs about the mean of the data.
  • The next animation shows the same results (with contours) modeling a set of 2D Gaussian samples.

gauss2D

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s