Some Observation Theory: LSE, WLSE and BLUE in Octave and R

In the edX Course DelftX: OT.1x Observation theory: Estimating the Unknown,a very systematic and conceptual approach in dealing with estimation problems was introduced. The approach involves a physical world, and a mathematical world. Conceptually, when we deal with a real-life problem, there are the following three stages: In the first stage our problem is defined in the real world, but we need to translate it to the mathematical world. In this mathematical world we can apply all our tricks and methods to get optimal numbers as a result. We call this ‘linking the mathematical model to the real world’. In the last stage, we need to go in the other direction: the results from the procedures in the mathematical world need to be transferred back to reality, where decisions need to be made. We call this the ‘unlinking’ or the disengagement of the ’mathematical model’and one particular type that is discussed in this course is the “Model of Observation Equations”.

The observations are denoted with the letter ‘y’. Consider observations in the case the measurements have not been taken yet, in which case, they are generally referred to as ‘observables, whose values should be interpreted as stochastic (or random) variables. The unknown parameter is indicated by the letter ’x’. For the functional model, the notation for this functional relation is denoted as ‘A’ and initially the linear relations are dealt with, in which case, ‘A’ becomes a matrix. Since all measurements will be affected by errors (either they are small or big), we need to include these errors also in the model. This is taken care of by the Stochastic model, which we need to describe the uncertainty in our observations due to the random errors.

With the above introduction from the course itself let’s consider solving the following problems that appeared as assignments in the same course and visualize the outputs from the models fitted. The following figure shows the summary of the theory to be used to solve the parameter estimation problems.Here the Least Square Estimator (LSE), Weighted Least Square Estimator (WLSE) with a given weight matrix and the Best Linear Unbiased Estimators (BLUE) will be considered.

obt.png

ob1

ob2

ob3.png

Here are the equations that are going to be used.
ob8
ob4.png

ob5.png
Here is the observation dataset:

y_t_i
3.4608
5.6308
4.4962
9.7645
4.9611
8.7935
8.5578
4.9095
9.5994
6.0969
16.5790
8.7046
11.5990
15.5990
7.8286
32.8450
30.5830
28.6850
43.0150
31.5090

The octave output is shown below:
ob6.png

Next figure shows the R output of the 97% confidence interval for the BLUE.

ob7.png

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s