Gaussian process regression.¶

Marcus Frean 2013, for COMP421

Gaussian process regression. Reads a datafile consisting of a matrix in which each row is a training item. The final column in the target and preceding columns are the input.

Hyperparameters theta control a squared exponential covariance function, and are chosen from Gaussian distributions. These theta are exponentiated to give the length scales in each input dimension, the overall vertical scale of the covariance function (i.e. the max), and the amount of noise assumed to be corrupting the observed target values. That is: length scale in 1st dim of input = exp(theta[0]) length scale in 2nd dim of input = exp(theta[1]) : : length scale in n-th dim of inpt = exp(theta[D-1]) Vertical scale of covariance fn = exp(theta[D]) Assumed variance of target noise = exp(theta[D+1]) NOTE THE EXPONENTIATION OF THETA, which allows us to put simple Gaussian priors on it.

The blue plot shows predictions and error bars (actually variance...) for the initial randomly chosen hyperparameters theta. The red shows those after optimizing theta.