Machine learning - Introduction to Gaussian processes

What people said

Recommends juliohm · 2013-07 · Cross Validated Stack Exchange · on “Understanding a paper on Gaussian Processes”

These two lectures by Nando de Freitas follow the same notation and will give you a full understanding of Gaussian Processes: Introduction to GP [this video] , Active learning with GP [LINK] .

Cites Antoni Parellada · 2016-09 · Cross Validated Stack Exchange · on “Simulating the posterior of a Gaussian process”

For the first time (excuse imprecission / mistakes) I took a look at Gaussian processes [LINK] , and more specifically, watched this video by Nando de Freitas [this video] . The notes are available online here [LINK] . At some point he draws $10$ random samples from a multivariate normal generated by constructing a covariance matrix based on a Gaussian kernel (exponential of squared distances in the $x$ axis). These random samples form the prior smooth plots that become less spread as data become available. Ultimately, the objective is to predict by modifying the covariance matrix, and obtaining the conditional Gaussian distribution at the points of interest. [LINK] The entire code is available a

Cites Maxim · 2017-12 · Cross Validated Stack Exchange · on “Bayesian optimizations of hyper-parameters - magical curve based on two data points?”

Definition A Gaussian Process (GP) is a Gaussian distribution over functions: $$f(x) \sim GP(m(x), k(x, x^\prime))$$ where $m(x)$ is the mean function and $k(x, x^\prime)$ is a kernel function. The solid blue line in the middle on your pictures is the mean $m(x) = \mathbb{E}[f]$, while the two lines around it are functions that are one standard deviation away from the mean: $\mathbb{E}[f] \pm \sigma[f]$. Each iteration is an evaluation of an outcome for some $x$. These are the blue dots on your pictures, after two and three evaluations. As you can see, the standard deviation collapses at these points, because their outcomes have been calculated , i.e., there's nothing to vary. All other poin

Cites koryakinp · 2018-06 · Cross Validated Stack Exchange · on “Estimating function with Gaussian Procceses”

I do not have strong math background, but I am trying to understand Gaussian Processes by example using the lecture Machine learning - Introduction to Gaussian processes by Nando de Freitas . Here is the link to a relevant slide [this video] . Given the following data: $$f(1.02) = 0.79$$ $$f(1.99) = 0.94$$ $$f(4.04) = 0.65$$ I want to estimate $f(3) \pm \sigma$ . I am using Gaussian Kernel: $$K(x_1,x_2) = \sigma * exp\Biggl(\frac{-(x_1-x_2)^2}{2l}\Biggr)$$ with parameters $\sigma = 1.5$ and $l = 5$ . According to the lecture, I need to compute the following matrix: $$ \begin{bmatrix} K & K_* \\ K^T_* & K_{**} \end{bmatrix} $$ So, $\mu$ and $\sigma$ can be computed: $$\mu = K^T_* * K^{-1} * f$$ $$