$$
\newcommand{\R}{\mathbb{R}}
\newcommand{\SSS}{\mathbb{S}}
\newcommand{\E}{\mathbb{E}}
\newcommand{\C}{\mathbb{C}}
\newcommand{\N}{\mathbb{N}}
\newcommand{\Z}{\mathbb{Z}}
\newcommand{\P}{\mathbb{P}}
\newcommand{\Q}{\mathbb{Q}}
\newcommand{\frakX}{\mathfrak{X}}
\newcommand{\frakB}{\mathfrak{B}}
\newcommand{\frakW}{\mathfrak{W}}
\newcommand{\Id}{\mathrm{Id}}
\newcommand{\dist}{\mathrm{dist}}
\newcommand{\span}{\mathrm{Span}}
\newcommand{\diag}[1]{\mathrm{diag}(#1)}
\newcommand{\dim}[1]{\mathrm{dim} \, #1}
\newcommand{\tr}[1]{\mathrm{tr}(#1)}
\newcommand{\partder}[2]{\frac{\partial #1}{\partial #2}}
\newcommand{\rank}[1]{\mathrm{rank} \, #1}
\newcommand{\inner}[1]{\langle #1 \rangle}
\newcommand{\innerbig}[1]{\left \langle #1 \right \rangle}
\newcommand{\abs}[1]{\lvert #1 \rvert}
\newcommand{\norm}[1]{\lVert #1 \rVert}
\newcommand{\ker}[1]{\text{ker}(#1)}
\newcommand{\supp}[1]{\text{supp}(#1)}
\newcommand{\calA}{\mathcal{A}}
\newcommand{\calB}{\mathcal{B}}
\newcommand{\calC}{\mathcal{C}}
\newcommand{\calD}{\mathcal{D}}
\newcommand{\calE}{\mathcal{E}}
\newcommand{\calF}{\mathcal{F}}
\newcommand{\calG}{\mathcal{G}}
\newcommand{\calH}{\mathcal{H}}
\newcommand{\calI}{\mathcal{I}}
\newcommand{\calJ}{\mathcal{J}}
\newcommand{\calK}{\mathcal{K}}
\newcommand{\calL}{\mathcal{L}}
\newcommand{\calM}{\mathcal{M}}
\newcommand{\calN}{\mathcal{N}}
\newcommand{\calO}{\mathcal{O}}
\newcommand{\calP}{\mathcal{P}}
\newcommand{\calQ}{\mathcal{Q}}
\newcommand{\calR}{\mathcal{R}}
\newcommand{\calS}{\mathcal{S}}
\newcommand{\calT}{\mathcal{T}}
\newcommand{\calU}{\mathcal{U}}
\newcommand{\calV}{\mathcal{V}}
\newcommand{\calW}{\mathcal{W}}
\newcommand{\calX}{\mathcal{X}}
\newcommand{\calY}{\mathcal{Y}}
\newcommand{\calZ}{\mathcal{Z}}
\newcommand{\bh}
\newcommand{\bf}
\newcommand{\Pker}{P_{\ker{L}^{\perp}}}
$$
Gaussian Field and SPDE
In this post, we will extend the previous discussion on Gaussian Field to Elliptical SPDE on Euclidean domain and compact Riemannian manifold with no boundaries. This turns out to be the natural way of defining general Gaussian field, which may have non-stationary second order structure, and/or its hyperparameters of the Covariance operator is dependent on spatial locations. Examples for Matérn class of kernels will be provided from Borovitskiy et al. 2020.
Posted on October 14, 2020
Gaussian Field and Covariance Operator
We will have a close look at Gaussian Process (Fields) in different views, with motivations from intuitive explanations given in Rasmussen & Williams 2003, and theoretical construction from Lototsky & Rozovsky 2017. This will serve as the preliminary setup for studying Elliptical SPDE driven by Gaussian white noise in the next post.
Posted on October 14, 2020
Green's function, RKHS and Regularisation
With the spectral perspective of RKHS introduced previously, we now look at a special and important category of reproducing kernels, which is the Green's functions to positive systems of differential equations. We will have examples on the eigenvalue problem for Dirichlet Laplace operator; and also on the Heat equation in Euclidean space. An intuitive generalisation is then provided. Finally, we will discuss informally the extension to discrete differential system and RKHS defined by Hermitian matrices.
Posted on October 7, 2020
Part 4: Covariance operator for reproducing kernels
Another important operator defined on a RKHS is Covariance operator, which is essential for doing kernel PCA, kernel CCA, and independence testing etc. We will see that under certain conditions, the centered Covariance operator is Hilbert-Schmidt, moreover, sharing the same spectrum as the HS Integral operator introduced in Part 3.
Posted on September 30, 2020
Part 3: Mercer's theorem and the spectral perspective of RKHS
In this article, we provide another formulation of RKHS in terms of the spectrum of the Integral operator corresponding to a reproducing kernel, which is given by Mercer's Theorem and the property of compact self-adjoint operators on separable Hilbert space.
Posted on September 28, 2020