$\newcommand{\hy}{\hat{y}} \newcommand{\hmu}{\hat{\mu}} \newcommand{\R}{\mathbf{R}} \newcommand{\hmu}{\hat{\mu}} \newcommand{\rR}{\mathbf{r}^TR^{-1}} \newcommand{\rRr}{\mathbf{r}^TR^{-1}\mathbf{r}} \newcommand{\oRr}{\mathbf{1}^TR^{-1}\mathbf{r}} \newcommand{\ymu}{(\mathbf{y}-\mathbf{1}\hmu)} \newcommand{\hyexp}{\hmu+\rR\ymu} \newcommand{\oRy}{\mathbf{1}^T\mathbf{R} ^{-1}\mathbf{y}} \newcommand{\yRo}{\mathbf{y}^T\mathbf{R} ^{-1}\mathbf{1}} \newcommand{\oRo}{\mathbf{1}^T\mathbf{R}^{-1}\mathbf{1}} \newcommand{\hmuexp}{\frac{\oRy}{\oRo}} \newcommand{\st}{\sigma^2} \newcommand{\mt}{\mu^2}$ This post shows a derivation of the DACE predictor’s MSE discussed in Stochastic Processes for Expensive Black-Box Optimization.

The mean squared error of a predictor at $\mathbf{x}$ based on the stochastic Gaussian process is

\begin{equation}MSE(\mathbf{x})=E[(\hat{y}(\mathbf{x})-y(\mathbf{x}))^2] =\sigma \big[1 - {\mathbf{r}(\mathbf{x})}^{T}\mathbf{R}^{-1}\mathbf{r}(\mathbf{x})-\frac{(1-\mathbf{1}^T\mathbf{R}^{-1}\mathbf{r}(\mathbf{x}))^2}{(\mathbf{1}^T\mathbf{R}\mathbf{1})}\big]\;, \label{eq:final-mse} \end{equation}

where $\mu$ is process’s mean and $\sigma^2\mathbf{R}$ is its covariance matrix over a sample $\mathcal{D}={(x^{(i)},y^{(i)})}_{1\leq i \leq n}$. For brevity, we will drop $\mathbf{x}$ henceforth. Before proceeding with the proof, let’s recall some terms and their definitions that will be useful in the proof. \begin{equation} \hy=\hyexp \end{equation}

\begin{equation} \hmu=\hmuexp \end{equation}

From the above, we have $E[y^2]=\sigma^2 + \mu^2$, $E[\mathbf{y}\mathbf{y}^{T}]=\sigma^2\R+\mu^2\mathbf{1}\mathbf{1}^T$.

Thus, we can expand the MSE term as

\begin{equation} MSE= \sigma^2 + \mu^2 + E[{\hy}^2] - 2 E[y\hy]\;. \label{eq:mse} \end{equation} Where \begin{equation} E[\hy^2]=\frac{\st}{\oRo}+\mt + \st (\rRr) - \st \frac{(\oRr)^2}{\oRo} \label{eq:h2} \end{equation} and

\begin{equation} -2E[y\hy]=-2\st(\rRr)-2\mt-2\st\frac{\oRr}{\oRo}+2\st\frac{(\oRr)^2}{\oRo}\;. \label{eq:h3} \end{equation}

Plugging Equations \ref{eq:h2} and \ref{eq:h3} into Eq. \ref{eq:mse} results in Eq. \ref{eq:final-mse}.