Asymptotic Variance of Estimation; Mean Squared Error

For the statistical model introduced in Section 3.4.2 we now investigate the asymptotic behavior of the variance ${\rm Var\,}\,\widehat\theta_n$ if $n\to\infty$.

Theorem 3.19   $\;$ Define $\sigma^2=\sum_{i=1}^\ell \pi_i(\varphi_i-\theta)^2$ and let ${\mathbf{Z}}=({\mathbf{I}}-({\mathbf{P}}-{\boldsymbol{\Pi}}))^{-1}$ be the fundamental matrix of ${\mathbf{P}}$ defined by $(\ref{def.fun.mat})$. Then

$$\lim\limits_{n\to\infty}\;n\,{\rm Var\,}\,\widehat\theta_n =\sigm... ...g}}({\boldsymbol{\varphi}})({\mathbf{Z}}-{\mathbf{I}}){\boldsymbol{\varphi}}\,.$$ (78)

Proof

 

• Clearly,

  $$n^2\,{\rm Var\,}\widehat\theta_n = {\mathbb{E}\,} \Bigl(\sum\limi... ...)^2- \Bigl(\sum\limits_{k=0}^{n-1}{\mathbb{E}\,}\varphi({\mathbf{X}}_k)\Bigr)^2$$ (79)

  
  
  and thus
  $$n^2\,{\rm Var\,}\widehat\theta_n = \sum\limits_{k=0}^{n-1}{\mathb... ... \Bigl(\sum\limits_{k=0}^{n-1}{\mathbb{E}\,}\varphi({\mathbf{X}}_k)\Bigr)^2\,.$$

• This representation will now be used to show (78) for the case ${\boldsymbol{\alpha}}_0={\boldsymbol{\pi}}$.
  • In this case we observe
    $$\Bigl(\sum\limits_{k=0}^{n-1}{\mathbb{E}\,} \varphi({\mathbf{X}}_k)\Bigr)^2=(n\theta)^2$$   and$$\qquad \sum\limits_{k=0}^{n-1}{\mathbb{E}\,} \varphi^2({\mathbf{X}}_k)=n\sum\limits_{i=1}^\ell\pi_i\varphi_i^2\,.$$

  • Furthermore, by the stationarity of the Markov chain $\{{\mathbf{X}}_n\}$,
    $$\sum\limits_{0\le k<k^\prime\le n-1}{\mathbb{E}\,}\Bigl( \varphi(... ...k){\mathbb{E}\,} \Bigl(\varphi({\mathbf{X}}_0)\varphi({\mathbf{X}}_k)\Bigr)\,,$$
    where
    $${\mathbb{E}\,}\Bigl( \varphi({\mathbf{X}}_0)\varphi({\mathbf{X}}_... ...top {\,{\rm diag}}({\boldsymbol{\varphi}}){\mathbf{P}}^k{\boldsymbol{\varphi}}$$
    and ${\mathbf{P}}^k={\mathbf{P}}^{(k)}=(p_{ij}^{(k)})$ denotes the matrix of the $k$-step transition probabilities.
  • A combination of the results above yields
    

    $$\frac{1}{n}\; {\rm Var\,}\Bigl(\sum\limits_{k=0}^{n-1}\varphi({\mathbf{X}}_k)\Bigr) = \sum\limits_{i=1}^\ell\pi_i\varphi_i^2+2 {\boldsymbol{\pi}}^\top ... ...its_{k=1}^{n-1}\;\frac{n-k}{n}\;{\mathbf{P}}^k{\boldsymbol{\varphi}} -n\theta^2$$

    $$= \sigma^2+2 {\boldsymbol{\pi}}^\top {\,{\rm diag}}({\boldsymbol{\v... ...bol{\varphi}} -\;\frac{n-1}{2}\;{\boldsymbol{\Pi}}{\boldsymbol{\varphi}}\Biggr)$$

    $$= \sigma^2+2 {\boldsymbol{\pi}}^\top {\,{\rm diag}}({\boldsymbol{\v... ...;\bigl({\mathbf{P}}^k -{\boldsymbol{\Pi}}\bigr)\Biggr){\boldsymbol{\varphi}}\,,$$

    
    

  • where the second equality is due to the identity
    $$\theta^2={\boldsymbol{\pi}}^\top\,{\,{\rm diag}}({\boldsymbol{\varphi}}){\boldsymbol{\Pi}}{\boldsymbol{\varphi}}\,.$$

  • Taking into account the representation formula (76) for ${\mathbf{Z}}-{\mathbf{I}}$ this implies (78).
• It is left to show that (78) is also true for an arbitrary initial distribution ${\boldsymbol{\alpha}}$.
  • At this point we will use a more precise notation: We will write ${\mathbf{X}}^{({\boldsymbol{\alpha}})}_0,{\mathbf{X}}^{({\boldsymbol{\alpha}})}_1,\ldots$ instead of ${\mathbf{X}}_0,{\mathbf{X}}_1,\ldots$ and $\widehat\theta^{({\boldsymbol{\alpha}})}_n$ instead of $\widehat\theta_n$.
  • It suffices to show that

    $$\lim\limits_{n\to\infty}\;n\,\Bigl({\rm Var\,}\,\widehat\theta_n^... ...bol{\pi}})}- {\rm Var\,}\,\widehat\theta_n^{({\boldsymbol{\alpha}})}\Bigr)=0\,.$$ (80)

    
    

  • For this purpose we introduce the following notation: For $0<r<n-1$ let
    $$Y^{(\cdot)}_r=\sum\limits_{k=0}^{r-1}\varphi({\mathbf{X}}^{(\cdot)}_k)$$   und$$\qquad Z^{(\cdot)}_{rn}=\sum\limits_{k=r}^{n-1}\varphi({\mathbf{X}}^{(\cdot)}_k)\,.$$

  • Then, by (79),
    

    $${n^2\,\Bigl({\rm Var\,}\,\widehat\theta_n^{({\boldsymbol{\pi}})}- {\rm Var\,}\,\widehat\theta_n^{({\boldsymbol{\alpha}})}\Bigr)}$$

    $$= \Bigl({\mathbb{E}\,}\bigl(Y^{({\boldsymbol{\pi}})}_r + Z^{({\bold... ...ol{\alpha}})}_r + {\mathbb{E}\,} Z^{({\boldsymbol{\alpha}})}_{rn}\bigr)^2\Bigr)$$

    $$= \Bigl({\mathbb{E}\,}\bigl(Y^{({\boldsymbol{\pi}})}_r\bigr)^2-\big... ...})}_r\bigr)^2 +\bigl({\mathbb{E}\,} Y^{({\boldsymbol{\alpha}})}_r\bigr)^2\Bigr)$$

    $$+2{\mathbb{E}\,}\Bigl(\bigl(Y^{({\boldsymbol{\pi}})}_r-{\mathbb{E... ...})}_{rn}-{\mathbb{E}\,}\bigl(Z^{({\boldsymbol{\alpha}})}_{rn}\bigr)\bigr)\Bigr)$$

    $$+ \Bigl({\mathbb{E}\,}\bigl(Z^{({\boldsymbol{\pi}})}_{rn}\bigr)^2... ...\bigr)^2-\bigl({\mathbb{E}\,} Z^{({\boldsymbol{\alpha}})}_{rn}\bigr)^2\Bigr)\,,$$

    
    
    where we denote the three summands in the last expression by $I_r$, $II_{rn}$ and $III_{rn}$, respectively.
  • $I_r$ does not depend on $n$ and hence $\lim_{n\to\infty} n^{-1} I_r=0$.
  • As the state space $E$ is finite we obtain for $c=\max_{i\in E} \vert\varphi(i)\vert<\infty$ that
    $$\frac{1}{n}\; II_{rn}\le 4rc\,{\mathbb{E}\,}\Bigl(\;\frac{1}{n}\;... ...-{\mathbb{E}\,}\bigl(Z^{({\boldsymbol{\alpha}})}_{rn}\bigr)\bigr\vert\Bigr)\,.$$

  • This implies $\lim_{n\to\infty} n^{-1} II_{rn}=0$ for any $r>0$, as
    $$\frac{1}{n}\;\bigl\vert Z^{(\cdot)}_{rn} -{\mathbb{E}\,}\bigl(Z^{(\cdot)}_{rn}\bigr)\bigr\vert\le 2c$$
    with probability $1$ for all $n>r$ and
    $$\lim\limits_{n\to\infty}\;\frac{1}{n}\;\bigl\vert Z^{({\boldsymbo... ...rn} -{\mathbb{E}\,}\bigl(Z^{({\boldsymbol{\alpha}})}_{rn}\bigr)\bigr\vert=0\,.$$

  • Furthermore, for $n>r>0$ we have the following estimate
    

    $$\frac{1}{n}\; III_{rn} \le \frac{1}{n}\;\sum\limits_{i=1}^\ell \Bigl({\mathbb{E}\,}\bigl(Z^{... ...\mathbb{E}\,} Z^{(\delta_i)}_{0,n-r}\bigr)^2 \Bigr) \vert\pi_i-\alpha_{ri}\vert$$

    $$\le \underbrace{\sup\limits_{n>0}\max\limits_{j\in\{1,\ldots,\ell\}}\... ..._{0n}\Bigr)^2}_{<\infty} \,\sum\limits_{i=1}^\ell\vert\pi_i-\alpha_{ri}\vert\,,$$

    
    
    where it is easy to see that the supremum is finite.
  • Due to the ergodicity of the Markov chain ${\mathbf{X}}^{({\boldsymbol{\alpha}})}_0,{\mathbf{X}}^{({\boldsymbol{\alpha}})}_1,\ldots$, the last summand will become arbitrarily small for sufficiently large $r$. This completes the proof of (80).
    
    $\Box$

Remarks

 

• Recall that
  • by Theorem 2.1 from ,,Statistik I'' we obtain for the mean squared error ${\mathbb{E}\,}(\bigl(\,\widehat\theta_n-\theta\bigr)^2)$ of the MCMC estimator defined in (70) $\widehat\theta_n=\widehat\theta({\mathbf{X}}_1,\ldots,{\mathbf{X}}_n)$ for $\theta$ that

    $${\mathbb{E}\,}(\bigl(\,\widehat\theta_n-\theta\bigr)^2) =\bigl({\mathbb{E}\,}\,\widehat\theta_n-\theta\bigr)^2 +{\rm Var\,}\,\widehat\theta_n\,,$$ (81)

    
    

  • i.e., the mean squared error of the MCMC estimator $\widehat\theta_n$ is equal to the sum of the squared bias $({\mathbb{E}\,}\,\widehat\theta_n-\theta)^2$ and the variance ${\rm Var\,}\,\widehat\theta_n$ of the estimator $\widehat\theta_n$.
• Both summands on the right hand side of (81) converge to 0 if $n\to\infty$ but with different rates of convergence.
  • In Theorem 3.19 we showed that ${\rm Var\,}\,\widehat\theta_n=O(n^{-1})$.
  • On the other hand, by Theorem 3.18 we get that $({\mathbb{E}\,}\,\widehat\theta_n-\theta)^2=O(n^{-2})$.
• Consequently, the asymptotic behavior of the mean squared error ${\mathbb{E}\,}(\bigl(\,\widehat\theta_n-\theta\bigr)^2)$ of $\widehat\theta_n$ is crucially influenced by the asymptotic variance ${\rm Var\,}\,\widehat\theta_n$ of the estimator, whereas the bias plays a minor role.

• In other words: It can make sense to choose the simulation matrix ${\mathbf{P}}$ such that
  • the asymptotic variance $\lim_{n\to\infty}n{\rm Var\,}\,\widehat\theta_n$ is as small as possible,
  • even if this results in a certain increase of the asymptotic bias $\lim_{n\to\infty}n\bigl({\mathbb{E}\,}\,\widehat\theta_n-\theta\bigr)$.

In order to investigate this problem more deeply we introduce the following notation: Let

$$V(\varphi,{\mathbf{P}},{\boldsymbol{\pi}})=\lim_{n\to\infty}n{\rm Var\,}\,\widehat\theta_n\,,$$

where $\varphi:E\to\mathbb{R}$ is an arbitrary function and $({\mathbf{P}},{\boldsymbol{\pi}})$ is an arbitrary reversible pair.

Theorem 3.20    

• Let ${\mathbf{P}}_1=(p_{1,ij})$ and ${\mathbf{P}}_2=(p_{2,ij})$ be two transition matrices on $E$ such that $({\mathbf{P}}_1,{\boldsymbol{\pi}})$ and $({\mathbf{P}}_2,{\boldsymbol{\pi}})$ are reversible.
  • For arbitrary $i,j\in E$ such that $i\not= j$ let $p_{1,ij}\ge p_{2,ij}$,
  • i.e., outside the diagonal all entries of the transition matrix ${\mathbf{P}}_1$ are greater or equal than the corresponding entries of the transition matrix ${\mathbf{P}}_2$.
• Then, for any function $\varphi:E\to\mathbb{R}$,

  $$V(\varphi,{\mathbf{P}}_1,{\boldsymbol{\pi}})\le V(\varphi,{\mathbf{P}}_2,{\boldsymbol{\pi}})\,.$$ (82)

  
  

Proof

 

• Let ${\mathbf{P}}=(p_{ij})$ be a transition matrix such that the pair $({\mathbf{P}},{\boldsymbol{\pi}})$ is reversible. It suffices to show that

  $$\frac{\partial}{\partial p_{ij}}\;V(\varphi,{\mathbf{P}},{\boldsymbol{\pi}})\le 0\,, \mbox{$\forall\, i,j\in E$\ with $i\not= j$.}$$ (83)

  
  

• By Theorem 3.19,

  $$\frac{\partial}{\partial p_{ij}}\;V(\varphi,{\mathbf{P}},{\boldsy... ...i}})\;\frac{\partial {\mathbf{Z}}}{\partial p_{ij}}\; {\boldsymbol{\varphi}}\,,$$ (84)

  
  
  where ${\mathbf{Z}}$ denotes the fundamental matrix of ${\mathbf{P}}$ introduced by (74).
  • On the other hand, as ${\mathbf{Z}}{\mathbf{Z}}^{-1}={\mathbf{I}}$, we get that
    $$\Bigl(\frac{\partial {\mathbf{Z}}}{\partial p_{ij}}\Bigr){\mathbf... ...{Z}} \Bigl(\frac{\partial {\mathbf{Z}}^{-1}}{\partial p_{ij}}\Bigr)={\,{\bf0}}$$
    and thus
    $$\frac{\partial {\mathbf{Z}}}{\partial p_{ij}}=-{\mathbf{Z}}\;\frac{\partial {\mathbf{Z}}^{-1}}{\partial p_{ij}}\;{\mathbf{Z}}\,.$$

  • Taking into account (84) this implies

    $$\frac{\partial}{\partial p_{ij}}\;V(\varphi,{\mathbf{P}},{\boldsy... ...tial {\mathbf{Z}}^{-1}}{\partial p_{ij}}\;{\mathbf{Z}}{\boldsymbol{\varphi}}\,.$$ (85)

    
    

• As the pair $({\mathbf{P}},{\boldsymbol{\pi}})$ is reversible, by the representation formula (75) for the fundamental matrix ${\mathbf{Z}}=(z_{ij})$ that was derived in Lemma 3.3 we obtain for arbitrary $i,j\in E$
  $$\pi_i z_{ij} = \pi_i\delta_{ij}+\sum\limits_{k=1}^\infty \Bigl(\p... ...limits_{k=1}^\infty \Bigl(\pi_jp_{ji}^{(k)}-\pi_j\pi_i\Bigr) = \pi_j z_{ji}\,.$$
  • This implies
    

    $${\boldsymbol{\pi}}^\top\, {\,{\rm diag}}({\boldsymbol{\varphi}}){\mathbf{Z}} = \Bigl(\sum\limits_{i=1}^\ell \pi_i\varphi_i z_{i1},\ldots,\sum\limits_{i=1}^\ell \pi_i\varphi_i z_{i\ell}\Bigr)$$

    $$= \Bigl(\pi_1\sum\limits_{i=1}^\ell z_{1i}\varphi_i ,\ldots,\pi_\ell\sum\limits_{i=1}^\ell z_{\ell i }\varphi_i \Bigr)$$

    $$= \bigl({\mathbf{Z}}{\boldsymbol{\varphi}}\bigr)^\top{\,{\rm diag}}({\boldsymbol{\pi}})\,.$$

    
    

  • Thus, by (85),

    $$\frac{\partial}{\partial p_{ij}}\;V(\varphi,{\mathbf{P}},{\boldsy... ...{\partial {\mathbf{P}}}{\partial p_{ij}}\;{\mathbf{Z}}{\boldsymbol{\varphi}}\,,$$ (86)

    
    
    where the last equality is due to the fact that
    $$\frac{\partial {\mathbf{Z}}^{-1}}{\partial p_{ij}}=-\;\frac{\partial {\mathbf{P}}}{\partial p_{ij}}$$
    which is an immediate consequence of the definition (74) of ${\mathbf{Z}}$.
• As ${\mathbf{P}}=(p_{ij})$ is a stochastic matrix and $({\mathbf{P}},{\boldsymbol{\pi}})$ is reversible
  • only the entries $p_{ij}$ where $i<j$ (or alternatively the entries $p_{ij}$ where $i>j$) can be chosen arbitrarily. This can be seen as follows.
  • For every pair $i,j\in E$ such that $i\not= j$ the entries $p_{ji}$, $p_{ii}$ and $p_{jj}$ can be expressed via $p_{ij}$ in the following way:
    $$p_{ji}=\frac{\pi_i}{\pi_j}\;p_{ij}\,,\qquad p_{ii}=c-p_{ij}\,,\qquad p_{jj}=c^\prime-\;\frac{\pi_i}{\pi_j}\;p_{ij}\,,$$
    where $c$ and $c^\prime$ are constants that do not depend on $p_{ij}$.
  • For arbitrary $i^\prime,j^\prime\in E$ the entry $\bigl({\,{\rm diag}}({\boldsymbol{\pi}})(\partial {\mathbf{P}}/\partial p_{ij})\bigr)_{i^\prime,j^\prime}$ of the matrix product ${\,{\rm diag}}({\boldsymbol{\pi}})(\partial {\mathbf{P}}/\partial p_{ij})$ is given by
    $$\Bigl({\,{\rm diag}}({\boldsymbol{\pi}}) \;\frac{\partial {\... ...rime)=(j,i)$,}\\ [3\jot] 0\,, &\mbox{else} \end{array}\right.$$

  • This implies that the matrix ${\,{\rm diag}}({\boldsymbol{\pi}})(\partial {\mathbf{P}}/\partial p_{ij})$ is non-negative definite, i.e., for all ${\mathbf{x}}\in\mathbb{R}^\ell$
    $${\mathbf{x}}^\top{\,{\rm diag}}({\boldsymbol{\pi}}) \;\frac{\partial {\mathbf{P}}}{\partial p_{ij}}\;{\mathbf{x}}\le 0\,.$$

  • By (86) this yields for arbitrary $i,j\in E$ such that $i\not= j$
    $$\frac{\partial}{\partial p_{ij}}\;V(\varphi,{\mathbf{P}},{\boldsy... ...ial {\mathbf{P}}}{\partial p_{ij}}\;{\mathbf{Z}}{\boldsymbol{\varphi}}\le 0\,.$$

• This completes the proof of (83).
  
  $\Box$

Remarks

$\;$ As a particular consequence of Theorem 3.20 we get that

• the simulation matrix ${\mathbf{P}}$ of the Metropolis algorithm (i.e. if we consider equality in (50) ) minimizes the asymptotic variance $V(\varphi,{\mathbf{P}},{\boldsymbol{\pi}})$
• within the class of all Metropolis-Hastings algorithms having an arbitrary but fixed ,,potential transition matrix'' ${\mathbf{Q}}=\bigl(q_{ij}\bigr)$.