Recursive Construction of the ,,Past''

• Recall that
  • in Section 2.1.3 we showed that a stationary Markov chain $X_0,X_1,\ldots$ with transition matrix ${\mathbf{P}}=(p_{ij})$ and stationary initial distribution ${\boldsymbol{\alpha}}=(\alpha_1,\ldots,\alpha_\ell)^\top$ can be constructed as follows, where
  • we started with a sequence $Z_0,Z_1,\ldots$ of independent and on $[0,1]$ uniformly distributed random variables and defined
    $$\displaystyle X_0=k$$   if and only if$$\qquad Z_0\in\Bigl(\sum_{i=1}^{k-1} \alpha_i,\sum_{i=1}^{k}\alpha_i\Bigr]\,,$$
    for all $k=1,\ldots,\ell$, i.e.

    $$X_0=\sum_{k=1}^\ell k {1\hspace{-1mm}{\rm I}}\Bigl(\sum_{i=1}^{k-1}\alpha_i <Z_0\le \sum_{i=1}^{k}\alpha_i\Bigr)\,.$$ (92)

    
    

  • The random variables $X_1,X_2,\ldots$ were defined by the recursion formula

    $$X_n=\varphi(X_{n-1},Z_n) \mbox{for $n=1,2,\ldots,$}$$ (93)

    
    
    where the function $\varphi:E\times[0,1]\to E$ was given by

    $$\varphi(i,z)=\sum_{k=1}^\ell k {1\hspace{-1mm}{\rm I}}\Bigl(\sum_{j=1}^{k-1}p_{ij} <z\le \sum_{j=1}^{k}p_{ij}\Bigr)\,.$$ (94)

    
    

• If the pair $({\mathbf{P}},{\boldsymbol{\alpha}})$ is reversible, then the stationary Markov chain $X_0,X_1,\ldots$ constructed in (92)-(94) can be tracked back into the past in the following way.
  • First of all we extend the sequence $Z_0,Z_1,\ldots$ of independent and on $[0,1]$ uniformly distributed random variables to a sequence $\ldots,Z_{-1},Z_0,Z_1,\ldots$ of independent and identically random variables that is unbounded in both directions.
  • Note that due to the assumed independence of $\ldots,Z_{-1},Z_0,Z_1,\ldots$ this expansion does not pose any problems as the underlying probability space can be constructed via an appropriate product space, product-$\sigma$-algebra, and product measure.
  • The random variables $X_{-1},X_{-2},\ldots$ are now constructed recursively setting

    $$X_{n-1}=\varphi(X_n,Z_{n-1}) \mbox{for $n=0,-1,\ldots,$}$$ (95)

    
    
    where the function $\varphi:E\times[0,1]\to E$ is defined in (94).

Theorem 2.14    

• Let $X_0,X_1,\ldots:\Omega\to E$ be a reversible Markov chain with state space $E$, transition matrix ${\mathbf{P}}=(p_{ij})$ and stationary initial distribution ${\boldsymbol{\alpha}}=(\alpha_1,\ldots,\alpha_\ell)^\top$.
• Then the sequence $\ldots,X_{-1},X_0,X_1,\ldots:\Omega\to E$ defined by $(\ref{fir.ste.sim.rev})$- $(\ref{nth.ste.sim.ver})$ is
  • a stationary Markov chain with transition matrix ${\mathbf{P}}$ and the one-dimensional marginal distribution ${\boldsymbol{\alpha}}$,
  • i.e., for arbitrary $k\in \mathbb{Z}=\{\ldots,-1,0,1,\ldots\}$, $i_k,i_{k+1},\ldots,i_n\in E$ and $m\ge 1$
    

    $${ P(X_k=i_k,X_{k+1}=i_{k+1},\ldots,X_{n-1}=i_{n-1},X_n=i_n)}$$

    $$= P(X_{k+m}=i_k,X_{k+m+1}=i_{k+1},\ldots,X_{n+m-1}=i_{n-1},X_{n+m}=i_n)$$

    $$= \alpha_{i_k}p_{i_ki_{k+1}}\ldots p_{i_{n-1}i_n}\,.$$

    
    

The proof of Theorem 2.14 is quite similar to the ones given for Theorems 2.11 and 2.13 and is therefore omitted.