Stationary Initial Distributions

- Recall
- If is an irreducible and aperiodic Markov chain with (finite) state space and (quasi-positive) transition matrix ,
- then the limit distribution
is the uniquely determined probability solution of the following
matrix equation (see Theorem 2.5):

- If the Markov chain is not assumed to be irreducible there can be more than one solution for (59).
- Conversely, it is possible to show that

- Let be an irreducible transition matrix, where .
- For arbitrary but fixed the entries
of
the stochastic
-dimensional matrices
where

converge to a limit

which does not depend on . The vector is a solution of the matrix equation and satisfies . - The distribution given by - is the only probability solution of .

A *proof* of Theorem 2.10 can be found in
Chapter 7 of E. Behrends (2000) *Introduction to Markov
Chains*, Vieweg, Braunschweig.

**Remarks**-
- Besides the invariance property
,
the Markov chain with stationary initial distribution
exhibits still another invariance property for
*all*finite dimensional distributions that is considerably stronger. - In this context we consider the following notion of a (strongly)
*stationary sequence*of random variables.

- Besides the invariance property
,
the Markov chain with stationary initial distribution
exhibits still another invariance property for
**Definition**-
- Let be an arbitrary sequence of random variables mapping into (which is not necessarily a Markov chain).
- The sequence of -valued random variables is called
*stationary*if for arbitrary and

- Let be a Markov chain with state space .
- Then is a stationary sequence of random variables if and only if the Markov chain has a stationary initial distribution.

**Proof**-
- The necessity of the condition follows immediately
- Conversely, suppose now that is a stationary initial distribution of the Markov chain .

**Remarks**-
- For some Markov chains, whose transition matrices exhibit a specific structure, we already calculated their stationary initial distributions in Sections 2.2.2 and 2.2.3.
- Now we will discuss two additional examples of this type.
- In these examples the state space is
*infinite*requiring an additional condition apart from quasi-positivity (or irreducibility and aperiodicity) in order to ensure the ergodicity of the Markov chains. - Namely, a so-called
*contraction condition*is imposed that prevents the probability mass to ,,migrate towards infinity''.

- In these examples the state space is

**Examples**-
*Queues*

see. T. Rolski, H. Schmidli, V. Schmidt, J. Teugels (2002)*Stochastic Processes for Insurance and Finance*. J. Wiley & Sons, Chichester, S. 147 ff.

- We consider the example already discussed in
Section 2.1.2
- of the recursively defined Markov chain
with and

- where the random variables
are independent and
identically distributed and the transition matrix
is given by

- of the recursively defined Markov chain
with and
- It is not difficult to show that
- Furthermore
- Thus, for Markov chains with (countably) infinite state space,
- irreducibility and aperiodicity do
*not always*imply ergodicity, - but, additionally, a certain
*contraction condition*needs to be satisfied, - where in the present example this condition is the requirement of
a
*negative drift*, i.e.,

.

- irreducibility and aperiodicity do
- If the conditions (65) are satisfied and
, then
- the equation has a uniquely determined probability solution ,
- which coincides with but which in general cannot be determined explicitly.
- However, there is a simple formula for the
*generating function*of , where

- Namely, we have

where and is the generating function of .

*Proof*of (68)

- We consider the example already discussed in
Section 2.1.2
*Birth and death processes with one reflecting barrier*

- We modify the example of the death and birth process discussed in
Section 2.2.3 now considering the infinite state space
and the transition matrix

where , and is assumed for all . - The linear equation system
is
of the form

- Similarly to the birth and death processes with two reflecting barriers one can show that
- As we assume and for all birth and death processes with one reflecting barrier are obviously irreducible.
- Furthermore, if for some then birth and death processes with one reflecting barrier are also aperiodic (as well as ergodic if the contraction condition (72) is satisfied).

- We modify the example of the death and birth process discussed in
Section 2.2.3 now considering the infinite state space
and the transition matrix