Markov Chains

- Markov chains can describe the (temporal) dynamics of objects,
systems, etc.
- that can possess one of
*finitely*or*countably*many possible configurations at a given time, - where these configurations will be called the
*states*of the considered object or system, respectively.

- that can possess one of
- Examples for this class of objects and systems are
- the current prices of products like insurance policies, stocks or bonds, if they are observed on a discrete (e.g. integer) time scale,
- the monthly profit of a business,
- the current length of the checkout lines (so-called ``queues'') in a grocery store,
- the vector of temperature, air pressure, precipitation and wind velocity recorded on an hourly basis at the meteorological office Ulm-Kuhberg,
- digital maps, for example describing the momentary spatial dispersion of a disease.
- microscopical 2D or 3D images describing the current state (i.e. structural geometrical properties) of biological tissues or technical materials such as metals or ceramics.

**Remarks**-
- In this course we will focus on
*discrete-time*Markov chains, i.e., the temporal dynamics of the considered objects, systems etc. will be observed*stepwise*, e.g. at integer points in time. - The algorithms for Markov Chain Monte Carlo simulation we will discuss in part II of the course are based on exactly these discrete-time Markov chains.
- The number of potential states can be very high.
- For mathematical reasons it is therefore convenient to consider the
case of infinitely many states as well. As long as the infinite case
is restricted to
*countably*many states, only slight methodological changes will be necessary.

- In this course we will focus on

- Specification of the Model and Examples
- Ergodicity and Stationarity
- Reversibility; Estimates for the Rate of Convergence
- Definition and Examples
- Recursive Construction of the ,,Past''
- Determining the Rate of Convergence under Reversibility
- Multiplicative Reversible Version of the Transition Matrix; Spectral Representation
- Alternative Estimate for the Rate of Convergence; Contrast
- Dirichlet-Forms and Rayleigh-Theorem
- Bounds for the Eigenvalues and