Introduction

• Markov chains

  • are a fundamental class of stochastic models for sequences of non-independent random variables, i.e. of random variables possessing a specific dependency structure.
  • have numerous applications e.g. in insurance and finance.
  • play also an important role in mathematical modelling and analysis in a variety of other fields such as life sciences.
• Questions of scientific interest often exhibit a degree of complexity resulting in great difficulties if the attempt is made to find an adequate mathematical model that is solely based on analytical formulae.
• In these cases Markov chains can serve as an alternative analytical tool as they are crucial for the construction of computer algorithms for the Markov Chain Monte Carlo simulation (MCMC) of the mathematical models under consideration.

This course on Markov chains and Monte Carlo simulation will be based on the methods and models introduced in the course ``Wahrscheinlichkeitsrechnung''. Knowledge from ``Statistik I'' and ``Statistik II'' can be useful but is not required.

• The main focus of this course will be on the following topics:
  • discrete-time Markov chains with finite state space
  • stationarity and ergodicity
  • Markov Chain Monte Carlo (MCMC)
  • reversibility and coupling algorithms
• Notions and results introduced in ``Wahrscheinlichkeitsrechnung'' will be used frequently. Hence, the lecture notes ``Wahrscheinlichkeitsrechnung'', WS 2003/2004, will be an important reference; see
  http://www.mathematik.uni-ulm.de/stochastik/lehre/ws03_04/wr/skript/skript.html
• References to these lecture notes will be labelled by the prefix ``WR'' in front of the number specifying the corresponding section, theorem, lemma, etc.

• The following list contains only a small collection of introductory texts that can be recommended for in-depth studies of the subject complementing the lecture notes.
  • D. Aldous, J.A. Fill (2002) Reversible Markov Chains and Random Walks on Graphs. manuscript
  • E. Behrends (2000) Introduction to Markov Chains. Vieweg, Braunschweig
  • P. Bremaud (1999) Markov Chains, Gibbs Fields, Monte Carlo Simulation, and Queues. Springer, New York
  • B. Chalmond (2003) Modeling and Inverse Problems in Image Analysis. Springer, New York
  • O. Häggström (2002) Finite Markov Chains and Algorithmic Applications. Cambridge University Press, Cambridge
  • U. Krengel (2002) Einführung in die Wahrscheinlichkeitstheorie und Statistik. Vieweg, Braunschweig
  • S.I. Resnick (1992) Adventures in Stochastic Processes. Birkhäuser, Boston
  • T. Rolski, H. Schmidli, V. Schmidt, J. Teugels (2002) Stochastic Processes for Insurance and Finance. Wiley, Chichester
  • H. Thorisson (2002) Coupling, Stationarity, and Regeneration. Springer, New York
  • G. Winkler (2003) Image Analysis, Random Fields and Markov Chain Monte Carlo Methods. Springer, Berlin