Institute for Applied Mathematics  / Probability Theory and Stochastic Analysis  / Teaching
University of Bonn Institute for Applied Mathematics
 
NoLinebreak

Markov Processes, Winter term 2025/26

Tuesdays 12.15-14.00, KHS, and Thursdays 12.15-14.00, GHS, We. 10.

Lecture course: Andreas Eberle

Exercises: Francis Lörler

Tutorial classes:

  • Mondays 12-14, 0.006
  • Wednesdays 8-10, 0.006

Exam: oral, 19.2., 26.2., 6.3., 26.3.

The course will cover Chapters 1-6 in the lecture notes 

Markov Processes 2025

Topics to be covered include

  • Markov chains in discrete time (Generator, martingales, recurrence and transience, Harris Theorem, ergodic averages, central limit theorem)
  • Markov chains in continuous time and piecewise deterministic Markov processes (Construction, generator, forward and backward equations, interacting particle systems on finite graphs)
  • General Markov processes (Semigroups and generators, Feller and L2 approach, martingale problem, Brownian motion with different boundary behaviours, h transform, diffusions, interacting particle systems on Zd)
  • Long time behaviour (invariant measures, ergodic theory, phase transitions)
  • Limits (Weak convergence, Donsker invariance principle, limits of martingale problems)

Optionally, the course can be combined with the two hour course 

V5F6 - Selected Topics in Applied Probability - Mixing times and Markov Chain Monte Carlo methods 

This course studies related questions from more a more applied perspective and covers in particular material from Chapters 7 and 8 in the lecture notes above.

Prerequisites: Conditional expectations, martingales, Brownian motion. Most importantly, you should be familiar with the general definition of conditional expectations and conditional probability laws. Some background on functional analysis can be helpful but is not assumed.

My lecture notes of the foundations course on Stochastic Processes are available here. There you find all the necessary background material. Alternatively, you may consult the more compact book Probability Theory by Varadhan. Sections up to 5.5 in this book have been covered in previous courses and will be assumed. Section 5.7 and Chapter 6 will be covered in this course.


Further Material:

Problem Sheets

Simulations

  • Euler-Maruyama approximation of diffusion processes (ipynb, pdf)

Remark on exercises with a programming part: These can be done in a programming language/system of your choice - just submit a printout of your listings. If you do not have another preference, we recommend Python with Jupyter Notebooks for which examples and model solutions will be provided here.

Installation: We recommend the distribution Anaconda. It already contains all packages that we will need (at first, these are mainly NumPy for numerical computations and matplotlib for 2D-plots), as well as many others. A brief intoduction and help for installation can be found here and here. Once you have installed and started Jupyter notebooks, you should go through and execute the following example notebook (ipynb-format) which contains a short introduction to Python that concludes with a simulation of random walks:

  • Introduction to Python (ipynb , html ); Notebook including output (html)

A good next reference are the Scipy lecture notes that contain introductions to Python, NumPy and matplotlib. Further useful references:

 

November 2025 Andreas Eberle eberle@uni-bonn.de