Home | deutsch  |  Impressum  |  Data Protection  |  Sitemap  |  Intranet  |  KIT
Institute of Stochastics

Secretariat
Kollegiengebäude Mathematik (20.30)
Room 2.056 und 2.002

Address
Hausadresse:
Karlsruher Institut für Technologie (KIT)
Institut für Stochastik
Englerstr. 2
D-76131 Karlsruhe

Postadresse:
Karlsruher Institut für Technologie (KIT)
Institut für Stochastik
Postfach 6980
D-76049 Karlsruhe

Office hours:
Mo-Fr 10:00 - 12:00

Tel.: +49 721 608 43270/43265

Fax.: +49 721 608 46066

Markovsche Entscheidungsprozesse / Markov Decision Processes (Summer Semester 2012)

Lecturer: Prof. Dr. Nicole Bäuerle
Classes: Lecture (0159900), Problem class (0160000)
Weekly hours: 2+1


There will be no exercise on Friday, the 13th of July. The exercise on Friday, the 20th of July will be held as normal and the corresponding exercise sheet can be downloaded from the "Studierendenportal".

Schedule
Lecture: Wednesday 8:00-9:30 Z 1
Problem class: Friday 11:30-13:00 Z 1
Lecturers
Lecturer Prof. Dr. Nicole Bäuerle
Office hours: by appointment.
Room 2.016 Kollegiengebäude Mathematik (20.30)
Email: nicole.baeuerle@kit.edu

Content

Suppose a system is given which can be controlled by sequential decisions. The state transitions are random and we assume that the system state process is Markovian which means that previous states have no influence on future states. Given the current state of the system (which could be for example the wealth of an investor) the controller or decision maker has to choose an admissible action (for example a possible investment). Once an action is chosen
there is a random system transition according to a stochastic law (for example a change in the asset value) which leads to a new state. Markov Decision Processes theory deals with controlling the process in such a way that the expected discounted reward of the system is maximized.
We will consider problems with finite and infinite horizon, stopping problems and problems with partial observation. Applications in the area of portfolio optimization, queueing and operations research are considered.


Prerequisite

Excellent knowledge of probability theory. Basic knowledge in Markov chains is helpful.

References

Bäuerle, N. and Rieder, U. (2011): Markov Decision Processes with applications to finance. Springer-Verlag.

Bertsekas, D. P. (2001) Dynamic programming and optimal control. Vol.II. Athena Scientific.

Bertsekas, D. P. (2005) Dynamic programming and optimal control. Vol.I. Athena Scientific.

Hernández-Lerma, O. and Lasserre, J. B. (1996) Discrete-time Markov control processes. Springer-Verlag.

Hernández-Lerma, O. and Lasserre, J. B. (1999) Further topics on discrete-time Markov control processes. Springer-Verlag.

Puterman, M. L. (1994) Markov decision processes: discrete stochastic dynamic programming. John Wiley & Sons.

Ross, S. (1983) Introduction to stochastic dynamic programming. Academic Press.

Schäl, M. (1990) Markoffsche Entscheidungsprozesse. B. G. Teubner.