Learning objectives

The aim of the course is to present some classes of probabilistic models of particular relevance in applications, in particular dynamic models. The emphasis is placed, in addition to mathematical rigor, on developing the ability to grasp the essential aspects of a real phenomenon and translate them into a model whose analysis, analytical or numerical, is accessible.The main topic of the course is the theory of Markov chains, both in discrete and continuous time. Each development of the theory is accompanied by the presentation of examples of applicative interest, motivated by economics, physical and biological sciences, but also by computational problems that emerge in the search for efficient algorithms. In the final part of the course the notions of conditional expectation and martingale will be introduced.At the end of the course, the student will have the tools to use a wide range of probabilistic models in both theoretical and applicative contexts, understanding their limits and effective applicability, also from a computational point of view. He will also be able to have a unifying and abstract vision of classes of problems with similar characteristics, and to face the reading of advanced texts.

Prerequisites and basic notions

Basic competencies of Mathematical Analysis, Linear Algebra, and Probability are required. In particular, the following knowledge is expected:
-- sequences and series of real numbers; limits; differential and integral calculus for functions of one or several variables; first order linear differential equations (or systems of equations);
-- matrices and linear systems; eigenvalues and eigenvectors;
-- probability space; conditional probability; discrete and absolutely continuous random variables and random vectors: probability density functions and expected value; law of large numbers.

Program

1. Conditional expectation and conditional distribution. Martingale. Stopping theorem and convergence theorem.
2. Discrete-time Markov chains. Markov property and transition probabilities. Irreducibility, aperiodicity. Stationary distributions. Reversible distributions.
3. Hitting times. One step analysis. Convergence to the stationary distribution. Law of large numbers for Markov chains. Markov Chain Monte Carlo methods: Metropolis algorithm and Gibbs sampler.
4. Reducible Markov chains. Transient and recurrent states. Absorption probabilities.
5. Continuous-time Markov chains. The Poisson process and its properties. Continuous-time Markov property. Semigroup associated with a Markov chain: continuity and differentiability; generator. Kolmogorov equations. Stationary distributions. Dynkin's formula. Probabilistic construction of a continuous-time Markov chain.

Bibliography

Didactic methods

All the topics will be illustrated in class. Additional material, such as exercises, lecture notes, and further references, will be available on the Moodle page of the course.

Learning assessment procedures

The exam consists of a written test and an oral test, both of which must be passed in the same exam session. To pass the exam, students must achieve an overall grade of at least 18/30.
-- The written test, with a maximum duration of 150 minutes, requires solving exercises based on the course content. The written test is considered passed if the student obtains a score of at least 16/30.
-- The oral test must be taken in the same exam session in which the written test was passed. It consists of a discussion of the theoretical topics covered in the course. If deemed sufficient, the oral test may be awarded a score from 0 to 4 points (out of 30), which will be added to the written test score. If the oral test is not passed or not taken, the grade of the written test is cancelled.
During the exam, the use of any artificial intelligence tool is strictly prohibited. Academic integrity requires students to demonstrate their own level of learning, as only through independent work and critical thinking can they acquire the skills and knowledge that will be assessed.

Evaluation criteria

To pass the exam, the student must demonstrate:
-- to have understood the theoretical notions, showing detailed knowledge of definitions and statements, as well as of some proofs;
-- to be able to apply theory to problem-solving;
-- to be able to express themselves clearly and rigorously, using technical terminology appropriately.

Exam language

Italiano

Sustainable Development Goals - SDGs

This initiative contributes to the achievement of the Sustainable Development Goals of the UN Agenda 2030. More information on sustainability