Learning outcomes

The Stochastic Systems course aims at giving an introduction to the basic concepts underlying the rigorous mathematical description of the temporal dynamics for random quantities. The course prerequisites are those of a standard course in Probability, for Mathematics / Physics. It is supposed that students are familiar with the basics Probability calculus, in the Kolmogorov assiomatisation setting, in particular with respect to the concepts of density function, probability distribution, conditional probability, conditional expectation for random variables, measure theory (basic ), characteristic functions of random variables, convrgence theorems (in measure, almost everywhere, etc.), central limit theorem and its (basic) applications, etc. The Stochastic Systems course aims, in particular, to provide the basic concepts of: Filtered probability space, martingale processes, stopping times, Doob theorems, theory of Markov chains in discrete and continuous time (classification of states, invariant and limit,measures, ergodic theorems, etc.), basics on queues theory and an introduction to Brownian motion. A part of the course is devoted to the computer implementation of operational concepts underlying the discussion of stochastic systems of the Markov chain type, both in discrete and continuous time. A part of the course is dedicated to the introduction and the operational study, via computer simulations, to univariate time series. It is important to emphasize how the Stochastic Systems course is organized in such a way that students can concretely complete and further develop their own: capacity of analysis, synthesis and abstraction; specific computational and computer skills; ability to understand texts, even advanced, of Mathematics in general and Applied Mathematics in particular; ability to develop mathematical models for physical and natural sciences, while being able to analyze its limits and actual applicability, even from a computational point of view; skills concerning how to develop mathematical and statistical models for the economy and financial markets; capacity to extract qualitative information from quantitative data; knowledge of programming languages or specific software.

Program

The entire course will be available online. In addition, a number of the lessons/all the lessons (see the course
schedule) will be held in-class.

1. Conditional Expectation and Conditional Distribution. Martingale. Stopping theorem and convergence theorem.

2. Discrete-time Markov chains. Markov properties and transition probability. Irreducibility, aperiodicity. Stationary distributions. Reversible distributions.

3. Hitting times. One step analysis. Convergence to the stationary distribution. Law of large numbers for Markov chains. MCMC: Metropolis algorithm and Gibbs sampler.

4. Reducible Markov chains. Transient and recurring states. Probability of absorption.

5. Continuous-time Markov chains. The Poisson process and its properties. Continuous-time Markov property. Semigroup associated with a Markov chain: continuity and differentiability; generator. Kolmogorov equations. Stationary distributions. Dynkin's formula. Probabilistic construction of a continuous-time Markov chain.

Examination Methods