The teaching is organized as follows:

Learning outcomes

Module 1 ( Discrete time Markov Chains )

Basics of the theory of discrete time Markov chain with finite or countable state space and examples of application.


Module 2 (Practice session of Stochastic systems)

Approximation and computation of invariant probabilities, Metropolis algorithm, simulation of queues and renewal processes with the use of Matlab.

Module 3 Introduction to Time Series analysis: the lessons aims to provide to the student a general framework to analyze time series as the outcome of a discrete time model fed by a white noise and an exogenous input. The lesson are completed by the use of a dedicated software in order to apply the theoretical aspects.

Program

Module 1
Markov chains with finite space state:
Definitions, transition matrix, transition probability in n steps, Chapman -Kolmogorov equation, finite joint densities, Canonocal space and Kolmogorov theorem (without proof).
State classification, invariant probabilities, Markov-Kakutani theorem, example of gambler's ruin, regular chains, criterion, limit probabilities and Markov theorem, reversible chains, Metropolis algorithm and Simulated annealing, numerical generation of a discrete random variable and algorithm for generation an omogeneus Markov chains with finite state space.

Markov chains with countable space state:
Equivalent definitions of transient and recurrent state, positive recurrence, periodicity, solidarity property, canonical decomposition of the state space, invariant measures, existence theorem, example of the unlimited random walk. Ergodicity and limit theorems.

Elements of Martingales associated to discrete time Markov chains:
Natural filtration, stopping times, conditional expectation given a random variable, strong Markov property, martingales. Optional stopping Theorem, example of gambler's ruin.

Module 2 Approximation and computation of invariant probabilities, Metropolis algorithm, simulation of queues with the use of Matlab.

Module 3 Elements of time series analysis :
Main scope of time series analysis: modelling, prediction and simulation.
Identification problem main components: a priori Knowledge, experiment design, goodness criteria, model, filtering and validation.
Model: main variables and correspondent schema. (AR, ARX, ARMA, output-error).
Goodness Criteria: least square, Maximum Likelihood, Maximum a posteriori.
Filtering: Linear parameter model, frequency filtering.
Matlab : main purpose and examples.

Bibliography

Examination Methods

Module 1 Oral exam

Module 2 Discussion of the solution of given homeworks.

Module 3 Written exam