The Stochastic Systems course aims at giving an introduction to the basic concepts underlying the rigorous mathematical description of the temporal dynamics for random quantities.

The course prerequisites are those of a standard course in Probability, for Mathematics / Physics.

It is supposed that students are familiar with the basics Probability calculus, in the Kolmogorov assiomatisation setting, in particular with respect to the concepts of density function, probability distribution, conditional probability, conditional expectation for random variables, measure theory (basic ), characteristic functions of random variables, etc.

The Stochastic Systems course aims, in particular, to provide the basic concepts of: Filtered probability space, martingale processes, stopping times, Doob theorems, theory of Markov chains in discrete and continuous time (classification of states, invariant and limit,measures, ergodic theorems, etc.), basics on queues theory and an introduction to Brownian motion.

A part of the course is devoted to the computer implementation of operational concepts underlying the discussion of stochastic systems of the Markov chain type, both in discrete and continuous time.

A part of the course is dedicated to the introduction and the operational study, via computer simulations, to univariate time series.

Stochastic Systems [ Applied Mathematics ]
AA 2015/2016

Syllabus

• Conditional Expectetion ( from Chap.1 of [BMP] )
• Definitions and basic properties
• Conditional expectations and conditional laws

• Introduction to stochastic processes ( From Chap.1 di [BMP] )
• Filtered probability space, filtrations
• Adapted stochastic process (wrt a given filtration)
• Martingale (first definitions and examples: Markov chains)
• Kolmogorov characterization theorem
• Stopping times

• Martingale ( From Chap.3 of [BMP]
• Definition of martingale process, resp. super, resp. lower, martingale
• Fundamental properties
• Stopping times for martingale processes
• Convergence theorems for martingales

• Markov chains (MC) ( From Chap.4 of [Beichelet] , Chap.5 of di [Baldi] )
• Transition matrix for a MC
• Construction and existence for MC
• Omogeneous MC (with respect to time and space)
• Canonical MC
• Classification of states for a given MC ( and associated classes )
• Chapman-Kolmogorov equation
• Recurrent, resp. transient, states ( classification criteria )
• Irriducible and recurrent chains
• Invariant (stationary) measures, ergodic measures, limit measures ( Ergodic theorem )
• Birth and death processes (discrete time)

• Continuous time MC ( From Chap.5 of [Beichelt] )
• Basic definitions
• Chapman-Kolmogorov equations
• Absolute and stationary distributions
• States classifications
• Probability and rates of transition
• Kolmogorov differential equations
• Stationary laws
• Birth and death processes ( first steps in continuous time )
• Queque theory (first steps in continuous time)

• Point, Counting and Poisson Processes ( From Chap.3 of [Beichelt] )
• Basic definitions and properties
• Stochastic point processes (SPP) and Stochastic Counting Processes (SCP)
• Marked SPP
• Stationarity, intensity and composition for SPP and SCP
• Homogeneous Poisson Processes (HPP)
• Non Homogeneous Poisson Processes (nHPP)
• Mixed Poisson Processes (MPP)

• Birth and Death processes (B&D) ( From Chap.5 of [Beichelt] )
• Birth processes
• Death processes
• B&D processes
° Time-dependent state probabilities
° Stationary state probabilities
° Inhomogeneous B&D processes

• An introduction to quequing theory (From Chap.5 of [Beichelt] )
• Basic concepts
• Classification A/B/s/m by Kendall
• Explicitly studied examples:
° M/M/+\infty
° M/M/s/0
# partial results for M/M/+\infty e M/G/+\infty
° M/M/s/+\infty
• Erlang's loss formula
• Little's formula


• Brownian Motion (BM) ( From Chap.7 of [Beichelt] )
• Definitions and basic properties
• Transformations of 1-dimensional BM
° exponential martingale
° variance martingale


Bibliography

Text used in the course are:

[Baldi] P. Baldi, Calcolo delle Probabilità, McGraw-Hill Edizioni (Ed. 01/2007)

[Beichelt] F. Beichelt, Stochastic Processes in Science, Engineering and Finance, Chapman & Hall/CRC, Taylor & Francis group, (Ed. 2006)

[BPM] P. Baldi, L. Matzliak and P. Priouret, Martingales and Markov Chains – Solve Exercises and Elements of Theory, Chapman & Hall/CRC (English edition, 2002)


Further interesting books are:

N. Pintacuda, Catene di Markov, Edizioni ETS (ed. 2000)

Brémaud, P., Markov Chains. Gibbs Fields, Monte Carlo Simulation, and Queues, Texts in Applied Mathematics, 31. Springer-Verlag, New York, 1999

Duflo, M., Random Iterative Models, Applications of Mathematics, 34. SpringerVerlag, Berlin, 1997

Durrett, R., Probability: Theory and Examples, Wadsworth and Brooks, Pacific Grove CA, 1991

Grimmett, G. R. and Stirzaker, D. R., Probability and Random Processes. Solved Problems. Second edition. The Clarendon Press, Oxford University Press, New York, 1991

Hoel, P. G., Port, S. C. and Stone, C. J., Introduction to Stochastic Processes, Houghton Mifflin, Boston, 1972

The course is diveded into the following three parts

1) Theory of stochastic systems
2) Introduction to time-series analysis
3) Computer exercises ( mainly based on the theory of Markov Chains, in discrete as well in continuous time )

The exam will be subdivided into the following three parts

* a written exam concerning point (1)
* a project presented in agreement with the programme developed with prof. Marco Caliari (point 3)
* exercises and a project concerning point (2)

The programme concerning the written exam, with respect to point (1), is the one reported in the Program section.
The project to be presented with prof. Caliari has to be decided with him.
The project to be presented with respect to point (2), will be chosen, by each student, within the the following list

========
Projects
========

1-Compare the following methods of estimate and/or elimination of time series trends

*First order differences study
*Smoothing with moving average filter
*Fourier transform
*Exponential Smoothing
*Polynomial Data fitting

2- Describe and provide a numerical implementation of the one-step predictor for the following models

FIR(4)
ARX(3,1)
OE(3,1)
ARMA(2,3)
ARMAX(2,1,2)
Box-Jenkins(nb,nc,nd,nf)

3- Compare the Prediction Error Minimization (PEM) and the Maximum Likelihood (ML) approach for the identification of the model parameters (it requires a personal effort in the homes ML)

4- Provide a concrete implementation for the k-fold cross-validation, e.g. using Matlab/Octave, following the example-test that has been given during the lessons

5-Detailed explanation of (at least) one of the following test
*Shapiro-Wilk
*Kolmogorov-Smirnov
*Lilliefors