Stochastic systems (2013/2014)

Course code
Laura Maria Morato
Academic sector
Language of instruction
Teaching is organised as follows:
Activity Credits Period Academic staff Timetable
Esercitazioni 1 I semestre Marco Caliari
Catene di Markov in tempo discreto 3 I semestre Laura Maria Morato
Analisi di serie temporali 2 I semestre Federico Di Palma

Lesson timetable

Learning outcomes

Module 1 ( Discrete time Markov Chains )

Basics of the theory of discrete time Markov chain with finite or countable state space and examples of application.

Module 2 (Practice session of Stochastic systems)

Approximation and computation of invariant probabilities, Metropolis algorithm, simulation of queues and renewal processes with the use of Matlab.

Module 3 Introduction to Time Series analysis: the lessons aims to provide to the student a general framework to analyze time series as the outcome of a discrete time model fed by a white noise and an exogenous input. The lesson are completed by the use of a dedicated software in order to apply the theoretical aspects.


Module 1
Markov chains with finite space state:
Definitions, transition matrix, transition probability in n steps, Chapman -Kolmogorov equation, finite joint densities, Canonocal space and Kolmogorov theorem (without proof).
State classification, invariant probabilities, Markov-Kakutani theorem, example of gambler's ruin, regular chains, criterion, limit probabilities and Markov theorem, reversible chains, Metropolis algorithm and Simulated annealing, numerical generation of a discrete random variable and algorithm for generation an omogeneus Markov chains with finite state space.

Markov chains with countable space state:
Equivalent definitions of transient and recurrent state, positive recurrence, periodicity, solidarity property, canonical decomposition of the state space, invariant measures, existence theorem, example of the unlimited random walk. Ergodicity and limit theorems.

Elements of Martingales associated to discrete time Markov chains:
Natural filtration, stopping times, conditional expectation given a random variable, strong Markov property, martingales. Optional stopping Theorem, example of gambler's ruin.

Module 2 Approximation and computation of invariant probabilities, Metropolis algorithm, simulation of queues with the use of Matlab.

Module 3 Elements of time series analysis :
Main scope of time series analysis: modelling, prediction and simulation.
Identification problem main components: a priori Knowledge, experiment design, goodness criteria, model, filtering and validation.
Model: main variables and correspondent schema. (AR, ARX, ARMA, output-error).
Goodness Criteria: least square, Maximum Likelihood, Maximum a posteriori.
Filtering: Linear parameter model, frequency filtering.
Matlab : main purpose and examples.

Assessment methods and criteria

Module 1 Oral exam

Module 2 Discussion of the solution of given homeworks.

Module 3 Written exam

Reference books
Activity Author Title Publisher Year ISBN Note
Analisi di serie temporali LJung System Identification, Theory for the User (Edizione 2) Prentice Hall PTR 1999
Teaching aids
Title Format (Language, Size, Publication date)
Elaborato 1 - appello del 5 febbraio  pdfpdf (it, 54 KB, 30/01/14)
Elaborato 2 - testo e linee guida per l'appello del 19 febbraio  pdfpdf (it, 75 KB, 13/02/14)
Elaborato 3 - testo e linee guida per l'appello del 25 luglio  pdfpdf (it, 701 KB, 18/06/14)
Errata Corrige I - Esercitazione IV  pdfpdf (it, 60 KB, 17/02/14)
Esercitazione 1 del 20-11: predizione e simulazione  zipzip (it, 1315 KB, 17/12/13)
Esercitazione 2 del 27-11: identificazione su errore di predizione  zipzip (it, 649 KB, 17/12/13)
Esercitazione 3 del 04-12: Identificazione ML e MAP  zipzip (it, 1292 KB, 17/12/13)
Esercitazione 4 del 11-12: Validazione Modelli  zipzip (it, 628 KB, 17/02/14)
Esercitazione 5 del 18-12: Simulazione d'esame  zipzip (it, 661 KB, 17/12/13)