# Stochastic systems (2013/2014)

Course code
4S00254
Credits
6
Coordinator
Laura Maria Morato
MAT/06 - PROBABILITY AND STATISTICS
Language of instruction
Italian
Teaching is organised as follows:
Activity Credits Period Academic staff Timetable
Esercitazioni 1 I semestre Marco Caliari
Catene di Markov in tempo discreto 3 I semestre Laura Maria Morato
Analisi di serie temporali 2 I semestre Federico Di Palma

### Lesson timetable

I semestre
Activity Day Time Type Place Note
Esercitazioni Friday 10:30 AM - 12:30 PM practice session Laboratory Alfa
Catene di Markov in tempo discreto Monday 2:30 PM - 4:30 PM lesson Lecture Hall G from Oct 1, 2013  to Nov 30, 2013
Catene di Markov in tempo discreto Friday 2:30 PM - 4:30 PM lesson Lecture Hall G from Oct 1, 2013  to Nov 30, 2013
Analisi di serie temporali Monday 2:30 PM - 4:30 PM lesson Lecture Hall G from Dec 1, 2013  to Jan 31, 2014
Analisi di serie temporali Wednesday 9:30 AM - 12:30 PM lesson Laboratory Gamma from Dec 4, 2013  to Jan 31, 2014
Analisi di serie temporali Wednesday 9:30 AM - 11:30 AM lesson Laboratory Alfa from Nov 27, 2013  to Nov 27, 2013

### Learning outcomes

Module 1 ( Discrete time Markov Chains )

Basics of the theory of discrete time Markov chain with finite or countable state space and examples of application.

Module 2 (Practice session of Stochastic systems)

Approximation and computation of invariant probabilities, Metropolis algorithm, simulation of queues and renewal processes with the use of Matlab.

Module 3 Introduction to Time Series analysis: the lessons aims to provide to the student a general framework to analyze time series as the outcome of a discrete time model fed by a white noise and an exogenous input. The lesson are completed by the use of a dedicated software in order to apply the theoretical aspects.

### Syllabus

Module 1
Markov chains with finite space state:
Definitions, transition matrix, transition probability in n steps, Chapman -Kolmogorov equation, finite joint densities, Canonocal space and Kolmogorov theorem (without proof).
State classification, invariant probabilities, Markov-Kakutani theorem, example of gambler's ruin, regular chains, criterion, limit probabilities and Markov theorem, reversible chains, Metropolis algorithm and Simulated annealing, numerical generation of a discrete random variable and algorithm for generation an omogeneus Markov chains with finite state space.

Markov chains with countable space state:
Equivalent definitions of transient and recurrent state, positive recurrence, periodicity, solidarity property, canonical decomposition of the state space, invariant measures, existence theorem, example of the unlimited random walk. Ergodicity and limit theorems.

Elements of Martingales associated to discrete time Markov chains:
Natural filtration, stopping times, conditional expectation given a random variable, strong Markov property, martingales. Optional stopping Theorem, example of gambler's ruin.

Module 2 Approximation and computation of invariant probabilities, Metropolis algorithm, simulation of queues with the use of Matlab.

Module 3 Elements of time series analysis :
Main scope of time series analysis: modelling, prediction and simulation.
Identification problem main components: a priori Knowledge, experiment design, goodness criteria, model, filtering and validation.
Model: main variables and correspondent schema. (AR, ARX, ARMA, output-error).
Goodness Criteria: least square, Maximum Likelihood, Maximum a posteriori.
Filtering: Linear parameter model, frequency filtering.
Matlab : main purpose and examples.

### Assessment methods and criteria

Module 1 Oral exam

Module 2 Discussion of the solution of given homeworks.

Module 3 Written exam

 Activity Author Title Publisher Year ISBN Note Analisi di serie temporali LJung System Identification, Theory for the User (Edizione 2) Prentice Hall PTR 1999
 Title Format (Language, Size, Publication date) Elaborato 1 - appello del 5 febbraio pdf (it, 54 KB, 30/01/14) Elaborato 2 - testo e linee guida per l'appello del 19 febbraio pdf (it, 75 KB, 13/02/14) Elaborato 3 - testo e linee guida per l'appello del 25 luglio pdf (it, 701 KB, 18/06/14) Errata Corrige I - Esercitazione IV pdf (it, 60 KB, 17/02/14) Esercitazione 1 del 20-11: predizione e simulazione zip (it, 1315 KB, 17/12/13) Esercitazione 2 del 27-11: identificazione su errore di predizione zip (it, 649 KB, 17/12/13) Esercitazione 3 del 04-12: Identificazione ML e MAP zip (it, 1292 KB, 17/12/13) Esercitazione 4 del 11-12: Validazione Modelli zip (it, 628 KB, 17/02/14) Esercitazione 5 del 18-12: Simulazione d'esame zip (it, 661 KB, 17/12/13)