Knowledge of basic notions of: estimation theory (maximum likelihood estimation, a-posteriori estimation); neural-based model identification; stochastic processes (mean, autocovariance, spectral density, optimal prediction); identification of ARMAX models. Ability to solve identification and prediction problems ranging from model formulation to the use of computer tools (Matlab) for parameter estimation and model simulation.
System Identification deals with methodologies that enable the construction of mathematical models of systems and signals based on experimental data. In presence of complex systems whose behavior can be hardly reduced to known "laws of nature", the use of identification techniques is often the only way to obtain models to be used in the context of forecasting, simulation, and control. The methods presented in the course are widely used in heterogeneous fields such as automation, biomedical engineering, econometry, hydrology, geophysics and telecommunications. Some basic notions of probability, estimation theory and stochastic processes are recalled. The main properties (stability, input-output description in the time and frequenct domains) of linear discrete-time systems are introduced. In the context of parametric estimation, the issues of model validation and model complexity are extensively discussed. Neural based identification is also illustrated and discussed, pointing out pros and cons with respect to standard approaches. The study of dynamic systems addresses three main topics: the optimal prediction of stationary stochastic processes (Wiener filtering), the identification of linear discrete-time systems, and spectral estimation (both nonparametric and maximum-entropy).
Probability: basic notions
independence, conditional probability, total probability and Bayes theorems;
Bernoulli trials, Poisson events;
the notion of random variable (R.V.), cumulative distribution function, probability density function, functions on one R.V.;
mode, median, moments of a R.V.;
joint random variables: distribution, density, moments, independence, incorrelation, functions of random variables;
Law of Lrge Numbers, Gaussian R.V., Central Limit Theorem.
Statistics: basic notions
notion of estimator; properties of estimators;
sample moments and their main properties;
confidence interval for the sample mean, Student's t.
Identification of linear-in-parameter models:
the least squares method, normal equations, identifiability;
Best Linear Unbiased Estimator: estimator, variance of parameters;
validation and choice of complexity: chi-square test, F-test, FPE, AIC, and MDL criteria.
maximum likelihood estimation: properties and examples;
a-posteriori estimation, Bayes estimator;
cross-validation, model complexity and the bias-variance dilemma;
identification of nonlinear-in-parameter models.
Radial basis function neural networks;
Multi-layer perceptron networks;
generalization, overfitting, selection of network size.
Stochastic processes and optimal prediction:
mean, autocorrelation, autocovariance, independence, incorrelation;
white noise, random walk, MA, AR, and ARMA processes, Yule-Walker equations;
stationarity, power spectral density, nonparametric spectral estimatiom;
spectral factorization, optimal prediction.
Identification of dynamic systems:
classes of dynamic models: output error, ARX, ARMAX;
prediction-error methods for system identification;
least-squares identification of ARX models: probabilistic analysis and persistent excitation.
Reccomended or required readings
Lecture notes (http://sisdin.unipv.it/labsisdin/teaching/teaching.php).
M. Bramanti. Calcolo delle probabilità e statistica. Esculapio.
A. Papoulis. Probability, Random Variables, and Stochastic Processes. McGraw-Hill.
L. Ljung. System Identification: Theory for the User. Prentice-Hall.