ECE 555
Control of Stochastic Systems

Displaying course information from Spring 2013.

Section Type Times Days Location Instructor
C DIS 1530 - 1650 T R   241 Everitt Lab  Mohamed Ali Belabbas
Web Page
Official Description Stochastic control models; development of control laws by dynamic programming; separation of estimation and control; Kalman filtering; self-tuning regulators; dual controllers; decentralized control. Course Information: Prerequisite: ECE 515 and ECE 534.
Subject Area Control Systems
Course Prerequisites Credit in ECE 515
Credit in ECE 534
Course Directors Tamer Başar
Detailed Description and Outline


  • Introduction: decision-making under uncertainty
  • Markov chain models: structure, steady-state probabilities, transience and recurrence, Lyopunov functions and stability
  • State space models: state, observation and control processes
  • Properties of linear stochastic systems: linear Gaussian systems; asymptotic properties, Gauss-Markov processes; quadratic costs
  • Controlled Markov chain models: finite state systems; Markov and stationary policies; cost of Markov policy; infinite state systems
  • Input-output models: elimination of state variables; impulse response and frequency response models
  • Dynamic programming: optimal control laws; complete and partial information; information state, dual control
  • Estimation and control of linear stochastic systems: linear Gaussian systems; Kalman filter; optimal linear-quadradic control; minimum variance control for input-output models
  • Identification and adaptive control: Bayesian and non-Bayesian approaches; maximum likelihood estimate; least-squares, prediction error and instrumental variable methods; recursive identification; ODE method; self-tuning regulator
P.R. Kumar and P. Varaiya, Stochastic Systems, Prentice-Hall, 1986.
Last updated: 2/13/2013