Short Description:
The course aims to present some topics in linear system theory, digital signal processing and stochastic processes to study the general framework called statistical signal processing. The course goal is to establish a firm foundation for the estimation theory (mainly parameter estimation), Wiener Filtering (approached from the direction of linear MSE estimation) and linear prediction. Some related topics such as AR, MA, ARMA, Harmonic processes, the linear decorrelating transform, series expansion of random processes, spectral factorization, causal – non causal IIR Wiener filters are also discussed along the path.
Related Courses:
EE 501, EE 531: Highly recommended
EE 430 (Undergraduate DSP Course), EE 230 (Undergraduate Probability Course), EE 306 (Undergraduate Stochastic Processes Course): A fairly complete knowledge of these courses is assumed.
Outline of Topics:
- Review of Some Linear Algebra Concepts: Review of Some Topics From EE 501 (1 Week)
List of some important topics (not all covered):
- Matrices as Transformations
- Linear Space, Linear Operators in Linear Space
- Equivalent representations with finite/infinite matrices
- Isomorphism between finite energy functions and finite power sequences (L2 ó l2 spaces)
- Representation of points in alternative coordinate systems, representation of operators in alternative coordinate systems
- Diagonalization of operators (Eigenfunctions ó Eigenvectors)
- Hermitian Operators ó Hermitian Matrices, Orthogonal Bases
Ref: Strang, Wolf, Lancaster
- Matrices as Linear Combiners
- Range and Null space of the combination process
- Linear independence of vectors (points in linear space)
- Projection to Range/Null Space, Direct Sums
Ref: EE 501
- Matrices as Equation Systems
- Linear constraints (equations), intersection of constraints
- Under-Over determined systems, Unique-None-Infinite solution systems
- LS solution for inconsistent equation systems (over-determined)
- Projection to range space,
- Pseudo Inverse, SVD
- Minimum norm solutions for systems with infinite solutions
- SVD and its properties.
Ref: EE 501
- Review of Some DSP Concepts: Review of EE 430 Fundamentals (2 hours)
- Basic Idea: Discrete time processing of continuous time signals
- Sampling Theorem (going to discrete time without any loss of information)
- Bandlimited Interpolation (going back to continuous time after processing)
- Discrete Time Operations:
- Z-Transform, discrete time LTI systems, convolution, convolution matrices, diagonalization of convolution matrices
- Random Processes: (4 Weeks) (Ideally should taken in parallel with EE 531)
- Random variables, random vectors (or a sequence of random variables), random processes
- Moment descriptors (especially 2nd order moment description of R.P’s, mean, variance, correlation, auto-correlation, power spectrum density etc.)
- Stationarity, Wide Sense Stationarity
- PSD and its properties, spectral factorization
- Linear Time Invariant Processing of WSS R.P’s
- Ergodicity
Ref: Therrien, Hayes, Papoulis, Ross
- Signal Modeling (2 Weeks)
- LS methods, Pade, Prony (Deterministic methods)
- AR, MA, ARMA Processes (Stochastic approach), Yule-Walker Equations, Non-linear set of equations for MA system fit,
- All-pole modeling
- Covariance Method
- Auto-correlation Method
- Harmonic Processes, Wold decomposition
- Decorrelating transforms such as Fourier Transforms for Harmonic Processes and KL transform in general.
- Applications: Signal Compression, Signal Prediction, System Identification, Spectrum Estimation.
Ref: Hayes, Papoulis
- Some Topics in Estimation Theory (5 weeks)
- Cost Functions: Mean Square, Mean absolute, max error
- MSE, ML, absolute error estimators
- Min MSE estimators
- Regression line, orthogonality
- Linear min MSE estimators
- Linear unbiased min MSE estimators
- Bias, consistency, efficiency, bias-error variance trade-off.
- Discussion of LS estimator for Ax=b + n systems.
- Wiener Filters as optimal estimators
- Linear predictors defined from Wiener filters
- Levinson-Durbin recursion for efficient solution of Wiener-Hopf equations. (time permits)
- Lattice Structures for efficient implementation of Wiener filters (time permits)
- IIR Wiener Filters
- Non-causal, Causal
Ref: Therrien, Hayes, Scharf
References:
Textbook for Signal Modeling Topic:
[Hayes] : M. H. Hayes, Statistical Signal Processing and Modeling, Wiley, New York, NY, 1996 (Level: moderate)
Textbook for Random Vectors and Processes Topics:
[Therrien] : Therrien, Charles W. , Discrete random signals and statistical signal processing, Prentice Hall, c1992. (Level: moderate)
[Scharf] : Louis L. Scharf, Statistical Signal Processing, Addison-Wesley Publishing Company, Inc., Reading, MA, 1991.(Level : advanced)
[Papoulis] : A. Papoulis, Probability, Random Variables, and Stochastic Processes, 3rd edition, McGraw Hill, 1991. (Level: important reference book, mostly advanced)
[Ross]: S. M. Ross, Introduction to probability models, 7th ed. Harcourt Academic Press, 2000. (Level: introductory but complete)
[Wolf] : Kurt Bernardo Wolf , Integral Transforms in Science and Engineering Plenum Pub Corp, January 1979 (Level: advanced)
[Lancaster]: P. Lancaster and M. Tismenetsky. The Theory of Matrices. Academic Press, Boston, 2nd edition, 1985. (Level: complete text, very valuable as a linear algebra reference)
|