Lecture Notes , Homework Sets , Learn Your Grades , Distribution of Grades

Announcements

  • EE 503 Overall Results
  • Notes on stochastic modelling is available here (summary of the related section in Hayes)
  • Midterm 1 is graded, solutions are available here. You can learn your grades from links above the announcements.
  • Final Exam: 9th Jan, 2008 , at 16:30, EA 310.
  • Midterm #2 is on 3rd Jan, 2008 (Thursday), at 17:30. 4th Jan 2008 (15:30-17:30). Coverage includes everything but the KL transform and the topics covered after KL.
  • Homework #6 is posted (due : first lecture after bayram)
  • MMSE Filtering Handout (two parts) from Statistical and Adaptive Signal Processing, by Manolakis (useful for Hw #5)
  • Homework #5 is posted (due : 30th Nov 2007)
  • Midterm #1 is on 16th Nov, 2007 (Friday during lecture hours)
  • Fall 2005 homework solutions from are available here. Homework problems are available here.
  • Homework #4 : Therrien: 2.4, 2.23, 2.22, 4.2, 4.23, 5.4, 5.7 (not to be collected, for exam study)
  • Check out: AR-MA-ARMA Processes JAVA Demo
  • Notes from Therrien on Rx diagonalization is placed in the homework sets folder.
  • Homework 1 is posted (There are some minor correction in the 3rd problem)

EE 503 Signal Analysis and Processing

(Fall 2007 – 2008)

 

 

Short Description:

 

The course aims to unify the knowledge of linear system theory, digital signal processing basics and stochastic processes into the framework of statistical signal processing. The course goal is to establish a firm foundation for estimation theory (parameter estimation, signal modeling), Wiener Filtering (approached from the direction of linear MSE estimation) and linear prediction. Some more advanced topics such as AR, MA, ARMA, Harmonic processes, linear decorrelating transform, series expansion of random processes, spectral factorization, causal – non causal IIR Wiener filters  are also introduced along the path.  

 

 

Outline of Topics:

 

  1. Review of Some Linear Algebra Concepts:
    1. Matrices as Transformations

                                                               i.      Linear Space, Linear Operators in Linear Space

                                                              ii.      Equivalent representations with finite/infinite matrices

                                                            iii.      Isomorphism between finite energy functions and finite power sequences (L2 ó l2 spaces)

                                                            iv.      Representation of points in alternative coordinate systems, representation of operators in alternative coordinate systems

                                                             v.      Diagonalization of operators (Eigenfunctions ó Eigenvectors)

                                                            vi.      Hermitian Operators ó Hermitian Matrices, Orthogonal Bases

                           Ref: Strang, Wolf, Lancaster

    1. Matrices as Linear Combiners

                                                               i.      Range and Null space of the combination process

                                                              ii.      Linear independence of vectors (points in linear space)

                                                            iii.      Projection to Range/Null Space, Direct Sums

                           Ref: Scharf

    1. Matrices as Equation Systems

                                                               i.      Linear constraints (equations), intersection of constraints

                                                              ii.      Under-Over determined systems, Unique-None-Infinite solution systems

                                                            iii.      LS solution for inconsistent equation systems (overdetermined)

1.        Projection to range space,

2.        Pseudo Inverse, SVD

                                                            iv.      Minimum norm solutions for systems with infinite solutions

                                                             v.      SVD and its properties.

                           Ref: Scharf

  1. Review of some DSP Concepts
    1. Basic Idea: Discrete time processing of continuous time signals

                                                               i.      Sampling Theorem (going to discrete time without any loss of information)

                                                              ii.      Bandlimited Interpolation (going back to continous time after processing)

    1. Discrete Time Operations:

                                                               i.      Z-Transform, discrete time LTI systems, convolution, convolution matrices, diagonalization of convolution matrices

  1. Review of some Random Processes Concepts:
    1. Random variables, random vectors (or a sequence of  random variables), random processes
    2. Moment descriptors (especially 2nd order moment description of R.P’s, mean, variance, correlation, auto-correlation, power spectrum density etc.) 
    3. Stationarity, Wide Sense Stationarity
    4. PSD and its properties, spectral factorization
    5. Linear Time Invariant Processing  of  WSS R.P’s
    6. Ergodicity

             Ref: Therrien, Hayes, Papoulis, Ross

  1. Signal Modeling
    1. LS methods, Pade, Prony (Deterministic methods)
    2. AR, MA, ARMA Processes (Stochastic approach), Yule-Walker Equations, Non-linear set of equations for MA system fit,

                                                               i.      All-pole modeling

1.        Covariance Method

2.        Auto-correlation Method

    1. Harmonic Processes, Wold decomposition
    2. Decorrelating transforms such as Fourier Transforms for Harmonic Processes and KL transform in general.
    3. Applications: Signal Compression, Signal Prediction, System Identification, Spectrum Estimation.

             Ref: Hayes, Papoulis

  1. Some Topics in Estimation Theory
    1. Cost Functions: Mean Square, Mean absolute, max error
    2. MSE, ML, absolute error estimators
    3. Min MSE estimators

                                                               i.      Regression line, orthogonality

    1. Linear min MSE estimators
    2. Linear unbiased min MSE  estimators
    3. Bias, consistency, efficiency, bias-error variance trade-off.
    4. Discussion of LS estimator for Ax=b + n systems.
    5. Wiener Filters as optimal estimators

                                                               i.      Linear predictors defined from Wiener filters

                                                              ii.      Levinson-Durbin recursion for efficient solution of Wiener-Hopf equations.

                                                            iii.      Lattice Structures for efficient implementation of Wiener filters

    1. IIR Wiener Filters

                                                               i.      Non-causal, Causal                    

     Ref: Therrien, Hayes, Scharf

 

References:

Textbook for Signal Modeling Topic:

 [Hayes] :  M. H. Hayes, Statistical Signal Processing and Modeling, Wiley, New York, NY, 1996 (Level: moderate)

 

Textbook for Random Vectors and Processes Topics:

[Therrien] : Therrien, Charles W. , Discrete random signals and statistical signal processing, Prentice Hall, c1992. (Level: moderate)

 

[Scharf] : Louis L. Scharf, Statistical Signal Processing, Addison-Wesley Publishing Company, Inc., Reading, MA, 1991.(Level : advanced)

 

[Papoulis] : A. Papoulis, Probability, Random Variables, and Stochastic Processes, 3rd edition, McGraw Hill, 1991. (Level: important reference book, mostly advanced)

 

[Ross]: S. M. Ross, Introduction to probability models, 7th ed. Harcourt Academic Press, 2000. (level : introductory but complete)

 

[Wolf] : Kurt Bernardo Wolf , Integral Transforms in Science and Engineering

Plenum Pub Corp, January 1979 (Level: advanced)

 

[Lancaster]: P. Lancaster and M. Tismenetsky. The Theory of Matrices. Academic Press, Boston, 2nd edition, 1985.  (Level: complete text, very valuable as a linear algebra reference)