Lecture Notes , Homework Sets , Learn Your Grades , Distribution of Grades

Announcements

  • The final exam is on Jan 13, (Wed.) at 16:30. The coverage for the final is everything except the last lecture on recursive estimation.
  • MT #2 results are anonunced. You can examine your MT1 and MT2 papers after we have the final exam.
  • Notes on KL Transform
  • Hw # 5 is assigned. (Due : 29th Dec.) (check the homework sets link above! Dec.28, 2009)
  • An example for the minimum MSE calculation for the non-causal IIR Wiener filter
  • MT #2: 18th Dec. 2009, Friday in class two hour exam.
  • MT #1 results are anonunced. (MT #1 Solutions)
  • Hw # 4 is assigned. (Due : 8th Dec.)
  • MT #1 : 22nd Nov, Sunday, 14:00-16:00, at D-126 (room where we have Friday lectures). Coverage includes everything up to, but not including signal modeling.
  • Hw3 (due: 17th Nov) is distributed.
  • Pre-midterm-I questions and solutions
  • MT #1 on Nov.3rd, (in class exam) Coverage: Problems of Therrien given at the end of chapters 2 and 3.
  • Hw2 (due: 27th Oct) is distributed.
  • Solution for Hw1 is placed in the Homework Sets folder.
  • Hw1 (due: 13th Oct) is distributed.
  • A problem sheet for linear algebra review is distributed (not to be collected).

 

EE 503 Signal Analysis and Processing

(Fall 2009 – 2010)

 

 

Short Description:

 

The course aims to unify the knowledge of linear system theory, digital signal processing basics and stochastic processes into the framework of statistical signal processing. The course goal is to establish a firm foundation for estimation theory (parameter estimation, signal modeling), Wiener Filtering (approached from the direction of linear MSE estimation) and linear prediction. Some more advanced topics such as AR, MA, ARMA, Harmonic processes, linear decorrelating transform, series expansion of random processes, spectral factorization, causal – non causal IIR Wiener filters  are also introduced along the path.  

 

 

Outline of Topics:

 

  1. Review of Some Linear Algebra Concepts:
    1. Matrices as Transformations

                                                               i.      Linear Space, Linear Operators in Linear Space

                                                              ii.      Equivalent representations with finite/infinite matrices

                                                            iii.      Isomorphism between finite energy functions and finite power sequences (L2 ó l2 spaces)

                                                            iv.      Representation of points in alternative coordinate systems, representation of operators in alternative coordinate systems

                                                             v.      Diagonalization of operators (Eigenfunctionsó Eigenvectors)

                                                            vi.      Hermitian Operators ó Hermitian Matrices, Orthogonal Bases

                           Ref: Strang, Wolf, Lancaster

    1. Matrices as Linear Combiners

                                                               i.      Range and Null space of the combination process

                                                              ii.      Linear independence of vectors (points in linear space)

                                                            iii.      Projection to Range/Null Space, Direct Sums

                           Ref: Scharf

    1. Matrices as Equation Systems

                                                               i.      Linear constraints (equations), intersection of constraints

                                                              ii.      Under-Over determined systems, Unique-None-Infinite solution systems

                                                            iii.      LS solution for inconsistent equation systems (overdetermined)

1.        Projection to range space,

2.        Pseudo Inverse, SVD

                                                            iv.      Minimum norm solutions for systems with infinite solutions

                                                             v.      SVD and its properties.

                           Ref: Scharf

  1. Review of some DSP Concepts
    1. Basic Idea: Discrete time processing of continuous time signals

                                                               i.      Sampling Theorem (going to discrete time without any loss of information)

                                                              ii.      Bandlimited Interpolation (going back to continous time after processing)

    1. Discrete Time Operations:

                                                               i.      Z-Transform, discrete time LTI systems, convolution, convolution matrices, diagonalization of convolution matrices

  1. Review of some Random Processes Concepts:
    1. Random variables, random vectors (or a sequence of  random variables), random processes
    2. Moment descriptors (especially 2nd order moment description of R.P’s, mean, variance, correlation, auto-correlation, power spectrum density etc.) 
    3. Stationarity, Wide Sense Stationarity
    4. PSD and its properties, spectral factorization
    5. Linear Time Invariant Processing  of  WSS R.P’s
    6. Ergodicity

             Ref: Therrien, Hayes, Papoulis, Ross

  1. Signal Modeling
    1. LS methods, Pade, Prony (Deterministic methods)
    2. AR, MA, ARMA Processes (Stochastic approach), Yule-Walker Equations, Non-linear set of equations for MA system fit,

                                                               i.      All-pole modeling

1.        Covariance Method

2.        Auto-correlation Method

    1. Harmonic Processes, Wold decomposition
    2. Decorrelating transforms such as Fourier Transforms for Harmonic Processes and KL transform in general.
    3. Applications: Signal Compression, Signal Prediction, System Identification, Spectrum Estimation.

             Ref: Hayes, Papoulis

  1. Some Topics in Estimation Theory
    1. Cost Functions: Mean Square, Mean absolute, max error
    2. MSE, ML, absolute error estimators
    3. Min MSE estimators

                                                               i.      Regression line, orthogonality

    1. Linear min MSE estimators
    2. Linear unbiased min MSE  estimators
    3. Bias, consistency, efficiency, bias-error variance trade-off.
    4. Discussion of LS estimator for Ax=b + n systems.
    5. Wiener Filters as optimal estimators

                                                               i.      Linear predictors defined from Wiener filters

                                                              ii.      Levinson-Durbin recursion for efficient solution of Wiener-Hopf equations.

                                                            iii.      Lattice Structures for efficient implementation of Wiener filters

    1. IIR Wiener Filters

                                                               i.      Non-causal, Causal                    

     Ref: Therrien, Hayes, Scharf

 

References:

Textbook for Signal Modeling Topic:

 [Hayes] :  M. H. Hayes, Statistical Signal Processing and Modeling, Wiley, New York, NY, 1996 (Level: moderate)

 

Textbook for Random Vectors and Processes Topics:

[Therrien] : Therrien, Charles W. , Discrete random signals and statistical signal processing, Prentice Hall, c1992. (Level: moderate)

 

[Scharf] : Louis L. Scharf, Statistical Signal Processing, Addison-Wesley Publishing Company, Inc., Reading, MA, 1991.(Level : advanced)

 

[Papoulis] : A. Papoulis, Probability, Random Variables, and Stochastic Processes, 3rd edition, McGraw Hill, 1991. (Level: important reference book, mostly advanced)

 

[Ross]: S. M. Ross, Introduction to probability models, 7th ed. Harcourt Academic Press, 2000. (level : introductory but complete)

 

[Wolf] : Kurt Bernardo Wolf , Integral Transforms in Science and Engineering

Plenum Pub Corp, January 1979 (Level: advanced)

 

[Lancaster]: P. Lancaster and M. Tismenetsky. The Theory of Matrices. Academic Press, Boston, 2nd edition, 1985.  (Level: complete text, very valuable as a linear algebra reference)