Course Description: Introduction to statistical and adaptive signal processing; FIR Wiener filter: linear mean square estimation, the orthogonality principle; The stochastic gradient approach: Least-Mean-Squares (LMS) and normalized LMS adaptive FIR filters; Linear prediction and autoregressive models: Gram-Schmidt orthogonalization and triangular matrix factorization, autoregressive model matching and the Yule-Walker equations; Applications: noise cancellation, system identification, equalization, spectrum estimation, line enhancing, speech analysis and synthesis, beamforming; Estimation of signal statistics: autocorrelation estimates and their statistical properties, the deterministic least-squares approach; Recursive Least Squares (RLS) adaptive filters: conventional RLS, QR-RLS; Comparative performance analysis of adaptive LMS and RLS filters: steady state error, tracking error, convergence rate and the role of orthogonalization; Order-recursive estimation: forward and backward prediction, lattice filter conÞguration, the Levinson and Schur algorithms, fast triangular factorization; Adaptive lattice filters: gradient and RLS; Classification of adaptive (RLS) filters by: architecture (triangular vs. linear, transversal vs. lattice), internal scaling/implementation (quotient, error-feedback, QR-based); Introduction to advanced adaptive filters: transform and subband domain, IIR (Laguerre-based), radial basis functions, backpropagation in neural nets, Voltera/Laguerre models.

Course Objectives: This course will use linear mean square estimation concepts as a starting point to explore some important areas of Modern Statistical Signal Processing. A student completing this course is expected to have a thorough understanding and a working knowledge of: FIR and IIR Wiener filtering, linear prediction and autoregressive model matching, the Levinson and Schur algorithms and the lattice filter configuration, autocorrelation estimation and the deterministic least squares method, LMS and RLS adaptive filters, order recursive (triangular and lattice) architectures. In addition, the student should acquire a firm grasp of: the Kalman filter, spectral and covariance factorization, performance analysis of adaptive fillters under non-stationary conditions, and (time permitting) a factual knowledge of some basic concepts concerning fundamentals of spectrum estimation, nonstationary spectrum analysis, IIR (Laguerre-based) lattice configuration, and nonlinear adaptive filtering.

Course Outline by Topical Areas:

Introduction to Statistical and Adaptive Signal Processing

Linear least squares filtering, spectral factorization and signal modeling, adaptive filters and their figures of merit, FIR vs. IIR adaptive filters, approaches to adaptive filtering: probabilistic (Wiener-Kalman) vs. deterministic (least-squares), applications.

Review of Wiener-Kalman Filter Theory

Linear prediction and the innovations process, the optimum filtering problem, error performance surface, normal equations and the principle of orthogonality, minimum mean-squared error, overview of Kalman filtering.

The Stochastic-Gradient Approach

Structure of the adaptive filter, the method of steepest descent, the LMS algorithm and its properties, operation in a non-stationary environment.

Finite-Order Linear Prediction

Finite order Wiener filter, autoregressive model matching, maximum-entropy approach, Yule-Walker equations, model-order selection, Gram-Schmidt orthogonalization, matrix interpretation of linear prediction.

Applications

Adaptive arrays, line enhancement, echo cancellation, equalization, spectrum analysis, system identification, adaptive control.

Autocorrelation/Spectrum Estimation

Autocorrelation estimates and their statistical properties, non-parametric spectrum estimation, performance and resolution.

The Deterministic Least Squares Approach

The linear-least-squares estimation problem, data windowing, the geometric (pseudo-probabilistic) interpretation of the least-squares criterion, autocorrelation estimates and their properties.

Adaptive Recursive Least Squares (RLS) Algorithms

Exponentially-weighted RLS, QR-RLS, convergence analysis, operation in a non-stationary environment, relationship to Kalman theory, comparison with the LMS.

Order-Recursive Estimation

One-step autocorrelation extension, Gram-Schmidt orthogonalization, forward and backward prediction, the Levinson and Schur algorithms, the lattice prediction filter for stationary and non-stationary processes, joint process estimation.

Adaptive Order-Recursive Filters

Order-update and time-update recursions, the basic building blocks for fast recursive algorithms, adaptive RLS lattice filter gradient adaptive lattice filter.

Linear Prediction for Non-stationary Signals

Stationary spectra associated with non-stationary signals, time-varying autoregressive models in transversal and lattice form, applications to multichannel and multirate signal processing.

Advanced Topics in Adaptive Filtering

Transform and subband domain, IIR (Laguerre-based), nonlinear.

Course Requirements:

Homework: 10-11 assignments - 40%

Examinations: Midterm - 30%; Final - 40%

Project: None

Notes:

Delivery charge: VHS tape duplication and delivery - $400 billed by Northeastern University.

Degree Applicability:

CE[AA]	CH[NA]	CS[AA]	EE[BDE]	EM[E]	ESM[NA]	MAT[NA]
MBA[NA]	ME[E]	MES[BE]	MSE[BE]	SE[NA]	SY[AA]

Click here for further information on degree applicability.

NTU Semester Credit Hours: 4
Number of Lecture Hours: 28 (100 minute) lectures
Days Class Meets on Campus: Monday/Wedneday

Contributing Scholar:
Hanoch Lev-Ari
ECE Department
Northeastern University
440 Dana Research Bldg.
360 Huntington Avenue
Boston, MA   02115
Phone: 617-373-3032

Fax: 617-373-8970
levari@ece.neu.edu
http://www.cdsp.neu.edu/info/faculty/levari/levari.html

Note: Contributing Scholars are responsible for the design, organization, content, and presentation of NTU courses. Online classroom management, student management, and other matters related to academic administration of courses are the responsibility of support "Faculty". Either person is often called "Instructor". To identify and differentiate between these roles, we use the terms "Contributing Scholar" and "Faculty".

Academic/Administrative Contact:  
Ms. Linda Alosso
Phone: 617-373-5621
          617-373-5621 Fax: 617-373-8574
l.alosso@neu.edu


Prerequisites: Graduate level courses in: (i) probability theory and stochastic processes, and (ii) modern signal processing

Textbooks: (Order Materials)

1.	Discrete Random Signals and Statistical Signal Processing, C. W. Therrien, Prentice Hall, 1992
2.	Adaptive Filter Theory, S. Haykin, Prentice Hall, 4th edition, 2002

Tuition for CC 763-F , 2004
Semester Credit Hours (SCH):	4
Tuition per 1 SCH, for Credit:	$1000.00
Total Tuition 4 SCH, for Credit:	$4000.00
Tuition per 1 SCH, Audit:	$745.00
Total Tuition 4 SCH, Audit:	$2980.00
*** Please note that the above tuition amounts do not include any delivery fees, textbook fees, or special fees that may be applicable - see Notes section above. ***