Adaptive Signal Processing

TÜLAY ADALI, PhD, is Professor of Electrical Engineering and Director of the Machine Learning for Signal Processing Laboratory at the University of Maryland, Baltimore County. Her research interests are in statistical and adaptive signal processing, with emphasis on nonlinear and complex-valued signal processing, and applications in biomedical data analysis and communications. Simon Haykin, PhD, is Distinguished University Professor and Director of the Cognitive Systems Laboratory in the Faculty of Engineering at McMaster University. A world-renowned authority on adaptive and learning systems, Dr. Haykin has pioneered signal-processing techniques and systems for radar and communication applications, culminating in the study of cognitive dynamic systems, which has become his research passion.

• Preface xiContributors xvChapter 1 Complex-Valued Adaptive Signal Processing 11.1 Introduction 11.1.1 Why Complex-Valued Signal Processing 31.1.2 Outline of the Chapter 51.2 Preliminaries 61.2.1 Notation 61.2.2 Efficient Computation of Derivatives in the Complex Domain 91.2.3 Complex-to-Real and Complex-to-Complex Mappings 171.2.4 Series Expansions 201.2.5 Statistics of Complex-Valued Random Variables and Random Processes 241.3 Optimization in the Complex Domain 311.3.1 Basic Optimization Approaches in RN 311.3.2 Vector Optimization in CN 341.3.3 Matrix Optimization in CN 371.3.4 Newton-Variant Updates 381.4 Widely Linear Adaptive Filtering 401.4.1 Linear and Widely Linear Mean-Square Error Filter 411.5 Nonlinear Adaptive Filtering with Multilayer Perceptrons 471.5.1 Choice of Activation Function for the MLP Filter 481.5.2 Derivation of Back-Propagation Updates 551.6 Complex Independent Component Analysis 581.6.1 Complex Maximum Likelihood 591.6.2 Complex Maximization of Non-Gaussianity 641.6.3 Mutual Information Minimization: Connections to ML and MN 661.6.4 Density Matching 671.6.5 Numerical Examples 711.7 Summary 741.8 Acknowledgment 761.9 Problems 76References 79Chapter 2 Robust Estimation Techniques for Complex-Valued Random Vectors 872.1 Introduction 872.1.1 Signal Model 882.1.2 Outline of the Chapter 902.2 Statistical Characterization of Complex Random Vectors 912.2.1 Complex Random Variables 912.2.2 Complex Random Vectors 932.3 Complex Elliptically Symmetric (CES) Distributions 952.3.1 Definition 962.3.2 Circular Case 982.3.3 Testing the Circularity Assumption 992.4 Tools to Compare Estimators 1022.4.1 Robustness and Influence Function 1022.4.2 Asymptotic Performance of an Estimator 1062.5 Scatter and Pseudo-Scatter Matrices 1072.5.1 Background and Motivation 1072.5.2 Definition 1082.5.3 M-Estimators of Scatter 1102.6 Array Processing Examples 1142.6.1 Beamformers 1142.6.2 Subspace Methods 1152.6.3 Estimating the Number of Sources 1182.6.4 Subspace DOA Estimation for Noncircular Sources 1202.7 MVDR Beamformers Based on M-Estimators 1212.7.1 The Influence Function Study 1232.8 Robust ICA 1282.8.1 The Class of DOGMA Estimators 1292.8.2 The Class of GUT Estimators 1322.8.3 Communications Example 1342.9 Conclusion 1372.10 Problems 137References 138Chapter 3 Turbo Equalization 1433.1 Introduction 1433.2 Context 1443.3 Communication Chain 1453.4 Turbo Decoder: Overview 1473.4.1 Basic Properties of Iterative Decoding 1513.5 Forward-Backward Algorithm 1523.5.1 With Intersymbol Interference 1603.6 Simplified Algorithm: Interference Canceler 1633.7 Capacity Analysis 1683.8 Blind Turbo Equalization 1733.8.1 Differential Encoding 1793.9 Convergence 1823.9.1 Bit Error Probability 1873.9.2 Other Encoder Variants 1903.9.3 EXIT Chart for Interference Canceler 1923.9.4 Related Analyses 1943.10 Multichannel and Multiuser Settings 1953.10.1 Forward-Backward Equalizer 1963.10.2 Interference Canceler 1973.10.3 Multiuser Case 1983.11 Concluding Remarks 1993.12 Problems 200References 206Chapter 4 Subspace Tracking for Signal Processing 2114.1 Introduction 2114.2 Linear Algebra Review 2134.2.1 Eigenvalue Value Decomposition 2134.2.2 QR Factorization 2144.2.3 Variational Characterization of Eigenvalues Eigenvectors of Real Symmetric Matrices 2154.2.4 Standard Subspace Iterative Computational Techniques 2164.2.5 Characterization of the Principal Subspace of a Covariance Matrix from the Minimization of a Mean Square Error 2184.3 Observation Model and Problem Statement 2194.3.1 Observation Model 2194.3.2 Statement of the Problem 2204.4 Preliminary Example: Oja’s Neuron 2214.5 Subspace Tracking 2234.5.1 Subspace Power-Based Methods 2244.5.2 Projection Approximation-Based Methods 2304.5.3 Additional Methodologies 2324.6 Eigenvectors Tracking 2334.6.1 Rayleigh Quotient-Based Methods 2344.6.2 Eigenvector Power-Based Methods 2354.6.3 Projection Approximation-Based Methods 2404.6.4 Additional Methodologies 2404.6.5 Particular Case of Second-Order Stationary Data 2424.7 Convergence and Performance Analysis Issues 2434.7.1 A Short Review of the ODE Method 2444.7.2 A Short Review of a General Gaussian Approximation Result 2464.7.3 Examples of Convergence and Performance Analysis 2484.8 Illustrative Examples 2564.8.1 Direction of Arrival Tracking 2574.8.2 Blind Channel Estimation and Equalization 2584.9 Concluding Remarks 2604.10 Problems 260References 266Chapter 5 Particle Filtering 2715.1 Introduction 2725.2 Motivation for Use of Particle Filtering 2745.3 The Basic Idea 2785.4 The Choice of Proposal Distribution and Resampling 2895.4.1 Choice of Proposal Distribution 2905.4.2 Resampling 2915.5 Some Particle Filtering Methods 2955.5.1 SIR Particle Filtering 2955.5.2 Auxiliary Particle Filtering 2975.5.3 Gaussian Particle Filtering 3015.5.4 Comparison of the Methods 3025.6 Handling Constant Parameters 3055.6.1 Kernel-Based Auxiliary Particle Filter 3065.6.2 Density-Assisted Particle Filter 3085.7 Rao–Blackwellization 3105.8 Prediction 3145.9 Smoothing 3165.10 Convergence Issues 3205.11 Computational Issues and Hardware Implementation 3235.12 Acknowledgments 3245.13 Exercises 325References 327Chapter 6 Nonlinear Sequential State Estimation for Solving Pattern-Classification Problems 3336.1 Introduction 3336.2 Back-Propagation and Support Vector Machine-Learning Algorithms: Review 3346.2.1 Back-Propagation Learning 3346.2.2 Support Vector Machine 3376.3 Supervised Training Framework of MLPs Using Nonlinear Sequential State Estimation 3406.4 The Extended Kalman Filter 3416.4.1 The EKF Algorithm 3446.5 Experimental Comparison of the Extended Kalman Filtering Algorithm with the Back-Propagation and Support Vector Machine Learning Algorithms 3446.6 Concluding Remarks 3476.7 Problems 348References 348Chapter 7 Bandwidth Extension of Telephony Speech 3497.1 Introduction 3497.2 Organization of the Chapter 3527.3 Nonmodel-Based Algorithms for Bandwidth Extension 3527.3.1 Oversampling with Imaging 3537.3.2 Application of Nonlinear Characteristics 3537.4 Basics 3547.4.1 Source-Filter Model 3557.4.2 Parametric Representations of the Spectral Envelope 3587.4.3 Distance Measures 3627.5 Model-Based Algorithms for Bandwidth Extension 3647.5.1 Generation of the Excitation Signal 3657.5.2 Vocal Tract Transfer Function Estimation 3697.6 Evaluation of Bandwidth Extension Algorithms 3837.6.1 Objective Distance Measures 3837.6.2 Subjective Distance Measures 3857.7 Conclusion 3887.8 Problems 388References 390Index 393