Nonlinear Dynamic Modeling of Physiological Systems

Vasilis Z. Marmarelis, PhD, received his diploma in electrical and mechanical engineering from the National Technical University of Athens and his MS in information science and PhD in engineering science (bio-information systems) from the California Institute of Technology. He is currently a professor in the faculty of the Biomedical and Electrical Engineering Departments at USC, where he served as chairman of Biomedical Engineering from 1990 to 1996. He is also Codirector of the Biomedical Simulations Resource (BMSR), a research center dedicated to modeling and simulation of physiological systems and funded by the National Institutes of Health through multimillion-dollar grants since 1985.

• Prologue xiii1 Introduction 11.1 Purpose of this Book 11.2 Advocated Approach 41.3 The Problem of System Modeling in Physiology 61.3.1 Model Specification and Estimation 101.3.2 Nonlinearity and Nonstationarity 121.3.3 Definition of the Modeling Problem 131.4 Types of Nonlinear Models of Physiological Systems 13Example 1.1. Vertebrate Retina 15Example 1.2. Invertebrate Photoreceptor 18Example 1.3. Volterra analysis of Riccati Equation 19Example 1.4. Glucose-Insulin Minimal Model 21Example 1.5. Cerebral Autoregulation 221.5 Deductive and Inductive Modeling 24Historical Note #1: Hippocratic and Galenic Views of 26Integrative Physiology2 Nonparametric Modeling 292.1 Volterra Models 312.1.1 Examples of Volterra Models 37Example 2.1. Static Nonlinear System 37Example 2.2. L–N Cascade System 38Example 2.3. L–N–M “Sandwich” System 39Example 2.4. Riccati System 402.1.2 Operational Meaning of the Volterra Kernels 41Impulsive Inputs 42Sinusoidal Inputs 43Remarks on the Meaning of Volterra Kernels 452.1.3 Frequency-Domain Representation of the Volterra Models 452.1.4 Discrete-Time Volterra Models 472.1.5 Estimation of Volterra Kernels 49Specialized Test Inputs 50Arbitrary Inputs 52Fast Exact Orthogonalization and Parallel-Cascade Methods 55Iterative Cost-Minimization Methods for Non-Gaussian 55Residuals2.2 Wiener Models 572.2.1 Relation between Volterra and Wiener Models 60The Wiener Class of Systems 62Examples of Wiener Models 63Comparison of Volterra/Wiener Model Predictions 642.2.2 Wiener Approach to Kernel Estimation 672.2.3 The Cross-Correlation Technique for Wiener Kernel Estimation 72Estimation of h0 73Estimation of h1 (𝜏) 73Estimation of h2 (𝜏1, 𝜏2) 74Estimation of h3 (𝜏1, 𝜏2, 𝜏3) 75Some Practical Considerations 77Illustrative Example 78Frequency-Domain Estimation of Wiener Kernels 782.2.4 Quasiwhite Test Inputs 80CSRS and Volterra Kernels 84The Diagonal Estimability Problem 85An Analytical Example 86Comparison of Model Prediction Errors 88Discrete-Time Representation of the CSRS Functional Series 89Pseudorandom Signals Based on m-Sequences 89Comparative Use of GWN, PRS, and CSRS 922.2.5 Apparent Transfer Function and Coherence Measurements 93Example 2.5. L–N Cascade System 96Example 2.6. Quadratic Volterra System 97Example 2.7. Nonwhite Gaussian Inputs 98Example 2.8. Duffing System 98Concluding Remarks 992.3 Efficient Volterra Kernel Estimation 1002.3.1 Volterra Kernel Expansions 101Model Order Determination 1042.3.2 The Laguerre Expansion Technique 107Illustrative Examples 1122.3.3 High-Order Volterra Modeling with Equivalent Networks 1222.4 Analysis of Estimation Errors 1252.4.1 Sources of Estimation Errors 1252.4.2 Estimation Errors Associated with the Cross-Correlation 127Technique Estimation Bias 128Estimation Variance 130Optimization of Input Parameters 131Noise Effects 134Erroneous Scaling of Kernel Estimates 1362.4.3 Estimation Errors Associated with Direct Inversion Methods 1372.4.4 Estimation Errors Associated with Iterative 139Cost-Minimization Methods Historical Note #2: Vito Volterra and Norbert Wiener 1403 Parametric Modeling 1453.1 Basic Parametric Model Forms and Estimation Procedures 1463.1.1 The Nonlinear Case 1503.1.2 The Nonstationary Case 1523.2 Volterra Kernels of Nonlinear Differential Equations 153Example 3.1. The Riccati Equation 1573.2.1 Apparent Transfer Functions of Linearized Models 158Example 3.2. Illustrative Example 1603.2.2 Nonlinear Parametric Models with Intermodulation 1613.3 Discrete-Time Volterra Kernels of NARMAX Models 1643.4 From Volterra Kernel Measurements to Parametric Models 167Example 3.3. Illustrative Example 1693.5 Equivalence Between Continuous and Discrete Parametric Models 171Example 3.4. Illustrative Example 1753.5.1 Modular Representation 1774 Modular and Connectionist Modeling 1794.1 Modular Form of Nonparametric Models 1794.1.1 Principal Dynamic Modes 180Illustrative Examples 1864.1.2 Volterra Models of System Cascades 191The L–N–M, L–N, and N–M Cascades 1944.1.3 Volterra Models of Systems with Lateral Branches 1984.1.4 Volterra Models of Systems with Feedback Branches 2004.1.5 Nonlinear Feedback Described by Differential Equations 202Example 1. Cubic Feedback Systems 204Example 2. Sigmoid Feedback Systems 209Example 3. Positive Nonlinear Feedback 213Example 4. Second-Order Kernels of Nonlinear 215Feedback Systems Nonlinear Feedback in Sensory Systems 216Concluding Remarks on Nonlinear Feedback 2204.2 Connectionist Models 2234.2.1 Equivalence between Connectionist and Volterra Models 223Relation with PDM Modeling 230Illustrative Examples 2324.2.2 Volterra-Equivalent Network Architectures for Nonlinear 235System Modeling Equivalence with Volterra Kernels/Models 238Selection of the Structural Parameters of the VEN Model 238Convergence and Accuracy of the Training Procedure 240The Pseudomode-Peeling Method 244Nonlinear Autoregressive Modeling (Open-Loop) 2464.3 The Laguerre-Volterra Network 246Illustrative Example of LVN Modeling 249Modeling Systems with Fast and Slow Dynamic (LVN-2) 251Illustrative Examples of LVN-2 Modeling 2554.4 The VWM Model 2605 A Practitioner’s Guide 2655.1 Practical Considerations and Experimental Requirements 2655.1.1 System Characteristics 266System Bandwidth 266System Memory 267System Dynamic Range 267System Linearity 268System Stationarity 268System Ergodicity 2685.1.2 Input Characteristics 2695.1.3 Experimental Characteristics 2705.2 Preliminary Tests and Data Preparation 2725.2.1 Test for System Bandwidth 2725.2.2 Test for System Memory 2725.2.3 Test for System Stationarity and Ergodicity 2735.2.4 Test for System Linearity 2745.2.5 Data Preparation 2755.3 Model Specification and Estimation 2765.3.1 The MDV Modeling Methodology 2775.3.2 The VEN/VWM Modeling Methodology 2785.4 Model Validation and Interpretation 2795.4.1 Model Validation 2795.4.2 Model Interpretation 281Interpretation of Volterra Kernels 281Interpretation of the PDM Model 2825.5 Outline of Step-by-Step Procedure 2835.5.1 Elaboration of the Key Step # 5 2846 Selected Applications 2856.1 Neurosensory Systems 2866.1.1 Vertebrate Retina 2876.1.2 Invertebrate Retina 3966.1.3 Auditory Nerve Fibers 3026.1.4 Spider Mechanoreceptor 3076.2 Cardiovascular System 3206.3 Renal System 3336.4 Metabolic-Endocrine System 3427 Modeling of Multiinput/Multioutput Systems 3597.1 The Two-Input Case 3607.1.1 The Two-Input Cross-Correlation Technique 3627.1.2 The Two-Input Kernel-Expansion Technique 3627.1.3 Volterra-Equivalent Network Models with Two Inputs 364Illustrative Example 3667.2 Applications of Two-Input Modeling to Physiological Systems 3697.2.1 Motion Detection in the Invertebrate Retina 3697.2.2 Receptive Field Organization in the Vertebrate Retina 3707.2.3 Metabolic Autoregulation in Dogs 3787.2.4 Cerebral Autoregulation in Humans 3807.3 The Multiinput Case 3897.3.1 Cross-Correlation-Based Method for Multiinput Modeling 3907.3.2 The Kernel-Expansion Method for Multiinput Modeling 3937.3.3 Network-Based Multiinput Modeling 3937.4 Spatiotemporal and Spectrotemporal Modeling 3957.4.1 Spatiotemporal Modeling of Retinal Cells 3987.4.2 Spatiotemporal Modeling of Cortical Cells 4018 Modeling of Neuronal Systems 4078.1 A General Model of Membrane and Synaptic Dynamics 4088.2 Functional Integration in the Single Neuron 4148.2.1 Neuronal Modes and Trigger Regions 417Illustrative Examples 4278.2.2 Minimum-Order Modeling of Spike-Output Systems 432The Reverse-Correlation Technique 432Minimum-Order Wiener Models 435Illustrative Example 4398.3 Neuronal Systems with Point-Process Inputs 4398.3.1 The Lag-Delta Representation of P–V or P–W Kernels 4458.3.2 The Reduced P–V or P–W Kernels 4468.3.3 Examples from the Hippocampal Formation 450Single-Input Stimulation in Vivo and Cross-Correlation  450TechniqueSingle-Input Stimulation in Vitro and Laguerre-Expansion 455TechniqueDual-Input Stimulation in the Hippocampal Slice 457Nonlinear Modeling of Synaptic Dynamics 4618.4 Modeling of Neuronal Ensembles 4639 Modeling of Nonstationary Systems 4679.1 Quasistationary and Recursive Tracking Methods 4689.2 Kernel Expansion Method 4699.2.1 Illustrative Example 4749.2.2 A Test of Nonstationarity 4759.2.3 Linear Time-Varying Systems with Arbitrary Inputs 4799.3 Network-Based Methods 4809.3.1 Illustrative Examples 4819.4 Applications to Nonstationary Physiological Systems 48410 Modeling of Closed-Loop Systems 48910.1 Autoregressive Form of Closed-Loop Model 49010.2 Network Model Form of Closed-Loop Systems 491Appendix I Function Expansions 495Appendix II Gaussian White Noise 499Appendix III Construction of the Wiener Series 503Appendix IV Stationarity, Ergodicity, and Autocorrelation Functions of Random Processes 505References 507Index 535

"...a perfect research tool, as reference book, and even as a textbook. I highly recommend it to everyone interested in nonlinear dynamics." (Journal of Intelligent & Fuzzy Systems, Vol. 16, No. 2, 2005) "...a well-written methodology book...a useful addition to [researchers, engineers and graduate students']...personal libraries." (E-STREAMS, September 2005)