Signal Theoretic Introduction to Random Processes

Roy M. Howard, PhD, is Adjunct Senior Research Fellow in the Department of Electrical and Computer Engineering at Curtin University, Perth, Australia. His research expertise includes modeling of stochastic processes, signal theory, and low noise amplifier design.

• Preface xiii1 A Signal Theoretic Introduction to Random Processes 11.1 Introduction 11.2 Motivation 21.3 Book Overview 82 Background: Mathematics 112.1 Introduction 112.2 Set Theory 112.3 Function Theory 132.4 Measure Theory 182.5 Measurable Functions 242.6 Lebesgue Integration 282.7 Convergence 372.8 Lebesgue–Stieltjes Measure 392.9 Lebesgue–Stieltjes Integration 502.10 Miscellaneous Results 612.11 Problems 623 Background: Signal Theory 713.1 Introduction 713.2 Signal Orthogonality 713.3 Theory for Dirichlet Points 753.4 Dirac Delta 783.5 Fourier Theory 793.6 Signal Power 823.7 The Power Spectral Density 843.8 The Autocorrelation Function 913.9 Power Spectral Density–Autocorrelation Function 953.10 Results for the Infinite Interval 963.11 Convergence of Fourier Coefficients 1033.12 Cramer’s Representation and Transform 1063.13 Problems 1254 Background: Probability and Random Variable Theory 1534.1 Introduction 1534.2 Basic Concepts: Experiments-Probability Theory 1534.3 The Random Variable 1604.4 Discrete and Continuous Random Variables 1624.5 Standard Random Variables 1654.6 Functions of a Random Variable 1654.7 Expectation 1664.8 Generation of Data Consistent with Defined PDF 1724.9 Vector Random Variables 1734.10 Pairs of Random Variables 1754.11 Covariance and Correlation 1864.12 Sums of Random Variables 1914.13 Jointly Gaussian Random Variables 1934.14 Stirling’s Formula and Approximations to Binomial 1944.15 Problems 1995 Introduction to Random Processes 2195.1 Random Processes 2195.2 Definition of a Random Process 2195.3 Examples of Random Processes 2215.4 Experiments and Experimental Outcomes 2255.5 Prototypical Experiments 2285.6 Random Variables Defined by a Random Process 2325.7 Classification of Random Processes 2335.8 Classification: One-Dimensional RPs 2365.9 Sums of Random Processes 2395.10 Problems 2396 Prototypical Random Processes 2436.1 Introduction 2436.2 Bernoulli Random Processes 2436.3 Poisson Random Processes 2466.4 Clustered Random Processes 2556.5 Signalling Random Processes 2576.6 Jitter 2626.7 White Noise 2656.8 1/f Noise 2726.9 Birth–Death Random Processes 2756.10 Orthogonal Increment Random Processes 2786.11 Linear Filtering of Random Processes 2826.12 Summary of Random Processes 2836.13 Problems 2857 Characterizing Random Processes 2897.1 Introduction 2897.2 Time Evolution of PMF or PDF 2917.3 First-, Second-, and Higher-Order Characterization 2927.4 Autocorrelation and Power Spectral Density 2977.5 Correlation 3087.6 Notes on Average Power and Average Energy 3107.7 Classification: Stationarity vs Non-Stationarity 3167.8 Cramer’s Representation 3237.9 State Space Characterization of Random Processes 3357.10 Time Series Characterization 3477.11 Problems 3478 PMF and PDF Evolution 3698.1 Introduction 3698.2 Probability Mass/Density Function Estimation 3708.3 Non/Semi-parametric PDF Estimation 3728.4 PMF/PDF Evolution: Signal Plus Noise 3788.5 PMF Evolution of a Random Walk 3818.6 PDF Evolution: Brownian Motion 3848.7 PDF Evolution: Signalling Random Process 3888.8 PDF Evolution: Generalized Shot Noise 3908.9 PDF Evolution: Switching in a CMOS Inverter 3968.10 PDF Evolution: General Case 4008.11 Problems 4059 The Autocorrelation Function 4179.1 Introduction 4179.2 Notation and Definitions 4179.3 Basic Results and Independence Information 4199.4 Sinusoid with Random Amplitude and Phase 4219.5 Random Telegraph Signal 4239.6 Generalized Shot Noise 4249.7 Signalling Random Process-Fixed Pulse Case 4349.8 Generalized Signalling Random Process 4419.9 Autocorrelation: Jittered Random Processes 4539.10 Random Walk 4569.11 Problems 45710 Power Spectral Density Theory 48110.1 Introduction 48110.2 Power Spectral Density Theory 48110.3 Power Spectral Density of a Periodic Pulse Train 48510.4 PSD of a Signalling Random Process 48710.5 Digital to Analogue Conversion 50110.6 PSD of Shot Noise Random Processes 50510.7 White Noise 50910.8 1/f Noise 51010.9 PSD of a Jittered Binary Random Process 51310.10 PSD of a Jittered Pulse Train 51710.11 Problems 52511 Order Statistics 55311.1 Introduction 55311.2 Ordered Random Variable Theory 55711.3 Identical RVs With Uniform Distribution 57411.4 Uniform Distribution and Infinite Interval 58411.5 Problems 59012 Poisson Point Random Processes 62112.1 Introduction 62112.2 Characterizing Poisson Random Processes 62312.3 PMF: Number of Points in a Subset of an Interval 62512.4 Results From Order Statistics 63012.5 Alternative Characterization for Infinite Interval 63412.6 Modelling with Unordered or Ordered Times 63612.7 Zero Crossing Times of Random Telegraph Signal 63812.8 Point Processes: The General Case 63912.9 Problems 63913 Birth–Death Random Processes 64913.1 Introduction 64913.2 Defining and Characterizing Birth–Death Processes 64913.3 Constant Birth Rate, Zero Death Rate Process 65613.4 State Dependent Birth Rate - Zero Death Rate 66213.5 Constant Death Rate, Zero Birth Rate, Process 66513.6 Constant Birth and Constant Death Rate Process 66713.7 Problems 66914 The First Passage Time 67714.1 Introduction 67714.2 First Passage Time 67714.3 Approaches: Establishing the First Passage Time 68114.4 Maximum Level and the First Passage Time 68514.5 Solutions for the First Passage Time PDF 69014.6 Problems 695Reference Material 709References 717Index 721

"This is a useful textbook for upper-undergraduate and graduate-level courses in applied mathematics as well as electrical and communications engineering departments. The book is also an excellent reference for research engineers and scientists who need to characterize random phenomena in their research." (Zentralblatt MATH, 2016)