Advances in Electromagnetics Empowered by Artificial Intelligence and Deep Learning

Sawyer D. Campbell is an Assistant Research Professor in the Pennsylvania State University Department of Electrical Engineering where he is also the associate director of the Computational Electromagnetics and Antennas Research Lab. Douglas H. Werner is the director of the Computational Electromagnetics and Antennas Research Lab as well as a faculty member of the Materials Research Institute at Penn State.

• About the Editors xixList of Contributors xxPreface xxviSection I Introduction to AI-Based Regression and Classification 11 Introduction to Neural Networks 3Isha Garg and Kaushik Roy1.1 Taxonomy 31.1.1 Supervised Versus Unsupervised Learning 31.1.2 Regression Versus Classification 41.1.3 Training, Validation, and Test Sets 41.2 Linear Regression 51.2.1 Objective Functions 61.2.2 Stochastic Gradient Descent 71.3 Logistic Classification 91.4 Regularization 111.5 Neural Networks 131.6 Convolutional Neural Networks 161.6.1 Convolutional Layers 171.6.2 Pooling Layers 181.6.3 Highway Connections 191.6.4 Recurrent Layers 191.7 Conclusion 20References 202 Overview of Recent Advancements in Deep Learning and Artificial Intelligence 23Vijaykrishnan Narayanan, Yu Cao, Priyadarshini Panda, Nagadastagiri Reddy Challapalle, Xiaocong Du, Youngeun Kim, Gokul Krishnan, Chonghan Lee, Yuhang Li, Jingbo Sun, Yeshwanth Venkatesha, Zhenyu Wang, and Yi Zheng2.1 Deep Learning 242.1.1 Supervised Learning 262.1.1.1 Conventional Approaches 262.1.1.2 Deep Learning Approaches 292.1.2 Unsupervised Learning 352.1.2.1 Algorithm 352.1.3 Toolbox 372.2 Continual Learning 382.2.1 Background and Motivation 382.2.2 Definitions 382.2.3 Algorithm 382.2.3.1 Regularization 392.2.3.2 Dynamic Network 402.2.3.3 Parameter Isolation 402.2.4 Performance Evaluation Metric 412.2.5 Toolbox 412.3 Knowledge Graph Reasoning 422.3.1 Background 422.3.2 Definitions 422.3.3 Database 432.3.4 Applications 432.3.5 Toolbox 442.4 Transfer Learning 442.4.1 Background and Motivation 442.4.2 Definitions 442.4.3 Algorithm 452.4.4 Toolbox 462.5 Physics-Inspired Machine Learning Models 462.5.1 Background and Motivation 462.5.2 Algorithm 462.5.3 Applications 492.5.4 Toolbox 502.6 Distributed Learning 502.6.1 Introduction 502.6.2 Definitions 512.6.3 Methods 512.6.4 Toolbox 542.7 Robustness 542.7.1 Background and Motivation 542.7.2 Definitions 552.7.3 Methods 552.7.3.1 Training with Noisy Data/Labels 552.7.3.2 Adversarial Attacks 552.7.3.3 Defense Mechanisms 562.7.4 Toolbox 562.8 Interpretability 562.8.1 Background and Motivation 562.8.2 Definitions 572.8.3 Algorithm 572.8.4 ToolBox 582.9 Transformers and Attention Mechanisms for Text and Vision Models 582.9.1 Background and Motivation 582.9.2 Algorithm 592.9.3 Application 602.9.4 Toolbox 612.10 Hardware for Machine Learning Applications 622.10.1 Cpu 622.10.2 Gpu 632.10.3 ASICs 632.10.4 Fpga 64Acknowledgment 64References 64Section II Advancing Electromagnetic Inverse Design with Machine Learning 813 Breaking the Curse of Dimensionality in Electromagnetics Design Through Optimization Empowered by Machine Learning 83N. Anselmi, G. Oliveri, L. Poli, A. Polo, P. Rocca, M. Salucci, and A. Massa3.1 Introduction 833.2 The SbD Pillars and Fundamental Concepts 853.3 SbD at Work in EMs Design 883.3.1 Design of Elementary Radiators 883.3.2 Design of Reflectarrays 923.3.3 Design of Metamaterial Lenses 933.3.4 Other SbD Customizations 963.4 Final Remarks and Envisaged Trends 101Acknowledgments 101References 1024 Artificial Neural Networks for Parametric Electromagnetic Modeling and Optimization 105Feng Feng, Weicong Na, Jing Jin, and Qi-Jun Zhang4.1 Introduction 1054.2 ANN Structure and Training for Parametric EM Modeling 1064.3 Deep Neural Network for Microwave Modeling 1074.3.1 Structure of the Hybrid DNN 1074.3.2 Training of the Hybrid DNN 1084.3.3 Parameter-Extraction Modeling of a Filter Using the Hybrid DNN 1084.4 Knowledge-Based Parametric Modeling for Microwave Components 1114.4.1 Unified Knowledge-Based Parametric Model Structure 1124.4.2 Training with l 1 Optimization of the Unified Knowledge-Based Parametric Model 1154.4.3 Automated Knowledge-Based Model Generation 1174.4.4 Knowledge-Based Parametric Modeling of a Two-Section Low-Pass Elliptic Microstrip Filter 1174.5 Parametric Modeling Using Combined ANN and Transfer Function 1214.5.1 Neuro-TF Modeling in Rational Form 1214.5.2 Neuro-TF Modeling in Zero/Pole Form 1224.5.3 Neuro-TF Modeling in Pole/Residue Form 1234.5.4 Vector Fitting Technique for Parameter Extraction 1234.5.5 Two-Phase Training for Neuro-TF Models 1234.5.6 Neuro-TF Model Based on Sensitivity Analysis 1254.5.7 A Diplexer Example Using Neuro-TF Model Based on Sensitivity Analysis 1264.6 Surrogate Optimization of EM Design Based on ANN 1294.6.1 Surrogate Optimization and Trust Region Update 1294.6.2 Neural TF Optimization Method Based on Adjoint Sensitivity Analysis 1304.6.3 Surrogate Model Optimization Based on Feature-Assisted of Neuro-TF 1304.6.4 EM Optimization of a Microwave Filter Utilizing Feature-Assisted Neuro-TF 1314.7 Conclusion 133References 1335 Advanced Neural Networks for Electromagnetic Modeling and Design 141Bing-Zhong Wang, Li-Ye Xiao, and Wei Shao5.1 Introduction 1415.2 Semi-Supervised Neural Networks for Microwave Passive Component Modeling 1415.2.1 Semi-Supervised Learning Based on Dynamic Adjustment Kernel Extreme Learning Machine 1415.2.1.1 Dynamic Adjustment Kernel Extreme Learning Machine 1425.2.1.2 Semi-Supervised Learning Based on DA-KELM 1475.2.1.3 Numerical Examples 1505.2.2 Semi-Supervised Radial Basis Function Neural Network 1575.2.2.1 Semi-Supervised Radial Basis Function Neural Network 1575.2.2.2 Sampling Strategy 1615.2.2.3 SS-RBFNN With Sampling Strategy 1625.3 Neural Networks for Antenna and Array Modeling 1665.3.1 Modeling of Multiple Performance Parameters for Antennas 1665.3.2 Inverse Artificial Neural Network for Multi-objective Antenna Design 1755.3.2.1 Knowledge-Based Neural Network for Periodic Array Modeling 1835.4 Autoencoder Neural Network for Wave Propagation in Uncertain Media 1885.4.1 Two-Dimensional GPR System with the Dispersive and Lossy Soil 1885.4.2 Surrogate Model for GPR Modeling 1905.4.3 Modeling Results 191References 193Section III Deep Learning for Metasurface Design 1976 Generative Machine Learning for Photonic Design 199Dayu Zhu, Zhaocheng Liu, and Wenshan Cai6.1 Brief Introduction to Generative Models 1996.1.1 Probabilistic Generative Model 1996.1.2 Parametrization and Optimization with Generative Models 1996.1.2.1 Probabilistic Model for Gradient-Based Optimization 2006.1.2.2 Sampling-Based Optimization 2006.1.2.3 Generative Design Strategy 2016.1.2.4 Generative Adversarial Networks in Photonic Design 2026.1.2.5 Discussion 2036.2 Generative Model for Inverse Design of Metasurfaces 2036.2.1 Generative Design Strategy for Metasurfaces 2036.2.2 Model Validation 2046.2.3 On-demand Design Results 2066.3 Gradient-Free Optimization with Generative Model 2076.3.1 Gradient-Free Optimization Algorithms 2076.3.2 Evolution Strategy with Generative Parametrization 2076.3.2.1 Generator from VAE 2076.3.2.2 Evolution Strategy 2086.3.2.3 Model Validation 2096.3.2.4 On-demand Design Results 2096.3.3 Cooperative Coevolution and Generative Parametrization 2106.3.3.1 Cooperative Coevolution 2106.3.3.2 Diatomic Polarizer 2116.3.3.3 Gradient Metasurface 2116.4 Design Large-Scale, Weakly Coupled System 2136.4.1 Weak Coupling Approximation 2146.4.2 Analog Differentiator 2146.4.3 Multiplexed Hologram 2156.5 Auxiliary Methods for Generative Photonic Parametrization 2176.5.1 Level Set Method 2176.5.2 Fourier Level Set 2186.5.3 Implicit Neural Representation 2186.5.4 Periodic Boundary Conditions 2206.6 Summary 221References 2217 Machine Learning Advances in Computational Electromagnetics 225Robert Lupoiu and Jonathan A. Fan7.1 Introduction 2257.2 Conventional Electromagnetic Simulation Techniques 2267.2.1 Finite Difference Frequency (FDFD) and Time (FDTD) Domain Solvers 2267.2.2 The Finite Element Method (FEM) 2297.2.2.1 Meshing 2297.2.2.2 Basis Function Expansion 2297.2.2.3 Residual Formulation 2307.2.3 Method of Moments (MoM) 2307.3 Deep Learning Methods for Augmenting Electromagnetic Solvers 2317.3.1 Time Domain Simulators 2317.3.1.1 Hardware Acceleration 2317.3.1.2 Learning Finite Difference Kernels 2327.3.1.3 Learning Absorbing Boundary Conditions 2347.3.2 Augmenting Variational CEM Techniques Via Deep Learning 2347.4 Deep Electromagnetic Surrogate Solvers Trained Purely with Data 2357.5 Deep Surrogate Solvers Trained with Physical Regularization 2407.5.1 Physics-Informed Neural Networks (PINNs) 2407.5.2 Physics-Informed Neural Networks with Hard Constraints (hPINNs) 2417.5.3 WaveY-Net 2437.6 Conclusions and Perspectives 249Acknowledgments 250References 2508 Design of Nanofabrication-Robust Metasurfaces Through Deep Learning-Augmented Multiobjective Optimization 253Ronald P. Jenkins, Sawyer D. Campbell, and Douglas H. Werner8.1 Introduction 2538.1.1 Metasurfaces 2538.1.2 Fabrication State-of-the-Art 2538.1.3 Fabrication Challenges 2548.1.3.1 Fabrication Defects 2548.1.4 Overcoming Fabrication Limitations 2558.2 Related Work 2558.2.1 Robustness Topology Optimization 2558.2.2 Deep Learning in Nanophotonics 2568.3 DL-Augmented Multiobjective Robustness Optimization 2578.3.1 Supercells 2578.3.1.1 Parameterization of Freeform Meta-Atoms 2578.3.2 Robustness Estimation Method 2598.3.2.1 Simulating Defects 2598.3.2.2 Existing Estimation Methods 2598.3.2.3 Limitations of Existing Methods 2598.3.2.4 Solver Choice 2608.3.3 Deep Learning Augmentation 2608.3.3.1 Challenges 2618.3.3.2 Method 2618.3.4 Multiobjective Global Optimization 2678.3.4.1 Single Objective Cost Functions 2678.3.4.2 Dominance Relationships 2678.3.4.3 A Robustness Objective 2698.3.4.4 Problems with Optimization and DL Models 2698.3.4.5 Error-Tolerant Cost Functions 2698.3.5 Robust Supercell Optimization 2708.3.5.1 Pareto Front Results 2708.3.5.2 Examples from the Pareto Front 2718.3.5.3 The Value of Exhaustive Sampling 2728.3.5.4 Speedup Analysis 2738.4 Conclusion 2758.4.1 Future Directions 275Acknowledgments 276References 2769 Machine Learning for Metasurfaces Design and Their Applications 281Kumar Vijay Mishra, Ahmet M. Elbir, and Amir I. Zaghloul9.1 Introduction 2819.1.1 ML/DL for RIS Design 2839.1.2 ML/DL for RIS Applications 2839.1.3 Organization 2859.2 Inverse RIS Design 2859.2.1 Genetic Algorithm (GA) 2869.2.2 Particle Swarm Optimization (PSO) 2869.2.3 Ant Colony Optimization (ACO) 2899.3 DL-Based Inverse Design and Optimization 2899.3.1 Artificial Neural Network (ANN) 2899.3.1.1 Deep Neural Networks (DNN) 2909.3.2 Convolutional Neural Networks (CNNs) 2909.3.3 Deep Generative Models (DGMs) 2919.3.3.1 Generative Adversarial Networks (GANs) 2919.3.3.2 Conditional Variational Autoencoder (cVAE) 2939.3.3.3 Global Topology Optimization Networks (GLOnets) 2939.4 Case Studies 2949.4.1 MTS Characterization Model 2949.4.2 Training and Design 2969.5 Applications 2989.5.1 DL-Based Signal Detection in RIS 3029.5.2 DL-Based RIS Channel Estimation 3039.6 DL-Aided Beamforming for RIS Applications 3069.6.1 Beamforming at the RIS 3069.6.2 Secure-Beamforming 3089.6.3 Energy-Efficient Beamforming 3099.6.4 Beamforming for Indoor RIS 3099.7 Challenges and Future Outlook 3099.7.1 Design 3109.7.1.1 Hybrid Physics-Based Models 3109.7.1.2 Other Learning Techniques 3109.7.1.3 Improved Data Representation 3109.7.2 Applications 3119.7.3 Channel Modeling 3119.7.3.1 Data Collection 3119.7.3.2 Model Training 3119.7.3.3 Environment Adaptation and Robustness 3129.8 Summary 312Acknowledgments 313References 313Section IV Rf, Antenna, Inverse-scattering, and other Em Applications of Deep Learning 31910 Deep Learning for Metasurfaces and Metasurfaces for Deep Learning 321Clayton Fowler, Sensong An, Bowen Zheng, and Hualiang Zhang10.1 Introduction 32110.2 Forward-Predicting Networks 32210.2.1 FCNN (Fully Connected Neural Networks) 32310.2.2 CNN (Convolutional Neural Networks) 32410.2.2.1 Nearly Free-Form Meta-Atoms 32410.2.2.2 Mutual Coupling Prediction 32710.2.3 Sequential Neural Networks and Universal Forward Prediction 33010.2.3.1 Sequencing Input Data 33110.2.3.2 Recurrent Neural Networks 33210.2.3.3 1D Convolutional Neural Networks 33210.3 Inverse-Design Networks 33310.3.1 Tandem Network for Inverse Designs 33310.3.2 Generative Adversarial Nets (GANs) 33510.4 Neuromorphic Photonics 33910.5 Summary and Outlook 340References 34111 Forward and Inverse Design of Artificial Electromagnetic Materials 345Jordan M. Malof, Simiao Ren, and Willie J. Padilla11.1 Introduction 34511.1.1 Problem Setting 34611.1.2 Artificial Electromagnetic Materials 34711.1.2.1 Regime 1: Floquet–Bloch 34811.1.2.2 Regime 2: Resonant Effective Media 34911.1.2.3 All-Dielectric Metamaterials 35011.2 The Design Problem Formulation 35111.3 Forward Design 35211.3.1 Search Efficiency 35311.3.2 Evaluation Time 35411.3.3 Challenges with the Forward Design of Advanced AEMs 35411.3.4 Deep Learning the Forward Model 35511.3.4.1 When Does Deep Learning Make Sense? 35511.3.4.2 Common Deep Learning Architectures 35611.3.5 The Forward Design Bottleneck 35611.4 Inverse Design with Deep Learning 35711.4.1 Why Inverse Problems Are Often Difficult 35911.4.2 Deep Inverse Models 36011.4.2.1 Does the Inverse Model Address Non-uniqueness? 36011.4.2.2 Multi-solution Versus Single-Solution Models 36011.4.2.3 Iterative Methods versus Direct Mappings 36111.4.3 Which Inverse Models Perform Best? 36111.5 Conclusions and Perspectives 36211.5.1 Reducing the Need for Training Data 36211.5.1.1 Transfer Learning 36211.5.1.2 Active Learning 36311.5.1.3 Physics-Informed Learning 36311.5.2 Inverse Modeling for Non-existent Solutions 36311.5.3 Benchmarking, Replication, and Sharing Resources 364Acknowledgments 364References 36412 Machine Learning-Assisted Optimization and Its Application to Antenna and Array Designs 371Qi Wu, Haiming Wang, and Wei Hong12.1 Introduction 37112.2 Machine Learning-Assisted Optimization Framework 37212.3 Machine Learning-Assisted Optimization for Antenna and Array Designs 37512.3.1 Design Space Reduction 37512.3.2 Variable-Fidelity Evaluation 37512.3.3 Hybrid Optimization Algorithm 37812.3.4 Robust Design 37912.3.5 Antenna Array Synthesis 38012.4 Conclusion 381References 38113 Analysis of Uniform and Non-uniform Antenna Arrays Using Kernel Methods 385Manel Martínez-Ramón, José Luis Rojo Álvarez, Arjun Gupta, and Christos Christodoulou13.1 Introduction 38513.2 Antenna Array Processing 38613.2.1 Detection of Angle of Arrival 38713.2.2 Optimum Linear Beamformers 38813.2.3 Direction of Arrival Detection with Random Arrays 38913.3 Support Vector Machines in the Complex Plane 39013.3.1 The Support Vector Criterion for Robust Regression in the Complex Plane 39013.3.2 The Mercer Theorem and the Nonlinear SVM 39313.4 Support Vector Antenna Array Processing with Uniform Arrays 39413.4.1 Kernel Array Processors with Temporal Reference 39413.4.1.1 Relationship with the Wiener Filter 39413.4.2 Kernel Array Processor with Spatial Reference 39513.4.2.1 Eigenanalysis in a Hilbert Space 39513.4.2.2 Formulation of the Processor 39613.4.2.3 Relationship with Nonlinear MVDM 39713.4.3 Examples of Temporal and Spatial Kernel Beamforming 39813.5 DOA in Random Arrays with Complex Gaussian Processes 40013.5.1 Snapshot Interpolation from Complex Gaussian Process 40013.5.2 Examples 40213.6 Conclusion 403Acknowledgments 404References 40414 Knowledge-Based Globalized Optimization of High-Frequency Structures Using Inverse Surrogates 409Anna Pietrenko-Dabrowska and Slawomir Koziel14.1 Introduction 40914.2 Globalized Optimization by Feature-Based Inverse Surrogates 41114.2.1 Design Task Formulation 41114.2.2 Evaluating Design Quality with Response Features 41214.2.3 Globalized Search by Means of Inverse Regression Surrogates 41414.2.4 Local Tuning Procedure 41814.2.5 Global Optimization Algorithm 42014.3 Results 42114.3.1 Verification Structures 42214.3.2 Results 42314.3.3 Discussion 42314.4 Conclusion 428Acknowledgment 428References 42815 Deep Learning for High Contrast Inverse Scattering of Electrically Large Structures 435Qing Liu, Li-Ye Xiao, Rong-Han Hong, and Hao-Jie Hu15.1 Introduction 43515.2 General Strategy and Approach 43615.2.1 Related Works by Others and Corresponding Analyses 43615.2.2 Motivation 43715.3 Our Approach for High Contrast Inverse Scattering of Electrically Large Structures 43815.3.1 The 2-D Inverse Scattering Problem with Electrically Large Structures 43815.3.1.1 Dual-Module NMM-IEM Machine Learning Model 43815.3.1.2 Receiver Approximation Machine Learning Method 44015.3.2 Application for 3-D Inverse Scattering Problem with Electrically Large Structures 44115.3.2.1 Semi-Join Extreme Learning Machine 44115.3.2.2 Hybrid Neural Network Electromagnetic Inversion Scheme 44515.4 Applications of Our Approach 45015.4.1 Applications for 2-D Inverse Scattering Problem with Electrically Large Structures 45015.4.1.1 Dual-Module NMM-IEM Machine Learning for Fast Electromagnetic Inversion of Inhomogeneous Scatterers with High Contrasts and Large Electrical Dimensions 45015.4.1.2 Nonlinear Electromagnetic Inversion of Damaged Experimental Data by a Receiver Approximation Machine Learning Method 45415.4.2 Applications for 3-D Inverse Scattering Problem with Electrically Large Structures 45915.4.2.1 Super-Resolution 3-D Microwave Imaging of Objects with High Contrasts by a Semi-Join Extreme Learning Machine 45915.4.2.2 A Hybrid Neural Network Electromagnetic Inversion Scheme (HNNEMIS) for Super-Resolution 3-Dimensional Microwave Human Brain Imaging 47315.5 Conclusion and Future work 48015.5.1 Summary of Our Work 48015.5.1.1 Limitations and Potential Future Works 481References 48216 Radar Target Classification Using Deep Learning 487Youngwook Kim16.1 Introduction 48716.2 Micro-Doppler Signature Classification 48816.2.1 Human Motion Classification 49016.2.2 Human Hand Gesture Classification 49416.2.3 Drone Detection 49516.3 SAR Image Classification 49716.3.1 Vehicle Detection 49716.3.2 Ship Detection 49916.4 Target Classification in Automotive Radar 50016.5 Advanced Deep Learning Algorithms for Radar Target Classification 50316.5.1 Transfer Learning 50416.5.2 Generative Adversarial Networks 50616.5.3 Continual Learning 50816.6 Conclusion 511References 51117 Koopman Autoencoders for Reduced-Order Modeling of Kinetic Plasmas 515Indranil Nayak, Mrinal Kumar, and Fernando L. Teixeira17.1 Introduction 51517.2 Kinetic Plasma Models: Overview 51617.3 EMPIC Algorithm 51717.3.1 Overview 51717.3.2 Field Update Stage 51917.3.3 Field Gather Stage 52117.3.4 Particle Pusher Stage 52117.3.5 Current and Charge Scatter Stage 52217.3.6 Computational Challenges 52217.4 Koopman Autoencoders Applied to EMPIC Simulations 52317.4.1 Overview and Motivation 52317.4.2 Koopman Operator Theory 52417.4.3 Koopman Autoencoder (KAE) 52717.4.3.1 Case Study I: Oscillating Electron Beam 52917.4.3.2 Case Study II: Virtual Cathode Formation 53217.4.4 Computational Gain 53417.5 Towards A Physics-Informed Approach 53517.6 Outlook 536Acknowledgments 537References 537Index 543