Introduction to Online Control

Elad Hazan is Professor of Computer Science at Princeton University. His research focuses on the design and analysis of algorithms for basic problems in machine learning and optimization. He is a pioneer of online nonstochastic control theory. Karan Singh is Assistant Professor of Operations Research at Carnegie Mellon University, and has previously worked at Google Brain and Microsoft Research. He works on the foundations of machine learning, control, and reinforcement learning.

• Symbols; Part I. Background in Control and RL: 1. Introduction; 2. Dynamical systems; 3. Markov decision processes; 4. Linear dynamical systems; 5. Optimal control of linear dynamical systems; Part II. Basics of Online Control: 6. Regret in control; 7. Online nonstochastic control; 8. Online nonstochastic system identification; Part III. Learning and Filtering: 9. Learning in unknown linear dynamical systems; 10. Kalman filtering; 11. Spectral filtering; Part IV. Online Control with Partial Observation: 12. Policy classes for partially observed systems; 13. Online nonstochastic control with partial observation; References; Index.

'We are in a golden age for control and decision making. A proliferation of new applications including self-driving vehicles, humanoid robots, and artificially intelligent drones opens a new set of challenges for control theory to address. Hazan and Singh have written the definitive book on the New Control Theory - non-stochastic control. The phrase 'a paradigm shift' has become cliche from overuse, but here it is truly well deserved; the authors have revisited the foundations by focusing on building controllers that perform nearly as well as if they knew future disturbances in advance, rather than relying on probabilistic or worst-case models. The non-stochastic control approach has extended one of the most profound ideas in mathematics of the 20th century, online (no-regret) learning, to master sequential decision making with continuous actions. This leads to high performance in benign environments and resilience in adversarial ones. The book, authored by pioneers in the field, presents both foundational concepts and the latest research, making it an invaluable resource.' Drew Bagnell, Carnegie Mellon University and Aurora