Theory of Neural Information Processing Systems provides an explicit, coherent, and up-to-date account of the modern theory of neural information processing systems. It has been carefully developed for graduate students from any quantitative dsicipline, including mathematics, computer science, physics, engineering or biology, and has been thoroughly class-tested by the authors over a period of some 8 years. Exercises are presented throughout the text and notes on historical background and further reading guide the student into the literature. All mathematical details are included and appendices provide further background material, including probability theory, linear algebra and stochastic processes, making this textbook accessible to a wide audience.
I INTRODUCTION TO NEURAL NETWORKS ; 1. General introduction ; 2. Layered networks ; 3. Recurrent networks with binary neurons ; II ADVANCED NEURAL NETWORKS ; 4. Competitive unsupervised learning processes ; 5. Bayesian techniques in supervised learning ; 6. Gaussian processes ; 7. Support vector machines for binary classification ; III INFORMATION THEORY AND NEURAL NETWORKS ; 8. Measuring information ; 9. Identification of entropy as an information measure ; 10. Building blocks of Shannon's information theory ; 11. Information theory and statistical inference ; 12. Applications to neural networks ; IV MACROSCOPIC ANALYSIS OF DYNAMICS ; 13. Network operation: macroscopic dynamics ; 14. Dynamics of online learning in binary perceptrons ; 15. Dynamics of online gradient descent learning ; V EQUILIBRIUM STATISTICAL MECHANICS OF NEURAL NETWORKS ; 16. Basics of equilibrium statistical mechanics ; 17. Network operation: equilibrium analysis ; 18. Gardner theory of task realizability ; APPENDICES ; A. Historical and bibliographical notes ; B. Probability theory in a nutshell ; C. Conditions for central limit theorem to apply ; D. Some simple summation identities ; E. Gaussian integrals and probability distributions ; F. Matrix identities ; G. The delta-distribution ; H. Inequalities based on convexity ; I. Metrics for parametrized probability distributions ; J. Saddle-point integration ; REFERENCES
The book provides an excellent class-tested material for graduate courses in artificial neural networks. It is completely self-contained and includes also thorough introduction to the discussed discipline-specific areas of mathematics...Therefore, this book represents a good reference source of applicable ideas for a wide audience including students, researchers and application specialists as well.