Combinatorial Methods in Density Estimation

• 1. Introduction.- §1.1. References.- 2. Concentration Inequalities.- §2.1. Hoeffding’s Inequality.- §2.2. An Inequality for the Expected Maximal Deviation.- §2.3. The Bounded Difference Inequality.- §2.4. Examples.- §2.5. Bibliographic Remarks.- §2.6. Exercises.- §2.7. References.- 3. Uniform Deviation Inequalities.- §3.1. The Vapnik-Chervonenkis Inequality.- §3.2. Covering Numbers and Chaining.- §3.3. Example: The Dvoretzky-Kiefer-Wolfowitz Theorem.- §3.4. Bibliographic Remarks.- §3.5. Exercises.- §3.6. References.- 4. Combinatorial Tools.- §4.1. Shatter Coefficients.- §4.2. Vapnik-Chervonenkis Dimension and Shatter Coefficients.- §4.3. Vapnik-Chervonenkis Dimension and Covering Numbers.- §4.4. Examples.- §4.5. Bibliographic Remarks.- §4.6. Exercises.- §4.7. References.- 5. Total Variation.- §5.1. Density Estimation.- §5.2. The Total Variation.- §5.3. Invariance.- §5.4. Mappings.- §5.5. Convolutions.- §5.6. Normalization.- §5.7. The Lebesgue Density Theorem.- §5.8. LeCam’s Inequality.- §5.9. Bibliographic Remarks.- §5.10. Exercises.- §5.11. References.- 6. Choosing a Density Estimate.- §6.1. Choosing Between Two Densities.- §6.2. Examples.- §6.3. Is the Factor of Three Necessary?.- §6.4. Maximum Likelihood Does not Work.- §6.5. L2 Distances Are To Be Avoided.- §6.6. Selection from k Densities.- §6.7. Examples Continued.- §6.8. Selection from an Infinite Class.- §6.9. Bibliographic Remarks.- §6.10. Exercises.- §6.11. References.- 7. Skeleton Estimates.- §7.1. Kolmogorov Entropy.- §7.2. Skeleton Estimates.- §7.3. Robustness.- §7.4. Finite Mixtures.- §7.5. Monotone Densities on the Hypercube.- §7.6. How To Make Gigantic Totally Bounded Classes.- §7.7. Bibliographic Remarks.- §7.8. Exercises.- §7.9. References.- 8.The Minimum Distance Estimate: Examples.- §8.1. Problem Formulation.- §8.2. Series Estimates.- §8.3. Parametric Estimates: Exponential Families.- §8.4. Neural Network Estimates.- §8.5. Mixture Classes, Radial Basis Function Networks.- §8.6. Bibliographic Remarks.- §8.7. Exercises.- §8.8. References.- 9. The Kernel Density Estimate.- §9.1. Approximating Functions by Convolutions.- §9.2. Definition of the Kernel Estimate.- §9.3. Consistency of the Kernel Estimate.- §9.4. Concentration.- §9.5. Choosing the Bandwidth.- §9.6. Choosing the Kernel.- §9.7. Rates of Convergence.- §9.8. Uniform Rate of Convergence.- §9.9. Shrinkage, and the Combination of Density Estimates.- §9.10. Bibliographic Remarks.- §9.11. Exercises.- §9.12. References.- 10. Additive Estimates and Data Splitting.- §10.1. Data Splitting.- §10.2. Additive Estimates.- §10.3. Histogram Estimates.- §10A. Bibliographic Remarks.- §10.5. Exercises.- §10.6. References.- 11. Bandwidth Selection for Kernel Estimates.- §11.1. The Kernel Estimate with Riemann Kernel.- §11.2. General Kernels, Kernel Complexity.- §11.3. Kernel Complexity: Univariate Examples.- §11.4. Kernel Complexity: Multivariate Kernels.- §11.5. Asymptotic Optimality.- §11.6. Bibliographic Remarks.- §11.7. Exercises.- §11.8. References.- 12. Multiparameter Kernel Estimates.- §12.1. Multivariate Kernel Estimates—Product Kernels.- §12.2. Multivariate Kernel Estimates—Ellipsoidal Kernels.- §12.3. Variable Kernel Estimates.- §12.4. Tree-Structured Partitions.- §12.5. Changepoints and Bump Hunting.- §12.6. Bibliographic Remarks.- §12.7. Exercises.- §12.8. References.- 13. Wavelet Estimates.- §13.1. Definitions.- §13.2. Smoothing.- §13.3. Thresholding.- §13.4. Soft Thresholding.- §13.5. BibliographicRemarks.- §13.6. Exercises.- §13.7. References.- 14. The Transformed Kernel Estimate.- §14.1. The Transformed Kernel Estimate.- §14.2. Box-Cox Transformations.- §14.3. Piecewise Linear Transformations.- §14.4. Bibliographic Remarks.- §14.5. Exercises.- §14.6. References.- 15. Minimax Theory.- §15.1. Estimating a Density from One Data Point.- §15.2. The General Minimax Problem.- §15.3. Rich Classes.- §15.4. Assouad’s Lemma.- §15.5. Example: The Class of Convex Densities.- §15.6. Additional Examples.- §15.7. Tuning the Parameters of Variable Kernel Estimates.- §15.8. Sufficient Statistics.- §15.9. Bibliographic Remarks.- §15.10. Exercises.- §15.11. References.- 16. Choosing the Kernel Order.- §16.1. Introduction.- §16.2. Standard Kernel Estimate: Riemann Kernels.- §16.3. Standard Kernel Estimates: General Kernels.- §16.4. An Infinite Family of Kernels.- §16.5. Bibliographic Remarks.- §16.6. Exercises.- §16.7. References.- 17. Bandwidth Choice with Superkernels.- §17.1. Superkernels.- §17.2. The Trapezoidal Kernel.- §17.3. Bandwidth Selection.- §17.4. Bibliographic Remarks.- §17.5. Exercises.- §17.6. References.- Author Index.

From the reviews of the first edition: "This book is built around a new look on the important problem of bandwidth selection in density estimation. This new method has been launched in two recent papers of the two authors in the Annals of Statistics. It is based on ideas of minimum distance methods and convergence theory for empirical measures, uniformly over certain classes. The methods aim at finding estimators with universal properties that is valid for all (or nearly all) densities. The book is self-contained because a lot of fundamental inequalities and essential combinatorial techniques are collected in the first part of the book. There is a rich choice of exercises, some of which may be quite hard. This makes it interesting for classroom teaching. It is an attractive book that certainly provides inspiration for further research." Short Book Reviews, Vol. 21, No. 2, August 2001 "The book deals with probability density estimation from an i.i.d. sample, but the approach is different from those used in other texts on this topic. ... It is the aim of the book to study universal performance properties of these estimates. ... it is well written following the same idea throughout and contains many exercises which complete the different topics. ... I enjoyed reading this nicely written book which can certainly be recommended to all mathematically orientated statisticians interested in the subject." (Ulrich Stadtmuller, Mathematical Reviews, Issue 2002 h) "This carefully written monograph focuses on nonparametric estimation of a density from i.i.d. data, with the goodness-of-fit being measured in terms of the L1-norm. ... The book is recommended to those who want to get an overview of the state of the art of this approach." (W. Stute, Zentralblatt MATH, Vol. 964, 2001)