Limit theorems for empirical processes

Abstract

The empirical measure P _n for iid sampling on a distribution P is formed by placing mass n ⁻¹ at each of the first n observations. Generalizations of the classical Glivenko-Cantelli theorem for empirical measures have been proved by Vapnik and červonenkis using combinatorial methods. They found simple conditions on a class C to ensure that sup {|P _n (C) − P(C)|: C ∈ C} converges in probability to zero. They used a randomization device that reduced the problem to finding exponential bounds on the tails of a hypergeometric distribution. In this paper an alternative randomization is proposed. The role of the hypergeometric distribution is thereby taken over by the binomial distribution, for which the elementary Bernstein inequalities provide exponential boundson the tails. This leads to easier proofs of both the basic results of Vapnik-červonenkis and the extensions due to Steele. A similar simplification is made in the proof of Dudley's central limit theorem forn ^1/2(P P _n −P)— a result that generalizes Donsker's functional central limit theorem for empirical distribution functions.