The Generalized Cross Entropy Method, with Applications to Probability Density Estimation

• DOI: 10.1007/s11009-009-9133-7
• Publication Date: 2009-05-15T01:14:08Z
• Authors: Zdravko I. Botev; Dirk P. Kroese
• Keywords: Density estimation; Cross-entropy method; Kernel density estimation; Empirical likelihood

Nonparametric density estimation aims to determine the sparsest model that explains a given set of empirical data and which uses as few assumptions as possible. Many of the currently existing methods do not provide a sparse solution to the problem and rely on asymptotic approximations. In this paper we describe a framework for density estimation which uses information-theoretic measures of model complexity with the aim of constructing a sparse density estimator that does not rely on large sample approximations. The effectiveness of the approach is demonstrated through an application to some well-known density estimation test cases.