A Two-Moment Inequality with Applications to Rényi Entropy and Mutual Information.
Journal Article (Journal Article)
This paper explores some applications of a two-moment inequality for the integral of the rth power of a function, where 0<1. The first contribution is an upper bound on the Rényi entropy of a random vector in terms of the two different moments. When one of the moments is the zeroth moment, these bounds recover previous results based on maximum entropy distributions under a single moment constraint. More generally, evaluation of the bound with two carefully chosen nonzero moments can lead to significant improvements with a modest increase in complexity. The second contribution is a method for upper bounding mutual information in terms of certain integrals with respect to the variance of the conditional density. The bounds have a number of useful properties arising from the connection with variance decompositions.
Full Text
Duke Authors
Cited Authors
- Reeves, G
Published Date
- November 2020
Published In
Volume / Issue
- 22 / 11
Start / End Page
- E1244 -
PubMed ID
- 33287012
Pubmed Central ID
- PMC7712232
Electronic International Standard Serial Number (EISSN)
- 1099-4300
International Standard Serial Number (ISSN)
- 1099-4300
Digital Object Identifier (DOI)
- 10.3390/e22111244
Language
- eng