A Two-Moment Inequality with Applications to Rényi Entropy and Mutual Information.

Journal Article (Journal Article)

This paper explores some applications of a two-moment inequality for the integral of the rth power of a function, where 0<1. The first contribution is an upper bound on the Rényi entropy of a random vector in terms of the two different moments. When one of the moments is the zeroth moment, these bounds recover previous results based on maximum entropy distributions under a single moment constraint. More generally, evaluation of the bound with two carefully chosen nonzero moments can lead to significant improvements with a modest increase in complexity. The second contribution is a method for upper bounding mutual information in terms of certain integrals with respect to the variance of the conditional density. The bounds have a number of useful properties arising from the connection with variance decompositions.

Full Text

Duke Authors

Cited Authors

  • Reeves, G

Published Date

  • November 2020

Published In

Volume / Issue

  • 22 / 11

Start / End Page

  • E1244 -

PubMed ID

  • 33287012

Pubmed Central ID

  • PMC7712232

Electronic International Standard Serial Number (EISSN)

  • 1099-4300

International Standard Serial Number (ISSN)

  • 1099-4300

Digital Object Identifier (DOI)

  • 10.3390/e22111244


  • eng