Skip to main content
release_alert
Welcome to the new Scholars 3.0! Read about new features and let us know what you think.
cancel

Two-moment inequalities for Rényi entropy and mutual information

Publication ,  Conference
Reeves, G
Published in: Ieee International Symposium on Information Theory Proceedings
August 9, 2017

This paper explores some applications of a two-moment inequality for the integral of the r-th power of a function, where 0 < r < 1. The first contribution is an upper bound on the Rényi entropy of a random vector in terms of the two different moments. When one of the moments is the zeroth moment, these bounds recover previous results based on maximum entropy distributions under a single moment constraint. More generally, evaluation of the bound with two carefully chosen nonzero moments can lead to significant improvements with a modest increase in complexity. The second contribution is a method for upper bounding mutual information in terms of certain integrals with respect to the variance of the conditional density. The bounds have a number of useful properties arising from the connection with variance decompositions.

Duke Scholars

Published In

Ieee International Symposium on Information Theory Proceedings

DOI

ISSN

2157-8095

ISBN

9781509040964

Publication Date

August 9, 2017

Start / End Page

664 / 668
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Reeves, G. (2017). Two-moment inequalities for Rényi entropy and mutual information. In Ieee International Symposium on Information Theory  Proceedings (pp. 664–668). https://doi.org/10.1109/ISIT.2017.8006611
Reeves, G. “Two-moment inequalities for Rényi entropy and mutual information.” In Ieee International Symposium on Information Theory  Proceedings, 664–68, 2017. https://doi.org/10.1109/ISIT.2017.8006611.
Reeves G. Two-moment inequalities for Rényi entropy and mutual information. In: Ieee International Symposium on Information Theory  Proceedings. 2017. p. 664–8.
Reeves, G. “Two-moment inequalities for Rényi entropy and mutual information.” Ieee International Symposium on Information Theory  Proceedings, 2017, pp. 664–68. Scopus, doi:10.1109/ISIT.2017.8006611.
Reeves G. Two-moment inequalities for Rényi entropy and mutual information. Ieee International Symposium on Information Theory  Proceedings. 2017. p. 664–668.

Published In

Ieee International Symposium on Information Theory Proceedings

DOI

ISSN

2157-8095

ISBN

9781509040964

Publication Date

August 9, 2017

Start / End Page

664 / 668