Skip to main content
construction release_alert
The Scholars Team is working with OIT to resolve some issues with the Scholars search index
cancel

Information-theoretic limits for the matrix tensor product

Publication ,  Journal Article
Reeves, G
Published in: IEEE Journal on Selected Areas in Information Theory
November 1, 2020

This article studies a high-dimensional inference problem involving the matrix tensor product of random matrices. This problem generalizes a number of contemporary data science problems including the spiked matrix models used in sparse principal component analysis and covariance estimation and the stochastic block model used in network analysis. The main results are single-letter formulas (i.e., analytical expressions that can be approximated numerically) for the mutual information and the minimum mean-squared error (MMSE) in the Bayes optimal setting where the distributions of all random quantities are known. We provide non-asymptotic bounds and show that our formulas describe exactly the leading order terms in the mutual information and MMSE in the high-dimensional regime where the number of rows n and number of columns d scale with d = O(nα) for some α < 1/20. On the technical side, this article introduces some new techniques for the analysis of high-dimensional matrix-valued signals. Specific contributions include a novel extension of the adaptive interpolation method that uses order-preserving positive semidefinite interpolation paths, and a variance inequality between the overlap and the free energy that is based on continuous-time I-MMSE relations.

Duke Scholars

Published In

IEEE Journal on Selected Areas in Information Theory

DOI

EISSN

2641-8770

Publication Date

November 1, 2020

Volume

1

Issue

3

Start / End Page

777 / 798
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Reeves, G. (2020). Information-theoretic limits for the matrix tensor product. IEEE Journal on Selected Areas in Information Theory, 1(3), 777–798. https://doi.org/10.1109/JSAIT.2020.3040598
Reeves, G. “Information-theoretic limits for the matrix tensor product.” IEEE Journal on Selected Areas in Information Theory 1, no. 3 (November 1, 2020): 777–98. https://doi.org/10.1109/JSAIT.2020.3040598.
Reeves G. Information-theoretic limits for the matrix tensor product. IEEE Journal on Selected Areas in Information Theory. 2020 Nov 1;1(3):777–98.
Reeves, G. “Information-theoretic limits for the matrix tensor product.” IEEE Journal on Selected Areas in Information Theory, vol. 1, no. 3, Nov. 2020, pp. 777–98. Scopus, doi:10.1109/JSAIT.2020.3040598.
Reeves G. Information-theoretic limits for the matrix tensor product. IEEE Journal on Selected Areas in Information Theory. 2020 Nov 1;1(3):777–798.

Published In

IEEE Journal on Selected Areas in Information Theory

DOI

EISSN

2641-8770

Publication Date

November 1, 2020

Volume

1

Issue

3

Start / End Page

777 / 798