Skip to main content

Revisiting Graph Contrastive Learning from the Perspective of Graph Spectrum

Publication ,  Conference
Liu, N; Wang, X; Bo, D; Shi, C; Pei, J
Published in: Advances in Neural Information Processing Systems
January 1, 2022

Graph Contrastive Learning (GCL), learning the node representations by augmenting graphs, has attracted considerable attentions. Despite the proliferation of various graph augmentation strategies, some fundamental questions still remain unclear: what information is essentially encoded into the learned representations by GCL? Are there some general graph augmentation rules behind different augmentations? If so, what are they and what insights can they bring? In this paper, we answer these questions by establishing the connection between GCL and graph spectrum. By an experimental investigation in spectral domain, we firstly find the General grAph augMEntation (GAME) rule for GCL, i.e., the difference of the high-frequency parts between two augmented graphs should be larger than that of low-frequency parts. This rule reveals the fundamental principle to revisit the current graph augmentations and design new effective graph augmentations. Then we theoretically prove that GCL is able to learn the invariance information by contrastive invariance theorem, together with our GAME rule, for the first time, we uncover that the learned representations by GCL essentially encode the low-frequency information, which explains why GCL works. Guided by this rule, we propose a spectral graph contrastive learning module (SpCo), which is a general and GCL-friendly plug-in. We combine it with different existing GCL models, and extensive experiments well demonstrate that it can further improve the performances of a wide variety of different GCL methods.

Duke Scholars

Published In

Advances in Neural Information Processing Systems

ISSN

1049-5258

Publication Date

January 1, 2022

Volume

35

Related Subject Headings

  • 4611 Machine learning
  • 1702 Cognitive Sciences
  • 1701 Psychology
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Liu, N., Wang, X., Bo, D., Shi, C., & Pei, J. (2022). Revisiting Graph Contrastive Learning from the Perspective of Graph Spectrum. In Advances in Neural Information Processing Systems (Vol. 35).
Liu, N., X. Wang, D. Bo, C. Shi, and J. Pei. “Revisiting Graph Contrastive Learning from the Perspective of Graph Spectrum.” In Advances in Neural Information Processing Systems, Vol. 35, 2022.
Liu N, Wang X, Bo D, Shi C, Pei J. Revisiting Graph Contrastive Learning from the Perspective of Graph Spectrum. In: Advances in Neural Information Processing Systems. 2022.
Liu, N., et al. “Revisiting Graph Contrastive Learning from the Perspective of Graph Spectrum.” Advances in Neural Information Processing Systems, vol. 35, 2022.
Liu N, Wang X, Bo D, Shi C, Pei J. Revisiting Graph Contrastive Learning from the Perspective of Graph Spectrum. Advances in Neural Information Processing Systems. 2022.

Published In

Advances in Neural Information Processing Systems

ISSN

1049-5258

Publication Date

January 1, 2022

Volume

35

Related Subject Headings

  • 4611 Machine learning
  • 1702 Cognitive Sciences
  • 1701 Psychology