Skip to main content
Journal cover image

Manifold adversarial training for supervised and semi-supervised learning.

Publication ,  Journal Article
Zhang, S; Huang, K; Zhu, J; Liu, Y
Published in: Neural networks : the official journal of the International Neural Network Society
August 2021

We propose a new regularization method for deep learning based on the manifold adversarial training (MAT). Unlike previous regularization and adversarial training methods, MAT further considers the local manifold of latent representations. Specifically, MAT manages to build an adversarial framework based on how the worst perturbation could affect the statistical manifold in the latent space rather than the output space. Particularly, a latent feature space with the Gaussian Mixture Model (GMM) is first derived in a deep neural network. We then define the smoothness by the largest variation of Gaussian mixtures when a local perturbation is given around the input data point. On one hand, the perturbations are added in the way that would rough the statistical manifold of the latent space the worst. On the other hand, the model is trained to promote the manifold smoothness the most in the latent space. Importantly, since the latent space is more informative than the output space, the proposed MAT can learn a more robust and compact data representation, leading to further performance improvement. The proposed MAT is important in that it can be considered as a superset of one recently-proposed discriminative feature learning approach called center loss. We conduct a series of experiments in both supervised and semi-supervised learning on four benchmark data sets, showing that the proposed MAT can achieve remarkable performance, much better than those of the state-of-the-art approaches. In addition, we present a series of visualization which could generate further understanding or explanation on adversarial examples.

Duke Scholars

Published In

Neural networks : the official journal of the International Neural Network Society

DOI

EISSN

1879-2782

ISSN

0893-6080

Publication Date

August 2021

Volume

140

Start / End Page

282 / 293

Related Subject Headings

  • Supervised Machine Learning
  • Benchmarking
  • Artificial Intelligence & Image Processing
  • 4905 Statistics
  • 4611 Machine learning
  • 4602 Artificial intelligence
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Zhang, S., Huang, K., Zhu, J., & Liu, Y. (2021). Manifold adversarial training for supervised and semi-supervised learning. Neural Networks : The Official Journal of the International Neural Network Society, 140, 282–293. https://doi.org/10.1016/j.neunet.2021.03.031
Zhang, Shufei, Kaizhu Huang, Jianke Zhu, and Yang Liu. “Manifold adversarial training for supervised and semi-supervised learning.Neural Networks : The Official Journal of the International Neural Network Society 140 (August 2021): 282–93. https://doi.org/10.1016/j.neunet.2021.03.031.
Zhang S, Huang K, Zhu J, Liu Y. Manifold adversarial training for supervised and semi-supervised learning. Neural networks : the official journal of the International Neural Network Society. 2021 Aug;140:282–93.
Zhang, Shufei, et al. “Manifold adversarial training for supervised and semi-supervised learning.Neural Networks : The Official Journal of the International Neural Network Society, vol. 140, Aug. 2021, pp. 282–93. Epmc, doi:10.1016/j.neunet.2021.03.031.
Zhang S, Huang K, Zhu J, Liu Y. Manifold adversarial training for supervised and semi-supervised learning. Neural networks : the official journal of the International Neural Network Society. 2021 Aug;140:282–293.
Journal cover image

Published In

Neural networks : the official journal of the International Neural Network Society

DOI

EISSN

1879-2782

ISSN

0893-6080

Publication Date

August 2021

Volume

140

Start / End Page

282 / 293

Related Subject Headings

  • Supervised Machine Learning
  • Benchmarking
  • Artificial Intelligence & Image Processing
  • 4905 Statistics
  • 4611 Machine learning
  • 4602 Artificial intelligence