Nonlinear feedforward networks with stochastic outputs: infomax implies redundancy reduction.

Journal Article (Journal Article)

We prove that maximization of mutual information between the output and the input of a feedforward neural network leads to full redundancy reduction under the following sufficient conditions: (i) the input signal is a (possibly nonlinear) invertible mixture of independent components; (ii) there is no input noise; (iii) the activity of each output neuron is a (possibly) stochastic variable with a probability distribution depending on the stimulus through a deterministic function of the inputs (where both the probability distributions and the functions can be different from neuron to neuron); (iv) optimization of the mutual information is performed over all these deterministic functions. This result extends that obtained by Nadal and Parga (1994) who considered the case of deterministic outputs.

Full Text

Duke Authors

Cited Authors

  • Nadal, JP; Brunel, N; Parga, N

Published Date

  • May 1998

Published In

Volume / Issue

  • 9 / 2

Start / End Page

  • 207 - 217

PubMed ID

  • 9861986

International Standard Serial Number (ISSN)

  • 0954-898X


  • eng

Conference Location

  • England