Nonlinear feedforward networks with stochastic outputs: infomax implies redundancy reduction.
Journal Article (Journal Article)
We prove that maximization of mutual information between the output and the input of a feedforward neural network leads to full redundancy reduction under the following sufficient conditions: (i) the input signal is a (possibly nonlinear) invertible mixture of independent components; (ii) there is no input noise; (iii) the activity of each output neuron is a (possibly) stochastic variable with a probability distribution depending on the stimulus through a deterministic function of the inputs (where both the probability distributions and the functions can be different from neuron to neuron); (iv) optimization of the mutual information is performed over all these deterministic functions. This result extends that obtained by Nadal and Parga (1994) who considered the case of deterministic outputs.
Full Text
Duke Authors
Cited Authors
- Nadal, JP; Brunel, N; Parga, N
Published Date
- May 1998
Published In
Volume / Issue
- 9 / 2
Start / End Page
- 207 - 217
PubMed ID
- 9861986
International Standard Serial Number (ISSN)
- 0954-898X
Language
- eng
Conference Location
- England