Skip to main content

Expression of fractals through neural network functions

Publication ,  Journal Article
Dym, N; Sober, B; Daubechies, I
Published in: IEEE Journal on Selected Areas in Information Theory
May 1, 2020

To help understand the underlying mechanisms of neural networks (NNs), several groups have studied the number of linear regions l of piecewise linear (PwL) functions, generated by deep neural networks (DNN). In particular, they showed that l can grow exponentially with the number of network parameters p, a property often used to explain the advantages of deep over shallow NNs. Nonetheless, a dimension argument shows that DNNs cannot generate all PwL functions with l linear regions when l > p. It is thus natural to seek to characterize specific families of functions with l > p linear regions that can be constructed by DNNs. Iterated Function Systems (IFS) recursively construct a sequence of PwL functions Fk with a number of linear regions which is exponential in k. We show that Fk can be generated by a NN using only O(k) parameters. IFS are used extensively to generate natural-looking landscape textures in artificial images as well as for compression of natural images. The surprisingly good performance of this compression suggests that human visual system may lock in on self-similarities. The combination of this phenomenon with the capacity of DNNs to efficiently approximate IFS may contribute to the success of DNNs, particularly striking for image processing tasks.

Duke Scholars

Published In

IEEE Journal on Selected Areas in Information Theory

DOI

EISSN

2641-8770

Publication Date

May 1, 2020

Volume

1

Issue

1

Start / End Page

57 / 66
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Dym, N., Sober, B., & Daubechies, I. (2020). Expression of fractals through neural network functions. IEEE Journal on Selected Areas in Information Theory, 1(1), 57–66. https://doi.org/10.1109/JSAIT.2020.2991422
Dym, N., B. Sober, and I. Daubechies. “Expression of fractals through neural network functions.” IEEE Journal on Selected Areas in Information Theory 1, no. 1 (May 1, 2020): 57–66. https://doi.org/10.1109/JSAIT.2020.2991422.
Dym N, Sober B, Daubechies I. Expression of fractals through neural network functions. IEEE Journal on Selected Areas in Information Theory. 2020 May 1;1(1):57–66.
Dym, N., et al. “Expression of fractals through neural network functions.” IEEE Journal on Selected Areas in Information Theory, vol. 1, no. 1, May 2020, pp. 57–66. Scopus, doi:10.1109/JSAIT.2020.2991422.
Dym N, Sober B, Daubechies I. Expression of fractals through neural network functions. IEEE Journal on Selected Areas in Information Theory. 2020 May 1;1(1):57–66.

Published In

IEEE Journal on Selected Areas in Information Theory

DOI

EISSN

2641-8770

Publication Date

May 1, 2020

Volume

1

Issue

1

Start / End Page

57 / 66