Skip to main content

Image Separation With Side Information: A Connected Auto-Encoders Based Approach.

Publication ,  Journal Article
Pu, W; Sober, B; Daly, N; Zhou, C; Sabetsarvestani, Z; Higgitt, C; Daubechies, I; Rodrigues, MRD
Published in: IEEE transactions on image processing : a publication of the IEEE Signal Processing Society
January 2023

X-radiography (X-ray imaging) is a widely used imaging technique in art investigation. It can provide information about the condition of a painting as well as insights into an artist's techniques and working methods, often revealing hidden information invisible to the naked eye. X-radiograpy of double-sided paintings results in a mixed X-ray image and this paper deals with the problem of separating this mixed image. Using the visible color images (RGB images) from each side of the painting, we propose a new Neural Network architecture, based upon 'connected' auto-encoders, designed to separate the mixed X-ray image into two simulated X-ray images corresponding to each side. This connected auto-encoders architecture is such that the encoders are based on convolutional learned iterative shrinkage thresholding algorithms (CLISTA) designed using algorithm unrolling techniques, whereas the decoders consist of simple linear convolutional layers; the encoders extract sparse codes from the visible image of the front and rear paintings and mixed X-ray image, whereas the decoders reproduce both the original RGB images and the mixed X-ray image. The learning algorithm operates in a totally self-supervised fashion without requiring a sample set that contains both the mixed X-ray images and the separated ones. The methodology was tested on images from the double-sided wing panels of the Ghent Altarpiece, painted in 1432 by the brothers Hubert and Jan van Eyck. These tests show that the proposed approach outperforms other state-of-the-art X-ray image separation methods for art investigation applications.

Duke Scholars

Published In

IEEE transactions on image processing : a publication of the IEEE Signal Processing Society

DOI

EISSN

1941-0042

ISSN

1057-7149

Publication Date

January 2023

Volume

32

Start / End Page

2931 / 2946

Related Subject Headings

  • Artificial Intelligence & Image Processing
  • 4607 Graphics, augmented reality and games
  • 4603 Computer vision and multimedia computation
  • 1702 Cognitive Sciences
  • 0906 Electrical and Electronic Engineering
  • 0801 Artificial Intelligence and Image Processing
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Pu, W., Sober, B., Daly, N., Zhou, C., Sabetsarvestani, Z., Higgitt, C., … Rodrigues, M. R. D. (2023). Image Separation With Side Information: A Connected Auto-Encoders Based Approach. IEEE Transactions on Image Processing : A Publication of the IEEE Signal Processing Society, 32, 2931–2946. https://doi.org/10.1109/tip.2023.3275872
Pu, Wei, Barak Sober, Nathan Daly, Chao Zhou, Zahra Sabetsarvestani, Catherine Higgitt, Ingrid Daubechies, and Miguel R. D. Rodrigues. “Image Separation With Side Information: A Connected Auto-Encoders Based Approach.IEEE Transactions on Image Processing : A Publication of the IEEE Signal Processing Society 32 (January 2023): 2931–46. https://doi.org/10.1109/tip.2023.3275872.
Pu W, Sober B, Daly N, Zhou C, Sabetsarvestani Z, Higgitt C, et al. Image Separation With Side Information: A Connected Auto-Encoders Based Approach. IEEE transactions on image processing : a publication of the IEEE Signal Processing Society. 2023 Jan;32:2931–46.
Pu, Wei, et al. “Image Separation With Side Information: A Connected Auto-Encoders Based Approach.IEEE Transactions on Image Processing : A Publication of the IEEE Signal Processing Society, vol. 32, Jan. 2023, pp. 2931–46. Epmc, doi:10.1109/tip.2023.3275872.
Pu W, Sober B, Daly N, Zhou C, Sabetsarvestani Z, Higgitt C, Daubechies I, Rodrigues MRD. Image Separation With Side Information: A Connected Auto-Encoders Based Approach. IEEE transactions on image processing : a publication of the IEEE Signal Processing Society. 2023 Jan;32:2931–2946.

Published In

IEEE transactions on image processing : a publication of the IEEE Signal Processing Society

DOI

EISSN

1941-0042

ISSN

1057-7149

Publication Date

January 2023

Volume

32

Start / End Page

2931 / 2946

Related Subject Headings

  • Artificial Intelligence & Image Processing
  • 4607 Graphics, augmented reality and games
  • 4603 Computer vision and multimedia computation
  • 1702 Cognitive Sciences
  • 0906 Electrical and Electronic Engineering
  • 0801 Artificial Intelligence and Image Processing