Skip to main content

High-resolution single-photon imaging with physics-informed deep learning.

Publication ,  Journal Article
Bian, L; Song, H; Peng, L; Chang, X; Yang, X; Horstmeyer, R; Ye, L; Zhu, C; Qin, T; Zheng, D; Zhang, J
Published in: Nature communications
September 2023

High-resolution single-photon imaging remains a big challenge due to the complex hardware manufacturing craft and noise disturbances. Here, we introduce deep learning into SPAD, enabling super-resolution single-photon imaging with enhancement of bit depth and imaging quality. We first studied the complex photon flow model of SPAD electronics to accurately characterize multiple physical noise sources, and collected a real SPAD image dataset (64 × 32 pixels, 90 scenes, 10 different bit depths, 3 different illumination flux, 2790 images in total) to calibrate noise model parameters. With this physical noise model, we synthesized a large-scale realistic single-photon image dataset (image pairs of 5 different resolutions with maximum megapixels, 17250 scenes, 10 different bit depths, 3 different illumination flux, 2.6 million images in total) for subsequent network training. To tackle the severe super-resolution challenge of SPAD inputs with low bit depth, low resolution, and heavy noise, we further built a deep transformer network with a content-adaptive self-attention mechanism and gated fusion modules, which can dig global contextual features to remove multi-source noise and extract full-frequency details. We applied the technique in a series of experiments including microfluidic inspection, Fourier ptychography, and high-speed imaging. The experiments validate the technique's state-of-the-art super-resolution SPAD imaging performance.

Duke Scholars

Altmetric Attention Stats
Dimensions Citation Stats

Published In

Nature communications

DOI

EISSN

2041-1723

ISSN

2041-1723

Publication Date

September 2023

Volume

14

Issue

1

Start / End Page

5902
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Bian, L., Song, H., Peng, L., Chang, X., Yang, X., Horstmeyer, R., … Zhang, J. (2023). High-resolution single-photon imaging with physics-informed deep learning. Nature Communications, 14(1), 5902. https://doi.org/10.1038/s41467-023-41597-9
Bian, Liheng, Haoze Song, Lintao Peng, Xuyang Chang, Xi Yang, Roarke Horstmeyer, Lin Ye, et al. “High-resolution single-photon imaging with physics-informed deep learning.Nature Communications 14, no. 1 (September 2023): 5902. https://doi.org/10.1038/s41467-023-41597-9.
Bian L, Song H, Peng L, Chang X, Yang X, Horstmeyer R, et al. High-resolution single-photon imaging with physics-informed deep learning. Nature communications. 2023 Sep;14(1):5902.
Bian, Liheng, et al. “High-resolution single-photon imaging with physics-informed deep learning.Nature Communications, vol. 14, no. 1, Sept. 2023, p. 5902. Epmc, doi:10.1038/s41467-023-41597-9.
Bian L, Song H, Peng L, Chang X, Yang X, Horstmeyer R, Ye L, Zhu C, Qin T, Zheng D, Zhang J. High-resolution single-photon imaging with physics-informed deep learning. Nature communications. 2023 Sep;14(1):5902.

Published In

Nature communications

DOI

EISSN

2041-1723

ISSN

2041-1723

Publication Date

September 2023

Volume

14

Issue

1

Start / End Page

5902