Skip to main content

From speaker verification to multispeaker speech synthesis, deep transfer with feedback constraint

Publication ,  Conference
Cai, Z; Zhang, C; Li, M
Published in: Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH
January 1, 2020

High-fidelity speech can be synthesized by end-to-end text-to-speech models in recent years. However, accessing and controlling speech attributes such as speaker identity, prosody, and emotion in a text-to-speech system remains a challenge. This paper presents a system involving feedback constraints for multispeaker speech synthesis. We manage to enhance the knowledge transfer from the speaker verification to the speech synthesis by engaging the speaker verification network. The constraint is taken by an added loss related to the speaker identity, which is centralized to improve the speaker similarity between the synthesized speech and its natural reference audio. The model is trained and evaluated on publicly available datasets. Experimental results, including visualization on speaker embedding space, show significant improvement in terms of speaker identity cloning in the spectrogram level. In addition, synthesized samples are available online for listening.

Duke Scholars

Published In

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

DOI

EISSN

1990-9772

ISSN

2308-457X

Publication Date

January 1, 2020

Volume

2020-October

Start / End Page

3974 / 3978
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Cai, Z., Zhang, C., & Li, M. (2020). From speaker verification to multispeaker speech synthesis, deep transfer with feedback constraint. In Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH (Vol. 2020-October, pp. 3974–3978). https://doi.org/10.21437/Interspeech.2020-1032
Cai, Z., C. Zhang, and M. Li. “From speaker verification to multispeaker speech synthesis, deep transfer with feedback constraint.” In Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, 2020-October:3974–78, 2020. https://doi.org/10.21437/Interspeech.2020-1032.
Cai Z, Zhang C, Li M. From speaker verification to multispeaker speech synthesis, deep transfer with feedback constraint. In: Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH. 2020. p. 3974–8.
Cai, Z., et al. “From speaker verification to multispeaker speech synthesis, deep transfer with feedback constraint.” Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, vol. 2020-October, 2020, pp. 3974–78. Scopus, doi:10.21437/Interspeech.2020-1032.
Cai Z, Zhang C, Li M. From speaker verification to multispeaker speech synthesis, deep transfer with feedback constraint. Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH. 2020. p. 3974–3978.

Published In

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

DOI

EISSN

1990-9772

ISSN

2308-457X

Publication Date

January 1, 2020

Volume

2020-October

Start / End Page

3974 / 3978