NeuroSpex+: Dual-Task Training of Neuro-Guided Speaker Extraction with Speech Envelope and Waveform

Dashanka De Silva; Siqi Cai; Saurav Pahuja; Tanja Schultz; Haizhou Li

doi:10.21437/Interspeech.2025-1304

NeuroSpex+: Dual-Task Training of Neuro-Guided Speaker Extraction with Speech Envelope and Waveform

Dashanka De Silva
, Siqi Cai^*
, Saurav Pahuja
, Tanja Schultz
, Haizhou Li

^*Corresponding author for this work

University of Bremen
National University of Singapore
The Chinese University of Hong Kong, Shenzhen

Research output: Contribution to journal › Conference article › peer-review

Abstract

Neuro-guided speaker extraction, i.e. NeuroSpex, aims to isolate the speech signal a listener is attending to in a multitalker environment using reference cues derived from cortical activity, such as electroencephalography (EEG). Despite remarkable progress, there remains untapped potential. In this study, we propose NeuroSpex+, a novel neuro-guided speaker extraction model that integrates an additional task of reconstructing the target speech envelope. By simultaneously optimizing the model for both the target speech envelope and speech waveform, NeuroSpex+ reinforces the mask generation for speaker extraction. Experimental results demonstrate that the proposed model significantly outperforms baselines, improving overall signal quality.

Original language	English
Pages (from-to)	5568-5572
Number of pages	5
Journal	Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH
DOIs	https://doi.org/10.21437/Interspeech.2025-1304
State	Published - 2025
Externally published	Yes
Event	26th Interspeech Conference 2025 - Rotterdam, Netherlands Duration: 17 Aug 2025 → 21 Aug 2025

Keywords

EEG
Speaker extraction
selective auditory attention
speech envelope

Access to Document

10.21437/Interspeech.2025-1304

Cite this

@article{6bfb20f4a4504c7ebecdfa6dfe87d5bc,

title = "NeuroSpex+: Dual-Task Training of Neuro-Guided Speaker Extraction with Speech Envelope and Waveform",

abstract = "Neuro-guided speaker extraction, i.e. NeuroSpex, aims to isolate the speech signal a listener is attending to in a multitalker environment using reference cues derived from cortical activity, such as electroencephalography (EEG). Despite remarkable progress, there remains untapped potential. In this study, we propose NeuroSpex+, a novel neuro-guided speaker extraction model that integrates an additional task of reconstructing the target speech envelope. By simultaneously optimizing the model for both the target speech envelope and speech waveform, NeuroSpex+ reinforces the mask generation for speaker extraction. Experimental results demonstrate that the proposed model significantly outperforms baselines, improving overall signal quality.",

keywords = "EEG, Speaker extraction, selective auditory attention, speech envelope",

author = "\{De Silva\}, Dashanka and Siqi Cai and Saurav Pahuja and Tanja Schultz and Haizhou Li",

note = "Publisher Copyright: {\textcopyright} 2025 International Speech Communication Association. All rights reserved.; 26th Interspeech Conference 2025 ; Conference date: 17-08-2025 Through 21-08-2025",

year = "2025",

doi = "10.21437/Interspeech.2025-1304",

language = "英语",

pages = "5568--5572",

journal = "Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH",

issn = "2308-457X",

publisher = "International Speech Communication Association",

}

TY - JOUR

T1 - NeuroSpex+

T2 - 26th Interspeech Conference 2025

AU - De Silva, Dashanka

AU - Cai, Siqi

AU - Pahuja, Saurav

AU - Schultz, Tanja

AU - Li, Haizhou

PY - 2025

Y1 - 2025

N2 - Neuro-guided speaker extraction, i.e. NeuroSpex, aims to isolate the speech signal a listener is attending to in a multitalker environment using reference cues derived from cortical activity, such as electroencephalography (EEG). Despite remarkable progress, there remains untapped potential. In this study, we propose NeuroSpex+, a novel neuro-guided speaker extraction model that integrates an additional task of reconstructing the target speech envelope. By simultaneously optimizing the model for both the target speech envelope and speech waveform, NeuroSpex+ reinforces the mask generation for speaker extraction. Experimental results demonstrate that the proposed model significantly outperforms baselines, improving overall signal quality.

AB - Neuro-guided speaker extraction, i.e. NeuroSpex, aims to isolate the speech signal a listener is attending to in a multitalker environment using reference cues derived from cortical activity, such as electroencephalography (EEG). Despite remarkable progress, there remains untapped potential. In this study, we propose NeuroSpex+, a novel neuro-guided speaker extraction model that integrates an additional task of reconstructing the target speech envelope. By simultaneously optimizing the model for both the target speech envelope and speech waveform, NeuroSpex+ reinforces the mask generation for speaker extraction. Experimental results demonstrate that the proposed model significantly outperforms baselines, improving overall signal quality.

KW - EEG

KW - Speaker extraction

KW - selective auditory attention

KW - speech envelope

UR - https://www.scopus.com/pages/publications/105020058101

U2 - 10.21437/Interspeech.2025-1304

DO - 10.21437/Interspeech.2025-1304

M3 - 会议文章

AN - SCOPUS:105020058101

SN - 2308-457X

SP - 5568

EP - 5572

JO - Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

JF - Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

Y2 - 17 August 2025 through 21 August 2025

ER -

NeuroSpex+: Dual-Task Training of Neuro-Guided Speaker Extraction with Speech Envelope and Waveform

Abstract

Keywords

Access to Document

Other files and links

Fingerprint

Cite this