Deep attention gated dilated temporal convolutional networks with intra-parallel convolutional modules for end-to-end monaural speech separation

Monaural speech separation techniques are far from satisfactory and are a challenging task due to interference from multiple sources. Recently the deep dilated temporal convolutional networks (TCN) has proven to be very effective in sequence modeling. This work explores how to extend TCN to result a new, state-of-the-art monaural speech separation method. First, a new gating mechanism is introduced and added to generate a gated TCN. The gated activation controls the flow of information. Further in order to combine multiple training models to reduce the performance variance and improve the effect of speech separation, we propose to use the principle of ensemble learning in the gated TCN architecture by replacing the convolutional modules corresponding to each dilated factor with multiple identical branches of the convolutional components. For the sake of objectivity, we propose to train the network by directly optimizing in a permutation invariant training (PIT) style of the utterance level signal-to-distortion ratio (SDR). Our experiments with the public WSJ0-2mix data corpus resulted in an 18.2 dB improvement in SDR, indicating that our proposed network can improve the performance of speaker separation tasks.