SPEECH SEPARATION BY MODIFIED DEEP NON-LINEAR FILTERING MODEL

Andrii Tsemko, Ivan Karbovnyk

Abstract


Background. Automatic Speech Recognition (ASR) systems in single-channel experimental setups perform poorly in real-world scenarios when several talkers are placed near a microphone. While for multi-channel setups, speech (or source) separation can be solved by algorithmic approaches such as beamforming or machine learning approaches using deep non-linear filtering models, there is no such monaural algorithm that can efficiently separate speech.

Materials and Methods. We focus on a successive model designed for multi-channel speech extraction with a steering mechanism called deep non-linear filtering and modify it for single-channel speech separation. This LSTM-based (LSTM - Long short-term memory) model without a steering mechanism demonstrates high-quality speech extraction from a fixed angle, which makes the model promising for single-channel separation. We modified such a model by appending a parallel output head that generates a second gain mask in addition to the existing one. We train the model on the Libri2Mix dataset in noisy conditions to separate speeches and extract them without background noise. Additionally, we investigate the influence of bidirectional LSTM layers versus the unidirectional version. As the DNLF (Deep non-linear filtering) model is built with bidirectional LSTM layers, it cannot be used for real-time separation processing.

Results and Discussion. Our trained DNLF-SS-UNI (Deep non-linear filtering speech separation unidirectional) model with unidirectional LSTM layers demonstrates 3.41 dB of SI-SDR (Scale-invariant signal-to-distortion ratio) and 5.34 dB of SI-SDRi (Scale-invariant signal-to-distortion ratio improvement) on test subset of Libri2Mix dataset with background noise, while its bidirectional version called DNLF-SS-BI (Deep non-linear filtering speech separation bidirectional) achieves 5.82 dB and 7.76 dB for SI-SDR and SI-SDRi, respectively. However, the bidirectional version of the model requires approximately 2.3 times more parameters than its unidirectional counterpart.

Conclusion. The use of bidirectional LSTM layers in speech separation models of architectures such as deep non-linear filtering is crucial, as a model with unidirectional LSTM layers instead shows a performance drop of approximately 2.4 dB for SI-SDR and SI-SDRi metrics.

Keywords: recurrent neural networks, speech separation, machine learning


Full Text:

PDF

References


[1] Tesch, K., & Gerkmann, T. (2023). Insights into Deep Non-linear Filters for Improved Multi-channel Speech Enhancement. IEEE/ACM Transactions of Audio, Speech and Language Processing, vol 31., 563-575. https://doi.org/10.1109/TASLP.2022.3221046

[2] Luo, Y., & Mesgarani, N. (2019). Conv-TasNet: Surpassing ideal time-frequency magnitude masking for speech separation. IEEE/ACM Trans. Audio, Speech, Language Process, 27(8), 1256–1266. https://doi.org/10.1109/TASLP.2019.2915167.

[3] Lea, C., Vidal, R., Reiter, A., & Hager, G. D. (2016). Temporal Convolutional Networks: A Unified Approach to Action Segmentation. https://doi.org/10.48550/arXiv.1608.08242

[4] Efthymios., T., Zhepei., W., Paris., S. (2020). Sudo rm -rf: Efficient Networks for Universal Audio Source Separation. IEEE 30th International Workshop on Machine Learning for Signal Processing (MLSP). https://doi.org/10.1109/MLSP49062.2020.9231900

[5] Tsemko, A., Karbovnyk, I. (2025). Edge-ready speech separation with Sudo-TasNet. Advances in cyber-physical systems, Vol. 10, No. 2. https://doi.org/10.23939/acps2025.02.202

[6] Chenda, L., Zhuo, C., Yi, C., Cong, H., Tianya, Z., Keisuke, K., Marc, D., Shinji, W., Yanmin, Q. (2021). Dual-Path Modeling for Long Recording Speech Separation in Meetings. IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 30, 1508-1520. https://doi.org/10.1109/TASLP.2022.3165712

[7] Tsemko, A., Karbovnyk, I. (2024). Models and Methods for speech separation in digital systems. Advances in cyber-physical systems, Vol. 9, No. 2. https://doi.org/10.23939/acps2024.02.121

[8] Cosentino, J., Pariente, M., Cornell, S., Deleforge, A., & Vincent, E. (2020). LibriMix: An open-source dataset for generalizable speech separation. ArXiv preprint. arXiv:2005.11262. https://doi.org/10.48550/arXiv.2005.11262.

[9] Le Roux, J., Wisdom, S., Erdoğan, H., & Hershey, J. R. (2018). SDR – half-baked or well done? ArXiv preprint. arXiv:1811.02508. https://doi.org/10.48550/arXiv.1811.02508.

[10] Morten, K., Dong, Y., Zheng-Hua, T., & Jesper, J. (2017). Multitalker Speech Separation With Utterance-Level Permutation Invariant Training of Deep Recurrent Neural Networks. IEEE/ACM Trans. Audio, Speech and Lang. Proc. 25(10), 1901-1913. https://doi.org/10.1109/TASLP.2017.2726762.




DOI: http://dx.doi.org/10.30970/eli.34.14

Refbacks

  • There are currently no refbacks.