Learned complex masks for multi-instrument source separation

• DOI: 10.48550/arxiv.2103.12864
• Publication Date: 2021-01-01
• Authors: Jansson, Andreas; Bittner, Rachel M.; Montecchio, Nicola; Weyde, Tillman
• Keywords: FOS: Computer and information sciences; Sound (cs.SD); Audio and Speech Processing (eess.AS); 0202 electrical engineering, electronic engineering, information engineering; FOS: Electrical engineering, electronic engineering, information engineering; 02 engineering and technology; Computer Science - Sound; Electrical Engineering and Systems Science - Audio and Speech Processing

Music source separation in the time-frequency domain is commonly achieved by applying a soft or binary mask to the magnitude component of (complex) spectrograms. The phase component is usually not estimated, but instead copied from the mixture and applied to the magnitudes of the estimated isolated sources. While this method has several practical advantages, it imposes an upper bound on the performance of the system, where the estimated isolated sources inherently exhibit audible "phase artifacts". In this paper we address these shortcomings by directly estimating masks in the complex domain, extending recent work from the speech enhancement literature. The method is particularly well suited for multi-instrument musical source separation since residual phase artifacts are more pronounced for spectrally overlapping instrument sources, a common scenario in music. We show that complex masks result in better separation than masks that operate solely on the magnitude component.