Vocals-Bass Source Separation Results

In the evaluation of our approaches to VBSS, we compared six approaches to VBSS. Our first approach, dubbed “Smoothing Filters”, uses smoothing filters to create a binary mask. Our second approach, dubbed “Vocals Harmonics 1”, uses Gaussian envelopes to create a soft mask based on the harmonics of the vocals stem. Our third approach, dubbed “Vocals Harmonics 2”, combines Gaussian envelopes from “Vocals Harmonics 1” with a static cutoff frequency. Our fourth approach, dubbed “2D FFT Peak Picking”, uses the 2D FFT to create a soft mask based on the temporal periodicity of the bass stem. Our fifth approach, dubbed “Energy Thresholding”, applies a lowpass filter whose dynamic cutoff frequency is determined by the energy distribution of the spectrogram. Our sixth approach, dubbed “Ideal Soft Mask”, uses ideal soft masks, which are unattainable in practice but create an upper bound on the performance that we might expect from our other approaches. The SDR, SIR, and SAR values achieved by each of these approaches are summarized in the table below. Notable results are given further discussion following the table.

​Our smoothing filters approach to VBSS is generally our worst-performing approach. Due to its lack of sophistication, the smoothing filters approach has the benefit of executing quickly but offers little else. For this reason, the smoothing filters approach serves as an excellent baseline against which we can compare our other approaches. We believe that the smoothing filters approach performs poorly because it is based on a shaky assumption. Namely, this approach is based on the assumption that the vocals tend to exhibit vibrato (i.e., the vocals tend to waver over time) while the bass does not. However, the vocals do not exhibit vibrato all the time and may actually appear quite flat in the spectrogram, such as when a singer sustains a note for an extended period of time without wavering. The shortcomings of the smoothing filters approach are most apparent in the song “Motor Tapes - Shore”, where the vocals exhibit very little vibrato. As a result, many harmonics corresponding to the vocals stem appear in the recovered bass stem, which leads to poor SDR, SIR, and SAR values. Across all songs, the smoothing filters approach tends to produce the worst SIR values for the vocals stem and the worst SAR values for the bass stem relative to our other approaches.

Our vocals harmonics approaches to VBSS generally perform well. We believe that these approaches perform well because they are based on a well-founded assumption. Namely, these approaches are based on the assumption that the vocals are strongly harmonic while the bass is weakly harmonic, which is almost always true. Our “Vocals Harmonics 2” approach performs better across the board relative to our “Vocals Harmonics 1” approach with few exceptions. We attribute the success of “Vocals Harmonics 2” to how this approach handles inharmonicity in the vocals. Due to the inharmonicity of the vocals, the soft mask in the “Vocals Harmonics 1” approach becomes misaligned with the actual harmonics corresponding to the vocals stem at higher frequencies. As a result, the “Vocals Harmonics 1” approach struggles to include these high-frequency harmonics in the recovered vocals stem. On the other hand, “Vocals Harmonics 2” includes all time-frequency bins above a certain cutoff frequency in the recovered vocals stem. This approach guarantees that high-frequency harmonics corresponding to the vocals stem are included in the recovered vocals stem. The recovered bass stem is largely unaffected because most of the bass stem lies below the cutoff frequency. The inclusion of these high-frequency harmonics corresponding to the vocals stem makes a difference; the performance gap between “Vocals Harmonics 2” and “Vocals Harmonics 1” is most pronounced in the SDR values for the vocals stem.

The biggest performance gap between “Vocals Harmonics 2” and “Vocals Harmonics 1” occurs for the song “Carlos Gonzalez - A Place For Us”. The vocals for this song are challenging to recover because two voices are singing in unison, which creates two sets of harmonics corresponding to the vocals stem. “Vocals Harmonics 1” is only designed to detect and extract a single set of harmonics, so this approach struggles and produces a low SDR value for the vocals stem. On the other hand, “Vocals Harmonics 2” includes all time-frequency bins above a certain static cutoff frequency in the recovered vocals stem. It just so happens that this cutoff frequency is located very fortunately. The second set of harmonics only becomes especially prevalent above this cutoff frequency, and most of the bass stem is localized below the cutoff frequency. In large part due to the favorable location of the cutoff frequency, “Vocals Harmonics 2” actually ends up being the best approach for the song “Carlos Gonzalez - A Place For Us” by a significant margin. Overall, we consider “Vocals Harmonics 2” to be a contender for our best approach alongside “Energy Thresholding”.

​“2D FFT Peak Picking” performs reasonably well but is overshadowed by our other approaches. “2D FFT Peak Picking” rivals “Vocals Harmonics 1” in performance, typically achieving slightly higher SDR values for the vocals stem but producing slightly lower SDR values for the bass stem. However, “Vocals Harmonics 2” achieves higher SDR values for both the vocals stem and bass stem across all songs relative to “2D FFT Peak Picking”. The best results for “2D FFT Peak Picking” come from the song “Music Delta - Hendrix”, which contains a short and simple bass riff that repeats multiple times while the vocals do not repeat. In other words, “2D FFT Peak Picking” works best when the bass is largely periodic while the vocals are largely aperiodic. Given that the “2D FFT Peak Picking” approach is designed to exploit the strong periodicity of the bass stem relative to the weak periodicity of the vocals stem, we do not find these results surprising.