DSP Tools

Throughout this project, we applied several digital signal processing tools from both inside and outside of EECS 351.

​

Frequency-domain analysis

• Using the short-time Fourier transform to obtain the time-frequency representations of songs

• Using the DFT to determine the fundamental frequency of the vocals stem (VBSS: Vocals Harmonics 1/2)

• Using the energy distribution in the frequency domain to determine a cutoff frequency (VBSS: Energy Thresholding)

​

Linear systems

• Using moving median filters is decidedly non-linear (HPSS/VBSS: Smoothing Filters)

• Using a moving median filter to denoise the vocals and bass soft masks is decidedly non-linear (VBSS: Vocals Harmonics 1/2)

• Using a moving average filter to denoise the bass soft mask (VBSS: Vocals Harmonics 1/2)

​

Filtering

• Using envelopes to extract harmonics resembles the behavior of a comb filter (VBSS: Vocals Harmonics 1/2)

• Using a low-pass filter to separate vocals from bass (VBSS: Energy Thresholding)

​

Image processing techniques

• Using the 2D FFT to extract signals that exhibit temporal periodicity (VBSS: 2D FFT Peak Picking)

​

Machine Learning

• Using a fully connected neural network to separate harmonics from percussion (HPSS: Fully Connected Neural Network)

• Using a U-Net architecture to separate harmonics from percussion (HPSS: U-Net Convolutional Neural Network)

• Using a mask inference architecture to separate harmonics from percussion (HPSS: Mask Inference Architecture)