Abstract
Soundprism, as proposed in this paper, is a computer system that separates single-channel polyphonic music audio played by harmonic sources into source signals in an online fashion. It uses a musical score to guide the separation process. To the best of our knowledge, this is the first online system that addresses score-informed music source separation that can be made into a real-time system. The proposed system consists of two parts: 1) a score follower that associates a score position to each time frame of the audio performance; 2) a source separator which reconstructs the source signals for each time frame, informed by the score. The score follower uses a hidden Markov approach, where each audio frame is associated with a 2-D state vector (score position and tempo). The observation model is defined as the likelihood of observing the frame given the pitches at the score position. The score position and tempo are inferred using particle filtering. In building the source separator, we first refine the score-informed pitches of the current audio frame by maximizing the multi-pitch observation likelihood. Then, the harmonics of each source's fundamental frequency are extracted to reconstruct the source signal. Overlapping harmonics between sources are identified and their energy is distributed in inverse proportion to the square of their respective harmonic number. Experiments on both synthetic and human-performed music show both the score follower and the source separator perform well. Results also show that the proposed score follower works well for highly polyphonic music with some degree of tempo variations.
Original language | English (US) |
---|---|
Article number | 5887382 |
Pages (from-to) | 1205-1215 |
Number of pages | 11 |
Journal | IEEE Journal on Selected Topics in Signal Processing |
Volume | 5 |
Issue number | 6 |
DOIs | |
State | Published - Oct 2011 |
Funding
Manuscript received September 30, 2010; revised February 22, 2011 and May 20, 2011; accepted June 05, 2011. Date of publication June 16, 2011; date of current version September 16, 2011. This work was supported by the National Science Foundation under Awards 0643752 and 0812314. The associate editor coordinating the review of this manuscript and approving it for publication was Prof. Gaël Richard.
Keywords
- Multi-pitch estimation
- Online algorithm
- Score following
- Source separation
ASJC Scopus subject areas
- Signal Processing
- Electrical and Electronic Engineering