In a nutshell

We are releasing Spleeter to help the research community in Music Information Retrieval (MIR) leverage the power of a state-of-the-art source separation algorithm. It comes in the form of a Python Library based on Tensorflow, with pretrained models for 2, 4 and 5 stems separation. Spleeter will be presented and live-demoed at the 2019 ISMIR conference in Delft.

A brief overview of source separation

While not a broadly known topic, the problem of source separation has interested a large community of music signal researchers for a couple of decades now. It starts from a simple observation: music recordings are usually a mix of several individual instrument tracks (lead vocal, drums, bass, piano etc..). The task of music source separation is: given a mix can we recover these separate tracks (sometimes called stems)? This has many potential applications: think remixes, upmixing, active listening, educational purposes, but also pre-processing for other tasks such as transcription.

From a Mix of many instruments, a source separation engine like Spleeter outputs a set of individual tracks or stems.

Interestingly, our brain is very good at isolating instruments. Just focus on one of the instrument of this track (say the lead vocal for instance) and you will be able to hear it quite distinctively from the others. Yet that’s not really separation, you still hear all the other parts. In many cases, it may not be possible to exactly recover the individual tracks that have been mixed together. The challenge is thus to approximate them the best we can, that is to say as close as possible to the originals without creating too much distortions.

For years, a lot of strategies have been explored, by dozens of brilliant research teams from all over the world. If you’re interested in this fascinating journey you should go read this literature overview, or this one. The pace of progress has recently made some giant leaps, mainly due to advances in machine learning methods. To keep track, people have been comparing their algorithm in international evaluation campaigns. That’s how we know that Spleeter performances match those of the best proposed algorithms.

Additionally, Spleeter is very fast. If you are running the GPU version you can expect separating 100x faster than real-time which makes it a good option to process large datasets.

What can I do with Spleeter ?

Quite a lot I’d say. If you’re a researcher working on Music Information Retrieval and have always considered that source separation artifacts made it unsuitable as a pre-processing step in your pipeline... Well, you should probably reconsider and try Spleeter. If you are a music hacker and want to build something awesome using Spleeter, then go ahead. Actually Spleeter is MIT-Licensed so you are really free to use it in any way you want. It goes without saying that if you plan to use Spleeter on copyrighted songs, make sure you get proper authorization from right owners beforehand.

How can I use Spleeter ?

Under the hood, Spleeter is a fairly complex and crafted engine but we’ve worked hard to make it really easy to use. The actual separation can be achieved with a single command line, and it should work on your laptop regardless of your Operating System. For more advanced users, there is a python API class called Separator that you can manipulate directly into your usual pipeline.

We’ve tried hard to come up with a thorough documentation. Don’t hesitate to give us feedback, point out issues or suggest improvement through the traditional github tools!