When Microsoft Songsmith promised to provide a custom backing track to anything that you sang, the Series of Tubes known as the Internet quickly used the tool to record versions of popular songs. They were laughably bad, filled with cheesy beats and backing synths plucking out "oom-pa-pa" rhythms.

As a research problem, Songsmith was a success; as a creative tool, it was (let's just admit it) a big fat failure.

But Parag Chordia, the director of the Music Intelligence Lab at Georgia Tech, sees promise in such tools, enough so that he turned his own research into an iPhone app called LaDiDa that takes what Songsmith did and makes it mobile... and cool.

The education of a digital sideman



"LaDiDa in a way encapsulates all my interests," Chordia tells Ars. His research projects to date have focused on machine listening and computational creativity, but Chordia also plays Indian classical music and is committed to making music more participatory.

Chordia and some of his fellow professors have spun off a company from their Georgia Tech research to commercialize tools that make it easy even for non-musicians to create tunes. The success of games like Rock Band and Guitar Hero were the first wave; they tapped into the primal rush that comes from creating music, but they didn't allow actual creation.

LaDiDa tries to enable this by providing sophisticated backing tracks in various styles. Sing into the iPhone over a metronomic beat, and the software figures out what key you're in, what mode is being used (major, minor, mixolydian, etc.), whether song sections are being repeated, and what chord progressions might work well underneath the melody line.

Rather than programming this knowledge into the software, Chordia and his team instead fed it huge amounts of data. Their machine learning program was given a huge song database in MIDI format and set to work on analyzing the data to learn just how songs are built. "We don't give it any rules," says Chordia. Instead, the goal was to find out whether "we can train a computer to learn by example."

The software developed a statistical model of pop music—what chord progressions match up with which melody notes, what arpeggios and instruments to use, how songs are structured. The engine was now ready to be "run forward," to generate music out the individual backing tracks out of musical samples created by a composer on Chordia's team.

To create the music, the software first turns someone's singing from pressure waves into pitches. The result is a data set of pitch match against time, which doesn't tell the computer much until several key analyses are performed. What key is the song in? (Humans may sing in all sorts of non-chromatic keys, starting a quarter tone up from Middle C, for instance.) What mode is the song in? (Major, minor, etc.) Are there any repeating structural elements? (Verse, chorus.)

The software then creates a backing track in one of several predefined styles. An internal sampler/sequencer creates the music, altering pitch and rhythm to match the proper tempo and key signature. It then applies an optional autotune effect to the original singer and a bit of reverb; the whole piece is then played back and can be e-mailed or posted to Twitter or Facebook.

Comparing the product against Microsoft's research project, Chordia says that LaDiDa just sounds better. "To do this kind of thing successfully, it's not just a technology play; it requires musical aesthetics," he says. In addition, modeling the songs has advanced significantly, and "our models are better." If the end result doesn't sound fantastic, people won't use the product; they might even take to mocking it on YouTube.

Changing the world



Is LaDiDa really a "new paradigm for creating music," as Chordia argues? After playing with the software for several days, we're not convinced that an iPhone can really change the world, but it's undeniably fun to mess around with the software. While capable of generating some good results, it's hugely important to sing with the metronome; unfortunately, our version consistently had sound output problems that made the metronome glitch out during playback. (A reboot seemed to clear things up.)

Also, we quickly learned that producing backing tracks to existing tunes can be a bad, even terrible idea. Let's just say that there now exists in some dark corner of my iPhone a version of "Danny Boy" so execrable that it makes Vogon poetry sound like Shakespeare. (The CEO of khu.sh, the company that produces LaDiDa, appears to get better results).

But when it comes to noodling around with new ideas, the software is a blast. Singing the first paragraph of this very article into LaDiDa produced something that, while perhaps not "music" is any real sense, does not make one's ears bleed. (Well, not much. Listen if you dare.) The software also produces backing tracks to any melody line played on musical instruments, as we discovered when coming across someone's Star-Trek-theme-song-plucked-on-guitar.

The design is simple, meant to be "low floor, high ceiling." That is, anyone can use the program without any real knowledge of music or software, but it's meant to be useful to those with such experience. In Chordia's view, music creation has been too much of an elite activity, and he's out to go all Web 2.0 on songwriting.

"Just as journalism is being transformed by tools that enable users, music will be transformed too," he says. The first wave was home production studios, laptop tools like Logic and Reason, but the next wave will be "broader and much more profound." It will enable to average person, even those without musical training, to truly make music.

The music "business" is an anomaly; looked at historically, "very few cultures have done it this way." Chordia wants to reconnect people with music, and his team has already extended its engine beyond the iPhone. Next up: a Web service that provides similar tools for webcam users. Sing it your computer from anywhere on the planet and get back not just a song but a music video.

Further down the line, truly collaborative music making with such tools may be possible, with one person generating the rhythm, another performing the solo. The music may also get more real-time, generating live accompaniment as you sing.

For now, though, the team has LaDiDa, an iPhone app going for $2.99 a pop. It plans to sell more "styles" from within the app, even launching a dev program where artists can create their own custom musical styles (and sell them).

LaDiDa may not yet be a truly serious music creation tool, but it's not hard to envision a day in the future when anyone with a decent voice, some autotuning, and a computerized backing band starts climbing up the pop charts. When that happens, Chordia—and Microsoft—will have had something to do with it.