When I first saw videos of Bruno Zamborlin’s Mogees project , I thought "cool, someone has made an app for that old contact-mic trick." I’ve seen plenty of performances done with contact mics and drummers have been trigger-progressing sequencer patterns from pads since the 80's. But then I began watching the expressions of wonderment and joy on the faces of those trying out Mogees. From non-musician to experienced percussionists, that look of entrancement revealed that there was something much deeper going on here. Mogees give anyone nearly transparent access to a percussive world of the sounds and objects around them while hiding the incredibly complex algorithms it takes to get them there. That's when I realized that its real magic is in taking some pretty esoteric “computer music” technology and making it resonate with people on a street level. Bruno, can you please explain, what is Mogees? I came up with this technology that I call Mogees. The idea is to turn every physical object into a musical instrument. To somehow inject musicality into the acoustic properties of the objects that are around us. The technology is composed by a piezo transducer sensor and software. The idea is that the piezo transducer transforms the vibrations that we make when we touch the object into an audio signal that is then sent to the computer or to the phone. Together with my friend Carmine Emanuele Cella, we wrote the software that analyses such vibrations and extracts some meaningful information such as frequency, amplitude, time decay and so on. And then applies machine learning techniques to estimate how we are interacting with the object in real time and decide the note to play. In terms of sound synthesis, Carmine and I employ a technique that we called ‘physical-inspired’. Which means that Mogees sounds something like a physical object, and the sound engine is fed by the real vibrations of the object, but instead of attempting to recreate exactly the behavior of a real-world object with complex physical equations, we create a virtual one based on musical rules.

Can you give us a little background on yourself? I’m 30 years old. I’m Italian. I studied computer science in Italy, and then I moved to Paris, I worked at IRCAM, which is the music research center where Max MSP was born. I worked there for a little more than three years as developer before starting my PhD between IRCAM and Goldsmiths. What kind of work were you involved with at IRCAM? Before IRCAM, I was already familiar Max. I learned it a bit by myself and then by working at an institute in Florence called Centro Tempo Reale, funded by the composer Luciano Berio. But my big education came when I moved over to IRCAM. That’s where I really started specifically studying gesture analysis and audio synthesis in a much more scientific way. The project I worked on there for three years is called Gesture Follower . It’s a project by Frédéric Bevilacqua, who is the head of the real-time musical interaction team at IRCAM and would later become one of my PhD supervisors. He taught me everything I know about Max, gesture recognition and interaction design. Together with Norbert Schnell, they really have been my mentors for the whole time, like they were my parents really. So the Gesture Follower existed already by the time I got there, but it was a Max patch that used a lot of FTM, which is a bundle of externals for Max MSP developed at IRCAM by Norbert Schnell. It was a big, complex patch, quite hard to decode and to understand. Basically my job, at the beginning at least, was to move all this code over to C++ code, coded into one single Max object called gf . I did that during the first six months of the project, and then for the rest of the time I’ve been improving the technology, adding new features, and working a lot with artists and musicians. Can you give us an example of how the Gesture Follower works? Sure. First, you can use any sensor you want to actually capture these gestures. It can be an accelerometer, a gyroscope, a video camera, an audio signal, et cetera. Then you basically teach to the system a bunch of gestures that you want the system to learn. So, let’s say you teach like five gestures. And then, when you perform, and the system will tell you how similar you are to each of the gestures you recorded previously. So it tells you, “I think your gesture is 80 percent similar to the first one and 20 percent to the second one,” for example. And it also tracks the temporal position, so it tells you where you are in the gesture — at the beginning, middle, or the end of the gesture. Then you can use this information to time-stretch audio files and video files and control them through your gestural performance. Violinist Mari Kimura is the only person that I’m aware of, that used the Gesture Follower by training the gestures to the system in real time, during the performance itself. She was training the Gesture Follower during the concert and then using it as a sort of a looper whose tempo was perfectly in sync with her performance. It was quite impressive. She is quite a performer. And then we did a few sound installations, most of them using the Gesture Follower. It was actually quite interesting to develop one technology and then apply it to so many different domains. I used it for dance, percussion, trumpet, interactive installations in a park — different contexts, but all using the same core technology. That's similar to what I’m doing with Mogees. If you watch the videos, they are all quite different but they actually rely on exactly the same technology. The whole point of Mogees is working in the street, finding the object that you like, stick the microphone onto it, and then start playing this object. So, you're based in England now. How did that come about? After three years working at IRCAM, I wanted to do a PhD. I got a full-time scholarship in London at the Goldsmith University of London's. I managed to do a joint PhD, in Arts and Computer Science, so I had two supervisors at IRCAM's Real-Time Musical Interactions team and two supervisors at Goldsmiths' Embodied AudioVisual Group. So Frédéric Bevilacqua had been my boss for three years at IRCAM, and then the supervisor of my PhD together with the other two supervisors at Goldsmiths. That's a lot of work! I was going back and forth from Paris to London on the Eurostar train, every month for years!

Plaid and Bruno Zamborlin - EL EX (Mogees trailer)

How did you ever have time to develop Mogees? Well, about the second year of the PhD, I came up with the basic idea for Mogees, and started just using them on my own gigs, my own performances. Then when I started uploading my videos on YouTube, it got quite a few views. The very first video that I did was in 2012, and it’s the first one that basically helped me decide to open a company and to make Mogees become my full-time activity for the last two years. This first video was actually a video with a prototype that I developed in Max MSP. I remember that I uploaded the video and then went back to Italy for the holidays. When I got back to London about a week later, there were almost 300,000 views on YouTube, and my inbox was full of emails, “can I buy one?” or “can I help you to sell it, can we commercialize it?” At that point I decided, OK, I really want to try to make this, from a Max patch that was perfectly working but was kind of complex into something which was super simple. So I started the porting to the iPhone app. It was really important for me to make it work on the iPhone. The whole point of Mogees is working in the street, finding the object that you like, stick the microphone onto it, and then start playing this object. So being mobile was definitely a requirement. But basically everything started from that video, really. And that’s one of the reasons why I still haven't finished my PhD! [Laughs] So, you originally designed and prototyped Mogee in Max? Yes. But the final version is for iPhone, so it’s pure C++ and Objective C code.