Designing Musical User Interfaces

Using harmonies and chords to go beyond visuals and enhance the user experience

We leverage the sounds of our morning alarm or the distinct beep of an unlocked car to communicate with our technology. Yet there’s an assumption that user interfaces communicate mostly through a screen, overlooking the power of sonic language.

I've been a designer for 16 years and I compose music as a hobby. These two skill sets have helped me reinforce the idea that user interfaces are meant to incorporate at least sound and vision. In my current job as a UX Designer at Udemy, my team has been working on a revamp of our learning experience. In a brainstorm session, the concept of incorporating sound in the interstitial screens of a course was surfaced. Excited, I started playing around with some synths and midi samples to create auditory feedback on lecture progress and completion. We experimented with different instruments, chords, and tempo. The challenge was to use audio to meaningfully illustrate progress while representing our values. What sound represents us? We ended up with some short and subtle motifs using a marimba and a harp in A Major.

This experience let me wondering… what if instead of using beeps and zings as auditory feedback on interfaces, we applied harmonies, notes, or chord progressions as symbolic sounds? What if we chose an instrument or set of instruments that speak to our brand and reflect the voice of our product? What if music was used in such a way that the user intuitively understood its underlying message?

Although hearing is one of our primary senses, most interfaces today are primarily visual. Sound feedback can enhance user interactions, yet there is a reliance almost entirely on how things appear on the screen. Auditory feedback aids the user by enabling them to look away from the device to complete multiple tasks. This feedback is also helpful by demonstrating that an action has been registered, is in progress, or has been completed without the use of a screen. Designing with audio is not easy, though. There are many aspects to consider if you want to keep your experience pleasant, meaningful, and practical.

I enjoyed the experience so much that I decided to compile a collection of musical sounds that others could implement in their productions. I ended up making over 200 audio samples of harmonies, sequences, SFX, speech, and chord progressions on 8 different instruments.

You can download the full pack here. But if you want to know a bit more about my background, my recommendations on how to design musical interfaces, and my process for creating these sounds, then keep on reading!