In this article we present an API and a set of JavaScript modules for the synchronized scheduling and aligned playback of predetermined sequences of events such as notes, audio segments, and parameter changes as well as media streams (e.g. audio buffers) based on the Web Audio API logical time. The API has been designed to facilitate the development on both ends, the implementation of modules which generate event sequences or media streams as well as the integration of such modules into complex audio applications that require flexible scheduling, playback and synchronization.

This paper presents a set of web-native tools for visualis- ing and interacting with time-based objects. These visu- alisations are rendered as part of the document using web standard technologies, allowing for an easy integration and interaction with the elements on the same document with- out the help of any non-native technologies such as Adobe Flash, Microsoft’s Silverlight or Oracle’s Java.

At Sonoport, we ported our Dynamic Sound Engine from Adobe's Flash technology to Web Audio API. The difference in approaches to threading, scheduling, and parameters between Flash and Web Audio API created a few challenges for us. These differences and some peculiarities of the Web Audio API required workarounds to be able to implement our Dynamic Sound Engine in Web Audio API. In this paper we discuss three of these workarounds dealing with creating parameters, scheduling operations, and playback position of buffers, and explain how these work-arounds, although not optimal solutions, allowed us to support our use cases. Finally, we consider how the upcoming AudioWorker change in the Web Audio API specification is expected to impact these workarounds.

As technologies evolve and mature, proper tooling is needed to increase adoption and combat complexity. Web audio is no exception. An overture on the state of web audio tools, we will explore using both browser developer tools that take advantage of privileged platform code, and cross-browser drop-in libraries to provide introspection of the state of an audio context and nodes. With these tools, we’ll cover common debugging scenarios like audio graph construction, node-to-node signal transformations, signal visualizations, garbage collection, and resource consumption. These tools can help beginners understand a complex new API, save a developer’s time debugging, and help browser implementers optimize their platform. Attendees will leave with the knowledge of what tools are out there, workflows for inspecting, fixing, and optimizing a web audio environment, and how to make their own tools for a specific use case.

There are many existing native libraries and frameworks for audio feature extraction used in multimedia information retrieval. Many are dependent on highly optimised low level code to cope with the high performance requirements of realtime audio analysis. In this paper, we present a new audio feature extractor library, Meyda, for use with the JavaScript Web Audio API, and detail its benchmarking results. Meyda provides the first library for audio feature extraction in the web client, which will enable music information retrieval systems, complex visualisations and a wide variety of technologies and creative projects that previously were relegated to native software. The Meyda project, including source code and documentation is released under an MIT license.

A large collection of Digital Audio Workstation (DAW) plugins is available on the Internet in open source form. This paper explores their reuse in browser environments, focusing on hosting options that do not require manual installation. Two options based on Emscripten and PNaCl are introduced, implemented, evaluated, and released as open source. We found that ported DAW effect and sound synthesizer plugins complement and integrate with the Web Audio API, and that the existing preset patch collections make the plugins readily usable in online contexts. The latency measures are higher than in native plugin implementations, but expected to reduce with the emerging AudioWorker node.

The introduction of the Web Audio API has enriched the web landscape enormously. It gives game developers the ability to add precisely timed, high performant sound effects and to create realistic spatialized sound landscapes. For many web developers it is the first time they encounter audio programming which leads to many interesting experiments when the world of web and audio collide. Traditional web developers start to become really interested in audio programming and educate themselves on the topic of synthesizers and audio effects. However, currently there are only few applications that try to create more sophisticates audio production tools in the browser. In this talk I want to show how I used the Web Audio API and other emerging web standards like WebRTC to create a collaborative digital audio workstation. It allows users to record, arrange and create their own songs in collaboration with other users in real time.

The web has supported multiple media since its inception - however, only recently has it become a viable platform for building audio applications. The talk will examine the journey of audio in the web platform, the intersection of interesting technologies that make this a pivotal point for audio and the web, and will highlight the opportunities unlocked by web audio and where we go from here. Chris Wilson is a Developer Advocate on the Google Chrome team. He started working on web browsers in 1993 when he co-authored the original Windows version of NCSA Mosaic before working on Internet Explorer for fifteen years at Microsoft. He has separate and combined passions for digital audio, music and the web, and co-edits the Web Audio and Web MIDI specifications at the W3C. He also specializes in playing many different instruments badly.

Monday afternoon, January 26, 2015 - IRCAM - Igor Stravinsky Room

14.00 Keynote #2 The First Computer Music Programming Language - Chris Lowis MUSIC was a programming language developed by Max Mathews at Bell Labs in 1957. In this talk we'll learn more about Max Mathews, the origins of computer music, and by building a compiler for MUSIC in JavaScript hear what some of the very first computer music compositions sounded like.

Chris Lowis is an invited expert on the W3C's Audio Working group. He studied acoustics and signal processing at the Institute of Sound and Vibration Research in Southampton, and recently worked at the R&D department at the BBC. He loves to use the Web Audio API to bring old synthesisers back to life, and to write about audio on the web in his newsletter Web Audio Weekly.

14.45 - 16.15 Languages & Environments | Session Moderator: Matthew Paradis

14.45 Can Web Audio be Liberated from the Von Neumann Style? - Emilio Jesús Gallego Arias The audio programming world is extremely prolific in domain-specific languages. Programmers and musicians prefer a custom-tailored environment where their high-level ideas can be better expressed and implemented thanks to specific domain knowledge.

In particular, strongly typed functional programming has become an ideal platform for audio programming, thanks to a concise, high-level writing style, high reusability, and ability to avoid mistakes and improve error handling.

- How may Web Audio benefit from these ideas?

- How may such languages benefit from Web Audio?

In this talk, we will use the functional audio programming language Faust as the reference DSL, to discuss a formal theory of interoperable and efficient audio components. Particular emphasis will be made on the use of strong, functional-style type-systems.

We will address performance, security, and practical considerations, and reflect on the relationship of such theoretical framework to the actual standard.

15.05 Extending Csound to the Web - Victor Lazzarini, Edward Costello, Steven Yi, and John Ffitch - - Victor Lazzarini, Edward Costello, Steven Yi, and John Ffitch - PDF This paper discusses the presence of the sound and music computing system Csound in the modern world-wide web browser platform. It introduces the two versions of the system currently available, as pure Javascript code, and as portable Native Client binary module with a Javascript interface. Three example applications are presented, showing some of the potential uses of the system. The paper concludes with a discussion of the wider Csound application ecosystem, and the prospects for its future development.

15.25 BRAID: A Web Audio Instrument Builder with Embedded Code Blocks - Benjamin Taylor and Jesse Allison - - Benjamin Taylor and Jesse Allison - PDF Braid (Browser Audio Interface and Database) is a web audio instrument-building environment developed with the NexusUI platform. To identify the requirements of such an environment, the utility of NexusUI as an audio interface engine for web browser-based projects is reviewed. The addition of inline web audio within a drag-and-drop interface-building environment is discussed. A consideration of a modified Model-View-Controller architecture to integrate DSP code and interface is followed by an examination of the workflow of designing browser-based instruments within Braid. Finally, a database for saving and sharing web audio instruments for performance or audience distribution is described.

15.45 Interactive Music with Tone.js - Yotam Mann - - Yotam Mann - PDF This paper discusses the features, architecture and implementation of Tone.js, a Web Audio framework to facilitate the creation of interactive music specifically suited to the affordances of the browser.

16.15 Demo / Poster Session #1 & Coffee Break (Gallery, Level -2)

WAVE Project Demo and Enhanced Published Score - Benjamin Matuszewski and Samuel Goldszmidt The presentation will focus on how emerging web standards can provide score based tools for educational and musicological purposes.

Among others, SVG, Web Audio API, javascript libraries (d3, etc.), offer new ways to consider and extend an already edited and commonly available score: the ""pdf"" version provided by publishers. Moreover published scores often contain specific notation idioms for contemporary music, not easily available in digital notation formats.

While working on raw images of the score, an application can enhance it with metadata like shape and text annotations. We can also consider, in conjunction with the Web Audio API, to synchronise a score and its performances, further allowing to highlight some aspects of the composition by audio manipulation.

We will show demos of these approaches within different applications based on Webern Opus 27, Bach first Prelude of the Well-Tempered Clavier.

Noteflight: A Web-standards-based Compositional Community - Joseph Berkovitz http://www.noteflight.com

Web-Based Visualizations and Acoustic Rendering for Multimodal Data from Orchestra Performances using Repovizz - Oscar Mayor In the demo a set of fully working web-based prototypes developed in the context of the EU FP7 PHENICX Project (http://phenicx.upf.edu) will be presented. The Phenicx project is about innovating the classical music experience providing them with a multimodal, multi-perspective and multilayer interactive engagement, before, during and after the concert. In this demo we present some prototypes that are related with the post concert experience.

We have recorded a set of classical pieces performed by top level orchestras, including some data modalities like multi-channel audio, video, motion capture sensors, midi and text. Once all data streams have been time-synchronized, we have performed the following analysis on the data:

- Low-level and high-level audio descriptors for each individual audio source

- Description of conductor gestures based on the motion capture sensors - Score to performance alignment

- Audio source separation

- Musical structure analysis of the performed piece

Then all these data is uploaded to the repovizz web repository (repovizz.upf.edu) that allows visualization and sharing of the data over the network. A set of customized web-based visualizations have been designed to build the prototypes that will be shown in this demo. Multimodal data streams are accessed on-line using the repovizz web API and html5 is used for the visualizations of the multimodal data and descriptors extracted from the performances. The web-audio API is used to handle the audio rendering in the client to mix between the different audio channels obtained from the different recorded audio sources or from the automatic isolation of instruments performed in the analysis step.

The visualizations available include (all web-based): - Scrolling piano roll visualization of the musical score while audio is playing - Orchestra layout visualization showing instrument activity and loudness while playing audio

- Audio focus to hear individual instruments playing alone

- Multi-perspective video angle selection during the concert

- 3D render of the conductor body

Here is a live example of the orchestra layout visualization including the isolation of instruments as an example of one of the prototypes that will be shown during the demo: http://repovizz.upf.edu/phenicx/RCO-Eroica/

Video showing some of the repovizz orchestra visualizations developed in the context of PHENICX: https://www.youtube.com/watch?v=c7pmDvkKY7A#t=168

Repovizz - Multimodal Online Database and Visualization Tool - Quim Llimona, Oscar Mayor, Esteban Maestre, Panos Papiotis Repovizz is an integrated online system capable of structural formatting and remote storage, browsing, exchange, annotation, and visualization of synchronous multi-modal, time-aligned data. Motivated by a growing need for data-driven collaborative research, repovizz aims to resolve commonly encountered difficulties in sharing or browsing large collections of multi-modal data. At its current state, repovizz is designed to hold time-aligned streams of heterogeneous data: audio, video, motion capture, physiological signals, extracted descriptors, annotations, et cetera. Most popular formats for audio and video are supported, while CSV-based formats are adopted for streams other than audio or video (e.g., motion capture or physiological signals). The data itself is structured via customized XML files, allowing the user to (re-) organize multi-modal data in any hierarchical manner, as the XML structure only holds metadata and pointers to data files. Datasets are stored in an online database, allowing the user to interact with the data remotely through a powerful HTML5 visual interface accessible from any standard web browser; this feature can be considered a key aspect of repovizz since data can be explored, annotated, or visualized from any location or device. Data exchange and upload/download is made easy and secure via a number of data conversion tools and a user/permission management system. Furthermore, the system allows real-time collaboration by means of "shared sessions", where changes in the visualization are synchronized across multiple users as if they were all on the same machine.

Listening Guides: Ten Year Report - Rodolphe Bailly This demo is a 10 year report of the production of online Listening Guides by the Cité de la musique and their use inside and outside the institution.

The purpose of the Listening Guides is to explain the language of Music in clear and simple terms, using an online synchronization tool between Music, Text, scores and annotations.

Music-Related Media Contents Synchronized over the Web: The IEEE 1599 Initiative - Adriano Barat, Stefano Baldan, Davide Andrea Mauro, Goffredo Haus, and Luca Andrea Ludovico http://emipiu.di.unimi.it/ - PDF

The Telemeta Platform and TimeSide Framework: Audio Archives Management and Automatic Analysis - Guillaume Pellerin The Telemeta platform and TimeSide framework: Audio archives management and Automatic analysis.

Telemeta is an open-source web audio platform for the management and access to digital sound archives and audio metadata. It is developed by the Parisson company and other open source developers since 2007.

Telemeta focuses on the enhanced and collaborative user-experience in accessing audio items and their associated metadata and on the possibility for the expert users to further enrich those metadata though hierarchical and structured fields, thesaurus and ontologies.

This platform has been deployed since 2011 in the context of ethnomusicological archives and hold the archives of the Center for Research in Ethnomusicology, which is the most important ethnic music collection in Europe. The platform is fully operational and is now used on a daily basis by researchers, teachers and archivists in the fields of ethnomusicology, anthropology, linguistics and acoustics.

The Telemeta audio engine relies on TimeSide, an open audio processing framework written in Python and JavaScript.

Not only TimeSide provides Telemeta with audio decoding, encoding and streaming methods but with a set of on-the-fly signal analysis methods as well.

Given these low and high level signal analysis capabilities, various audio signal representations can be computed and incorporated in the embedded HTML audio player. By additionally wrapping several audio features extraction libraries,various automatic annotation, segmentation and musicological analysis can be performed over the audio archives and metadata.

The TimeSide engine architecture is composed of several modules and makes it easy to develop and add new plugins. Through collaboration with academic research labs in computer science, speech processing and music information retrieval, new automatic analysis functionalities are brought to the platform regularly.

As component of a fully functional but continuously evolving web platform, the development of both Telemeta and TimeSide is secured with unitary testing process.

The Telemeta and TimeSide platform are available as open-source projects at the following addresses:

https://github.com/yomguy/Telemeta

https://github.com/yomguy/TimeSide

Real-Time Client-Side Physical Modeling Harpsichord - Thomas Cipierre / Laurent Pottier (CIEREC – EA3068) As part of the ANR Blanc Project FEEVER (ANR-13-BS02-0008, 10/2013, 42 months), the CIEREC (Saint-Étienne - France) was asked by the Museum of Art and Industry of Saint-Étienne to implement a real-time synthesized harpsichord for a special exhibition .

A standalone version was created by Laurent Pottier and Luc Faure, all DSP edited in FAUST in a physical modeling approach, with a Max/MSP GUI (DSP based on Julius Smith and Romain Michon previous works on the Faust-STK ). Our second goal was then to provide a client-side real-time harpsichord on the web, in order to share a convenient musical researching and/or production tool, as much as an educational support to anyone, even with no prior computational and/or musical knowledge.

The idea was thus to implement a polyphonic Javascript version of our model, thanks to the faust2asmjs script (which consists of compiling our FAUST DSP in an optimized polyphonic asm.js version), create instances, use regular Web Audio API nodes, and control DSP parameters through a responsive user-friendly GUI in Polymer . Some issues were of course to face, from trying to optimize the amount of computing resources needed by the physical modeling approach (especially for near zero amplitude data), to trying to be the more "web-standard" we could concerning the GUI, despite the unbalanced implementations of W3C requirements through the mainly used web browsers (Web Audio API, Web MIDI API, Web Components).

You will find our still evolving web harpsichord version here : http://musinf.univ-st-etienne.fr/recherches/ClavecinHtml/web-harpsichord.html .

You will also find a quick video demonstration of our first stand-alone version (FAUST DSP - Max/MSP GUI) here : http://feever.fr/videos.

Delivering Object-Based 3D Audio using The Web Audio API - Chris Pike

Binaural Synthesis with the Web Audio API - Thibaut Carpentier - - Thibaut Carpentier - PDF

Real-Time Acoustic Auralization on the Web - Chinmay Prafulla Pendharkar Auralization is the technique user for creation and reproduction of spatially distributed sound based on mathematical models. In recent times auralization is gaining interest in the field of Room Acoustics as an effective way to hear how proposed spaces or rooms would distribute sonic events. Auralization using real-time sound sources takes this virtual reality one step further by processing the users voice and reproducing it to give a sense of envelopment similar to the space being modeled.

Traditionally most tools for creating these mathematical models, their outputs in form of impulse responses, or the tools for experiencing auralization are native apps. Web Audio enables parts of the usecases for real-time auralization by allowing user audio inputs and fast native convolution using the ConvolverNode. Earlier this year, I created a demo of this concept called Auralizr, based on a methodology derived and implemented in PureData by my Acoustician colleague Josefin Lindebrink. https://github.com/notthetup/auralizr

In this talk I'll explain how real-time auralization works on the Web and especially Mobile Web. I will also discuss some performance in terms of latency and how things can be improved. I will talk about some use case ideas and also some future plans we have with this project.

Birds of a Feather (Les Oiseaux de Mme Plumage): Dynamic Soundscapes using Real-Time Manipulation of Locally Relevant Birdsongs - Bill Walker and Brian Belet

17.00 - 18.30 Delivering & Listening - Igor Stravinsky Room | Session Moderator: Thibaut Carpentier

17.00 Delivering Object-Based 3D Audio using The Web Audio API and The Audio Definition Model - Peter Taylour, Chris Pike, and Frank Melchior - - Peter Taylour, Chris Pike, and Frank Melchior - PDF Presentation of an application that demonstrates object-based 3D audio rendering in the web browser using the Web Audio API. The application loads audio files containing object-based meta-data and provides head-tracked dynamic binaural rendering of the content to create an immersive 3D audio experience for headphone listeners. The user can interact with the rendering by muting individual audio objects and switching between the binaural rendering mode and conventional stereo rendering. The demo also includes visualisations of the object-based meta-data. Active objects are highlights on a 2D timeline visualisation and THREE.js is used to show the rendering location of each audio source in 3D space.

This application demonstrates a future of broadcast sound experiences over the web, where immersive content is rendered on the client and can be adapted to listener context, as page layout is adapted to device context today with responsive design.

17.20 Towards the Next Generation of Web-based Experiments: A Case Study Assessing Basic Audio Quality Following the ITU-R Recommendation BS.1534 (MUSHRA) - Michael Schoeffler, Fabian-Robert Stšter, Bernd Edler, and JŸrgen Herre - - Michael Schoeffler, Fabian-Robert Stšter, Bernd Edler, and JŸrgen Herre - PDF Listening tests are widely used to assess the quality of audio systems. The majority of such listening tests are conducted in controlled environments with selected participants and professional audio equipment. In the last few years, conducting listening tests over the Internet, as so called web-based experiments, has become popular. A recent study has shown that web-based experiments lead to comparable results as laboratory experiments.

Until now, it was only possible to implement a limited number of listening test types as web-based experiments because web standards were missing some crucial features (e.g. sample manipulation of audio streams). With the upcoming of the Web Audio API, a much wider range of listening test types can be implemented as new audio processing features have been introduced. This talk will demonstrate which new possibilities are enabled by the Web Audio API. To this end, the ITU-R Recommendation BS.1534 (MUSHRA) is taken as an example.

17.40 Spatially Distributed Sound Computing and Rendering using the Web Audio Platform - Lonce Wyse - - Lonce Wyse - PDF Large multi-channel spatial audio systems have historically been a playground for universities and well-funded studios, but only a dream for independent composers.

Similarly, "parallel computers" were locked in research facilities where only a few musicians ever gained access to the computational power to convolve hundreds of separate audio streams with spatially specific room impulse responses. Mobile devices in the hands of audiences can quickly configure themselves into these kinds of systems at very affordable (and distributed) cost and little effort, making powerful and expressive spatially distributed musical platforms accessible to anyone today. We describe some software systems and artistic works that have been recently developed to explore some of the spatial audio capabilities of the mobile device browser platform.