Genesis Digital Audio Workstation

First Post | RSS Feed | Genesis on GitHub

Genesis is a work-in-progress digital audio workstation with some ambitious goals: peer-to-peer multiplayer editing, complete plugin safety, and a built-in peer-to-peer community which shares plugins, projects, and samples.

This is the development blog.

Posts

Live Coding: Building the Sequencer 2015 Oct 25 In this live coding session, we: Troubleshoot a problem with the audio pipeline.

Rename Piano Roll to Sequencer.

Create the rows of the piano and label them.

View Comments

SIMD Test 2015 Oct 21 I was aware of Single Instruction Multiple Data optimizations, but I wasn't sure how to take advantage of them, or whether or not it was something I needed to directly take advantage of, versus relying on the compiler to do so for me. So naturally I wrote some code to explore. Results: Both GCC and Clang can perform simple SIMD optimization when -O3 is enabled. More difficult SIMD optimization problems were not explored. test-gcc-debug sum: 99897982687.000000 time: 0.446810 test-gcc-o3 sum: 99897982687.000000 time: 0.157358 test-gcc-simd sum: 99897982687.000000 time: 0.157701 test-gcc-debug-simd sum: 99897982687.000000 time: 0.213039 test-clang-debug sum: 99897982687.000000 time: 0.410983 test-clang-o3 sum: 99897982687.000000 time: 0.159946 test-clang-simd sum: 99897982687.000000 time: 0.160412 test-clang-debug-simd sum: 99897982687.000000 time: 0.204966 This is the C code that was being timed: #ifdef USE_SIMD for (int i = 0; i < size; i += 4) { v4si *a_ptr = (v4si *) &a[i]; v4si *b_ptr = (v4si *) &b[i]; v4si *c_ptr = (v4si *) &c[i]; *c_ptr = *a_ptr + *b_ptr; } #else for (int i = 0; i < size; i += 1) { c[i] = a[i] + b[i]; } #endif What follows here is what assembly this C code turns into, for each of these compiler settings: test-gcc-debug - no optimizations, no explicit SIMD

- no optimizations, no explicit SIMD test-gcc-o3 - optimization level 3, no explicit SIMD

- optimization level 3, no explicit SIMD test-gcc-simd - optimization level 3, explicit SIMD

- optimization level 3, explicit SIMD test-gcc-debug-simd - no optimization, explicit SIMD You can tell from the assembly that the output generated from -O3, without explicit SIMD, is the same as the output generated from -O3, with explicit SIMD. test-gcc-debug 41:test.c **** for (int i = 0; i < size; i += 1) { 165 .loc 1 41 0 166 0123 C745F800 movl $0, -8(%rbp) #, i 166 000000 167 012a EB2A jmp .L8 # 168 .L9: 42:test.c **** c[i] = a[i] + b[i]; 169 .loc 1 42 0 discriminator 3 170 012c 8B45F8 movl -8(%rbp), %eax # i, tmp120 171 012f 4898 cltq 172 0131 8B148500 movl a(,%rax,4), %edx # a, D.3584 172 000000 173 0138 8B45F8 movl -8(%rbp), %eax # i, tmp122 174 013b 4898 cltq 175 013d 8B048500 movl b(,%rax,4), %eax # b, D.3584 175 000000 176 0144 01C2 addl %eax, %edx # D.3584, D.3584 177 0146 8B45F8 movl -8(%rbp), %eax # i, tmp124 178 0149 4898 cltq 179 014b 89148500 movl %edx, c(,%rax,4) # D.3584, c 179 000000 41:test.c **** for (int i = 0; i < size; i += 1) { 180 .loc 1 41 0 discriminator 3 181 0152 8345F801 addl $1, -8(%rbp) #, i 182 .L8: 41:test.c **** for (int i = 0; i < size; i += 1) { 183 .loc 1 41 0 is_stmt 0 discriminator 1 184 0156 817DF8FF cmpl $99999999, -8(%rbp) #, i 184 E0F505 185 015d 7ECD jle .L9 #, 186 .LBE3: 43:test.c **** } test-gcc-o3 163 .L4: 44:test.c **** for (int i = 0; i < size; i += 1) { 45:test.c **** c[i] = a[i] + b[i]; 167 .loc 1 45 0 discriminator 3 168 00c0 660F6F80 movdqa a(%rax), %xmm0 # MEM[symbol: a, index: ivtmp.29_77, offset: 0B], MEM[symbol: a, index: ivtm 168 00000000 169 00c8 4883C010 addq $16, %rax #, ivtmp.29 170 00cc 660FFE80 paddd b-16(%rax), %xmm0 # MEM[symbol: b, index: ivtmp.29_77, offset: 0B], vect__33.14 170 00000000 171 00d4 0F298000 movaps %xmm0, c-16(%rax) # vect__33.14, MEM[symbol: c, index: ivtmp.29_77, offset: 0B] 171 000000 172 00db 483D0084 cmpq $400000000, %rax #, ivtmp.29 172 D717 173 00e1 75DD jne .L4 #, test-gcc-simd 163 .L4: 34:test.c **** 35:test.c **** #ifdef USE_SIMD 36:test.c **** for (int i = 0; i < size; i += 4) { 37:test.c **** v4si *a_ptr = (v4si *) &a[i]; 38:test.c **** v4si *b_ptr = (v4si *) &b[i]; 39:test.c **** v4si *c_ptr = (v4si *) &c[i]; 40:test.c **** *c_ptr = *a_ptr + *b_ptr; 168 .loc 1 40 0 discriminator 3 169 00c0 660F6F80 movdqa a(%rax), %xmm0 # MEM[symbol: a, index: ivtmp.25_86, offset: 0B], MEM[symbol: a, index: ivtm 169 00000000 170 00c8 4883C010 addq $16, %rax #, ivtmp.25 171 .LVL10: 172 00cc 660FFE80 paddd b-16(%rax), %xmm0 # MEM[symbol: b, index: ivtmp.25_86, offset: 0B], D.2962 172 00000000 173 00d4 0F298000 movaps %xmm0, c-16(%rax) # D.2962, MEM[symbol: c, index: ivtmp.25_86, offset: 0B] 173 000000 174 .LBE26: 36:test.c **** for (int i = 0; i < size; i += 4) { 175 .loc 1 36 0 discriminator 3 176 00db 483D0084 cmpq $400000000, %rax #, ivtmp.25 176 D717 177 00e1 75DD jne .L4 #, test-gcc-debug-simd 168 .L9: 169 .LBB4: 37:test.c **** v4si *a_ptr = (v4si *) &a[i]; 170 .loc 1 37 0 discriminator 3 171 012c 8B45F8 movl -8(%rbp), %eax # i, tmp120 172 012f 4898 cltq 173 0131 48C1E002 salq $2, %rax #, tmp121 174 0135 48050000 addq $a, %rax #, tmp122 174 0000 175 013b 488945D8 movq %rax, -40(%rbp) # tmp122, a_ptr 38:test.c **** v4si *b_ptr = (v4si *) &b[i]; 176 .loc 1 38 0 discriminator 3 177 013f 8B45F8 movl -8(%rbp), %eax # i, tmp124 178 0142 4898 cltq 179 0144 48C1E002 salq $2, %rax #, tmp125 180 0148 48050000 addq $b, %rax #, tmp126 180 0000 181 014e 488945D0 movq %rax, -48(%rbp) # tmp126, b_ptr 39:test.c **** v4si *c_ptr = (v4si *) &c[i]; 182 .loc 1 39 0 discriminator 3 183 0152 8B45F8 movl -8(%rbp), %eax # i, tmp128 184 0155 4898 cltq 185 0157 48C1E002 salq $2, %rax #, tmp129 186 015b 48050000 addq $c, %rax #, tmp130 186 0000 187 0161 488945C8 movq %rax, -56(%rbp) # tmp130, c_ptr 40:test.c **** *c_ptr = *a_ptr + *b_ptr; 188 .loc 1 40 0 discriminator 3 189 0165 488B45D8 movq -40(%rbp), %rax # a_ptr, tmp131 190 0169 660F6F08 movdqa (%rax), %xmm1 # *a_ptr_38, D.2868 191 016d 488B45D0 movq -48(%rbp), %rax # b_ptr, tmp132 192 0171 660F6F00 movdqa (%rax), %xmm0 # *b_ptr_39, D.2868 193 0175 660FFEC1 paddd %xmm1, %xmm0 # D.2868, D.2868 194 0179 488B45C8 movq -56(%rbp), %rax # c_ptr, tmp133 195 017d 0F2900 movaps %xmm0, (%rax) # D.2868, *c_ptr_40 196 .LBE4: 36:test.c **** for (int i = 0; i < size; i += 4) { 197 .loc 1 36 0 discriminator 3 198 0180 8345F804 addl $4, -8(%rbp) #, i 199 .L8: 36:test.c **** for (int i = 0; i < size; i += 4) { 200 .loc 1 36 0 is_stmt 0 discriminator 1 201 0184 817DF8FF cmpl $99999999, -8(%rbp) #, i 201 E0F505 202 018b 7E9F jle .L9 #, How to get that nice assembly output gcc -std=c11 -S -D_POSIX_C_SOURCE=199309L -fverbose-asm -g -D USE_SIMD test.c -o test.s as -alhnd test.s From this nice StackOverflow answer. View Comments

Live Coding: Rendering the User's Project 2015 Oct 21

In this live coding session, we: Add a UI for viewing and cancelling a rendering job.

Troubleshoot and fix a segfault that happened during cleanup.

Troubleshoot and fix rendered audio sounding glitchy. View Comments

Live Coding 2015 Oct 20 I've been experimenting with broadcasting myself coding Genesis live. Here are some archived videos. In these videos we worked on: Move audio graph code out of project code.

Prepare codebase for ability to render project.

Fix memory leaks. If you, like me, do not have Flash installed, you can use the excellent livestreamer project to view the content. I think I'm going to try to start doing these streams daily. I'm not really sure where to announce when the stream starts. I'll try twitter. My username is andy_kelley. View Comments

Render Project Dock 2015 Oct 15 Today I refactored the audio file format code and added this dock which is used to start a render of the project. The options here are persisted in the user settings file. Next up is making the Render button actually start a render. View Comments

Project Settings Pane 2015 Oct 14 Here is a screenshot of the Project Settings Pane. Changing the sample rate does in fact change the sample rate of the underlying audio pipeline. Next up is implementing rendering to disk, which uses these settings. View Comments

libsoundio integration complete 2015 Oct 06 I am pleased to report that libsoundio integration is complete and it has indeed solved the buffer underrun problems and other issues I was running into before. In this screenshot you can see that Genesis is connected to multiple sound drivers at once - JACK, PulseAudio, ALSA, and Dummy. On top of that it is listening and will automatically refresh when you insert or remove a device such as a USB microphone. View Comments

libsoundio 2015 Sep 03 I struggled with buffer underruns for a long time. I seemed to be dealing with some enigmatic audio problems. After many failed attempts to solve the audio problems I was having, I decided to tackle the problem head on. I extracted the audio engine code and started a new project, one in which I was dedicated to becoming an expert at how audio input and output is handled on every platform. Hopefully after solving the crap out of sound I/O in a way independent from the other issues I was having, I would be able to solve the Genesis problems. Well, 3 months and a lot of hard work later I have completed libsoundio 1.0.0. libsoundio is a lightweight abstraction over various sound drivers. It provides a well-documented API that operates consistently regardless of the sound driver it connects to. It performs no buffering or processing on your behalf; instead exposing the raw power of the underlying backend. libsoundio is appropriate for games, music players, digital audio workstations, and various utilities. libsoundio is serious about robustness. It even handles out of memory conditions correctly. Features & Limitations Supported backends: JACK PulseAudio ALSA CoreAudio WASAPI Dummy (silence)

Exposes both raw devices and shared devices. Raw devices give you the best performance but prevent other applications from using them. Shared devices are default and usually provide sample rate conversion and format conversion.

Exposes both device id and friendly name. id you could save in a config file because it persists between devices becoming plugged and unplugged, while friendly name is suitable for exposing to users.

Supports optimal usage of each supported backend. The same API does the right thing whether the backend has a fixed buffer size, such as on JACK and CoreAudio, or whether it allows directly managing the buffer, such as on ALSA or PulseAudio.

C library. Depends only on the respective backend API libraries and libc. Does not depend on libstdc++, and does not have exceptions, run-time type information, or setjmp.

Errors are communicated via return codes, not logging to stdio.

Supports channel layouts (also known as channel maps), important for surround sound applications.

Ability to monitor devices and get an event when available devices change.

Ability to get an event when the backend is disconnected, for example when the JACK server or PulseAudio server shuts down.

Detects which input device is default and which output device is default.

Ability to connect to multiple backends at once. For example you could have an ALSA device open and a JACK device open at the same time.

Meticulously checks all return codes and memory allocations and uses meaningful error codes.

Compare libsoundio to: PortAudio, RtAudio, or SDL2. View Comments

Playback 2015 May 28 Download Video This is another major milestone for the project: we have playback. Lock-free multi-threaded programming is extremely difficult. Here are a couple of data structures I implemented to solve some problems: #ifndef ATOMIC_DOUBLE_HPP #define ATOMIC_DOUBLE_HPP #include <atomic> using std::atomic; class AtomicDouble { public: AtomicDouble() { assert(_value.is_lock_free()); } void add(double x) { double current_value = _value.load(); while (!_value.compare_exchange_weak(current_value, current_value + x)) {} } void store(double x) { _value.store(x); } double load() const { return _value.load(); } private: atomic<double> _value; }; #endif The play head position uses this to keep track of where is being played. #ifndef ATOMIC_VALUE_HPP #define ATOMIC_VALUE_HPP #include <atomic> using std::atomic_int; #ifndef ATOMIC_INT_LOCK_FREE #error "require atomic int to be lock free" #endif // single reader, single writer atomic value template<typename T> class AtomicValue { public: AtomicValue() { _in_use_index = 0; _active_index = 0; _write_index = -1; } ~AtomicValue() {} T *write_begin() { assert(_write_index == -1); int in_use_index = _in_use_index.load(); int active_index = _active_index.load(); if (in_use_index != 0 && active_index != 0) _write_index = 0; else if (in_use_index != 1 && active_index != 1) _write_index = 1; else _write_index = 2; return &_values[_write_index]; } void write_end() { assert(_write_index != -1); _active_index.store(_write_index); _write_index = -1; } T *get_read_ptr() { _in_use_index.store(_active_index.load()); return &_values[_in_use_index]; } T *write(const T &value) { T *ptr = write_begin(); *ptr = value; write_end(); return ptr; } private: T _values[3]; atomic_int _in_use_index; atomic_int _active_index; int _write_index; AtomicValue(const AtomicValue ©) = delete; AtomicValue &operator=(const AtomicValue ©) = delete; }; #endif This structure uses triple buffering to provide a single reader, single writer atomic value, where the value can be anything. In Genesis, it is used to hold the data structure that contains all the MIDI events. When a user edits the project, the pointer atomically flips to an unused buffer for writing, similar to how screen rendering works. View Comments

Gradients and Samples 2015 May 11 The Resources Tree now has a scroll bar and reads samples from your configured sample directories. I also made it a lot easier in my codebase to make a fancy gradient background. The gradient is computed in a shader on the GPU: Vertex Shader #version 150 core in vec3 VertexPosition; out float MixAmt; uniform mat4 MVP; void main(void) { MixAmt = clamp(0, VertexPosition.y, 1); gl_Position = MVP * vec4(VertexPosition, 1.0); } Fragment Shader #version 150 core in float MixAmt; out vec4 FragColor; uniform vec4 ColorTop; uniform vec4 ColorBottom; void main(void) { FragColor = ColorBottom * MixAmt + ColorTop * (1 - MixAmt); } View Comments

Dock Demo 2015 May 09 Download Video Now you can drag docks around into any configuration for editing. By the way, here's what the config file looks like so far (it lives in ~/.genesis/config ). // Genesis DAW configuration file // This config file format is a superset of JSON. See // https://github.com/andrewrk/liblaxjson for more details. { // your display name user_name: "andy", // your user id user_id: "02f6a090170886e5f6eecf1b22bf250527205e779828e1d2c6052ca681ef8985", // open this project on startup open_project_id: "908ad8b5f7c0ff2cc22a60c5240a4f475596c3d2193293fe0c192f8bfab08ac0", // these perspectives are available for the user to choose from perspectives: [ { name: "Default", dock: { dock_type: "Horiz", split_ratio: 0.307832, child_a: { dock_type: "Tabs", tabs: ["Resources"], }, child_b: { dock_type: "Tabs", tabs: ["Track Editor", "Mixer"], }, }, }, ], // open these windows on startup open_windows: [ { // which perspective this window uses perspective: 0, // position of the window on the screen left: 243, top: 111, width: 1102, height: 855, maximized: false, // whether to show dockable pane tabs when there is only one always_show_tabs: true, }, ], } View Comments

Project File Format 2015 Apr 26 The first 16 bytes of every Genesis project file are: ca2f 5ef5 00d8 ef0b 8074 18d0 e40b 7a4f I got this from /dev/random . This uniquely identifies the file as a Genesis project file. The default file extension is .gdaw . After this contains an ordered list of transactions. A transaction is a set of edits. There are 2 types of edits: put and delete. A put contains a key and a value, each of which is a variable number of bytes. A delete contains only a key, which is a variable number of bytes. A transaction looks like this: Offset Description 0 uint32be length of transaction in bytes including this field 4 uint32be crc32 of this transaction 8 uint32be number of put edits in this transaction 12 uint32be number of delete edits in this transaction 16 the put edits in this transaction - the delete edits in this transaction A put edit looks like: Offset Description 0 uint32be key length in bytes 4 uint32be value length in bytes 8 key bytes - value bytes A delete edit looks like: Offset Description 0 uint32be key length in bytes 4 key bytes That's it. To read the file, apply the transactions in order. To update a file, append a transaction. Periodically "garbage collect" by creating a new project file with a single transaction with all the data, and then atomically rename the new project file over the old one. View Comments

Beginnings of Track Editor 2015 Apr 25 Started work on drawing the Track Editor. I also planned out how the undo and redo framework would work, even when there are multiple users working on the same project. View Comments

Resources Tree Widget Implemented 2015 Apr 25 You can also see the menu mnemonics in action. The tree view updates live as you plug or unplug devices. View Comments

Menu Widget Implemented 2015 Apr 25 Got the main menu widget working. Everything is OpenGL. View Comments

Milestone: Basic Synth and MIDI Keyboard Support 2015 Mar 30 Today I reached a motivating milestone: I can play a polyphonic sine wave using a USB MIDI keyboard. Download Video It's just a simple sine wave and there are no parameters to configure. I wasn't sure how to modulate the volume when there were multiple notes being pressed at the same time. Should it do simple addition? Then the synth could output samples above 1.0. That seems broken. Should it average the notes together? Then when you play 2 notes at once, each note is half volume of a single note. If one of the simultaneous notes had a low velocity, that seems weird. Another problem is the clicking when you release a note. It's because the sine wave abruptly ends instead of gracefully fading towards zero: These are problems for another day. Moving on. I also got a delay example working. It connects your default recording device to a delay (echo) filter to your default playback device. Like the synth example, it's a C program which only depends on libgenesis. I took all the error checking code out for clarity. #include "genesis.h" #include <stdio.h> #include <stdlib.h> int main(int argc, char **argv) { struct GenesisContext *context; genesis_create_context(&context); // block until we have audio devices list genesis_refresh_audio_devices(context); int playback_device_index = genesis_get_default_playback_device_index(context); int recording_device_index = genesis_get_default_recording_device_index(context); struct GenesisAudioDevice *out_device = genesis_get_audio_device(context, playback_device_index); struct GenesisAudioDevice *in_device = genesis_get_audio_device(context, recording_device_index); struct GenesisNodeDescriptor *playback_node_descr; genesis_audio_device_create_node_descriptor(out_device, &playback_node_descr); struct GenesisNode *playback_node = genesis_node_descriptor_create_node(playback_node_descr); struct GenesisNodeDescriptor *recording_node_descr; genesis_audio_device_create_node_descriptor(in_device, &recording_node_descr); genesis_audio_device_unref(in_device); genesis_audio_device_unref(out_device); struct GenesisNode *recording_node = genesis_node_descriptor_create_node(recording_node_descr); struct GenesisNodeDescriptor *delay_descr = genesis_node_descriptor_find(context, "delay"); fprintf(stderr, "delay: %s

", genesis_node_descriptor_description(delay_descr)); struct GenesisNode *delay_node = genesis_node_descriptor_create_node(delay_descr); genesis_connect_audio_nodes(recording_node, delay_node); genesis_connect_audio_nodes(delay_node, playback_node); genesis_start_pipeline(context); for (;;) genesis_wait_events(context); } View Comments

Thread Safe Lock Free Queue Implementation 2015 Mar 27 I spent three days in a row working on a many writer, many reader, fixed-size, thread-safe, first-in-first-out lock-free queue. The queue makes no syscalls except under one condition: if a reader attempts to dequeue an item and the queue is empty, then it makes a syscall to go to sleep. And then if and only if this has happend, a writer will make a syscall to wake up a sleeping reader when it enqueues an item. Here is the code: #ifndef THREAD_SAFE_QUEUE #define THREAD_SAFE_QUEUE #include <linux/futex.h> #include <sys/time.h> #include <unistd.h> #include <sys/syscall.h> #include <sys/errno.h> #include <atomic> using std::atomic_int; using std::atomic_flag; #ifndef ATOMIC_INT_LOCK_FREE #error "require atomic int to be lock free" #endif // if this is true then we can send the address of an atomic int to the futex syscall static_assert(sizeof(int) == sizeof(atomic_int), "require atomic_int to be same size as int"); static inline int futex(int *uaddr, int op, int val, const struct timespec *timeout, int *uaddr2, int val3) { return syscall(SYS_futex, uaddr, op, val, timeout, uaddr2, val3); } static inline int futex_wait(int *address, int val) { return futex(address, FUTEX_WAIT, val, nullptr, nullptr, 0) ? errno : 0; } static inline int futex_wake(int *address, int count) { return futex(address, FUTEX_WAKE, count, nullptr, nullptr, 0) ? errno : 0; } // must call resize before you can start using it // size must be at least equal to the combined number of producers and consumers template<typename T> class ThreadSafeQueue { public: ThreadSafeQueue() { _items = nullptr; _size = 0; } ~ThreadSafeQueue() { free(_items); } // this method not thread safe int __attribute__((warn_unused_result)) resize(int size) { assert(size >= 0); if (size != _size) { T *new_items = (T *)realloc(_items, size); assert(new_items); // error handling not shown _items = new_items; _size = size; } _queue_count = 0; _read_index = 0; _write_index = 0; _modulus_flag.clear(); return 0; } // put an item on the queue. panics if you attempt to put an item into a // full queue. thread-safe. void enqueue(T item) { int my_write_index = _write_index.fetch_add(1); int in_bounds_index = my_write_index % _size; _items[in_bounds_index] = item; int my_queue_count = _queue_count.fetch_add(1); if (my_queue_count >= _size) panic("queue is full"); if (my_queue_count <= 0) futex_wake(reinterpret_cast<int*>(&_queue_count), 1); } // get an item from the queue. blocks if the queue is empty. thread-safe. // if the queue has 4 items and 8 threads try to dequeue at the same time, // 4 threads will block and 4 threads will return queue items. T dequeue() { for (;;) { int my_queue_count = _queue_count.fetch_sub(1); if (my_queue_count > 0) break; // need to block because there are no items in the queue int err = futex_wait(reinterpret_cast<int*>(&_queue_count), my_queue_count - 1); if (err == EACCES || err == EINVAL || err == ENOSYS) panic("futex wait error"); // one of these things happened: // * waiting failed because _queue_count changed. // * spurious wakeup // * normal wakeup // in any case, release the changed state and then try again _queue_count += 1; } int my_read_index = _read_index.fetch_add(1); int in_bounds_index = my_read_index % _size; // keep the index values in check if (my_read_index >= _size && !_modulus_flag.test_and_set()) { _read_index -= _size; _write_index -= _size; _modulus_flag.clear(); } return _items[in_bounds_index]; } // wakes up all blocking dequeue() operations. thread-safe. // after you call wakeup_all, the queue is in an invalid state and you // must call resize() to fix it. consumer_count is the total number of // threads that might call dequeue(). void wakeup_all(int consumer_count) { int my_queue_count = _queue_count.fetch_add(consumer_count); int amount_to_wake = -my_queue_count; if (amount_to_wake > 0) futex_wake(reinterpret_cast<int*>(&_queue_count), amount_to_wake); } private: T *_items; int _size; atomic_int _queue_count; atomic_int _read_index; atomic_int _write_index; atomic_flag _modulus_flag; }; #endif I wrote some unit tests and they all pass, even when I run 10 instances of the unit tests at once on repeat for 10 minutes. Coding with the C++11 atomics was quite handy. It depends on a Linux-specific feature called futex for causing threads to sleep and wake up. Other operating systems have similar features, and I will need to create an OS-specific port of this data structure for each operating system when the time comes. View Comments

Interesting Commits 2015 Mar 26 commit e17a50a491851c19158b0908102ea81b1c700564 Author: Andrew Kelley <superjoe30@gmail.com> Date: Thu Mar 5 16:34:02 2015 -0700 RingBuffer implementation I use mmap to put the same slice of memory in two places so that each write and read from the ring buffer only takes one memcpy . commit f3581e6989e98b8aeddd66d3ad452ec622a713a9 Author: Andrew Kelley <superjoe30@gmail.com> Date: Sat Mar 7 19:13:58 2015 -0700 alright C++ you win. gimme dat inheritance When working on widget user interface code, I realized that I was implementing inheritence without actually using C++'s inheritance features, and it was more error prone than just giving in and depending on libstdc++. The core backend and the GUI are decoupled. The core backend is in a shared library called libgenesis which does not link against any GUI-related libraries - not even libstdc++. Meanwhile, the GUI depends on libgenesis and puts a user-interface on top of it. libgenesis is intended to be a general-purpose utility library for doing digital audio workstation related things, such as using it as the backend for a headless computer-created music stream. 749f762f34fa6ba207374f8807c1be241810d768 Author: Andrew Kelley <superjoe30@gmail.com> Date: Tue Mar 10 12:45:51 2015 -0700 32-bit floats in memory instead of 64 I'm still not sure about this one. I've read some conflicting things about whether to use 32-bit float or 64-bit float for audio processing. I made this commit when I found out that JACK uses exclusively 32-bit floats for audio samples. commit bdaca9bea4fb046cdb7357da7476cb455ab5d5c1 Author: Andrew Kelley <superjoe30@gmail.com> Date: Tue Mar 10 19:37:00 2015 -0700 GLFW instead of SDL2 I experienced 2 issues with SDL: home button doesn't work on some keyboards

keyboard shortcuts ignore keyboard layout GLFW has a narrower focus and smaller footprint. I hoped that this meant the bug surface area would be smaller. Sadly, it has the same bugs. However it is under more active development with a faster release schedule, and less bloat in between Genesis and the hardware is nice. Next I got a list_devices example program working: $ ./list_devices playback device: 44100 Hz GK104 HDMI Audio Controller Digital Stereo (HDMI) playback device: 44100 Hz Built-in Audio Analog Stereo (default) recording device: 44100 Hz Monitor of GK104 HDMI Audio Controller Digital Stereo (HDMI) recording device: 44100 Hz Monitor of Built-in Audio Analog Stereo recording device: 44100 Hz Built-in Audio Analog Stereo (default) controller device: Midi Through Port-0 (default) 6 devices found And a normalize_audio example which reads an audio file of any format, increases the volume as much as possible without clipping, and then saves to any format. This demonstrates the audio import and export capabilities. $ ./normalize_audio ~/tmp/quiet.ogg ~/tmp/out.flac Channels: 2 (Stereo) 222208 frames (5.04 seconds) Amplification factor: 18.638 out codec: FLAC (Free Lossless Audio Codec) out sample format: signed 24-bit integer out sample rate: 44100 View Comments

Vaporware Grand Plans 2015 Mar 16 Real-Time Safety and Multithreading Most operating systems do not attempt to make any real-time guarantees. This means that various operations do not guarantee a maximum bound on how long it might take. For example, when you allocate memory, it might be very quick, or the kernel might have to do some memory defragmentation and cache invalidation to make room for your new segment of memory. Writing to a hard drive might cause it to have to warm up and start spinning. This can be a disaster if one of these non-bounded operations is in the audio rendering pipeline, especially if the latency is low. The buffer of audio going to the sound card might empty before it gets filled up, causing a nasty audio glitch sound known as a "buffer underrun". In general, all syscalls are suspect when it comes to real-time guarantees. The careful audio programmer will avoid them all. libgenesis meets this criteria with one exception. libgenesis takes advantage of hardware concurrency by using one worker thread per CPU core to render audio. It carefully uses a lock-free queue data structure to avoid making syscalls, but when there is not enough work to do and some threads are sitting idly by, those threads need to suspend execution until there is work to do. So if there is more work to be done than worker threads, no syscalls are made. However, if a worker thread has nothing to do and needs to suspend execution, it makes a FUTEX_WAIT syscall, and then is woken up by another worker thread making a FUTEX_WAKE syscall. Compatibility libgenesis follows semver. Major version is bumped when API or ABI compatibility is broken. Minor version is bumped when new features are added. Patch version is bumped only for bug fixes. Until 1.0.0 is released no effort is made toward backward compatibility whatsoever. Genesis Audio Studio has an independent version from libgenesis. Major version is bumped when a project file will no longer generate the same output audio as the previous version. Minor version is bumped when new features are added. Patch version is bumped only for bug fixes. Until 1.0.0 is released no effort is made toward project files being backward compatible to previous versions. Coordinates Positions in the audio project are in floating point whole notes. This is to take into account the fact that the tempo and time signature could change at any point in the audio project. You can convert from whole notes to frames by calling a utility function which takes into account tempo and time signature changes. Multiplayer and Peer-to-Peer When a user opens Genesis, there should be a pane which has a set of rooms that users can gather in and chat. For example, a lobby. Users can create other rooms, perhaps private, to join as well. Users can invite other users to join their open project. When a user joins the project, a peer-to-peer connection is established so the edits do not go through a third party. Further, if the peers are connected via LAN, network speeds will be very fast. The server(s) that provide this chat are also peers. Individuals or businesses could donate to the server space, similar to being a seeder in a torrent situation, by running the server software, adding their node to the pool. When two (or more) users are simultaneously working on a project, the playback head is not synchronized. The users are free to roam about the project, making changes here and there. However, each person will see "where" in the project the other person is working, and see the changes that they are making. So it would be trivial, for example, for both users to look at a particular bassline, both listening to it on loop, albeit at different offsets, while one person works on the drums, and the other person works on the bass rhythm. Plugin Registry and Safety Plugins must be provided as source code and published to the Genesis registry. The Genesis registry will not be a single server, but once again a peer-to-peer network. Downloading from the plugin registry will be like downloading a torrent. By default Genesis will act as a peer on LANs when other instances of Genesis request plugins over the LAN. It's not clear how this goal will be accomplished, but we will attempt to build a system where these constraints are met: Plugins are provided as source code that is guaranteed to build on all supported platforms. It's not possible to have a plugin that works on one person's computer and not another.

Plugins either have compile-time protection against malicious code and crashes (such as segfaults) or run-time protection. One idea: instead of one sandboxed process per plugin, have one sandboxed process that runs all the untrusted plugin code; the entire real-time execution path.

DRM will never be supported although paid plugins are not out of the question, as long as the constraint is met that if a user wants another user to join their project, the other user is able to use the plugin with no restrictions. Project Network Users could browse published projects and samples on the peer-to-peer network. A sample is a rendered project, so if you want to edit the source to the sample you always can. Publishing a project requires licensing it generously so that it is always safe to use projects on the network for any purpose without restriction. The network would track downloads and usages so that users can get an idea of popularity and quality. Projects would be categorized and tagged and related to each other for easy browsing and searchability. So, one user might publish some drum samples that they made as projects, one project per sample. Another user might use all of them, edit one of them and modify the effects, and then create 10 projects which are drum loops using the samples. A third user might use 2-3 of these drum loops, edit them to modify the tempo, and produce a song with them and publish the song. Yet another user might edit the song, produce a remix, and then publish the remix. This project, sample, and plugin network should be easily browsable directly from the Genesis user interface. It should be very easy to use content from the network, and equally easy to publish content to the network. It should almost be easier to save it open source than to save it privately. License Genesis is licensed with the Lesser General Public License. A full copy of the license text is included in the LICENSE file, but here's a non-normative summary: As a user you have access to the source code, and if you are willing to compile from source yourself, you can get the software for free. As a company you may freely use the source code in your software; the only restriction is that if you modify Genesis source code, you must contribute those modifications back to the Genesis project. View Comments

Opening an Audio File 2015 Mar 15 Next I implemented a text box complete with mouse interaction. I made sure to get all the expected things working: double click, triple click, dragging, ctrl+backspace, etc. I implemented a file finder widget using rucksack for image assets. I used libav to open an audio file and display the waveforms. I implemented ability to delete a selection, save, and playback audio with PortAudio. At this point I had been using the default Unity Ubuntu user interface. I noticed that when vsync is on, switching to another window or switching back was really laggy. This sucks. I filed a bug for that, and then switched to XFCE where I am now much happier, and the lag is gone. I have some complaints about PortAudio. First of all, it spits out a bunch of garbage to stderr that you can't do anything about. So I'm already biased against it. Secondly it does not give you the low level control that you need for a DAW application. For example it doesn't have PulseAudio support so I'm accessing the sound driver via Genesis to PortAudio to PulseAudio ALSA Wrapper to PulseAudio to ALSA. And finally, I'm getting some crashes, and my favorite way to fix crashes is to delete code. So I'm turning to the PulseAudio API directly. This means that when I want to support OSX I need to create a CoreAudio backend, and when I want to support Windows I need to create an ASIO or DirectSound backend. I might also want to create an ALSA backend for Linux installations that don't have PulseAudio. I haven't yet investigated what the audio situation is for the various BSDs. Oh yeah and a lot of linux audio community members say I should only support JACK because PulseAudio will have too high of a latency. JACK support is planned but one of my design philosophies is that Genesis should be easy to install and run right off the bat without having to separately set up and run a JACK server. So the final strategy will be: try JACK, and then fall back to PulseAudio. But for now, PulseAudio is a build requirement. After I got zooming and scrolling to work, I decided that it was time to create the core audio pipeline and start working on something that lets us create music rather than just edit audio files. git checkout -b audio-pipeline View Comments

Progress So Far 2015 Feb 06 I started working on Genesis in March 2013. The first commit: commit 6d1699c42fce480c0b7ec84b521e088b09a3af32 Author: Andrew Kelley <superjoe30@gmail.com> Date: Wed Mar 13 01:38:29 2013 -0400 hello world diff --git a/README.md b/README.md new file mode 100644 index 0000000..01ae3a5 --- /dev/null +++ b/README.md @@ -0,0 +1 @@ +# Digital Audio Workstation diff --git a/main.go b/main.go new file mode 100644 index 0000000..893d327 --- /dev/null +++ b/main.go @@ -0,0 +1,7 @@ +package main + +import "fmt" + +func main() { + fmt.Printf("Hello, world

") +} Yes, that's right. I thought I was going to write it in Go. This turned out to be a bad idea. Here's why: Go is garbage collected, which makes its performance characteristics less predictable - a serious problem with real-time audio processing.

I would have to write an abstraction layer between any C libraries and Go code.

Other people who want to re-use Genesis library code would be forced to use Go or be out of luck. Sure enough the next thing I tried to do was use a GTK Go-binding library and find it broken and inadequate. I fiddled with that for a bit and then lost enthusiasm. But I picked up the project in January 2014: commit ff958caab890738334f660b058ded310b759206a Author: Andrew Kelley <superjoe30@gmail.com> Date: Sat Jan 25 23:19:16 2014 -0500 switching to C++ and Qt. main window, does nothing Qt solves a lot of platform abstraction for the programmer. You might even say it solves too much. The downside is that it is a hulking behemoth of a dependency with a nontrivial build process. But, it enabled me to make some windows and fiddle around with some code. I added the ability to list LV2 plugins and open a PortAudio stream. During this time I came up with some feature requirements: Never require the user to restart the program.

Robust undo/redo support.

Ability to edit multiple projects at once. Copy and paste between them.

Support for N audio channels instead of hardcoded stereo.

Tight integration with an online sample/project sharing service. Make it really easy to save the project open source.

multiplayer support - Each person can be editing a different section. And then I lost interest for a while. I spent a lot of time working on GrooveBasin, prototyped a Grappling Hook-Based Video Game, and entered Ludum Dare 30. In November 2014, progress resumed. Well, for some definitions of progress: commit 8799dcb4c20758bf546ba0d7cccfb60b2c7c3d5a Author: Andrew Kelley <superjoe30@gmail.com> Date: Sun Nov 2 15:49:01 2014 -0700 delete all source code And then: commit 95f2f251645548196604af9d4087d5b9e043a60a Author: Andrew Kelley <superjoe30@gmail.com> Date: Sun Nov 2 16:06:47 2014 -0700 hello, rust diff --git a/src/main.rs b/src/main.rs new file mode 100644 index 0000000..6686bd0 --- /dev/null +++ b/src/main.rs @@ -0,0 +1,3 @@ +fn main() { + println!("Hello, world!") +} Rust seemed like a cool alternative to C++ because it had fancy new programming language features like not requiring .h files. More seriously, the 3 problems with using Go are resolved with Rust, since it can call C code directly, C code can call it, and there is no garbage collector. Further, the safety guarantees it made were promising. But nothing in life is free. GTK is sufficiently complicated that a Rust "safety wrapper" is in order. The one available was not complete. At this time I began to have a new philosophy: Stop depending on libraries that hold the project down. This led me to ditch the idea of using GTK or Qt and instead code the user interface toolkit from scratch. This would work out better anyway if I wanted to sandbox plugins and provide a user-interface API for them to use. commit e7856aad61bcdaec657832807997244eff20a528 Author: Andrew Kelley <superjoe30@gmail.com> Date: Fri Jan 16 08:55:19 2015 -0700 goodbye GTK, hello SDL2 Wait, never mind, let's try GLFW. commit 401b3c99b4849fb65616190e41dbfbe32794c6c4 Author: Andrew Kelley <superjoe30@gmail.com> Date: Fri Jan 16 10:10:01 2015 -0700 goodbye SDL2, hello GLFW At this point I felt like I needed a direction, something to work towards. So I created a goal: Open an audio file with libav. Display the waveform of the audio file in the display. Next I began to experience Rust development. I went back and forth between GLFW and glutin for window library as one or the other seemed to be better. I created groove-rs - bindings to libgroove. I wrote my own 3D math library because the existing one's API was too hard for me to understand. I fixed a lot of code after updating to the latest Rust compiler each day. Finally, I got font rendering working. It took me 16 days, but I felt like I had learned enough about Rust to finally let me develop efficiently. And then I tried to abstract the font rendering code into a widget concept, and everything broke down. The Rust compiler has many false negatives - situations where it is a compile error due to safety, but actually it's pretty obvious that there are no safety problems. This is so frustrating and demotivating that I realized the benefits of Rust did not outweigh the slow development pace that I had taken on. commit 611d1afd7439761a1be8edb8c3f434ade0c7fcdb Author: Andrew Kelley <superjoe30@gmail.com> Date: Sun Feb 1 00:42:02 2015 -0700 rust is too hard commit c45c8c32c4a2f22112c4ee15d8a42b0ad5415e89 Author: Andrew Kelley <superjoe30@gmail.com> Date: Mon Feb 2 18:08:44 2015 -0700 delete all code, switch to C++ I felt guilty for allowing myself get distracted from actually making progress all this time. From now on I would be lazer focused and in general avoid depending on other people's code. If it's a bug in my code, then I know how to fix it. New roadmap: Load all the audio file's channels into memory. Display all of the audio file's channels in a list. Render channels graphically. Ability to make a selection. Playback to speakers. Delete selection. Undo/Redo Multiplayer So I knew I would be switching to C++. But I did not want to fall into the same trap in C++-land that I did in Rust-land: instead of solving the actual problem of making a DAW, trying to understand how Rust or C++ works and get my code to compile. I considered switching to C instead of C++. Linus Torvalds has some things to say about that. But some C++ features are too good to ignore, such as template data structures. For example, in C if you want to create a list data structure, you have 3 options: Make it generic using preprocessor directives.

Make it generic using void * instead of actual types.

instead of actual types. Don't make it generic. Reimplement the same data structure over and over for each type. This sucks. Preprocessor directives are the devil, Too much void * leads to runtime memory safety errors that the compiler can't catch. And what if it wasn't a simple list data structure, but instead a hash table or something? This is crazy, we can't reimplement the same data structure a bunch of times. Another example. In C you use malloc to allocate memory. It typically looks something like malloc(sizeof(MyType) * count) . If you forget to multiply by count or sizeof, oops. segfault. Meanwhile if you can use templates then you can define a function like this: template<typename T> __attribute__((malloc)) static inline T * allocate(size_t count) { return reinterpret_cast<T*>(malloc(count * sizeof(T))); } Now you can't mess it up, the compiler does the work for you. So I wanted templates from C++. And atomics from C++11. But that's about it. Other than that, I want to code like I'm in C. The beauty of C-style coding is that the control flow is so simple, it's impossible to not know what code is doing. Contrast that with fancy C++ stuff and you never know what's going to happen or what each line of code is going to do. I figured out how to have my cake and eat it too. I discovered that you can compile with g++ and link with gcc, like so: g++ -nodefaultlibs -fno-exceptions -fno-rtti -std=c++11 -c main.cpp -o main.o gcc -o main main.o As long as you don't use anything from the C++ standard library, you end up with working code that was compiled with the C++ compiler and does not depend on libstdc++. With clang it's even easier, just don't put -lstdc++ on the linker line. So I moved on, happy with this choice of language. Over the course of 4 days, I implemented, with useful, reusable abstractions: A UTF-8 encoder/decoder and String class.

OpenGL shaders for text rendering.

A hash table implementation with robin hood hashing.

A Label graphics class to render text, which uses FreeType to load a TTF file and caches rendered symbols in a hash table. At this point I felt crazy for even considering Rust. I had accomplished more in 4 days what took me 16 days in Rust. But more importantly, my abstractions were holding up. View Comments