Electron , Javascript , MacOS ,

Electron , Javascript , MacOS , Tags: chrome, electron, javascript, macOS, soundflower



Building desktop applications used to be tedious in the past. With each platform having its own programming language, and a different way of doing things. Windows, Mac, Linux… We used to have java with “Swing” which was great and cross platform, but building apps with this was not really fast. Then JavaFX came which was better. But nowadays, we have awesome tools which allows us to leverage the power and ease of use of web technologies to build desktop apps. Electron provides us such capabilities and what is awesome, is that it is cross platform.

Recently, we’ve had a situation where I needed to record system sound on a mac alongside video to channel the resulting stream over WebRTC. The application was a desktop app, and obviously built with electron. And as most of you might know, electron apps run chromium and leverages web technologies to build desktop apps.

The Problem: How to Record System Sound on MacOS?

To stream a desktop in real-time with audio and video on electron, you will need the “desktopCapturer”. Then, precise constraints as required by Electron. Though this approach will work on Windows, it will not work on MacOS. MacOS will simply not allow the system audio to be captured, but has no problem with allowing the video stream after permissions are granted. This is a bug that has been present for years now, this post is about how to handle this issue.

This is a known issue and while searching, I came across several complaints about this issue. Some date back to a few years ago. Here you can find an open issue about Chrome not providing a possible means of recording system audio without external drivers. This report dates back to 2016, now as I’m writing this, we are in 2019. Digging deeper into this, I found that it was due to MacOS. The operating system didn’t allow system sound to be captured by default.

In my opinion, the fact that there is no proper solution to this is because the number of developers who need to implement system audio capture functionality into electron apps is not really significant. I also think it is important to highlight a point about various solutions to this issue found online. Most of the solutions I’ve seen developers bring up, were hacks which aimed at mixing two streams from different sources and mixing them. That is, capturing video from the desktop, capturing audio from the microphone. Then mixing both. This is an example of what I’m talking about. Though this is the approach we are using in this post, we will get video and audio streams both from the system.

The Solution

After trying a bunch of possible solutions, we finally came across a final solution. Though it requires a fairly complex installation of third-party software, it works and permits us to record system sound on MacOS. Let’s dive in.

If you like this post, you can chose to follow me on Twitter or Github and subscribe to this page’s notifications to stay updated with new articles or like my page on Facebook.

Installing Sound flower:

Sound Flower is an extension to the MacOS kernel, it is open source and it’s made to create a virtual audio output device on MacOS. To install soundflower, you need to follow these instructions. I recommend you follow the instructions attentively till the end. Because, after installation, you have to setup the software so as to enable it to capture MacOS audio.

Record System Sound With Javascript Sound Flower Installation

After the installation of SoundFlower, you can notice with your JavaScript that new audio output devices appear in the OS. As you can see below. The audio output device highlighted is the one we are interested in, and which we will use to record sound system on macOS.

Soundflower audio Devices

Record System Sound on MacOS with Video

Here are the steps we will use to record the system sound.

Get every media device on the computer

Filter these media devices and get only the sound flower audio output device we highlighted above,

Extract the audio stream from the sunflower virtual device we filtered out.

At this point, you have succeeded in recording system audio on MacOS, but we will add Video to it.

Extract the video stream

Mix the Video stream with the previous audio stream

Get the resulting stream and perform what ever you want with it.

Here is the code performing the steps I mentioned above:

//Getting user devices: navigator.mediaDevices.enumerateDevices() .then(devices => { //... //We filter the device which we are willing to get let audDevices = devices.filter(device => { return device.kind == "audiooutput" && device.label == "Soundflower (2ch)" && device.deviceId != "default" })[0]; //We get the user media corresponding to the audio device we are willing to get navigator.mediaDevices.getUserMedia({ audio: { deviceId:{exact: audDevice.deviceId} } }) }) //The next step is to get audio stream from the sunflower virtual device, //then mix it with the video stream we are already getting. .then(audioStream => { navigator.mediaDevices.getUserMedia({ audio: false, video: { mandatory:{ chromeMediaSource: "screen" } } }).then(videoStream => { let audioTracks = audioStream.getAudioTracks(); //We mix audio and video streams together if (audioTracks.length > 0){ videoStream.addTrack(audioTracks[0]); } //... Now you have a video stream with the appropriate audio stream "do what you want with it !!!" }) }) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 //Getting user devices: navigator . mediaDevices . enumerateDevices ( ) . then ( devices = > { //... //We filter the device which we are willing to get let audDevices = devices . filter ( device = > { return device . kind == "audiooutput" && device . label == "Soundflower (2ch)" && device . deviceId != "default" } ) [ 0 ] ; //We get the user media corresponding to the audio device we are willing to get navigator . mediaDevices . getUserMedia ( { audio : { deviceId : { exact : audDevice . deviceId } } } ) } ) //The next step is to get audio stream from the sunflower virtual device, //then mix it with the video stream we are already getting. . then ( audioStream = > { navigator . mediaDevices . getUserMedia ( { audio : false , video : { mandatory : { chromeMediaSource : "screen" } } } ) . then ( videoStream = > { let audioTracks = audioStream . getAudioTracks ( ) ; //We mix audio and video streams together if ( audioTracks . length > 0 ) { videoStream . addTrack ( audioTracks [ 0 ] ) ; } //... Now you have a video stream with the appropriate audio stream "do what you want with it !!!" } ) } )

You might be interested by this post about building offline web applications with Javascript using Service Workers and IndexDB.

Conclusion

We have seen how to record video, alongside system sound on MacOS with Javascript. Though this method works well, you can tweak it so that it get’s only sound or so it does whatever you want. If you have a method which is more efficient in recording sound and video in with JavasScript, please share in the comments section.

References

https://rogueamoeba.com/freebies/soundflower/

https://github.com/electron/electron/issues/8589