The Secret of Good Electron Apps June 25, 2019

tl;dr Check out https://github.com/jlongster/electron-with-server-example

Some people really hate Electron apps. The idea that an app includes an entire copy of the Chrome web browser sounds ridiculous. This feeling is validated when looking at the apps on your machine — they eat up memory, boot slowly, and aren't very responsive. It's hard enough to build good apps on web, why the heck are we bringing the web to desktop and causing even more problems?

I won't spend time arguing for Electron, but the incentives are obviously there given the success of it. We don't want to accept Electron's bloat though. Can we have our cake and eat it too?

Some of Electron's problems (large file size, slower boot up time) are inherent in the architecture and need to be solved at a lower-level. The bigger problems (memory hungry and sluggish) can be managed in user-land, but it takes a lot of care to do so. What if I told you there's a secret that automatically minimizes these problems?

The "secret" is to do the bulk of your work locally in a background process. The less you rely on the cloud, and the more powerful you make your background process, the more you can reap these benefits:

Data loads instantly. You never have to wait for data to load over the network, it loads from a local data source instead. That right there will be a huge speed increase.

You never have to wait for data to load over the network, it loads from a local data source instead. That right there will be a huge speed increase. Little need for caching. Since all your data is available instantly, the client doesn't have to be concerned about caching so much. Usually web apps need to build up a lot of local state in order to have good performance, and this is one source of memory bloat.

This ignores other benefits like your app working when offline, but that's for another post.

This is how I built Actual, a personal finance manager. It's 100% local, and syncing across devices is an optional feature that happens on the side. For a data-heavy app, the results speak for themselves:

That's a total of 239.1MB of memory when the app is resting (it will go up depending on the page, but this is the baseline). Still not ideal, but not too bad either. That's better than the 1371MB that Slack is current taking up on my machine. Slack is an outlier though, and Electron gets too much flack for its specific problems. Other apps like Notion and Airtable sit around 400-600MB, so we're still beating that by a good bit.

And that's before I've really optimized anything. I plan on rewriting one of the critical and memory-intensive pieces in Rust which should significantly reduce memory usage.

The background server can optimize its memory usage by only loading the appropriate data into memory as needed. The best thing to do is to use something like SQLite which is already highly optimized for this sort of stuff (seriously, just use SQLite). Moving things into the background process also frees up the UI to be as responsive as possible.

It also turns that that using a background server in Electron also provides an extraordinary developer experience. See the next section for all the crazy things you're allowed to do.

Even if your app is heavily web-based, you might need a background process if you need access to node APIs. It's the only place where it's safe to access them. This project will still be helpful to you to learn how to set it up.

Introducing electron-with-server-example

In order to encourage people to create truly local Electron apps, I created electron-with-server-example to demonstrate how to set all this up. This is the project I wish existed when I was starting with Electron.

The technical details are documented in the README of the project. A high-level overview:

It creates a normal node process to run the server in the background

to run the server in the background It creates an IPC channel using node-ipc to communicate between the backend and the UI directly

using node-ipc to communicate between the backend and the UI directly If in development mode, instead of running the server as a background process, it runs it inside another window for debugging

Wait a second, what's with the last point? It runs the server inside a window in development?

That's right! Once I got the background process working, I realized that we have all of Chrome's developer tools sitting right there, and the ability to create a window with node integration. That makes it easy to load the node process inside a window instead of in the background, allowing full access to Chrome's developer tools for debugging.

Just look at all the cool stuff you can do now!

Use the fancy console

I added logging inside server-ipc.js to log all the requests and responses, and I can inspect them with the fancy console:

Loading...

Step debugging

Naturally, you can use the stepping debugger. It's not ground breaking, but it's nice that it's always there instead of having to go through the work of setting up the node inspector when you need it.

Loading...

Perhaps my favorite feature is that the stellar performance tools are always available just a click away. Easily profile any part of your backend.

Loading...

You can even profile the startup time of your background process (which is probably the heaviest part of bootup) by simply starting the recording and reloading the window! Reloading will restart the server, which brings us to the next point.

Restart the server with Cmd+R

Another one of my favorites is that since the server is running in the window, simply reloading the window will restart the server! Cmd+R (or Ctrl+R if you are on Windows) will restart the server and there's no need to restart the frontend process.

This means you can make changes to the server, hit Cmd+R and you'll get the latest changes, no other work needed! The frontend will keep whatever state it's in and you can keep working against the latest server. Kinda like hot-patching the server in a way.

In the below image, after I change some code I hit Cmd+R to refresh the server and continue using the client.

Loading...

Inspecting & hot-patching server live

Usually I just add console logs or make tweaks and restart the server, but sometimes you're debugging something really tricky, and it would be really helpful to dig into the current server state to see what's going on. Maybe even change a few things while the server is running to see how it effects the problem.

In the console, node's require is available! This means you can require any of your server modules and hack on them in the console. For example, to get the server-handlers.js module, just do let handlers = require('./server-handlers') in the console.

Let's add some server state: a history of all the numbers passed to make-factorial (real apps would have much more complicated server state):

handlers . _history = [ ] handlers [ 'make-factorial' ] = async ( { num } ) => { handlers . _history . push ( num ) … }

We can now inspect this in the console by requiring the module and then looking at handlers._history . We can even replace the implementation of make-factorial entirely at run-time!

Loading...

Conclusion

Checkout electron-with-server-example to read about the implementation details and see all the code for a backend server in Electron.

If you use Visual Studio Code, you might be used to good integration of developer tools with a backend node server. While you could run the server separately yourself and tell Electron to connect to a process owned by VS Code, I find it much simpler to reuse Electron's developer tools which are already there. This also means you get to use whatever editor you like and you have full access to all the tools including the profiler.

I have been heavily using the above techniques over the last several years to develop Actual, and I'm very happy with it. It's probably the best node development experience I've ever done.

It's also crucial that we start developing real local apps. All of the technology is there, we just have to care about it. This article focused on the tech benefits of using a backend server and developing a 100% local app, but it doesn't even go into other benefits like being available regardless of network connection. In the future I'll write more about this and how syncing works in Actual.