Photo by Chris Ried on Unsplash

Why I Wrote Redux in Lua

Redux on an ESP8266

Refactoring My NodeMCU for IoT

This article goes over my challenges writing Redux in Lua with the ability to use middleware. All I wanted to do was cleanup some of my own legacy code for the NodeMCU’s ESP8266 and ended up making something more complicated than it originally seemed or needed to be.

History

When I bought this house in summer of 2017, I started work on custom software for my smart home. Along with that came the need for embedded systems such as the NodeMCU (ESP8266).

My very first, and only project was to controlling my garage door. I wrote it in two separate versions using C and Lua. I bought a leaf switch designed for this sort of thing so my software would know if the garage is open or closed. I used that data to find out if I forgot to close the garage. I’m sure you’ve felt the same thing before too!

I never did buy a lengthy garage door cable \to set it up even though I’d written all the software, tested it, and had everything else ready to go. Not too long after, I changed jobs and stopped all personal projects. I figured it’d be easier to just buy a smart garage door opener instead and later put an extra Nest Cam in the garage just for that little extra piece-of-mind.

Smart Garland Lights

Fast-forward to Christmas Eve 2018. We were throwing the family Christmas party again. While I had all our Christmas trees setup with WeMo smart plugs on a schedule, my wife and I never remembered to turn on the battery-powered garland lights over the fireplace mantle. I figured if it was at least hooked up with Alexa, we’d be more likely to turn it on.

After some initial testing, I found that the two pins hidden away on the top of the battery compartment actually took 3V to power the lights. That was exactly what the two D batteries provided in the battery compartment. All I needed to do now was put a relay between one of the wires connecting to the two pins, and then I‘d be able to control the loop!

Refactoring

I took the old garage door software and started refactoring. The way I wrote the Lua version, it would load up a simple web server with 3 endpoints:

GET /status

GET /garage-door/off

GET /garage-door/on

I know that’s not super secure nor did it use POST or PUT for those commands, but it was still my original prototype. The idea was to setup a Node.js service with the fauxmo library. When you asked Alexa to change the state of this device, the Node.js service would call the on or off endpoint on the NodeMCU web server. If I ever needed to debug something, I could simply use the /status endpoint.

Redux Envy

While I was working on modifying my old garage door code to work with the garland lights, I started seeing how the old code was pretty unreadable and how using Redux would clean it up. Since I’d written a Redux-like library to allow reducers to be used in React components, I figured writing Redux for Lua would be a piece of cake. Oh how wrong I was!

First, my Lua is rusty. I never really learned it, and the last time I really used it was when I worked on Pulsen, my first and only game,years ago.

Second, embedded environments are completely different to our standard OSes. I was limited to 64K of RAM and debugging was troublesome since I had to listen on the serial interface. While listening to the serial interface I wasn’t able to upload file changes at the same time. Every time there was a bug, I had to close the serial listener, send over the updated files, then restart the device.

I ran into a few other issues like “how do I send back JSON over HTTP?”. Or better yet “how do I change this socket protocol into HTTP?”. The library I hooked into on the NodeMCU was using HTTP sockets directly rather than how Express.js handles it. The NodeMCU ROM does have a webserver library available, but I remember it causing issues last time so I avoided it.

Oh yeah, table merging was another problem. In JavaScript, I could use Object.assign or the spread operator, but in Lua, I was completely lost. I eventually found a merge function in a StackOverflow answer that allowed me to combine Lua tables without bringing in a memory-heavy library. It’s required when combining various parts of Redux’s state.

At some point, I started getting weird error messages. I later found those occurred because I was running out of memory. Normally, minification would help, but I wasn’t size-constrained; I was memory constrained. I ended up compiling the Lua files, then transferred them. Finally, I had enough memory to test. Sadly, in this mode I was unable to debug errors because every error was some cryptic message on line 1. Talk about a big waste of my time.

The other issue concerned file imports. I was using dofile early on, but realized that put everything in global scope. I later changed to using loadfile which, if you didn’t know, ends up looking stupid like this:

local actions = loadfile('actions.lc')()

Yes loadfile returns a function. You then have to call that function to get your file data. And I believe if your file doesn’t exist, you get back nil , the null equivalent. I just went ahead and assumed all my file imports were there since I put them there.

Because of the NodeMCU’s proprietary Lua code, I wasn’t able to test anything other than syntax using the Lua interpreter in my host system. There’s also no emulator or debug board available for the ESP8266 chip. If I had one of those, oooh boy!

I think I finally got to sleep around 6 or 7 in the morning. I literally could’ve just manually turned on the garland Chrismas day and been done with it, but nooooo, I had to go and write a suite of IoT software including an entire state management library! If I’d just done the simple refactor, I wouldn’t have had to deal with out-of-memory issues and would’ve been able to sleep much earlier. Oh well, now I’ve got an article to write and a Redux library I’ll never use again.

What About Redux in Lua?

So what’s this Redux implementation? What’s so great about it? It’s full-on Redux. It has the 3 basic functions: dispatch , getState , and subscribe . The subscribe function is easy to write because it just add subscribers. dispatch is just as simple because it just calls each subscriber one-by-one and doesn’t even pass in state. And then getState , ooh boy. function() return state end . Can you get any simpler?

No. In fact, it only got more complicated. dispatch does a lot more than just call subscribers. You gotta take that action, call all reducers, and create a new state . Sadly, I didn’t have combineReducers to help me so I had to write in a way to take a list of named reducers in a Lua table (think object props in JS) and loop through them, making sure to add that new state slice into the proper location in state .

Everything Was Wrong!

This is where my entire design of this system was wrong. I needed to add side-effects. Redux isn’t very useful when it only stores values in state; that’s why the subscribe function exists. In react-redux, the side-effect is updating component props. In this project, it’s altering GPIO and sending back web requests.

There are two kinds of side-effects in Redux. Middleware, for when you need to know the action that was called and potentially call other actions, and subscribers, for when you just need to know when state has updated.

Implementing a reducer store the current GPIO state was kinda useless since even my /status endpoint could just ask for the current pin state directly without Redux being in the picture at all. On an embedded system like this, that would’ve saved me a lot of time. It was at this point I had an epiphany moment: writing Redux for this project was even more useless than I’d originally thought. On top of that, I spent a lot of time working on it too!

See, because I copy-paste-modified the garage code, I didn’t think anything of it except rewriting what was there. Sadly, I actually didn’t need part of that code; the part that stored the state in memory. It was a poor design choice in my original garage door implementation that caused me to spend all this time unnecessarily writing Redux for Lua. If I’d thought about my original implementation rather than simply rewrote it — thinking it would be fun — to use Redux, I would’ve saved myself a lot of time, gotten to bed sooner, and been happy about my super simple solution.

Why Use Redux Middleware?

Since I was already at this point, I wanted to finish what I started. In fact, I hadn’t run into memory issues just yet. Those would start sometime during the development of Redux middleware.

I had a few options. I could write the GPIO side-effects as a subscribe method which requires creating a reducer and storing GPIO state in Redux’s state. Or, I could write middleware so I can write something similar to a reducer that only deals with side-effects. This way, I don’t need to duplicate storing GPIO in state. I chose the latter, but left my reducer in there anyway, doh!

In addition to GPIO, I also needed a way to return a valid HTTP response. I could’ve used the subscribe method directly in my webserver, but there was an issue. I needed to know the current state of the pin. To do that, I could simply return the current GPIO value. In this particular case, I didn’t need to wait for Redux at all!

With the other two endpoints, I could’ve gotten away with returning nothing but an HTTP 200, but I wanted to also return the current GPIO state only after you changed it for easier debugging. This meant I needed my webserver response to come from Redux middleware since I needed to listen for a SEND_RESPONSE action that included socket in its payload so I could do socket:send(httpResponse) .

It was way too complex for what I needed and because I needed to call sendResponse , now my Redux actions needed to include a socket prop. I knew this system was code smell, but it was so late at this time, I didn’t want to bother thinking about it.

Adding Middleware Capabilities to Redux

The fun part of this whole Redux project was going to be the challenge of adding middleware. The previous implementation in my React components article didn’t include middleware capabilities so the middleware code was just begging to be written. Even if it was Lua, I was gonna get that functionality in there!

An interesting thing about Redux middleware, it’s a function that returns a function that returns a function. All middleware run asynchronously to one another. To achieve this, you’re provided with a next callback. After a little bit of thought, I realized the callback was just store.dispatch .

When a middleware function wants to return a value, it actually just dispatches another action; otherwise, it returns void. The action could trigger a reducer to write to state, or it could trigger another middleware function like in my case. It could even return nothing at all! Redux middleware is really amazing.

This is what that looks like from the dispatch function:

middlewareFunction(store)(store.dispatch)(action)

And this is what it looks like writing middleware:

Why this isn’t named dispatch instead of next ? Makes me wonder if I’m not understanding something about Redux. No big; my implementation still follows the middleware API.

Feel free to checkout the rest of the complicated source code here:

Conclusion

I did it! I finally wrote the middleware component to Redux and in a completely different language to boot. All in all, it was completely useless. After looking online just now, other folks already wrote their own Redux libraries for Lua too and if I rewrote this project today, it wouldn’t include it at all. I could clean up that by code removing the complexity entirely, not by using Redux.

The important piece I learned is that simply doing the minimal amount of work isn’t enough. Sometimes it’s better to reconsider the entire architecture and design before putting it to code. Obviously my past self did something stupid, and I suffered in the here-and-now because of it. Sad, but I still wrote something cool!

While I was writing those two middleware functions, I was thinking how great it would’ve been to use RxLua instead of writing my own middleware. As soon as I ran into memory limitations, that wasn’t gonna happen. Oh well. It would’ve been really cool to write my own Redux-Observable in Lua.

More Reads

If you like what you read, you should also checkout my other articles; especially the ones on Redux: