About this talk

How do you go offline with your app? In this talk we'll show you how to utilise Redux Offline to make your app offline-first. Optimistic updates, background synchronisation, and more.

Transcript

Jani: Hello. I'm trying to use my outside voice. How's everybody doing? Audience: [crosstalk 00:00:08] Jani: Yeah! Yeah! All right, so what we're gonna do today, slight change to the plan, instead of talking about Redux Offline, I don't know if anybody has seen this on Twitter, but basically, Max Stoiber, the guy who talked at the last reactivate meet up. He just recently reached his 20th thousand tweet. So I figure, he has a lot of material, everybody loves that kid. So we [00:00:30] might as well read his tweets aloud for half an hour. So ... Audience: [crosstalk 00:00:34] Jani: So, what I did, is that I wrote this little funny script that we can ... pop into the Twitter or into the console ... but my full screen doesn't really ... All right, lets do that ... (whistling) ... I've done this mistake [00:01:00] before. Hey! There we go! So what I did was I just turned Twitter into a presentation machine, so we are going to read random Max Stoiber tweets. "Update, 200 files later, styled component is still ... awesome." This guy tweets a lot about styled components, no? "This is how I got started in public speaking ..." This is pretty boring right? I think there is this other guy though, I don't know if you've heard of this guy, he's called Ken. Audience: [crosstalk 00:01:31] Jani: [00:01:30] Maybe he has some better Twitter material. (whistling) All right, so ... Audience: [crosstalk 00:01:46] Jani: "So Netherlands airport security frisked my two year old daughter ..." Oh, no, I'm ... All right, well his mom just sent him a snap, I don't know if I wanna see that. Oh movie reviews, [00:02:00] of course, "Hackshaw Ridge, holy fucking shit. You're fucking no repo having ass mother fuckers. Audience: [crosstalk 00:02:06] Jani: All right, so this was just a fun little thing, we're actually going to talk about Redux offline, I'm not insane. So, hi everybody, my name is Jani, I work at a company called Formidable Labs. You might have heard of us through out open source things that we do. We built Spectacled, the presentation editor, which I'm not using here because I just can't live without these cool animations [inaudible 00:02:29]. [00:02:30] But we also, we're also for consultancy, we build things in React, React Native, Node, all that kind of stuff. What I'm here to talk about today is building offline first apps with Redux Offline. So first a little show of hands, how many people here have actually built an application that you could call offline capable. All right, so that's a few hands, a few kind of like shaky hands. So this is kind of like a new field in way, but I believe it's a very important one and it's gonna hit you sooner or later. [00:03:00] So just for context, I started on writing offline capable React apps a couple years ago when I got introduced to React Native, which is basically just a way of making Native mobile apps with React. And I was really happy as somebody who considered themselves this web developer, to be able to ship a Native mobile app. And we did and we high fived and we had a lot of beers, and then we started to get first reviews. And it was like, why doesn't this app work offline, why's it telling me that you have no network connectivity, [00:03:30] I don't care I just want to see me stuff. So what I realised is that we did make it look like a real app, it had a little Tinder swipe thing and everything that everybody loves, but it didn't behave like a real mobile app should in that it works in variable network conditions. So, that was a mistake that I kinda took hard, and I took it home and I kind of thought about it for awhile. And now a couple years later, I've kind of decided to do something about it. The second thing why this is very important today is that there are new features and new capabilities [00:04:00] that web browsers are able to have thanks to these AP [inaudible 00:04:05] service worker. How many people here know what a service worker is? All right, so I think that's pretty much everybody, but if there was some person who I didn't see ... It's basically a progressive enhancement for mobile apps that allows you to run the script on the background that can do things like intercept network requests and do thing can do things in the background when your app is not open. And what that enables us to do is make web apps that work offline in mobile browsers. Unless you have [00:04:30] an iPhone, in which case, you know, it doesn't work because: Apple. So this is the kind of context and to build ... why would we want to build offline first applications. Well there's a lot of good reasons, I think the biggest one for most of us, the one that is easy to sell to our bosses and east to understand is that, in mobile context, poor network connectivity can actually lead to poor user experience and having apps that is able to work in variable [00:05:00] network conditions, but also longer periods of offline use, that's just going to improve the [inaudible 00:05:06] of the app. One example is that if I'm using Twitter for instance and I'm on the tube, I wanna like something and I don't want to be told, "Oh, you can't like this." I want to do that action and then later when I come back from the tube, that will eventually be synchronised to the back end. And I don't need to know about that, so that's a good user experience. But there's a whole nother set of users, like all [00:05:30] over the world, not necessarily here, in what we used to call the "developed world". But you know, where we have better network access. But there's a whole, another grade of users who have incredibly poor network access and they still might want to use your services and as a business they are important customer segment that you can access if I can make apps that work on their devices and their networks. So, that's another reason. Anyways, so lets talk about how might one go about building an offline [00:06:00] first app. And remember this is not in context of React, this in not in context of even web browsers, this is in context of how do you actually do this? So the first thing that you might think, like, "OK if network is a problem, how 'bout we check if we're online, and then it's we're online we do something, and then if are offline I do something else." You might easily see why this is not a great architecture, and is not a great [inaudible 00:06:24] pattern that we'll scale, but also at the same time it's truly impossible because networks aren't really query-able in that [00:06:30] sense and even if they were, between the time that you query and the time that you try to do the action, that reality might have changed. 'Cause you know like that's not at time of operation. So that's out of the window. Then you go, okay, well if we're gonna fail anyway, lets just do a fall back. Let's do something, try it, if it doesn't work out, tap into some sort of fall back store or some kind of fall back cache, and read stale [inaudible 00:06:54] from there. And then when we do get new data, put it in the cache. And I think this is a reasonable strategy [00:07:00] for a certain class of application, and in fact service worker makes this really easy, because all the service worker APA-dos is intercept requests and you can do this. But it makes it really hard to do right operation, like eventually capture an operation that needs to happen and then eventually make that consistent into another service. And it's also, like most of time, network is not really the only thing you want to have react when something happens, it's also the application state. So you think, "OK, well application state, I wanna actually do something to my app [00:07:30] state when I'm offline, or when I try to do a request that will not succeed. So, maybe we can push this database level, maybe we can essentially just store stuff into our database and then eventually when we come online we can synchronise that database." And there's honestly really good tools for this, there's Pouch DB, there's Realm Mobile Platform, there's so many great tools for this. But I think, one big problem with these is that essentially, it requires you to have a bespoke backend or a bespoke database receiver [00:08:00] on the other. And it's not really realistic that within the companies you work at that, and certainly not in the companies that we as consultants at Formidable work at, where we can control the entire ecosystem around our app. Like we need to be able to talk to varying data sources. And finally, databases are kinda complicated and if something doesn't go right, it's really hard to understand why and it's also really hard to debug or fix those issues, because you're so tied into a product that you have no control over it. And then finally I wanted [00:08:30] to shout out, how many people were at the React London conference? A few people at least. So there was Andrey Sitnik talk about Logux, which is a solution that ... for the same problem. But essentially, it takes it a lot further and it decouples that data storage and that eventual consistency model by shipping these transaction logs and using events or [inaudible 00:08:52] Super cool, check out Logux, Andrey is a great developer. But for me I wanted something a little bit different. [00:09:00] So, lets talk really briefly about why is this so hard? Why am I talking about this, shouldn't this kind of be an easy-ish thing? And I don't know if you know this meme, or this idea that, back in the day of document oriented websites, being web developer's really easy, you get a request, you get some data, you template some HTML, you send it over, that's your responsibility, done and dealt with you don't need to do anything. And I think that that's kind of [00:09:30] a little bit like false ... what's the word for that? Anyway, it's kind of like false idea and it's nostalgia. And I think in a couple of years we'll be nostalgizing about the days of the single page application, the days of when we only needed to manage app state for one app session lifetime, and not between app session lifetimes. And this is where the complexity comes in, essentially as in universe, entropy always [00:10:00] increases, chaos always increases and same thing happens for your application. Things happen in between and makes it really hard to manage over time, things like, what if your app code changes, but you need to [inaudible 00:10:11] to read back an older state? How do you make sure you can migrate that, essentially creating a distributed database. There are some hard problems in there. So, if you start looking at offline first apps, you kind [00:10:30] of like stumble, I at least stumbled 'cause I'm not a big thinker, I'm not a big academic reader, so I kind of stumbled on all these requirements accidentally one by one. And they fall into few categories but essentially you want network resilience, you want the idea that when a network request fails that you are able to retry it and you are able to do it in poor networks. You need to be able to persist your app state, you need to able to read data from a cache but also do the right operations that we talked about. You, essentially there's two models, there's [00:11:00] optimistic UI updates which is that you tell the user the thing went through even if it didn't, like Twitter does with the "likes", or sometimes you need to tell them, "Okay, well this didn't really yet go through, but we'll notify you when it does." And there's the two modalities that have slightly different requirements when it comes to building the offline system. There's also many different ways that you can load data, either lazily, like when you request something you put in cache sometimes. You might have a use case where you need to preload bunch of data so a user can go complete their [00:11:30] journey or complete their, whatever thing they're trying to achieve over time, like if they work in a, like if they're professional and they work in a poorly networked area or what not. And yeah there's a bunch of other stuff, complexity, you might want to background sync your state, because if the app user doesn't use the app very often, you might, it might, he might, he or she might go offline and stay offline until the end of the use session, and then that data doesn't get synced until months later when they're [00:12:00] returned, so you might want to actually background sync it. And we already talked about the versioning problems. So, there's a lot of stuff that actually arises when you start building these things. So what I wanted, what I would like in my life, is generally a tool that is easy to use, so it doesn't require weeks of configuration. I want something that works with any back end, because that's the reality for most app developers, whether mobile or web or whatever, it's that you don't always control the full ecosystem around it. [00:12:30] Because I'm a React developer and this talk was written for a React Conference and this is a React Meet-up, I wanted to be familiar to the developers on my team, so they don't need to learn an entirely new way of thinking of things. And finally, this term that I'm trying to launch: "reasonaboutable". I don't know if you know the meme, easy to reason about, like "Oh, functionable programme makes things easy to reason about." So, basically I wanted something that follows those principles, that you can actually [inaudible 00:12:56], you can debug it, you can, like you don't need to be a genius to understand [00:13:00] how it works as long as you have a [inaudible 00:13:02] and some patience. So, we already have this tool in our toolbox that most of you know. Should I even ask? How many of here know Redux? Hey! So Redux is a tool that most developers use or at least know how to use if they're not currently using it. So there's this interesting idea a friend of mine, Vile, he tweeted that Redux looks far [00:13:30] more useful when viewed not as an app state management solution, but as a protocol where different libraries can operate on shared state. So essentially, if you think of actions and get state and middle wares [inaudible 00:13:40] all that, that's just the way, it's a protocol for different libraries to try to achieve different things to make sure that they're not stepping on each other's toes by enhancing and adding behaviour on your state container. And this is why I turned to Redux when I wanted to make these apps work offline. This is a complicated drawing and I'm not going to walk you through it, [00:14:00] but basically, the really basic idea of the architecture that I actually wrote, like three different times for three different apps, just iterating on it because I didn't know how to generalise it at the time. It's basically the same idea, you fire, you do something, your user fires an action, you put it in the store, and essentially what you do, is when you're offline or even if you aren't you put this action into outbox, which is a special branch of your store, and then you have a middleware [00:14:30] that is essentially just reading stuff from that outbox and sending it over network. And that middleware has a really nice, that's a really nice place to encapsulate things like retry and network resiliency and all that. And finally, there is essentially lot of stuff around it, so cache pruning for instance, so making sure that your state doesn't grow too big. Scheme of migration, so when you're versioning your application and you're updating code live. Network listening, so you're not using [00:15:00] the user's battery trying to make network requests when they're clearly not going to succeed. All these kinds of things. And then finally, persistence, which is that your Redux stores, essentially a big [inaudible 00:15:10] tree, so why could we not, instead of having a database solution, having SQL light, having something, why can't we just take this store, serialise it, put it onto disc and then when we need it later we can just rehydrate it. So, this is the basic architecture. That's also the architecture of Redux offline, which is [00:15:30] probably not surprising given the title of this talk. So this is a library that essentially take this pattern that I've written a few times and I think we're kind of like, I'm getting to a place where I'm happy with the pattern, so I decided to put it in a library so we can start iterating on it, instead of rewriting it every time. So, really quickly. I don't want to look at too much code, but basic idea is that that state container, that is the protocol for different libraries to inter-operate on the same state, already has [00:16:00] a mechanism for extending it, and that's called enhancers. So what we can do is we can just import this offline enhancer from Redux Offline and then we can import some configuration and then we just build in Redux method called compose to basically just plug it into our store. And there I have [inaudible 00:16:18] middleware and offline, but you could have Redux dev tools, you could have sagas, this are composable with different libraries. And this software slightly simplistic. That configuration [00:16:30] that you saw that we imported there, that's like a sensible default that's going to work, 90% of it is going to work for 90% of the use cases, but there's, I can guarantee that every app will need to customise some of this. So instead of having a deep configuration API, I just basically wanted to have a simple API where you can override any individual part of the fairly complicated chain of events that needs to happen for this offline synchronisation to work. But so that you can still [00:17:00] use the basic orchestration layer and the basic components around Redux offline without having to throw it away and, "Oh fuck it, I'll have to rewrite it myself 'cause it doesn't do that one thing that I need." So, here's the thing, we can change how we detect network, how we persist the state, how we cause effects, effect is the magic word that just means, you want something to happen in another system somewhere, do an HTP request, whatever. The how do you retry things, and how do you know that when something is not going to [00:17:30] succeed, how do you know that you need to give up. So you can discard an actions. So, that's basically the entire configuration API and the type signatures here are in flow type, which Redux Offline has written entirely with flow type. And the offline action is kind of like a magic thing that we're going to look at next. So the idea here is that when your user does something and you fire an action, you typically have in Redux, you have this pattern that you do something [00:18:00] and then when it succeeds you do something and if it fails you do something else. Normally you can just do this as promise chains, you can do this as call backs, you can use whatever middleware you want, but when your state needs to be persistent, when these actions need to be basically saved into a disc loaded later, what we need is continuable transactions. So we can't use anything memory based, we can't use anything that creates temporary application state and then continues on that. So instead what we're gonna do is that we're gonna describe the entire context that is needed for [00:18:30] the application to be able to eventually preform this action, when it can. The three things that you can define [inaudible 00:18:39] offline key, which is just the thing that you can add to a vanilla Redux action, or the effect, which can be literally anything, here we have an example of a HTTP load that can be passed directly to fetch, but it can honestly be anything. And then you have action that is the commit action, which is something that you will do when the request succeeds [00:19:00] and then you have rollback which is something you do when it fails, when you give up, like I'm not longer going to retry this. With this you can actually implement quite sophisticated things, we talked about pessimistic, optimistic, all these different things, transactional state. What you can do for an optimistic scenario, when you handle your initial follow user request action, you do the optimistic update and when you rollback, you roll it back and tell the user something happened, you don't even need a commit action because [00:19:30] you've already optimistically updated it. For a pessimistic one, you follow user request, you can set a loading spinner or whatever indicator saying that this action is not yet completed and then in the commit action you remove that, and again in the rollback action you hide it and tell the user something went wrong. So it's a very very simple API, but it really enables all the different patterns that I've so far discovered. This of course looks a bit verbose and people say [00:20:00] that Redux is verbose, people complain about this. And I think that one thing that a lot of those people miss, is what everybody misses when they try to measure something by lines of code and see how good it is or how bad it is, which is that this is explicit this is simple. We don't use Redux in order to make really simple things super easy and fast to do. We use Redux to make things that are really really complicated to make them explicit, make them simple, make it understandable. [00:20:30] But it doesn't need to be this complicated, we can essentially simplify that a little bit, SO if in Redux you're using action creators that are rolled back, it's just an action creator that recall eagerly in order to populate the action into the meta tree, instead of calling it later. And the effect, we can use, you can have whatever function that just returns those descriptor object. And this actually, even though we do things like post, and follow user error, we aren't actually causing any effects in any other system where it's return descriptors that will [00:21:00] later be handled by middleware, which is the basic idea of Redux anyway. And then when you fire this action your reducers don't actually change at all. If you have an app that already knows how to do the typical triad of do something and then roll it back later, or do something and commit it later, that same reducer will work with this action, it just will do it later. So, I believe this is fairly simple. This has gotten surprisingly popular, so this is a couple of days ago, [00:21:30] Redux Offline, 3000 stars. I don't know what the current exchange rate between GitHub stars and US Dollars is, but I'm pretty sure I'm ready to retire soon. Audience: [crosstalk 00:21:38] Jani: No, I'm just joking, I'm only setting up a joke. 'Cause there's of course haters. This is Marcel, who we'll be talking about next. Audience: [crosstalk 00:21:46] Jani: Marcel, like every popular project attracts it's hater, like Taz here ... Anyway, that was just light interlude after lot of boring code. [00:22:00] Let's talk a little bit, how am I doing for time? Okay I have like five minutes. So I wanted to talk, 'cause offline is something that is really close to my heart, but as we saw in the beginning, not so many people raised hands when asked, "Are you already doing this?" I'm going finished the whole offline portion by just now saying that, if this doesn't seem relevant to you today, fine, and I'm very happy for you, 'cause this is not always fun. At the same time, lot of these requirements [00:22:30] will become mainstream, will become understood by the industry, and it will trickle down to the engineers as well, eventually. So this is something that most of us will have to do eventually. And I, honest to God hope that Redux Offline is not the solution for it. I honest to God hope that this is something that we begin with and then somebody comes up with something a lot better. But anyway, that's the stuff about offline. In order for it to, the general sort of interest, technology meet-up interest, I wanted to talk a few things that [00:23:00] I have learned about, not only when writing this library, but also over the years, accumulating the experience and the failures. What can we learn as software developers about this? So the one thing, really really hard trying to coin ... everybody go tweet reasonaboutability right now. So the idea, is really that I'm not smart, I am not super human, what I can do though is use this [inaudible 00:23:29] bugger and as long as [00:23:30] the software works in a way that is both predictable and modifiable without wrecking it and it doesn't have a lot of magic involved in it, then I'm really happy using it, even though it contains a lot of code. I'm still relatively young, relatively healthy, I can type, but I can't think better than I can. The other thing is familiarity, the idea is that this API hopefully, is familiar to already any team that wants to adopt it, which is great for [00:24:00] anybody who wants to make a product or a tool that people will adopt. I don't really have any vested interest in anybody adopting this, what I do have a vested interest in is, when I use it in my own team, is that I don't need to teach a bunch of people. And sometimes we as developers we feel that, the more extraordinarily snowflakey solutions that we make the smarter that other people will think that we are and I can guarantee that that is not true. Nobody will like you any better and I'm sure you're all lovely people and very likeable as it is, but nobody [00:24:30] will love you better because you decided to implement your own state machine or whatever in some way. Just do the thing that people are already doing. Next thing about flexibility, so the idea that you make something work for everybody is extraordinarily difficult and it also will drive you into the mud in terms of clarity or vision and doing the thing. But at the same time, figuring out what are the highly divergent needs of people and trying cater for them with a simple as possible [00:25:00] API. I think that will is a generally a really good software engineering pattern because it makes your code a lot more robust and long lived, instead of like, "Oh, I just have to rewrite this and throw it away." Finally, humility. I'm not a very humble person, but in terms of code, humility, it's like don't try to do everything, don't try to solve the world, just do one thing and do it well and know where to draw the limits, and also be honest about it. If you are, if the thing [00:25:30] that your making has some trade offs, don't try to tell people it doesn't. If it doesn't solve something and you claim that it does, then you're just going to end up looking foolish, no one will buy that, people are very evidence based in this industry. This is not a term that I'm trying to coin, I just wanted to put the "ity" there, because it made a really, really nice, kind of like ... Audience: [crosstalk 00:25:49] Jani: So, documentation. One thing that I in fact spend, I think, more time writing the documentation for Redux Offline than I did the first version that we released, [00:26:00] or I released. I think that it's so frustrating that when you see that somebody has solved their problem or has purported to solve a problem, but you don't know how to use it. Or maybe the have API docs, but you don't actually know how it's supposed to be used. And people using a great tool wrongly is still not a good outcome. So, write good documentation. Focus on the whole user experience, like when you write software, user [inaudible 00:26:24] software, you think about the whole user journey, you think about the UI text that go along with the product, [00:26:30] because that's actually the product. In the same way, documentation is part of software, it's not an external thing, it's not a secondary thing. If the documentation is not great, I don't care how brilliant [00:26:41] is, is basically what I'm saying. So, I think that's pretty close time. You can ... Oh God ... These things were going so well, no. So you can download Redux Offline from GitHub, or NPM, NPM is [inaudible 00:26:58] the Redux Offline. The documentation [00:27:00] in the repository, like I just talked about, its seen quite a lot of love, so it's hopefully something that, even it you don't think that this library is great for you, but if you want to understand the problem better, understand the trade offs and the challenges, just reading through that documentation will not take more than ten minutes, but it's written in, what I hope is readable and concise prose. So I do recommend reading it. There's also a link to a blog article. Blog article ... that's not even a thing. Its a blog. That basically described everything that I've said today, if you want to send [00:27:30] it to your colleagues, or whatever, like that's linked. You can follow me on Twitter, where I'm mostly shit posting these days, very little technical content, but you can see that. And Formidable Labs, I'm not even going to the "we're hiring" thing. I'm just gonna say that, we're very excited to be here in London, we're an American company originally, and London is such a crazy place, full of amazing software developers, and amazing community, I'm really really happy to be here. And if you do need any Reacts developed [00:28:00] then you know who to call. All right, thanks. Audience: (Clapping) Jani: Do we do a Q and A? Speaker 3: Yup! Does anyone have any questions for Jani? Yes, in the green [inaudible 00:28:23] Jani: OK, you can moderate. Speaker 4: So [inaudible 00:28:25]. Based on your [00:28:30] experience, when doing the offline is better to have the UI tell something about Ahmed is offline or it's just like, should be signs ...? Jani: Oh, that really depends on the context and the domain. If you're trying to buy a ticket to a show, you probably want to know when you actually got the ticket and not and say, "Hey you got a ticket!" But then turns out you didn't. Not a great idea. Twitter like, nobody really gives a shit if it doesn't eventually go through, like maybe your friend or our mom will be hurt, "I sent that dog picture, but [00:29:00] you didn't like it." No, I did I did I swear. So maybe that's the worst thing that can go wrong. It's case by case, but generally speaking, if you don't need to tell user boring things, like network or loading indicators, don't, because that's not what they care about. Speaker 3: Yes. Speaker 5: How does this fit in in your mind to things like Apollo and Relay in the sense of [inaudible 00:29:25] I think one thing you mentioned is that you don't control your API, so in that case, obviously you [00:29:30] can't use Graphcool but like how does this relate to that ... Jani: Really interesting question. So today, Redux Offline works perfectly in scenario, well perfectly, but works in scenarios where you're only doing data reading by Graphcool but you're not using mutations. For mutations, the problem isn't even Graphcool, you could actually, like that effect payload can be a Graphcool mutation that you eventually send, it just doesn't work with Relay or Apollo currently. Speaker 5: Right. Jani: I would really want this to work with Apollo, and also Relay modern looks really cool. So that would be something that [00:30:00] I would look at, but I don't have enough knowledge. So if anybody is interested in solving that problem, I'm really happy to take contributions. Speaker 3: Come on. Jani: Come on, don't be shy Speaker 3: There are no dumb questions. Jani: [inaudible 00:30:17] Speaker 6: Do you have like a roadmap, or anything that you'd like to improve? Jani: Honestly, so my roadmap right now, one tip that I would give us all software developers is that, don't release something into the wild when [00:30:30] it's the busiest month of your life. So, not generally speaking a good idea, so right now, my roadmap is surviving May. And then after that I think I'm going to do another iteration where biggest thing is going to be fixing bugs. This is new software, the production patterns behind it are super robust because I've actually shipped them to production, but the library itself is still a little bit sketchy in many ways. So I think that is the first one. After that Relay, supporting sagas officially, which we kinda do, kinda don't, [00:31:00] so that's a big thing. So, yeah. All right. Speaker 3: All right thanks Audience: (clapping)