About this talk

Watch how AWS Lambdas execute NodeJs, and if NodeJs' strengths are still effective in such an environment.

Transcript

Thanks for inviting me. I don't think I've attended any JS Roundabout events but I've basically been, this is what I wanted to talk about, NodeJS on Serverless. And hopefully, I don't know who is using Serverless. Is anyone using it? My guy. I have a friend. Okay. No, I'm all friends with you. I'm Ulpian. I do a lot of things. Most of the stuff that I do is programming. That's about it. I know I have Twitter and GitHub links but email is probably the only one that I really respond to. This talk really is about raising more questions than giving answers so I'm sorry about that. This is not about the Serverless framework if you have checked that out and if you have sort of seen it. And it's also learnings in the last year. They have leading a team. We've been doing this project and we just decided to say, Let's just use Lambdas for some reason, and it has quite a lot of things like DevOp stuff for a code talk, so bear with me if you really really really hate DevOps. But it's so good. So we started this project and we decided to go with AWS Lambda because for a set of reasons. I mean it's modular and scalable. We didn't want to maintain a lot of things, we wanted to free up developer time. Also the project was run partly by a team that was an agency and they kind of wanted to make sure they could reuse code and take this to other projects and also because it was cool. That's the other reason I have to list. And so usually what we have is, when we're setting up a project, we usually have something that kind of looks like that. I know that's very very simple, but it has everything in there. Maybe a source, maybe some docs, has a nice text folder, it's all in one place, our index JS God file is there and everything. So that's kind of what it looks like and that's what the team had been used to, that's what kind of we were being used to, and that's what, we knew that wasn't going to be the case and we were gonna move on. So we looked at AWS Lambda, this is the kind of response we got, which is AWS Lambda is a compute service that runs your back end code in response to events. My first reaction was... That's kind of everything. Everything responds to an event and everything is a compute. So anyway, what it really kind of is is essentially it's a cloud platform and it's running a sort of series of EC2 machines. In the background it's running Amazon Linux, it's the Linux kernel for v4 plus. That basically essentially manages the execution of containers, Linus containers. And container being an execution environment. And what it does is it takes your code, it basically puts it in one of these containers and you have a whole sort of configuration around how you wanna run this code and how it should be. It basically starts that up and tries to freeze it in case you have subsequent sort of requests coming in to try and make it as efficient as possible. However this is erratic because it doesn't know when it's going to die, it doesn't know when it's going to live either so it's kind of, you can't really depend on it but that's how it runs. So low low level, that's essentially how it kind of is as a summary. So it's essentially, the idea is that you can write your code with a team and not care about how this is gonna be configured in any real way. You can connect DNS and everything but in terms of the server and in terms of all the configuration around that, all the maintenance around that and all the scaling around that, you don't really need to care about that. You just write some code, some logic and it's going to sort of run. The problem is that that little piece where they say everything else runs fine, that piece is not totally true and I wouldn't trust anyone that tells me everything is gonna be fine. But essentially the cool thing is that events can be anything, it does run your code pretty well, actually. And those events can be HTP requests events from other Amazon services that they run, it could be a JSON file. It supports v6 of Node. And also very importantly, usage is measured in increments of 100,000 milliseconds. That's basically the execution time of the Lambda itself, so if you, essentially, if you have your code, you have some logic that's gonna run that, it basically takes in a request. The event could be an HTTP so you have an API service. That starts executing and then it's measuring per 100 milliseconds. So if you have a long process or if you have any kind of bottleneck or anything there that's gonna start costing you money. So when we looked at Lambdas we knew that this was gonna have to be a sort of microservice approach. We knew that we were gonna have to build small pieces of code that can communicate with each other. This definition here, which I think is pretty good, says microservices is an approach to application development in which a large application is built as a suite of modular services. Each module supports a specific business goal and uses a simple well-defined interface to communicate with other sets of services. That I think pretty much rounds it off. I know there's no real consensus on sort of what microservice is and everyone has a general idea. I think that pretty much rounds it off and I said okay, well lets go with what these guys are saying. But of course that is pretty much a kind of really idealised situation. And so what we found out was that first of all, from the idea of microservices and from the services that were on Amazon it kind of mapped really well. So if you're not aware of all the services on Amazon or what they do, they have a pretty good service called API gateway which is essentially like an HTP server load balancer. It does caching as well, it's very very powerful. And it's quite easily configurable. Then you have the, what API gateway calls APIs, which we believe map to resources pretty well in the routes and then the Lambdas themselves could be basically endpoints in an API. So if you think of this as a sort of vanilla rest kind of structure for an API. You can say okay, I've got my resources, I've got the endpoints that are gonna execute. Each of these endpoints kind of serves a specific business goal or a specific sort of action that can be well-defined and everybody will be fully aware of how to sculpt that and how to write that code. And then you have things like BPC for security and so on, so it kind of mapped pretty nicely. And so what happened is we started going from that big kind of code base to something a bit more like that. So you have this Serverless folder which essentially bundles up your code base and it puts it in there. That's what actually gets sent to AWS. And this time what we ended up having was the handler.js file, which is the sort of JavaScript that will actually hold the logic for that specific function that's gonna run in the cloud. And we have this source folder which is like our services and models. So we broke up the code base into kind of what we were defining as modules so that one resource would be one module and all the endpoints would be written as separate Lambdas in the handler. I know that says hander but it's handler. Then I set a standard for the classes and the services and the models, and each model has its own repo. So take into consideration by the way that this specific project wasn't Google, right? We weren't trying to figure out, like we weren't building Deep Mind, we weren't trying to create this amazing thing, it was actually a pretty small project. And the aim of this actually initially was to get into AWS Lambda because it was a small project. And I know that there is this counterintuity about if you're doing microservices you've got to have a big team or you've got to have at least a fairly big strong team that doesn't mind cleaning up things and maintaining things all the time, which is the kind of problem that we went into. But the aim was that if we built this kind of microservice that we didn't have to care about, then it could be something really beneficial for any other project that we'd be building or any other kind of thing that you would need. If you have a Lambda that does authorization for you, if you have a Lambda that does image manipulation. Joe you're mentioning like you have Go, which is running all this encryption stuff, then you can basically split it out across this whole cluster and it does a very specific thing and you call it whenever you need to. So that was initially what we were trying to do. And we went with that microservice approach with the hope that okay, we know what we're doing, we're gonna make this work. So this is kind of what a handler started to look like. This is pseudo, but we basically, because we were creating the models and some of the modules that we were creating were sort of in their own repos, they were NPM packages. Some of them were privately held. We thought that was the best way that we could essentially use this. So that's essentially what we did. And we have these services, we have even a sort of wrapper for the handler because Amazon basically gives you the handler function that you write gives you a context and that's about it and so we wanted to do some other things, like we wanted to, and you mentioned you didn't like generators, but we put COE in there 'cause it's version six. And we started to realise actually that we're running an API services, and when you run an API service you need things like a database, you need things like image manipulation services, you need your models to come in and do the validation and all that other stuff. And so we were starting to realise, and this is a year back so there's things that have changed, but we started to realise that we need to do essentially what I would call hacks to make this work right. Like wrapping up the entire function as a COE and then giving it to Amazon to deal with. It's not the best case scenario, but we needed that because we didn't want to work, it would start complicating the situation when we started working with connections and database objects and other IO operations and you have to remember there's no backbone to this so this is completely being held on Lambda, like from beginning to end. There's no ingle EC2 machine that's orchestrating anything. So it started to look like this where we're like injecting services on this, we're wrapping a generator and then the response, the self the call back would be would hold some other dependencies that are not really clear here. So things are getting really murky. And there's issues. I know this talk is really pessimistic, but it will get better. So some of the problems around this, some of my favourite ones anyway, is the connection pooling problem. So if you imagine that you have this platform that's spinning up these containers, well you won't have anything that lasts very long. So we have an unpredictable state of executions for the Lamdas, the blew up our connection pool. We had a Postgres database. I mean, you can imagine any other kind of IO operation and we hit this test, we were running all of our tests on the API and it just went pew, it just went away, it was 500s everywhere. So what we had to do was then actually build the connection and tear it down every single invocation on the Lamda, which again, feels bad, you can realise I haven't had a good sleep for a whole year. And the other one was VPC and cold starts. So essentially the other problem is that Lamdas go cold because they cost money to run and when they go cold what I mean is that the container which was first created is gonna go away. Because if you're not like Facebook or something, you're not going to get requests every single second or every 10 seconds or whatever it is. So they essentially go away and they need to start back up again when you make another request that comes in where there are no containers. So you imagine all these containers are being spun up, they die down and then somebody enters your website and wants to do something, that hits the API and then they've got to wait because that container's gotta spin up and to run that kind of operation does take computing power. So this isn't a smart solution and believe me, in the Serverless community there is not a smart solution for this. We basically ping it every five to 10 minutes. We're basically annoyingly nudging this as much as we can, and we keep pinging every five to 10 minutes to try and keep at least one container open so there's an invocation. So you can imagine our sweet little hearts at the beginning of this project thinking oh we're going to save money, and then we get to the point where it's like wait, so we're pinging it anyway, so we're automating it EC2, anyway, it gets dark. But a VPC cold start takes up to 10 seconds to warm itself up. So if you have these Lambdas, if you have this whole infrastructure that you've set up and you have a VPC around that 'cause you really wanna be secure, then it's gonna take up to 10 seconds to warm itself up which is slow, right? We get freaked out if it's 800 milliseconds or 500 or something, so 10 seconds is a long time. The other thing is, and I wanted to put some screenshots of people having problems on stack overflow and so on where they're like, oh my God, I wrote the perfect code but it comes back 500. And it's like well, the problem with that is the way that these containers run, the way that Amazon runs your Node.js process is that it obviously simply runs Node as it is. And what it does is you can change the execution time, however the way that Lambda realises that you're finished with that Lambda is by actually checking the call stack and the event loop. And if you have something in there, then what ends up happening is that it blows up because it's past its execution limit. And so the problem is that if you imagine a developer who's kind of just used to opening up a database connection, hoisting it above the application code and its some code that's checking for a pool or checking for something on the side, that's gonna keep jumping into the event loop. And it's gonna keep sending things in the call stack saying is it alive, is it alive? So Amazon comes, Linus comes to this point where it's like well it's never finishing but I have this hard limit and I've got to kill it and it just returns to 500, the API gateway's returning to 500. Your actual Lambda is dead and buried. So there are these kind of problems that I think people hadn't quite realised because they kind of shoehorned the concept of just a machine or a server into what is actually a container. And they get these problems. Then there's this, I'm not even gonna talk about this. If you have another talk, maybe I can give? 'Cause this is a long one. Yeah, testing is horrible. Let's just leave it at that. So this was getting annoying because there was a lot of issues and I think actually the problem is the way that we think about microservices, the way that we think about code and API services actually in general, and one of the reasons, when I was speaking to Phil as well where I mentioned to him actually why it's written on the event that the benefits of Node.js and why I was questioning that when it comes to Lambda is because actually Lambda and, I personally think all sort of microservices, are very good for executing some programme, some series of instructions for a very specific reason. And in a way, node is a fast sort of engine, you can execute code very quickly but node actually is one step before that. It's a really good job queue. It's an orchestrator for you. And so one of the problems was that when we were getting to put this up on AWS Lambda we weren't realising that what we were doing, we were putting already a very good job queue into a wider infrastructure that acted as a job queue. API gateway in a way is a certain queue. It takes certain requests and it does this really really well. So that's one of the conceptual problems that I think we were getting at. And I think that Node.js still is Amazing to use in Lambda and you should still do that. It's just that the way that you use it, and the way that you think about how the service is actually running, and to Amazon's credit they do mention that AWS Lambda is good for running web services in a paragraph, in a very vague product-selly paragraph. But most of the time they're actually talking about image manipulation, they're talking about encryption, they're talking about CPU intensive tasks. So actually you can imagine yourself building a web server with Node.js and then having any IO operation be some Lambda that does some heavy task. In our case were trying to run the entire API service on just Lambdas. And so that I think took a bit more lessons than we realised. So what I think is going to happen and I think what is happening right now is that the majority of developers that are using Serverless applications, they won't be going through the microservice approach because it's hard and we're lazy as people. So what they're going to do is still leverage Serverless architecture and the way that they're going to do that is for example, very recently a tool came up which is what you're seeing over there called UP. It's written by TJ because he writes everything. And basically it's essentially a proxy service. So if anyone's used Heroku, this is your way of doing Heroku like in, I don't know, 10 seconds, five minutes, something like that. You can have an entire express.js or koa.js sort of server running and what happens is it packages that entire application without any other configuration for Serverless or microservice sort of setup. You're running it as it is and it packages that up and then it basically proxies the request that you would get into your express.js or koa.js app, and it basically shoehorns this into a Lambda and it takes this out and deploys it. Now the reason why that works is because it actually turns out that it doesn't matter how many Lambdas you have. If you have one Lambda it's fine because ultimately AWS will spawn these out as containers. So if you have your entire application in one Lambda, it'll spin as many containers needed for that. Now there are caps for this by the way. I think it's something like a hundred, a thousand requests per second. There's a cap and I think if you call them up or something then they will increase this for you. But the point is that you only need one version of your code and it'll still function fine. If you have 10 Lambdas or if you have one, the scaling is the same. So that's one reason why it'll work. But really importantly, so basically what these guys are doing in the AWS Lambda world is useless because all we need is one container, not all of them. But this brought me this question and this point that it changes the way that we're writing JavaScript. And it changes the way that we're sort of designing our web services. And then the logic that we ultimately write is very different as well because the way that we're writing dependencies and the way that we are sort of pooling in libraries and so on becomes really different. I mean, when you're writing a web server and you have that one instance and the machine is going to be running fine, you can hoist your dependencies, you can pool connection, you can do all these things and you know it's going to be cached and you know it's gonna be fine. That's not the case now. And we have to think much more stateless, we have to think much more functional in order for this to work. But I think that people, as I said, because we're lazy, I think we're just gonna build out application the way we've been doing now and use a tool like UP to basically just push it up into Lambda and think that we're gonna be fine. So looking back on this, there was a sort of, like I've been thinking about a sort of proposal of what we could do. And just thinking about how, if we are sort of, if it's too hard to maintain the large series of code bases around modules to build our API service, how do we make a lazy way to make that possible? So I think that the way we do that is by we have to build functional stateless code bases and is anyone here aware with IOC, inversion of control? Right, so basically the idea is that we build out classes for example, or objects. And we push out the control not in the class itself, but in the module that we're requiring. If you have a code base and you're using a concept like IOC, it means that you can list out your dependencies, you don't really care where they come from, you're just injecting your dependencies, they come into all your classes, and then you purely care about the logic. It's still one code base and I'll show you an example of this. It's all one code base. But behind the scenes we can take out those modules because they're so cleanly decoupled and we can do the kind of boring and also just long work of taking this out and putting it on a Lambda and then doing the rest of the configuration stuff. So APIs that are sculpted around resources, Lambda handlers can become the endpoints. The gateway APIs can take those as resources the same way that we've done it. It's not a bad idea, it actually worked very well it was just really laborious. And also monitoring and metrics are easier, they make more sense when Lambdas are split. There are a couple of small projects that are using that tool which I showed you, UP, it just takes your entire thing and proxies it, but the problem is error handling becomes a huge issue and there's just now way to see what's actually happening, monitoring becomes an even bigger problem. Proxying services like that, they add 500 milliseconds to the round trip, up to, most of the time it is. When people say up to 500 it's like yeah, it's always there. So they always add that to the round trip, and also, I don't know, there's something dirty about the idea of shoehorning an entire application in Lambda, maybe I haven't got around to that yet. Also the nature of Node.js fits better with this concept, it was actually really interesting seeing your point about CSP because there's, I think things like that can definitely be used to kind of inspire the thinking of how do you get these two pieces to work together? How do you get them to communicate but do quite specific things? And also to use Node in its sort of maximum. So if you will allow me some shameless advertising, the people who are doing this project is the reason, they're hiring developers, they're looking for good developers, they also write good code. So you can visit that GitHub link. SLS Press is basically a framework that will allow you to, not in the way that I showed you by proxying, so you take a whole application and deploy it, not that way, but I think a more efficient way. They allow you to set up routes if you're used to express.js, you're used to this, you have a handler, you set up all your routes, you write all your functions in the way that you would with express.js, except that all of this goes to AWS Lambda and you start leveraging the beautiful world that is AWS. I hope I haven't put you off by my pessimistic talk about how bad AWS is, 'cause it's actually really good. But I'm just trying to show the minefields, and I'm also just trying to sketch out the thinking that kind of inspired me to think about well actually, where do we go with wall this? And I use Golang as well for example, and Golang's perfect, you can write a series of instructions, it runs very fast, it's very good. But I think when it comes to Node.js as JavaScript developers we, and I'm not trying to like, which all due respect, we're kind of all over the place when it comes to conceptually thinking about things because that's kind of the power of Node.js that you could potentially do anything you want to with JavaScript, but there is a danger to that in which we kind of murky the waters of the concepts of programming when we get into specific things like this. And that's exactly what happened with us. And so that was the talk, I hope there's a lot of food for thought and questions and so on. If you have any questions I'd be happy to take them. Thank you.