A crash course on Serverless with Node.js

66,835 reads

@ adnanrahic Adnan Rahić Dev/Avocado at Sematext.com. Co-Founder at Bookvar.co. Author of "Serverless JavaScript by Example"

Regardless of your developer background, it’s inevitable you’ve heard the term Serverless in the past year. The word has been buzzing around in my ears for longer than I dare say. For too long have I been putting off looking into it in more detail. Well here goes nothing.

reactions

Ready? Today we’ll go through the awesomeness and pain points of using Serverless. We’ll also define main keywords and topics that are crucial in getting started with the technology. Lastly we’ll go ahead and jump right into some code and write our own Serverless functions, emulate the environment locally and monitor performance! Keep in mind, you will need an AWS account to follow along with the code examples. Luckily, they have incredible free tiers so you don’t need to worry about breaking the bank when playing around with new things.

reactions

You were saying?

How come going Serverless is so cool all of a sudden? Is it good not to use servers anymore? I love servers, why not use them? Servers are awesome. You use the command line to tell them what to do. Why would anybody want to give that up? I was genuinely flabbergasted. But hey, taking a step back, I realize they’re not optimal. They’re a pain to manage in clusters. They don’t scale gracefully. These are only the first things that come to mind.

reactions

Let’s switch our mindset completely. Think about only using functions. No more managing servers. You only care about the code. Sounds rather cool. We as developers shouldn’t need to do the tedious work on the command line. Let the ops guys handle that. What do we even call this type of architecture? Only using functions? Small functions? Tiny services?

reactions

Functions as a Service (FaaS)

It’s called functions as a service, and it’s amazing. The concept is based on Serverless computing. It gives us the ability to deploy any individual piece of code, or function. The code runs and returns a value, in turn ending the process. Sounds simple, right? Well, it is. If you’ve ever written a REST API you’ll feel right at home. All the services and endpoints you would usually keep in one place are now sliced up into a bunch of tiny snippets, microservices. The goal is to completely abstract away servers from the developer and only bill based on the amount of times the functions have been invoked. Meaning services such as these are easy to scale.

reactions

But, all is not so sunny on this side of the fence. FaaS has been going through some teething issues. How do you think errors are handled? Not having a physical server to monitor is a bit of a mind bending experience. Having insight in your system is reasonably hard. Especially on a larger scale.

reactions

Wading into shallow waters

To get an understanding of how to write Serverless applications we first need to touch on the subject of what lies behind it all. The tools and services at our disposal that makes it all possible.

reactions

AWS Lambda

AWS Lambda is a compute service that lets you run code without provisioning or managing servers.

— AWS Documentation

Lambda is an event based system for running code in the cloud. You don’t worry about servers, only the code you write. It scales automatically and only charges you for the time it actually is running the code, the compute time. But, most importantly, it scales automatically! How awesome is that. No more worrying about if the EC2 instance you spun up is large enough to serve all your users.

reactions

AWS API Gateway

Lambda would be incomplete without the API Gateway. All lambda functions require an event to be triggered to invoke them. Gateway provides the REST endpoints which trigger the functions. Imagine you have the average Express app. You would usually create an

app.get()

reactions

app.get('/', function(req, res, next) { /* execute some code */ });

method for a particular route, like this:

When a user hits the

'/'

reactions

The Serverless framework

route an event will trigger the callback function. Gateway is the route, Lambda is the callback function.

Managing all of this is a pain. First time I tried it out, it took me the better part of a day to figure out what does what. With good reason. The docs are too complex, not beginner friendly at all. Serverless to the rescue!

reactions

Serverless is your toolkit for deploying and operating serverless architectures. Focus on your application, not your infrastructure.

The Serverless framework bundles all the tools you need into a manageable package, making it simple and straight forward to create and deploy serverless applications. It’s so awesome, it abstracts away all the tedious tasks you have to do in the AWS Console, such as creating functions and connecting them to events. The only downside is that you have to push code to AWS every time you wish to test your functions, while emulating the environment locally is a bit of pain.

reactions

The use cases when Serverless is the better choice are vast. Because of the easy scaling and low maintenance, any application you have in production where your user throughput varies rapidly is a valid contender to use serverless architecture. Lastly, if you suck at the Linux shell, and if DevOps is not your thing, you have every reason to try Serverless.

reactions

A new mindset

Serverless architecture is unforgiving. That’s a fact. Only setting it up takes a fair share of mental power. I’m not counting emulating it locally. That’s a whole other beast altogether.

reactions

The hostility requires us to change our approach. We have to live with the fact that we do not have overview of our whole system. But, humans adapt and overcome. In comes the Serverless framework like a knight in shining armor.

reactions

Let’s jump in and create a simple Serverless function.

reactions

Setting up Serverless is simple. You need to install it through npm and hook up your AWS account. Don’t worry, if you get intimidated by the AWS Console, it’s perfectly fine. I’ll break down the process and we’ll go through everything step by step.

reactions

1. First of all you need to install Serverless globally.

Fire up a terminal window and run:

reactions

$ npm install -g serverless

You’ve now installed the Serverless framework globally on your machine. The Serverless commands are now available to you from wherever in the terminal.

reactions

Note: If you’re using Linux, you may need to run the command as sudo.

reactions

2. Create an IAM User in your AWS Console

Open up your AWS Console and press the services dropdown in the top left corner. You’ll see a ton of services show up. Go ahead and write IAM in the search box and press on it.

reactions

You’ll be redirected to the main IAM page for your account. Proceed to add a new user.

reactions

Pick a funky name for your new IAM user and give the user programmatic access. Proceed to the next step.

reactions

Now you can add a set of permissions to the user. Because we are going to let Serverless create a delete various assets on our AWS account go ahead and check AdministratorAccess.

reactions

Proceeding to the next step you will see the user was created. Now, and only now will you have access to the users Access Key ID and Secret Access Key. Make sure to write them down or download the .csv file. Keep them safe, don’t ever show them to anybody. I’ve pixelized them even though this is a demo, to make sure you understand the severity of keeping them safe.

reactions

With that done we can finally move on to entering the keys into the Serverless configuration.

reactions

3. Enter IAM keys in the Serverless configuration

Awesome! With the keys saved you can set up Serverless to access your AWS account. Switch back to your terminal and type all of this in one line:

reactions

$ serverless config credentials --provider aws --key xxxxxxxxxxxxxx --secret xxxxxxxxxxxxxx

Hit enter! Now your Serverless installation knows what account to connect to when you run any terminal command. Let’s jump in and see it in action.

reactions

4. Create your first service

Create a new directory to house your Serverless application services. Fire up a terminal in there. Now you’re ready to create a new service. What’s a service you ask? View it like a project. But not really. It's where you define AWS Lambda Functions, the events that trigger them and any AWS infrastructure resources they require, all in a file called serverless.yml.

reactions

Back in your terminal type:

reactions

$ serverless create - - template aws - nodejs - - path my - service

The create command will create a new service. Shocker! But here’s the fun part. We need to pick a runtime for the function. This is called the template. Passing in

aws-node

reactions

5. Explore the service directory with a code editor

will set the runtime to Node.js. Just what we want. Thewill create a folder for the service. In this example, naming it my-service.

Open up the my-service folder with your favorite code editor. There should be three files in there. The serverless.yml contains all the configuration settings for this service. Here you specify both general configuration settings and per function settings. Your serverless.yml looks like this, only with a load of comments.

reactions

# serverless.yml service: my-service provider: name: aws runtime: nodejs6 .10 functions: hello: handler: handler.hello

The

functions

hello

handler

reactions

property lists all the functions in the service. You can seeis the only function currently in thefile. Theproperty points to the file and module containing the code you want to run in your function. By default this handler file is named. Very convenient indeed.

Opening up the handler.js you’ll see the handler module and function named hello. The function takes three parameters. The event parameter represents the event data passed to the function. The context tells us about the context of the function, it’s running time, state and other important info. The last parameter is a callback function which will send data back. In this example the response is sent back as the second parameter of the callback function. The first always represents an error. If there is no error

null

reactions

// handler.js module .exports.hello = ( event, context, callback ) => { const response = { statusCode : 200 , body : 'Go Serverless!' }; callback( null , response); };

is passed along.

This is all great, but we still can’t trigger the function. There is no event connected to it, hence no way to trigger the function. Let’s fix this. Jump back to the serverless.yml and uncomment the lines where you see

events:

reactions

# serverless.yml service: my-service provider: name: aws runtime: nodejs6 .10 functions: hello: handler: handler.hello events: # uncomment these lines - http: path: hello/get method: get

Watch out so you don’t mess up the indentation of the file,

events

handler

reactions

6. Deploying to AWS

should be directly beneath. Great, with that done we can finally deploy the function to AWS.

The deployment process is very straightforward. Within the service directory run this command in your terminal:

reactions

$ serverless deploy -v

You’ll see the terminal light up with a ton of messages. That’s the -v doing its magic. Gotta love those verbose logs!

reactions

But, most important for us is that it will log back the endpoint. Serverless has automagically created an API Gateway endpoint and connected it to the Lambda function. How awesome is that!? Hitting the endpoint in the browser will send back the text

Go Serverless!

reactions

Note: If you want to test the function through the command line you can run:

reactions

$ serverless invoke -f hello -l

This will return back the full response object as well as info regarding the state of the Lambda function, such as duration and memory usage.

reactions

Relieving the pain

It sucks that I have to deploy the function to AWS every time I want to test it out. Wouldn’t it be awesome if there was a way to emulate the environment locally?

reactions

With that awkward digression, voilà, Serverless Offline! Now I can finally test all the code locally before pushing it to AWS. That relieves a lot of stress on my back.

reactions

It’s surprisingly easy to add Serverless Offline to your services. Installing one npm module and adding two lines to the serverless.yml is all you need.

reactions

No better way to prove it to you than to show you.

reactions

1. Initialize npm in the service directory

Now you need to step inside the my-service directory and open up a terminal window in there. Once inside you can run:

reactions

$ npm init

2. Install Serverless Offline

With npm initialized there nothing more to do than just run the installation.

reactions

$ npm install serverless- offline --save-dev

The

--save-dev

reactions

flag will save the package as a development dependency.

Before moving on, you first need to let the terminal know it has a new command available. So within the serverless.yml file add a two new lines.

reactions

# serverless.yml service: my-service provider: name: aws runtime: nodejs6 .10 functions: hello: handler: handler.hello events: - http: path: hello/get method: get # adding these two lines plugins: - serverless-offline

3. Run it locally

To make sure you’ve installed everything correctly run:

reactions

$ serverless

You should see an option named

offline

reactions

among the various choices listed. If you do you’re good to go.

Note: If you want to see more helpful information about Serverless Offline run

serverless offline --help

reactions

in your terminal window.

With all that out of the way, go ahead and spin up the local emulation of Lambda and API Gateway.

reactions

$ serverless offline start

You’ll see all your routes listed in the terminal. Your Lambdas are now running on your localhost. The default port is

3000

reactions

. Feel free to open up a browser and check it out. Hitting the endpoint http://localhost:3000/hello/get will send back the same text as in the example above with the deployed function.

How awesome is this. Now we don’t need to constantly push code to AWS to see if it’s working. We can test it locally and only push when we’re sure it works.

reactions

Watching my back

In traditional applications when something breaks, you know about it. You also know where it broke. Got to love those stack traces! Regardless, the monitoring process of such apps is pretty straightforward. How does this relate to using Serverless? The logs on AWS CloudWatch are horrible. It took me an eternity to find failing functions for simple applications, imagine the horror with large scale applications.

reactions

What I found as a great alternative is Dashbird. It’s free and seems promising. They’re not asking for a credit card either, making it a “why not try it out” situation.

reactions

It takes 5 minutes to get up and running with the service mainly because of the great getting started tutorial they have.

reactions

Hooking Dashbird up with Serverless finally let’s me see what’s going on in my app. Pretty cool to have someone watching your back.

reactions

Errors are highlighted, and I can see the overall health of my system. What a relief. It also tracks the cost. Don’t worry about blowing the budget. Even real-time monitoring is included. Now that’s just cool.

reactions

reactions

Tools like this make it a walk in the park to manage large scale applications.

reactions

Wrapping up

reactions

What a journey. You have now witnessed the transition from traditional web development into the serverless revolution. With these simple tools we now have everything we need to create awesome, scalable, and reliable applications.

reactions

The only thing left holding us back is our own mindset. Realizing that functions are not equal to servers will be the turning point. But, we are going in the right direction. Tools like Serverless, and Dashbird ease the painful transition incredibly well. They have helped me a great deal on my path down the great unknown of serverless architecture.

reactions

I urge you to continue playing with these tools. Try to include them into your existing development process. You’ll feel relieved with how much support you suddenly have. It does wonders for the nerves as well.

reactions

If you want to take a look at all the code we wrote above, here’s the repository. Or if you want to read my latest articles, head over here.

reactions

Latest stories written by Adnan Rahić - Medium

Read the latest stories written by Adnan Rahić on Medium. Software engineer @bookvar_co. Coding educator @ACADEMY387…medium.com

reactions

Edit: The 22nd of December I published a video course on the subject of Serverless JavaScript. Feel free to take a peek below.

reactions

Serverless JavaScript by Example [Video] | PACKT Books

Become dexterous with live demonstrations on serverless web developmentwww.packtpub.com

reactions

Edit: The 2nd of January I published the follow-up for this tutorial. You can take a look at it below.

reactions

Building a Serverless REST API with Node.js and MongoDB

The Serverless movement has gained a bit of momentum in the past few months. Everybody seems to be talking about it…hackernoon.com

reactions

Hope you guys and girls enjoyed reading this as much as I enjoyed writing it.

Do you think this tutorial will be of help to someone? Do not hesitate to share. If you liked it, smash the clap below so other people will see this here on Medium.

reactions

Tags