Are you confused when scheduled Lambdas execute twice, SNS messages trigger an invocation three times, your handmade S3 inventory is out of date because events occurred twice? Bad news: Sooner or later, your Lambda function will be invoked multiple times. You have to be prepared! The reasons are retries on errors and event sources that guarantee at-least-once delivery (e.g., CloudWatch Events, SNS, …).

How do you know that your Lambda function is broken (or not idempotent)? If your function is given the same input (aka event) multiple times, the function MUST produce the same result. If your function produces different results with the same input, the implementation is not idempotent, and you are in big troubles.

You may ask yourself how to fix it? Let’s work with a concrete example. Imagine a Lambda function to ensure that a user can only make a specific number request per day. A request could be an upload to an S3 bucket, sending a message, whatever. In other words, the Lambda function implements rate limiting. To do so, you need to store some state. A good place to store the state is DynamoDB. Luckily, DynamoDB offers many features to fix your problem.

1. Iteration: The not idempotent implementation

The first iteration provides the most simple implementation. But also a broken implementation.

The input event looks like this:

{

"user" : "u1"

}



The function uses a DynamoDB table ratelimit with the primary key id (partition key). The id consists of the user and the current date (yyyy-mm-dd). Additionally, a calls attribute of type number is used to track the number of calls.

const AWS = require ( 'aws-sdk' );

const dynamodb = new AWS.DynamoDB({ apiVersion : '2012-08-10' });

const limit = 8 ;

exports.handler = ( event, context, cb ) => {

const date = new Date ().toISOString().slice( 0 , 10 );

const id = `iteration1: ${event.user} : ${date} ` ;

dynamodb.updateItem({

TableName: `ratelimit` ,

Key: {

id: { S : id},

},

UpdateExpression: 'ADD calls :one' ,

ExpressionAttributeValues: {

':one' : { N : '1' }

},

ReturnValues: 'ALL_NEW'

}, function ( err, data ) {

if (err) {

cb(err);

} else {

const calls = parseInt (data.Attributes.calls.N, 10 );

cb( null , { limited : calls > limit});

}

});

};



The implementation is not idempotent. The calls attribute is incremented even if the invocation is just a retry. The implementation would limit too early in this case. Let’s fix that!

2. Iteration: The mostly idempotent implementation

Let’s try to fix the first iteration. Add a request id to the event.

The input event looks like this:

{

"user" : "u1" ,

"request" : "r1"

}



All event sources provide some unique id that you can use as the request id. Some examples:

Kinesis: Records[].eventID

SNS: Records[].Sns.MessageId

API Gateway: requestContext.requestId

Scheduled CloudWatch Event: id