Still using AWS DynamoDB Console?

Try Dynobase to accelerate DynamoDB workflows with code generation, data exploration, bookmarks and more. Check it out .

The recent AWS Reinvent 2017 event saw some major announcements, including a couple of exciting ones for DynamoDB users. Global tables and On-Demand backups. In this post, I’ll be taking a quick look at on-demand backups, how they work and how we can schedule them to regularly take full backups.

What are On-Demand Backups?

On-Demand backups are a feature built into the DynamoDB service (Accessible via the API, AWS Management Console and CLI as usual), which allows you to take a full backup of a table at a point in time.

This task has no impact on performance or availability to your tables. All backups are automatically encrypted, cataloged, easily discoverable, and retained until you explicitly delete them.

Additionally, you can restore these backups to a new table at any point.

Along with data, the following is included in the backups:

Global secondary indexes (GSIs)

Local secondary indexes (LSIs)

Streams

Provisioned read and write capacity

The following is NOT included in the backups:

Auto scaling policies

AWS Identity and Access Management (IAM) policies

Amazon CloudWatch metrics and alarms

Tags

Stream settings

Time To Live (TTL) settings

What are the Costs like?

Charges vary between regions, but you are charged for storage of the backups only, which means the size of the table you are backing up. The price is approx 40% of standard DynamoDB storage costs. (Approx $0.10-0.11 per GB-month).

How does it work under the covers?

According to the AWS Docs:

When you create an on-demand backup, a time marker of the request is cataloged. The backup is created asynchronously by applying all changes until the time of the request to the last full table snapshot. Backup requests are processed instantaneously and become available for restore within minutes.

The above explains how the backups are taken without impacting performance.

How do you take backups?

Navigate to the DynamoDB Console Click on the new Backups option under Tables

Click on Create Backup

Select the table you wish to backup, and type in a name for the Backup. Click Create Backup Click okay on the success message

You should now see your backup



How do you restore backups?

In the Backups section in DynamoDB Console Select the backup you wish to restore, and click Restore Backup

Enter the table you wish to restore the backup to. This needs to be a new table. Click Restore

You should see the new table being restore back in DynamoDB



How do you schedule backups to be taken regularly?

Unfortunately, there is no native functionality that allows you to schedule these backups regularly, but they do recommend you to use CloudWatch Events and AWS Lambda together to trigger these backups. Lets see how we can set this up:

Create a new Lambda Function using NodeJS 6.10 using this code on github . (As at Dec 3rd 2017) Keep in mind, the native Lambda AWS SDK has not been updated to include the backup and restore APIs. Include node_modules/aws_sdk as part of the Lambda function code. This seems to be resolved now Use the following IAM Policy for the Lambda function Role Create a CloudWatch Event Rule Set a schedule for the backups to run (ie: Every 60 minutes) Set the target as the Lambda function Configure the input to be a constant (JSON text).

This allows you to configure backups for tables at independent intervals. ie: Seperate rules for each table / interval. Pass in constant JSON like this (seperated by comma):

Done! The Lambda should run as often as configured, and take backups of the configured input. You should see backups using the name format: [TABLE_NAME][YYYYMMDDTHHMMSS]

Final thoughts

Although a valid backup option for DynamoDB, it is worth validating whether this solution meets your needs.

Pros

No impact on performance

Backups taken in very little time

No limit on backups

Restores are very straight forward

Scheduling for backups is possible (including alerts when scheduling fails)

Cons

Storage of the backups can get expensive (compared to S3)

Backups are full snapshots only (no incremental options)

Not available in all regions (yet)

You can only restore to a new DynamoDB table

No ability to backup to a separate account (Essential for Disaster Recovery in case of security breach)

There is news that a new Point in Time backup / restore option will be available in DynamoDB in early 2018, looking forward to seeing that as a more complete solution. My big hope is that it will provide the ability to move the backups to an isolated account, for DR scenarios. Fingers crossed!

HTH