Blog

28 January 2015 Dylan Lingelbach

Streaming Dynamo Backups

Announcing our streaming DynamoDB backup library

We recently opened sourced our streaming, Node.js DynamoDB to S3 backup library.

You may ask:

Why create a library for DynamoDB backups? Isn't DynamoDB backed up by Amazon already? Doesn't AWS have support for exporting data?

Yes and yes. However AWS DynamoDB just guarantees that Amazon won't lose your data. What happens if you have a script that deletes your user's table? What if you need to look at the state of the data 2 weeks ago to help reproduce a tricky bug?

You can use Data Pipeline to save DynamoDB to S3, but what if you want to limit the rate at which you request data? Or maybe you want to run the backup locally or you don't want a pipeline for every table?

This is where dynamo-backup-to-s3 comes in.

It is a simple npm module that allows you to stream DynamoDB backups to S3 with a lot of flexibility.

Even though this library is small it's been a huge help to us.

Check it out on GitHub. Feel free to open issues/send PRs if you have suggestions!

Filed Under: Tech Community