Summary

As software lifecycles move faster, the database needs to be able to keep up. Practices such as version controlled migration scripts and iterative schema evolution provide the necessary mechanisms to ensure that your data layer is as agile as your application. Pramod Sadalage saw the need for these capabilities during the early days of the introduction of modern development practices and co-authored a book to codify a large number of patterns to aid practitioners, and in this episode he reflects on the current state of affairs and how things have changed over the past 12 years.

Your data platform needs to be scalable, fault tolerant, and performant, which means that you need the same from your cloud provider. Linode has been powering production systems for over 17 years, and now they’ve launched a fully managed Kubernetes platform. With the combined power of the Kubernetes engine for flexible and scalable deployments, and features like dedicated CPU instances, GPU instances, and object storage you’ve got everything you need to build a bulletproof data pipeline. If you go to dataengineeringpodcast.com/linode today you’ll even get a $60 credit to use on building your own cluster, or object storage, or reliable backups, or… And while you’re there don’t forget to thank them for being a long-time supporter of the Data Engineering Podcast!



Preamble

Hello and welcome to the Data Engineering Podcast, the show about modern data infrastructure

When you’re ready to launch your next project you’ll need somewhere to deploy it. Check out Linode at dataengineeringpodcast.com/linode and get a $20 credit to try out their fast and reliable Linux virtual servers for running your data pipelines or trying out the tools you hear about on the show.

Go to dataengineeringpodcast.com to subscribe to the show, sign up for the newsletter, read the show notes, and get in touch.

You can help support the show by checking out the Patreon page which is linked from the site.

To help other people find the show you can leave a review on iTunes, or Google Play Music, and tell your friends and co-workers

Your host is Tobias Macey and today I’m interviewing Pramod Sadalage about refactoring databases and integrating database design into an iterative development workflow

Interview

Introduction

How did you get involved in the area of data management?

You first co-authored Refactoring Databases in 2006. What was the state of software and database system development at the time and why did you find it necessary to write a book on this subject?

What are the characteristics of a database that make them more difficult to manage in an iterative context?

How does the practice of refactoring in the context of a database compare to that of software?

How has the prevalence of data abstractions such as ORMs or ODMs impacted the practice of schema design and evolution?

Is there a difference in strategy when refactoring the data layer of a system when using a non-relational storage system?

How has the DevOps movement and the increased focus on automation affected the state of the art in database versioning and evolution?

What have you found to be the most problematic aspects of databases when trying to evolve the functionality of a system?

Looking back over the past 12 years, what has changed in the areas of database design and evolution? How has the landscape of tooling for managing and applying database versioning changed since you first wrote Refactoring Databases? What do you see as the biggest challenges facing us over the next few years?



Contact Info

Website

pramodsadalage on GitHub

@pramodsadalage on Twitter

Parting Question

From your perspective, what is the biggest gap in the tooling or technology for data management today?

Links

The intro and outro music is from The Hug by The Freak Fandango Orchestra / CC BY-SA

Liked it? Take a second to support the Data Engineering Podcast on Patreon!