TypeORM Testing with Docker + GitHub Actions

How to approach unit, integration and E2E testing with a TypeORM/TypeScript/GraphQL API (Updated 2/1/2020)

Photo by Startup Stock Photos from Pexels

Introduction

Figuring out how to write automated tests for an ORM can be really difficult. There are many different ways to approach the problem and good explanations can be hard to find. This article is about my experience developing ORM tests, strategies, and how to get them working with GitHub actions (CI). I’m using TypeORM but the ideas in this article should be applicable to other ORM implementations.

I’ve been working with TypeORM for just over a year, I used it on a large commercial application, I’m using it again in 2020 with a Node.js TypeScript/Apollo GraphQL starter kit I’m developing called BrainStrike. I wanted to make a “nirvana” project with a tech stack that I believe in, some of the components are relatively new, so I had to deep dive into several technologies. BrainStrike is on GitHub, it’s a relatively simple educational project. I will be sharing more when it’s ready.

As a web developer, it’s always good to have a working recipe for developing modern applications, and it’s important to keep your tools up to date… stack decisions are incredibly important. I’m a huge fan of GraphQL and TypeScript, so those were obvious choices, I chose TypeORM for database connections because it is feature-rich and has good support for TypeScript.

I had several goals for this project:

Develop a full-stack example using all modern components. Server API + Client. Have it fully typed with absolutely minimum use of escape hatches like the any type. Working, practical end-to-end type safety. GraphQL + TypeScript are both typed languages, both go well with code generation. It should have a great developer experience. TypeORM modeling and GraphQL API’s offer a very good DX. I intend to learn more from developing the kit. (I will evaluate the quality of the experience when it’s done, but I’ve already learned a lot about what works and what doesn’t). Working examples of unit, integration, and end to end testing (written in TypeScript) for both client and server.

Today I’m going to be writing about that last goal. Testing.

All projects have different requirements, I don’t think it’s necessarily smart to deploy this kind of stack for every project and you should make considerations based on what development resources you have available, e.g., time, cash flow, team makeup, etc. I work on software for enterprise applications. I want to practice on a stack that will be viable for use in production, by teams, for several years.

This article assumes you have an existing project with Node.js and TypeORM, you may need to install Docker and get a free GitHub account if you don’t already have one. My examples are written in TypeScript.

What ‘s an ORM?, What is TypeORM? and why should I use it?

TypeORM is an Object/Relation Mapping tool for JavaScript and TypeScript. An ORM (Object/Relational Mapping) is an abstraction for interacting with databases, what’s really nice about TypeORM is that it supports modern JavaScript features and has type safety with TypeScript.

Most projects I’ve worked on over the last 10 years have not used an ORM. I had become accustomed to rolling my own mapping solutions because traditional ORM solutions had fallen out of favor.

I used ORMs in the 90’s/early 2000s. They developed a reputation for having bloated interfaces, leaky abstractions, and poor performance. I’ve been burned in the past, so I still have some fears about using TypeORM. The software industry is littered with broken promises, but for database modelling, things have improved tremendously.

Its still a complex problem, you have to stick to relational structures but the intention here to get my hands dirty and find the edge cases using a modern framework. I’ve lowered my expectations about what an ORM can do.

You should consider TypeORM if you have to interact with databases using JavaScript/TypeScript, especially if you’re starting a new project. If you need special database-specific features or need to integrate with an existing solution there will be some challenges, you may need to migrate your database or write your own query builders if you want bleeding-edge performance but TypeORM generated queries are relatively well optimized and improving.

GitHub Actions + TypeORM + Docker

What is testing?, What’s the difference between Unit, Integration and E2E tests— Why do we do it/ Why do we want it?

Automated testing is a huge topic that is beyond the scope of this article but I want to give a brief primer on the subject so that you have context for why it is important.

Unit tests are designed to automate testing of a single unit of code in isolation. Usually, you offer some inputs and test whether you’re getting the right outputs. Consider if you’re working with a team, and somebody changes something that introduces a bug in a particular unit. Your build system can be configured to prevent that code from being submitted with appropriate feedback in an error report.

Integration tests are as their name implies — they involve testing the interactions of various units integrated rather than separately in isolation.

E2E tests are sometimes referred to as functional tests, or browser tests or UI tests or smoke tests (yes, it’s annoying how many different terms there are). E2E tests are defined as testing the complete functionality of a system. They run slower and often simulate real user interaction, so in the case of a browser app, that involves simulating a user clicking around and interacting with an app. Here we’re testing a GraphQL API.

I’ve worked at companies with little to no automated testing and that was very challenging. In the last decade, testing has become more emphasized due to the Agile methodology.

It’s a point of frustration for developers. I’ve had Agile “coaches” come on to legacy systems, that are overcome with technical debt, advocate testing as a catchphrase (because it’s in the manifesto), pretend they’ve solved the problem just by suggesting it, then proceed to ignore automated testing and prioritize customer-facing features because it makes stakeholders happy, disregarding developer happiness and compounding technical debt… That strategy will inevitably come back to bite you, but they could still blame the developers, after all, they did advocate testing!

I’m wary of Agile obsessed managers if they haven’t evaluated the systems they’re managing. The software has to be as Agile as well — good leaders are hard to come by and if they’re simply pushing the manifesto, that’s another red flag. It means the responsibility will be shunted to the developer. It’s revealing when a manager has contempt for developers. That’s poor leadership, bosses are an antiquated concept in this millennium. Leaders should have technical strategies for testing, be helpful in the process, and allocate time for both automated testing as well as QA testing (if it’s available).

If you are trying to add testing to an existing system, it can be very challenging because the system could have been developed hastily/poorly with no consideration of testing, making effective testing nearly impossible.

Testing should be part of your initial design considerations and not an afterthought. I’ve learned that all code submissions where coverage changes should have appropriate tests because code written with tests in mind is different than code without.

You tend to create more appropriate abstractions when you know the code has to pass tests, especially if you have to write them. If the developer/manager is just trying to get a feature up with no tests, it’ll almost certainly become a rats nest with compounding technical debt.

Consider unit testing. A unit is the smallest possible testable software component. To make a unit testable, it not only needs to be small but also make special considerations about its dependencies, and you may have to inject a dependency into a unit to test it in isolation. Does your dependency lend itself well to testing? What if it’s an old module and it can’t be tested properly?

Integration and E2E testing need to make considerations for environments. You may be able to write a good integration test for your local machine, but can that test be deployed to another environment? Can it be spun up in a matrix of environments? What if you have to mock something, how complicated is it? What dependencies does the dependency have to mock?

Testing Pyramid

Unit, integration, and E2E/smoke tests are often referred to as the testing pyramid. Unit tests being at the bottom (well, technically, but I’ve denoted static checking with TypeScript as the first test layer), then integration and finally E2E, unit tests are considered the least expensive/fastest/highest value, things get progressively more expensive (and slower) as you move up the pyramid.

There is some debate about how much coverage you should have, some people say 50%, some people say 80%, a few say 100%. I fall into the 100% camp, I’m quite far off that goal myself but I always strive for 100% because it encourages thoughtful development, getting from 99% to 100% can be very challenging, but a huge amount can be learned by adding that last 1%. It’s often not achievable in production because things are moving too fast, but it is something to strive for, and I would encourage backlog tasks to bring up the coverage wherever possible.

Consider a starter kit, you want to have established patterns that make it easier to build up. A starter kit that has no testing will be a nightmare to retrofit. Testing should not be added later.

OK, so what about ORM testing?

By now you may be starting to ponder how you implement testing for a system that uses an ORM. It’s not trivial because by their nature an ORM can have many complex base taxonomies that are difficult to decouple. I was able to figure out how to unit test my datasource models relatively easily but when I got to integration testing, things got significantly hairier.

The main issue is that of mocking and stubbing, testing several ORM units together often involves spinning up several dependencies and many of those depend on things like connections and database introspection. Consider something like schema synchronization or migrations, to test that you will be jumping through hoops. If you want to mock these (especially with TypeScript) you will be writing an unreasonable amount of mocking code, and it will break frequently.

You don’t want to make unit tests that are unmanageable because they will not increase productivity. They will have a deleterious effect on your development efforts. I have worked on projects with bad unit testing and they had to be rewritten, and it took longer to fix the unit tests than it did to write the modules.

Like many people, when approaching this subject I Googled the phrase “how to unit test TypeORM without hitting a database”. I got presented with the most popular result: a GitHub issue from 2017 that is still active, and I read the whole thing. It’s an interesting discussion, but it took a while to see any kind of consensus about a practical approach let alone a “correct” one.

There were a few suggestions; you could attempt to mock everything (nightmarish, too much code), other users advocated using an SQL lite connection that runs in memory, that’s very fast, but the problem with that approach is that it’s limited to SQL lite features. My database of choice is Postgres and I used a timestamp column, but timestamp is not supported by SQL Lite.

Thankfully by the end of 2019, a popular approach was starting to emerge and it’s slightly counter-intuitive because it does involve hitting a database — a dedicated testing database. I’ve done some research into this technique and I know several tech companies in the greater Boston area have deployed it successfully.

OK, so we have an approach but what’s the best way to do it? This has to be incorporated into your build process because you want pull requests to be able to run the tests. You could have a dedicated server instance for testing somewhere and you could write your testing connections to connect to that. I wanted something a little easier to manage, and something that could live in my Git repository.

I realized that for my use case I could spin up a test database and destroy it quite easily using a Docker container. I just needed to write up a docker-compose.yml file that lives in Git, then wire up a CI workflow to start Docker before executing my tests. I had to make another decision about what to use for CI.

I thought this would be a good time to try GitHub actions. GitHub actions are great for a starter kit because if someone were to fork the repository in GitHub, they would get the same tests instantly.

So now that we’ve covered our rationale and decided on our approach, let’s get into the nitty-gritty!

Docker Compose

Docker is a tool for spinning up containers. I have created a docker-compose.yml file that pulls down Postgres and Postgres Admin images and I’m using a bash script to generate multiple tables with the relevant user info:

Notice this docker-compose file is using environment variables, it also references a startup script to generate 2 databases, brainstrike , and brainstrike_test . You don’t actually need PgAdmin but I find it useful when developing locally.

server/.env.example (update with your own information)

If you have Docker installed you can spin up this container with the command docker-compose up -d the -d stands for ‘detached’ when means its running in the background.

Unit Test Example

Before we get into an example of a TypeORM unit test, lets take a look at my entry file to see how I’m establishing connections and test connections. There are a few things going on this file but effectively it’s a series of functions that are used to start the server or in the case of a test, start a test server:

I have these very simple Card and Category TypeORM entities, I’m going to focus on just Card in this article. Below is an Apollo Datasource called CardAPI that takes a repos object in its constructor. The repos are a container for TypeORM repositories. I’ll be mocking this repos object and injecting it in the Card unit test.

CardAPI DataSource server/src/datasources/card.ts

I’ve chosen not use the test DB connection for the individual datasource Repository’s as mocking the repos object is relatively easy, better to test the unit in isolation — they won’t be running in an integrated or E2E environment.

If you look in the __tests__ folder we can find our first unit test for the Card datasource:

CardAPI Unit Test server/src/datasources/__tests__/card.ts

Straightforward enough, but for integration and E2E tests we’re going to use a test database connection.

I’m using a jest globalSetup file (configured via jest.config.js), this allows me to wipe the test database and re-run the synchronize and migrations clean:

Integration Test Example

BrainStrike Server server/src/__tests__/integration.ts

E2E (End-to-End) Test Example

BrainStrike Server server/src/__tests__/e2e.ts

The important thing with these tests are these are using our test database connection.

GitHub Action CI

The next important part is setting up our GitHub action to spin up the CI environment, this is simply a YAML file in the GitHub action workflows folder, note how it’s pulling our environment vars from a secrets object. These are configured via the settings tab under ‘Secrets’ within GitHub (see next screenshot)… I’m aware I’m duping some code here for our Docker and Node steps, they use the same env variables. Unfortunately YAML features like anchors and aliases are not yet supported by GitHub actions, otherwise I would’ve rolled those up in the same anchor.

The BrainStrike repository has a monorepo structure so there are separate folders for server and client with corresponding actions. Note how the server test has a on>push>paths configuration, this means this workflow watches the server folder for pushes before being triggered.

The code also establishes a matrix of tests for both Node 10.x and Node 12.x, these will run at the same time and if one fails, the entire test will fail.

configure GitHub secrets

server action triggered output

Conclusion + Next Steps

So there you have it, one approach to complete automated testing with TypeORM and GitHub actions!

Next you’ll want to finish your unit tests (these are only a partial example of a full test suite), if your Docker container is much larger and you’ve got more to build it can be quite slow to build, and it blows the container away for each test, you may want to look into how you can cache the Docker to speed up build times.

You may also want to consider putting your entire application into the Docker compose because it can be easier to deploy/test a whole container with a service like AWS.

I hope this article helps you- good luck with your project!

Resources

BrainStrike / Node.js / TypeScript / Apollo / TypeORM

GitHub actions / Docker