While working on a project using the Operator-sdk I found that test coverage was quite low. The reason was that of a lot of generated deepcopy files in the repo due to the Operator-sdk.

Initial Test Setup

First I am using the following repo to add colours to my tests for readability

go get -u github.com/rakyll/gotest

I test the pkg directory which I tested with the follow command

gotest -v -covermode=count -coverprofile=coverage.out ./pkg/...

This runs the test and outputs the test coverage to a file coverage.out you can check the coverage locally using the html flag

go tool cover -html=coverage.out

Or you can use the func flag which give you the functions percentage coverage in the cli

go tool cover -func=coverage.out

Refining the coverage

As I was never going to test the generated files I decided to see if I could remove them from our code coverage report. At first I looked at the go test -run flag in the following format.

gotest ./packagedirectory -run=testname

I found it would have too much maintenance to update. I tried using it with regex to try and hit all test but found that with a greedy regex expression like -run=^Test.+ I would hit all test but also all files so I was back where I started.

The directory pkg/apis was where generated deepcopy files were located. I moved the files that I was testing to a from pkg/apis to a sub package called types and filtered my tests by directory instead. So my test command looked like

gotest -v -covermode=count -coverprofile=coverage.out ./pkg/controller/... ./pkg/providers/... ./pkg/resources/... ./pkg/apis/integreatly/v1alpha1/types/...

This runs the tests in the directories listed above and updates the coverage.out with only the files in this directory. This increased my overall coverage by about 14%

CI/CD and coveralls

We were using Travis for CI/CD and setup coveralls with the following config in .travis.yml .

For more information on setting up

-repotoken=$COVERALLS_TOKEN

See the goveralls repo https://github.com/mattn/goveralls and https://docs.coveralls.io/go

With this setup we can then check our coverage as part of the PR , This doesn’t replace writing tests but does eliminate generated files from the coverage report.

Myblog