How to Test Monorepo After Split Before Actual Split

5 min

Mon, Feb 10

X comments

Edit Post

In 14 months old post How to Test Monorepo in 3 Layers we talked about testing monorepo in 3 layers. So you can be sure every package works as it should.



3 layers are testing in a monorepo, testing package directory, and testing after a split. The latter takes a huge amount of time. The time we don't have to spare in 2020.



How can we make it faster while keeping the test quality high?

Do you use Github Actions on your Github projects? Well, if you do, it might cut your commit feedback loop from 17 minutes to just 3 and make you super productive by accident.

We now use Github Actions on Symplify and Rector for over a month, and in January 2020, it helped us to merge amazing 177 pull-requests.

Warning: Cleaning Up Might Cause you to Spot more Bottle Necks

That's why so few developers like to tidy up. It usually only shows worse and worse mess in the code.

The worse mess in our code was the 3rd layer of monorepo testing - after split tests.

We had to wait till the full monorepo is split (+ 5 minutes), and CI runs on each split package (+ 2 minutes). So instead of 3 minutes, we're now back on 10 minutes. Also, the split testing can only happen after the PR is merged into the master branch.

Do your 1st and 2nd layer pass? All good? You have to wait till the split to find out it's not all good. Doh.

Short Reminder: What is the 3rd layer of Monorepo Testing?

Let's say you run:

git clone [email protected]:symfony/console.git composer install # all needed dependencies are installed, symfony/* and all external in /vendor directory vendor/bin/phpunit

And that's all! The standard way of testing packages.

But with monorepo, the symfony/console is just one of directories in big monorepo symfony/symfony repository:

symfony/src/Symfony/Component/Console symfony/src/Symfony/Component/EventDispatcher symfony/src/Symfony/Component/HttpKernel ...

To run unit tests on symfony/console only, we'd have to call PHPUnit on the directory:

vendor/bin/phpunit -d symfony/src/Symfony/Component/Console

But there is no:

symfony/src/Symfony/Component/Console/vendor/*

So we can't test it. That's why we have to wait after the split is done and trigger tests in a split repository - symfony/console the way we did at first:

git clone [email protected]:symfony/console.git # all needed dependencies are installed, symfony/* and all external composer install vendor/bin/phpunit

You can read more about it in original post.





Status Quo: It Sucks, but it's the Best Testing

3rd layer of monorepo testing sucks

it takes + 10 minutes

it doesn't test pull-requests - it's like tests without CI basically

it confusing to have few tests in one repository and one repository

It's so bad that the most monorepo projects don't have this 3rd layer - Symfony nor Laravel. And I don't blame them. I was about to drop it too because having feedback about one line of code from 2 sides with entirely different settings hurts my brain.

The problem is, this 3rd layer testing is the closest to the reality of how these packages are used. So missing it will cause you headaches with bugs, so simple to find by users of your packages that you won't believe them.

Could it be possible with GitHub Actions?

This bottleneck got me thinking... what do we need to simulate?

git clone [email protected]:symfony/console.git # all needed dependencies are installed, symfony/* and all external composer install vendor/bin/phpunit

Just locally for every package.

Albert Einstein always taught me:

"think in patterns, my young padawan",

so I knew there must be a way.

Then I suddenly knew what to do!





I won't lie, it took me 3-4 hours of trial and error and many toilet eureka visits to figure out the working model. What went wrong at first?

Symlink cannot be used , because real/relative paths are different from standard package testing

, because real/relative paths are different from standard package testing All packages can't run in one workflow , because it's super slow

, because it's super slow We cannot use shared vendor, the dependencies are conflicting

So What Worked?

Every package composer.json had to require mutually dependencies (e.g symplify/easy-coding-standard requires symplify/package-builder ) with *

had to require mutually dependencies (e.g requires ) with Every other package had to require other packages via composer path repository:





There was one more pit to fall into. How many repositories we have to add here?

{ "require": { - "symplify/package-builder": "^7.3" + "symplify/package-builder": "*" } }

Just this one, right?

+"repositories": [ + { + "type": "path", + "url": "../../packages/package-builder", + "options": { + "symlink": false + } + } +}

Well, I assumed so, but this would eventually download symplify/package-builder with local version, *but all the rest of `symplify/` packages from packagist**. In the end we'll have a mess like:

symplify/package-builder - local

- local symplify/autowire-array-parameter - from packagist before split !== different code that we use

We have to add all the possible local repositories, so the package always has the local version of the code.

Showcase: Symplify Workflow

It makes sense, but it sounds like a lot of manual work, right? Not for long!





You know me too well, me and manual work don't get on very well with each other.

I made a command for MonorepoBuilder that does all the work above for us:

vendor/bin/monorepo-builder localize-composer-paths

All we need to do is run it before composer update .





This simple idea is running split tests in 14 Symplify packages. How?

switch to the package directory

modify paths to use local path packages without symlink to prevent path false positives

symlink to prevent path false positives install dependencies

run vendor/bin/phpunit there

# .github/workflows/after_split_testing.yaml name: After Split Testing jobs: after_split_testing: runs-on: ubuntu-latest steps: - uses: actions/[email protected] - uses: shivammathur/[email protected] with: php-version: 7.4 coverage: none - run: | composer install --no-progress # this is the magic packages/monorepo-builder/bin/monorepo-builder localize-composer-paths # testing one package cd packages/easy-coding-standard # download dependencies composer update --no-progress # run tests vendor/bin/phpunit

And that's it! 1 extra new line in workflow is a worth hour of human-hours a week on one project.





See full workflow prevent repeating code.

See full pull-request.

A reminder: this all is possible thanks to super-fast Github Actions with 20 concurrent workers. On Travis with only 3 workers, this might take extra 15-25 minutes, which is utterly annoying.





Happy speed coding!