Version control is a contentious issue amongst developers. As an Unreal Engine developer you may be most familiar with Perforce, or if you're really behind the times you may still be using SVN, or even numbered folders on your desktop.

For some of you nothing in this article will be new. For others it might be a slight yet welcome adjustment to your existing process. For yet more people it will be full of entirely new, scary concepts. Trust me, they're not that bad once you start using it.

A brief history of version control.

Subversion

Version control started to get serious with the introduction of Subversion, or SVN. One of the first mature version control systems capable of branching and merging, SVN itself directly created new roles within the development industry as every team needed a Merge Boss.

This is where SVN's issues began. Merging often isn't possible as a stand-alone role. Merging a developer's commit took varying amounts of cooperation with the developer. Even under a peer-review system multiple merges are absolutely capable of leading to catastrophe. Using SVN in a team implied build servers such as Hudson, even for simple projects such as websites. So when I say SVN became a role, what I meant was SVN also became a significant bureaucratic hurdle in releasing changes from your team. Things were better, but at the same time they weren't.

SVN famously comes with its own lengthy book for introducing developers to the concepts behind version control and how to use SVN. An entire book.

Perforce/Mercurial

Aimed at a completely different crowd, Perforce and Mercurial tried to take into account the needs of large-data projects. Perforce... works. But for it to work well costs money. Having lost the version control race to Git, Mercurial simply suffers from a lack of support and new development.

Git

Git attempts to solve all of these problems. Created by Linus Torvalds in response to the difficulties of running version-control in kernel development, Git is open-source, free and has world-class hosted and self-hosted options. What it also has is a steep learning curve.

First, these are the issues that Git aimed to solve:

Decentralisation

Trunkless design

Branches that are a collection of commit tags

Various quality-of-life features

Decentralisation is a significant problem in version control, one would say it's the main problem. Solving it implies fundamental design choices in the way your version control system works. Git solves this through pull requests.

Found on Google Images.

Ideally a pull request is a proposed merge that has been fully prepared and can be merged into the remote repository with no further changes or work on the part of the repository owner receiving the contribution. Pull requests go through the usual process of peer review and in a good development setup have already been run through unit testing and other test frameworks.

These two ideas form the basis of Git and are the two concepts that developers must have a solid grasp of before using Git. Git is essentially a control freak's dream system.

The stated drawbacks to Git by Perforce users generally sound like:

No file locking.

Difficult to commit.

Difficult to resolve merge conflicts.

Remembering to pull changes from your target branch back into your development branch.

GitHub only offers small hosting volumes per-project.

Fortunately as of 2018 Git has both file-locking and Large File Storage options, both of which work well. LFS in particular is well-designed and allows a repo contributor to use a file pattern to absorb large files that match a name pattern or binary-type into the LFS system, which rather than trying to track sub-file changes, tracks entire files as a single change. Handily LFS can be easily partitioned into a different drive on your Git repository server.

Merge conflicts and commits are a little more difficult and, once again, require bureaucracy to resolve.

Commit message templates to the rescue. There are a variety of tricks, but the most common is:

The first line is your message and JIRA number. Every subsequent line contains a bullet-list of changes.

The simple reason for this is it makes it easier to quickly grep the log for a summary of what each branch contains, with more information available as needed. Detailed commit messages are important, even on solo projects.

Merge conflicts are a little bit more complicated. There's rarely one solution. Git is able to auto-resolve conflicts but they need to be reviewed by the committer. For distributed teams this can be an issue, as the remote developer whose code you're affecting can't see your commit until you've merged and pushed it to your remote branch. That sometimes means undoing a merge to do it differently. In an ideal world there are no commits to re-add code that was removed, just perfect commits that were rolled back and modified until they're correct. This is the major sticking-point for most developers: it's more work. But you get a better result, particularly in large teams.

Finally, self-hosting through GitLab is the panacea that really makes Git shine next to Perforce. GitLab is very similar in function to GitHub, but has the added security of being YOUR server. GitLab can be configured quickly to run as a locked-down private server, or as an open-source project host that invites user registrations and pull requests. It supports 2FA and other modern authentication features. It is Good.

Now you know everything worth knowing and that brings our brief history of version control to a close.

Unreal Engine And Where It Fits In.

Unreal Engine has a handful of specific challenges to version control: specifically it has a highly-referenced project structure that is easy to break through parallel sets of changes.

Units of work are difficult to quantify and change for each team or task, you don't know if you'll need to lock the level blueprint or wait for someone to finish working on it or not. Multiple blueprint developers leads to blueprint libraries segregated by who made them rather than category or purpose. New merges can lead to a developer working through a tree of dependencies re-attaching node wires to pins and forcing struct types to refresh before recompiling every blueprint that used them. In fact this can happen just by moving a project to a new workstation. It is a fragile system.

There are basic techniques for managing this. You ensure your developers are divided by role. You move units of work to the correct developer. You ensure that developers downstream from that change have something else to do while they wait for the changes to be made, etc.

It is no surprise that for these developers version control is a waking nightmare as version control multiplies the chance of the above issues occuring. For this reason most Unreal Engine developers like monolithic commit systems such as that used by Perforce.

But it doesn't have to be this way.

So what's wrong with Perforce?

Despite now supporting a distributed architecture, nobody seems to use Perforce that way. It's unwieldy, there are very few client options and I've anecdotally heard of developers permanently losing access to their repositories when the hardware it was hosted on died, even though they recovered the hard drive. Perforce isn't what I'd call a mature or fully-featured product and it's not as well thought-out as Git.

How To Use Git Better.

1. Set up your ignore files correctly. There are a plethora of folders and files that you shouldn't be including in your repository. Here's mine:

# Visual Studio user specific files .vs/ # VS projects. You should re-generate these locally. **/*.sln # Compiled Object files **/*.slo **/*.lo **/*.o **/*.obj # Fortran module files **/*.mod # These project files can be generated by the engine **/*.xcodeproj **/*.sln **/*.suo **/*.opensdf **/*.sdf **/*.VC.db **/*.VC.opendb # Binaries **/Binaries/* # Builds **/IOS/*.ipa # Don't ignore icon files in Build !**/Build/**/*.ico # Configuration files generated by the Editor **/Saved/* # Compiled source files for the engine to use **/Intermediate/* # Cache files for the editor to use **/DerivedDataCache/* # Other stuff **/.dropbox **/desktop.ini **/.DS_Store **/*.swp Content/StarterContent/ **/*Conflicted* .gitignore

Go nuts. Add more.

2. Check your LFS patterns. Most non-source things in an Unreal Engine project will end up on LFS. Fortunately this is all covered by a single line in your .gitattributes file:

*.uasset filter=lfs diff=lfs merge=lfs -text .gitattributes

Don't forget to run git lfs install on your project folder first.

3. Commit frequently using detailed commit messages and only including files that you intend to be present in that commit.

Much like blueprints, a commit is a unit of work. This makes it easy to know what to include in your commit and what not to include. Ideally if someone decides to remove your commit from the history, it shouldn't take other units of work with it. Even as a solo developer do yourself a huge favour and take care with your commits.

4. Talk to other developers. This really should go without saying but you need to be peer-reviewing each other's work and talking about what you're working on. Nobody likes redoing work and it's inevitable if you are not communicating.

5. Define your units of work by role. Importing assets to the game is a single unit of work. Implementing them is another. These are separate commits and can be undertaken by separate people. Just because it fell to one person doesn't mean it should be one commit. You should know how your commits will look before you've lifted a finger to develop because the unit of work was well-defined.

6. Set up continuous integration on a team server of some sort. Setting this up is where the Merge Boss role ended up, because doing less work is the natural evolution of the developer. It's not the easiest task but it will be the most rewarding.

Continuous integration allows you to automatically run your testing framework every time a commit or a merge is made. Your developer will probably get an email or notification every time he commits saying everything is broken, until he performs his last commit and it tells him that it all works again. That's the nature of unit testing: seeing it fail is an expected and desirable part of the process. You just saved him some time because he didn't have to load it and run the test regime.

The flip side to this is that your peer review process must ensure that everything has a unit test written for it where it is reasonable to do so. Unit tests can't run if they haven't been written. There are lint packages that can also validate this step for you and generate a notification from your CI server.

7. Understand when to branch. Actually this one is easy: every time. They're called "feature branches" for a reason. New feature? New branch. It will contain many units of work. Base your feature branch on the branch that you intend to merge back into.

Shamelessly stolen from Google Images.

8. Regularly pull changes from this target branch back into your feature branch. If you handle a few conflicts each day then you won't be handling a month of conflicts at the end.

9. Aim to finish your feature branch. It's not helping anyone by sitting there and getting further and further from the main development branch. The sooner you can wrap it up and get it merged back in, the happier you'll be.

10. Do your version control outside of the Unreal Engine editor. Whether you're using a visual client like SourceTree or the CLI you will find you have more control over your work and less problems if you dive into really using Git and not trying to use a simplified concept through a proxy.

11. Google everything when you are resolving a conflict. Don't be afraid to straight up duplicate your work directory and try things on a copy before proceeding. Not everything in version control is reversible. Not every command is a smart idea. If you haven't had to handle a bad conflict or a broken merge state in a while then it's almost guaranteed that you're going to make a mistake that you'll regret. Once again:

Got a version control issue? Duplicate your working directory.

And that's pretty much all you need to know. Happy versioning.