A High Level Decision

Let me start off by saying that I (and other team members) are still HUGE fans of Unity. Several of us on the team have been using Unity since v3.X days (including Pro licenses) and CONTINUE to use it for various projects. If I was starting a mobile project today, I would still reach for Unity.

We made a LOT of progress on prototyping ExtroForge with the Unity engine. Huge swaths of gameplay elements were engineered and integrated. We thrived with the rich variety of Asset Store packages available to us as well as the YEARS of online posts, blogs and tutorials. With at least two 15-year software engineering professionals on the team, we were efficient and proficient under Unity.

All that being said, we got to a point where a single aspect of the Unity Engine became a critical blocker – multiplayer networking. Several people on the team had experience with building multiplayer projects with Unity – one using the third party SmartfoxServer and the other with the built-in RakNet functionality. Extroforge, however, brought some specific needs to the table. – including Authoritative server logic, server-side physics and collision logic and high-performance (Extroforge is part FPS and part RTS in many ways). We prototyped multiplayer using the built-in networking and in the meantime, looked at some third party options. We were eagerly awaiting the Unity 5+ replacements for the networking – UNet.

The first iteration was less than impressive. Extremely limited (missing?) documentation and oddball (buggy? flaky?) performance. Simple replication of an animated model (location/rotation) was either non-functional or super-laggy. We experimented. We read forum posts. We waited. There were no official Unity tutorials. Some of the people that WERE making tutorial videos that we followed actually gave up and decided to wait until future releases. Then the Unity roadmap was published. With the implementation of the standalone simulation server pushed out to March of 2016 (plus some other elements undefined), we decided to look at other options. What a pleasant shock it was to fire up Unreal for the first time with a starter project and see that little side menu next to the “Play” button that allowed you to auto-start not just a dedicated server, but also as many ‘additional’ player clients as you wanted to. It was multiplayer nirvana.

Note that this was not SOLELY a decision based on multiplayer functionality built into the engine. We have struggled for some time to get the world/terrain/foliage/water to look the way we wanted. Our Team/Studio lead who was responsible for the look and feel decided to look at some other game engines to see how quickly he could achieve the desired results (our Unity results either looked too cartoonish or too “claylike”).

CryEngine was so beautiful “right out of the box” that he almost cried. But convincing the team into such a low-documentation, small community, high learning curve environment was a non-starter. With Unreal, the desired look and feel was quickly created and we were all MUCH more comfortable with the user community, documentation level and toolsets (including Blueprints).

Once we were convinced (via a few sample projects and prototyping sessions) that we could get the look-n-feel that we wanted – and with the multiplayer stability and scalability that Epic brings to the table (after YEARS in the multiplayer FPS space), we made the official decision to jump. It was a HARD decision. We had a lot of code and content that was Unity specific (including our procedural mesh based voxel implementation). We were so familiar with the Unity way of doing things – including the C# language. Blueprints and C++ gave us more than a moment’s pause…but we had committed.

C# versus C++ versus Blueprints…oh my.

The ExtroForge team had some technical depth. Both senior engineers had worked in C and C++ years ago – although had moved on to managed languages since – heavy in Java and C#. The rest of the team had mixed experiences in different high level languages. We were well experienced in both OOP and Component based design. C# was an ‘easy’ language to work with in Unity and the additions that GameObject and its related brethren brought to the table had become second nature to us. I’ll admit that the Blueprints concept in Unreal frightened me a bit. Interestingly enough, none of us had done any visual scripting before, so we had nothing to compare to. I had some bad experiences with wired prefabs in Unity – where we could easily lose a bunch of important settings. I got very used to establishing important values and settings in code. That fear, along with many many years of software development experience behind us, made me confident that we could/would blow the dust off our C++ skills and start that way. I was encouraged by the fact that most of the ‘starter’ sample projects that were available to download/install through the engine/editor had a Blueprint AND a C++ version. That encouragement was short lived, however. Not to sound demeaning – but because of the large number of ‘hobbyists’ that flocked to Unreal because of it’s price, there were a HUGE number of tutorials, forum posts, videos, etc related to accomplishing Unreal tasks in Blueprint land. There were SIGNIFICANTLY less in C++ land. At first I wasn’t too worried. Give me a good API doc (and the engine source) and I should be fine.

Yeah – not so much.

It was hard enough to remember or research the ‘right’ way to do things in an unfamiliar language – learning the “Unreal” way to do those same things sometimes was too much for my tiny brain. Even when I had a modicum of success on the C++ side (some custom structure/classes to embody our voxel world), I was unable to get Unreal to allow the use of these custom structs/classes in certain areas (like RPC calls). Oh, I could get past compile errors by adding the appropriate macros or proper ‘naming’ of my structs/classes – but even after that, RPC calls that used one of these custom structs simply produced empty results on the other side (a problem I STILL haven’t figured out – so I pass things around in Unreal specific structures and convert/de-convert on the other side). That issue aside, we made progress living in C++ land. I was able to dynamically reference mesh or materials (by name) at runtime so that I wouldn’t fall prey to any brittle wiring losses like that I experienced with Unity. Yay. Until we tried standalone non-editor builds. Then the dynamic techniques to grab those assets at runtime fell over with ugly errors. I’ve read all the articles and tried all the techniques – and still found myself unable (at runtime) to dynamically grab a material or mesh or blueprint by name at runtime in C++. So I started looking more closely at Blueprints to see how they could help.

One senior team member had already started wiring up some blueprint functionality – adding to the skeleton first person character BP that was tied to the C++ classes I had been working on. Creating little boxes…dragging little wires….all the colors!!! I was fascinated. Most of it was input/controls related as well as GUI/UI stuff – elements I figured were probably the best use for Blueprints. I started my foray into blueprints myself was for Material logic. I was AMAZED at the stuff we could easily do in seconds in a material blueprint that would have taken….DAYS writing shader code in Unity. It was easy, it was fast, and it was (dare I say it) satisfyingly FUN. While my code remained firmly in C++ land, we slowly learned to expose variables from the C++ side so they could be accessed from these input/UI blueprints as needed (still haven’t determined if you can access a BP variable/function from C++ side – but it looks like it is painful). I learned that I could create my critical settings and initialize arrays of important game elements in C++ (where it was safe in a non-binary-corruptible file) and expose it in to be used as necessary in Blueprints.

Sort of.

So interestingly enough….enums are now the bane of my existence. I love enums. Been a huge fan since before the dawn of time and have used them in every high level language since I was a wee engineer.

Many of the low level elements of our game uses enums at the base to represent all sorts of things – Building types, Voxel types, Resource types, etc, etc, etc. And I spent a bit of time Unreal-ifying these enums so that they could be read properly and used properly in Blueprint land (Unreal made some changes to make it easier back in v4.7? you used to have to use some oddball techniques to deal with them). At the heart and soul of things, Blueprint-land doesn’t know about enums – but it knows about 8 bit bytes – and it treats them as such (but you can still reference them as the Enum type you carefully crafted in blueprints). All that aside, the issue we have been having (v4.9) is that SOMETIMES blueprints want us to cast the Enum type to a byte and sometimes NOT. Sometimes a section of blueprints that previously worked WITHOUT a cast to a byte – now errors out and requires it. AHHHGGGG. It’s not unusual to make a bunch of C++ code changes (UNRELATED TO EXISTING ENUMS) and have the Blueprint editor FREAK OUT after a compile and show a trillion errors (all related to Enum references). At first I spent HOURS recreating seemingly perfect nodes with the exact same inputs and outputs to clear the errors. Then I learned I can simply close the editor and re-open and all is well. WTF? That doesn’t make for a good workflow – but we are managing. It’s possible that this (or part of this) is fixed in 4.91 (several Enum/UEnum related fixes in the release notes) so we will have to see. (UPDATE – nope – neither 4.9.1 or 4.9.2 seem to have fixed this…so we remain mindful).

We are drawing more and more of the C++ side and Blueprint side together – exposing C++ functions to blueprints and even creating custom blueprint nodes. I have learned that I really DO love blueprints – and their place in the development process. They represent a perfect mechanism for certain elements of game development and we appreciate them more and more every day. Most of what we are doing with them is tied to the player character actions – and we see no compelling reason to move that code to C++ land. Unreal – you made a convert out of me!!!

Asset Management

Unity asset management is a very mature beast. Drag a file in to the editor, copy a file to the Assets folder – it just worked. Move assets around to different directories – just drag and drop. The package import/export functionality (some flaky determination of dependencies aside) was top notch. There were even engine functions exposed (through editor scripts) that allowed you to code all sorts of neato-whizbang custom logic to manage your resources. Our experiences with Unreal, however were somewhat painful. Some of this was due to ignorance/inexperience on our part – but some is still a mystery. Trying to migrate content from one project to another – with different directory structures provided many wasted hours. Trying to migrate (using the tools Unreal provided) Blueprints based on C++ classes was an unmitigated disaster. C++ classes can’t be migrated – you have to copy on your own – but should you need to change class names (we migrated from a project where the we wanted to rename the autogenerated character controller classes) the migrated blueprints are permanently crippled do to un-changeable references to the original class. Moving content (in-editor) between folders was also a severe pain point – Unreal would explicitly leave copies of files behind or empty sub folders and all sorts of references would get broken (some un-fixable). We got burned so much by that last issue, we simply try NOT to move content around – being extremely careful where we place items from the very beginning. To be fair, Unity’s critical ‘asset database’ COULD get corrupted and when it did, you were in for a loooong period of re-importing all the assets. But Unity did this all automatically (detecting its own corruption and fixing itself) – whereas any detection of issues or fixing is done manually in Unreal.

As far as the Unity Asset Store versus the Unreal Marketplace is concerned – it’s simply a matter of maturity. Unity has had years to build up a large and thriving store filled with both free/inexpensive and pricy assets (of varying quality). As an author of several Unity store assets, I was happy with the content creation and selling process (and most importantly payout). Finding and importing new content in Unity was fairly painless. The Unreal marketplace is most assuredly smaller Much smaller. Tiny even. And the ‘average’ price for content is higher. No doubt about it. We’ve had no experience yet with selling (or even buying) assets on the store, using internally created assets or starter content as necessary.

Procedural mesh manipulation

Unity provided the ability to manipulate mesh data (creating from scratch or manipulating after the fact) out of the box. Vertices, vertex colors, Uvs – it was all accessible and well documented. Over the years, I used it for countless reasons – terrain deformation, mesh destruction/fracturing, and in ExtroForge, our voxel construction engine required it. It worked well, it was fast and it was ubiquitous.

I was surprised to learn that that functionality was ‘officially’ missing from Unreal when I started using it (4.7ish). Terrain deformation is not supported (at least with the built-in Landscape classes) and interestingly enough even terrain height data is only available through the editor – requiring trace/raycast to the terrain at runtime if you want to know the height at a point! There was a lengthy forum post where wicked-smart people had created (and posted) their own codebase to do some basic procedural mesh construction/manipulation, but it was far from complete. In v4.8, Unreal released an ‘Experimental’ ProceduralMeshComponent which closed the gap a bit. We were able to reproduce our voxel procedural mesh techniques in Unreal with little fuss (although I read about some others having issues where moving a procedural mesh component can leave its physics collider behind!). 4.9.X added some enhancements and it looks like they will continue to improve it. There are still some gaps between what Unreal provides and Unity (like manipulating meshes of already existing models) but at least they are heading in the right direction.

Lighting

Unreal adds an incredible number of open world lighting tools to our arsenal. Yes, Unity rolled out with realtime global illumination via the Enlighten engine, but it’s still limited for an open world where structures are being added and removed all the time. What we really benefited from was the distance field ambient occlusion, and the realtime indirect lighting via light propagation volumes. Looking into the distance, things in Unity tended to look flat. A distant mountain should cast a shadow on the valley below, but with Unity we just couldn’t achieve that without an insane shadow distance.

Unreal Engine had a solution for that that. Distance field shadows and ambient occlusion are designed to shade faraway things in an approximated kind of way that looks perfect from far away. We really were able to get the lush and well-lit / well shaded world that we wanted to with Unreal Engine.

With light propagation volumes, we were able to create items with emissive materials that would cast light onto nearby geometry. The resulting look is fantastic. Again – this is listed as ‘experimental’ (we had to make a setting in the engine property files to enable it), but so far – so good!

Open World Tools

In Unity, we used our own custom terrain height generation logic as well as custom texturing and foliage/feature additions. It was all done dynamically at run-time – and worked fairly well. We were a bit dismayed to find that we couldn’t modify terrain height at run-time in Unreal (as long as we were using the built-in landscape elements). However – there were some compensations. We are able to make great use of the new procedural open world tools in Unreal. The ability to assign random grasses and plants to certain terrain textures really helps to make the world look lush and full of life up close.

When it comes to populating the terrain with trees, the new procedural foliage placement tools make it easy to grow entire ecosystems based on rules not unlike our dynamic Unity based system. We still have work to do in being able to generate these maps/worlds completely dynamically in Unreal – but we look forward to the technical challenge.

Miscellaneous Features And Functions

Prior to v4.6, Unity’s UI system was a bit of a mess. The inefficient built-in system had been superseded by a handful of excellent (but relatively expensive) 3rd party assets. Unity’s new UI system (> 4.6) seemed to get up to speed nicely and provide just about everything a developer could need. Unreal’s UMG system feels similar – providing the same robust UI features – but with the added advantage of being able to wire and code via Blueprints. In my opinion, Blueprints are the PERFECT mechanism for wiring UI logic and flow. We’ve had no major issues in this area other than a little worry that the Widget component (that allows you to attach a UI widget to a 3D asset) is specially marked as ‘experimental’. Cross fingers – because it works well so far!

Unreal’s AI behavior tree and ‘perception’ features have no parallel in Unity other than 3rd party assets. I have tried several of those assets and usually ended up rolling my own FSM or HTN/GOAP in C# as necessary. We have barely touched the surface in our experimentation with it in Unreal but on the surface it has provided a very rapid way to add basic prototype NPC and creature AI to the game without the need to hand-create a lot of infrastructure.

One final thing…

I have been on the receiving end of several critical bugs in Unity that prevented some mobile game launches/rollouts. We were truly at the mercy of their release/update cycle. You could argue the same about Unreal (they certainly have their fair share of bugs!) except what often gets tossed out is the fact that Unreal provides source code to the engine. If there is a critical bug/issue, you COULD track it down and fix it on your own. If you needed a feature that the engine didn’t support, you COULD add it. For most ‘hobbyists’ , this is a pipe dream. For the ExtroForge team, however, this is a reality. We have already forked the engine code and added logic so that we could have multiple sets of Uvs in procedurally created meshes. This was a critical roadblock in our voxel logic so that we could apply voxel health/damage/power through the shader (material) without wasting CPU cycles. It was exciting to be able to tweak the engine that way (less than 20 lines of code) and it will be more exciting to see if we can actually pull and merge the 4.9.1 release into our engine codebase cleanly.

Well – there you have it. There is still a lot of love for Unity (although I haven’t opened it in over a month) – and we still actively watch for their progress on multiplayer networking stuffs and other key engine additions.

Our team spends every day exploring and learning more and more about Unreal and In the future, we will post followups related to other elements we can compare to Unity (once we have enough time under our belt).