Controllers, Ports, Mods and Languages

This is a fairly long post that covers a lot of things, so if you want, you can just jump down to information on Mods and Languages or my thoughts on the issues and work in porting to Mac and Linux. Otherwise, read on!

The last two weeks have been busy. You’d think that the time between calling a game finished and its actual release would be a time to relax, but there’s still plenty to do.

Between emails, crash fixes, compatibility testing, press coverage, and talking with distributors, I’m also trying to decide what’s next for Banished and Shining Rock Software. I’ve got a lot of new game ideas, but there’s also more I’d like to do with Banished.

Before I get into thoughts on the future of Banished, I’d like to talk a bit about Steam Dev Days.

Steam OS/Box/Controller



I spent a week in Seattle at Valve’s first developer conference – it was both a nice break from my office, and a chance to meet some other awesome game developers in person. In the past, being a 3D engine programmer, I’d be all about the talks on the latest graphics APIs, hardware, and rendering techniques. But strangely, as I’ve had to become a generalist taking care of all aspects of a game studio, I found myself attended all the marketing, business and community talks rather than sitting through anything technical.

Seeing a talk by Michael Abrash was cool – Who wouldn’t want to go see a talk buy a guy that wrote amazing assembly and code optimization books, worked on the Quake engine, and is now working on VR tech at Valve?

I did get a chance to play Banished with a Steam Controller, and to my amazement, without any button remapping, the controller just worked. It could do everything in the game from camera movement to placing objects to interacting with the user interface. Certainly the experience would be better with a dedicated button map, but overall the controller feels pretty good and the game was playable five seconds after I plugged in the controller.

I only put about 45 minutes into messing with the Steam Controller – I plan to do some more testing and playing at some point, probably when I’ve got time to add full native support for it.

A big push at the conference was obviously moving games toward Linux and OpenGL to support the Steam Box and Steam OS. Since I’ve been thinking about Mac and Linux versions of the game anyway, being an early entry into games that will run on Steam OS probably won’t hurt and not take much extra time. Plus playing Banished on my couch in front of a large TV is pretty amazing.

Porting to Mac and Linux

So porting. It’s not the most glamorous of coding, but I’ve done it many times on consoles. I haven’t ever coded on a Mac (unless an Apple IIgs counts), but really it can’t be much worse than my experience of getting a PS3 out of the box for the first time and making a game run on it. I’ve used GCC and Unix previously so the learning curve on Linux shouldn’t be quite as high.

I expected (or maybe I hoped) to port my game engine to other platforms when I started writing it so everything is already separated into platform specific code vs generic code. Lately I’ve been thinking about the effort it’s going to take to do the port. From my past experience, ports in well structured code take a few weeks for a basic implementation per platform, and then a few more to work out any issues and get things optimized. I can only hope to be so lucky. It’ll probably take a bit longer since this will be my first port of my own engine code.

There are certainly some challenges in doing a port. There are several systems that have implementations in the code that need to be written per platform, and data that needs to be changed to fit the target system.

First, video will have to have an implementation writing using OpenGL. It shouldn’t be too much of an issue – adding a new rendering system to Banished only takes about 90K of code (if the Direct X 9/11 implementations are used as a gauge.) I haven’t used OpenGL since 2001 or so, so I’ll have to learn the 3.X API but I figure that’d be easy. To me it’s still just shaders, data, textures, and drawing triangles. The shaders will have to be changed to GLSL instead of HLSL, but this is really no big deal either – the math and structure of the shaders doesn’t really change – it’s just syntax.

I’ll probably write an OpenGL renderer on PC first before moving it to the other platforms. This way I can flip renderers at runtime and make sure everything displays properly before trying to get things to display onto the other systems. I’ll also be able to then compare OpenGL with DX11 and see which is actual faster and end the debate once and for all. (Yeah, right!)

There’s some work to be done to get a window created and display graphics in it, or switching the display to fullscreen mode. Even if all platforms were to share OpenGL as the renderer, this code is always different per platform.

All text data in the game is currently in a unicode format – I’d like to change this fully to UTF8 at some point – but there’s not a whole lot of interaction with in-game text and OS interfaces, so at the minimum I can just convert strings to UTF8 as needed in platform functions, such as opening or creating a file. The file system in the engine has a platform implementation anyway, so this should be fairly easy.

The input system definitely needs a platform implementation as well. The keyboard and mouse have to be queried per frame to allow the higher level systems to react to user input. This generally isn’t much code and will probably only take one day per platform. I just have to figure out what to do on systems with only one mouse button and no scroll wheel….

The last major system that needs a platform implementation is audio. The audio system needs to be able to play sound effects and play streaming music. Neither are very complicated in Banished, but I’ve seen some very complicated audio interfaces over the years as I’ve worked with different consoles, so I’m not sure what sort of work will be required on this end. I’m currently using a windows only compressed format for audio, so obviously the data format will have to be changed for audio.

And speaking of differing data formats, that’s the final thing thing that has to be changed per platform.

The game engine precompiles all data from its original source format into a raw format that is ready for use by the hardware. A PNG or JPG image is compiled into a GPU-ready block-compressed format and stored. Models and animations are loaded from FBX and stored in a vertex format ready for use by the GPU. WAV files are compressed and stored in the xWMA format that xAudio uses. Text assets that describe game data and string tables that hold text are compiled into a binary format.

The reasoning here is that there’s no need to decompress a PNG to a full 32 bit format only to recompress it into block format that the graphics hardware uses. There’s no need to compress audio at load time, and there’s no reason to parse a text format when a simple binary load will work. To give you an idea of the time savings, compiling all the raw data from source assets takes about 3 minutes. Loading a save game and all the compiled assets takes just a few seconds.

But because all this data is precompiled, it potentially needs to be recompiled per platform. Some of the data is common, but there is definitely platform specific data as well.

The target platform might use a different endianness, and require byte order changes – sure this can be done at load time, but why do CPU work that can be done offline as a preprocess?

A block compressed texture works just as well with OpenGL as DirectX – however if I ever moved the game engine to consoles there would be a need to have different memory layouts for textures, or use totally different block compression techniques.

Audio data will be different as well. I could certainly move to a more commonly used format, like ADPCM or OGG that PC, Mac and Linux can use – however again I may one day port to a platform with proprietary audio system and would still need a different data format.

Pixel and vertex shaders are another resource that changes per platform. I currently store HLSL shaders in compiled binary format, and this format is different for OpenGL. Even if I stored the source text and compiled at runtime, there are text differences between the APIs.

I think allowing for different data for different platforms is good practice, as you never know what sort of system the code may be ported to in the future.

So what do I do about this data difference issue? I already have a tool that compiles all the game data into binary format when I make a full build of the game. The tool will have to be extended to output data for different platforms – I prepared for this somewhat, so mostly I just have to take the time to write the converter and serialization code for each platform specific resource. This shouldn’t take too long, but it will require a bit of work depending on how different the data is per platform.

The upside of all this work is that any future games will already just work on all platforms, and I only have to worry about the generic code that compiles on all systems.

That’s about it for the porting process, but I’ve also been thinking about the data compilation tool in terms of using it for a mod-kit.

Mods and Languages

The tool I wrote and use internally to create the data for the game is pretty much the perfect mod tool for it. My hope is to provide this tool and a bit of the core data so that the community can reskin the game, change game balance, and add new buildings, professions, and workplaces.

The tool knows how to deal with all the data in the game, and would allow users to change and add to the game data. That includes textures, models, animations, string tables, audio, music, fonts, character sets, user interface configuration, user interface graphics, sprites, vertex and pixel shaders, and render state. It also compiles game data that configures all the objects in game – workplaces, trees, plants, resources, animals, buildings, citizens, weather, environment, lighting, professions, toolbars, general game balance, and more.

In addition to being able to change almost anything in the game, the tool can also package a set of compiled resources into a single file which would be great for distributing mods.

I’ve had many people ask about translations for the game. The code for supporting multiple languages in Banished is only partially complete and could use some work. I also don’t have any plans for official translations yet. However the mod-kit would allow for community made translations distributed as mods.

I would love to see mods that extend the game. You could add an apiary that has beekeepers and produces honey, a miller that grinds wheat into flour, milk production could be added to cattle, and then a baker could take honey, eggs, milk, and flour and produce honey cakes. Or you could reskin the entire game for different time periods or locations. Or add a lumber mill and add a new class of houses and construction. Or make better character models and animations. I can think of so many directions to take Banished that I can’t do them all myself – but I think the community could do some amazing things with it.

As I wasn’t thinking about modding when I started coding Banished, there are some things the modkit won’t be able to do. You won’t be able to change behaviors or add new ones, and you can’t make new UI behaviors. You can’t make soldiers that fight off packs of wolves or change the AI of the people, or change the deer simulation. All that is in C++ code, and at the moment I don’t have a good way to allow plug-ins with new code to extend the functionality. This is something I’m considering, but I haven’t really examined the effort or implications this sort of code change would require.

There are a few things I still need to resolve for the mod kit, like making an interface to load multiple mods at once, and dealing with mods that override the same base resource. I also need to think about mods in relation to patches – if I make a patch for the game or extend the game with new functionality, the binary data formats may change, which will require mods to be recompiled with a new tool that goes with that particular data version.

When?

I don’t know when all this will actually happen. As its obvious from the rest of Banished’s development I’m not good at estimating how long it will take to get things done.

I’d like the mod support to be available soon, but the game needs to ship and I’m sure I’m going to have bugs and crashes and compatibility issues to deal with first. I may also take a bit of a break after these three years of development on a single project – A short break and some part time work on some new tools, tech, and game prototypes might be nice.

Anyway, 21 days left until the release of Banished! Are you counting down? I am!