Tech Stuff

As I’m in the middle of a bunch of changes, I don’t have any new and pretty screenshots. So I apologize in advanced to those who aren’t programmers or aren’t interested in the way the engine works….

Back when I started the game engine for Banished, I didn’t want to write an editor. This was mostly for development speed. Writing nice tools can take an immense amount of time. As a result of this, all the data is human readable text, other than inherently binary assets like textures and audio.

My goal was to build a resource system that could serialize a C++ class from a text definition that looks like the class itself, but also support reads and writes to a binary format for speed and size. I also wanted the ability to one day support a tool that could edit the data, and write it back out to text format without extra code being added to the classes.

So here’s an example of what I came up with. In the user interface, a sprite is defined by this (simplified) class.

class ImageDescription : public ElementDescription { public: virtual void Serialize(IO::Stream& stream); protected: ExternalPtr<SpriteSheet> _spriteSheet; String _spriteName; int _width; int _height; };

A ‘SpriteSheet’ is a object that contains many sprites on a single texture – in this case it’s wrapped by a pointer to the external resource. On disk an instance of the image sprite looks like this.

ImageDescription imageNewGame { Alignment _alignment = MidCenter; int _bottomPad = 4; int _leftPad = 8; int _rightPad = 8; int _topPad = 4; SpriteSheet _spriteSheet = "SpriteSheet.rsc"; String _spriteName = "NewGame" }

This nicely fills in all the fields. You’ll note that _width and _height are missing, and there are additional fields not listed in the class. Any missing fields are default constructed, and the others are in the base class. _alignment is an enum, and the system allows string definitions to define values for enums.

To serialize the class to any format, the code looks like this.

void ImageDescription::Serialize(IO::Stream& stream) { ParentClass::Serialize(stream); IO::Serializer s(stream); s.Serialize(_Field(_spriteSheet)); s.Serialize(_Field(_spriteName)); s.Serialize(_Field(_width)); s.Serialize(_Field(_height)); }

The data stream that is read in or written out by the Serialize() method can be many things. It can be a text parser that reads in text (ascii or wide), a text writer that writes to text, binary read, binary write, memory read, memory write, or any binary form in a compressed format. While I don’t actively use it, it can also be a stream that reads the types and addresses of each field so that an automatic property page can be built to edit the data.

The _Field() macro is used just to pull the name from the variable. _InternalField() can also be used to serialize data that isn’t meant to be written to text form, such as binary image data. Smart pointers can be read and written. All collections (arrays, sets, maps, etc), all integral types and custom data types can be serialized as well. The Serializer class handles headers and footers on the data to ensure data integrity, as well as supporting versioning the data – old data can be skipped, or new data can be written out.

The nicest thing about this system is that the text reader supports data derivation. If I want to make another image that is very similar to the one above, I can use inheritance, similar to C++ inheritance.

ImageDescription imageQuitGame : "imageNewGame" { String _spriteName = "QuitGame" }

The imageQuitGame inherits all the properties of imageNewGame, but overrides the actual sprite name. This cuts down on typing time, copy paste mistakes, and also allows changes to a single object to effect everything similar. The entire UI visual style can be changed by just editing a few base objects.

When the text data changes, the game will parse the new text and then save out a cached binary version. This cached version is then used until the text resource or some dependent file changes. At the same time, binary assets are transformed into a state ready for hardware to use – shader programs are compiled, textures are converted to DXT formats, WAV files are compressed. This gives great loading times as costly parsing, data conversion, and error checking is totally avoided once everything is cached.

While the underlying system to make this work is fairly complex, adding new objects is fast and hardly takes more time than writing the class definition and constructor. And I have a great editor – it’s the same program that I code in. It also makes pseudo data reflection available in C++.