I did some final tweaks to the modified cultural properties and made sure no terrible failures occurred while doing some long simulation runs. I’m not very happy with the end result though. The performance gain is very small (reduced CPU usage by less than 2%), and now there’s a quirkiness in the way polities spreads that irks me a lot. Many cell groups bordering polities gain and loose tribalism knowledge in a cyclical manner for a while and I can’t seem to find any obvious bug that would trigger this behavior. It all appears to be a natural, albeit unpredictable, consequence of the changes I made to cultural knowledges. It’s not really a big deal in the end, so I’m not planning on rolling back the changes I made any time soon. Though I might rethink that if the these quirkiness become troublesome in the future.

So that’s the end of performance improvements in CPU and Memory for 0.3.1. And thus I started working on fixing and optimizing the Save/Load process… Currently. Save file size tends to increase proportionally to the amount of cell groups present in the world, and each group takes about 2k of memory to store on file. So a file with a mere 600 groups (very early in the simulation) takes about 1.6MB. A very old world with almost all of its cells inhabited can easily generate a 100MB save file. The worst part is the time it takes to save the file. The game will take a couple of minutes to complete the save process and the simulation must halt until this process completes. I can basically go to the kitchen and brew myself a cup of coffee while I wait for the game to save.

It would be easy to blame the fact that I decided to store save data as an XML file instead of binary file. But, besides the fact that XML files serialized using .NET are only two times larger than binary files, they are faster to parse (so faster to load) than binary files. Also, it’s way easier to work with and debug XML files than binary files. There are many other ways I can try to improve the save process. The easiest one is to use serialization attributes to reduce in size XML tags and attribute names, which I already started doing, but this is just a very small improvement. A better approach is to prevent the save process from serializing data that could instead be regenerated using procedural generation methods. Although it will require some creative coding to get the most of this approach.

Finally, there’s one big thing I have been thinking a lot on implementing in the game, differential save files. Instead of trying to save data for all cell groups in a save file, it should be feasible to just store the data of groups that have been updated since the last save. This would mean that instead of having all world data in a single file, data would be spread across multiple files. This would speed up the saving process a great deal and make each individual save file way smaller. On the other hand, the load process might take longer since every time the game tries to load a world, it would need to read all the related save files since each file would only have a portion of the world’s data. Though, ideally, loading is something that needs to be done only once per game session. While saving can occur multiple times within a single session.

This last idea is highly experimental and I’m not really planning on implementing it for the next public version. Nevertheless I hope to take a stab at it in a not so distant future. In the meantime, my plan is to continue reducing the save file by overriding the serialization process while trying to avoid doing heavy rewrites. I expect to continue doing this for another week or two. Then 0.3.1 will be feature complete and I’ll move into the testing phase. I feel a little bit more optimistic than last week so it’s very possible the next version will be ready by mid November.