Second Reality Code Review: Part 1 (Introduction)

On July 23, 2013 the source code of Second Reality was released. Like many, I was eager to look at the internals of a demo that inspired me so much over the last 20 years.



I was expecting a monolithic mess of assembly but instead I found a surprisingly elaborated architecture, mixing several languages in an elegant way. The code is something like I had never seen before that perfectly represents two essential aspects of demomaking :

Team work.

Obfuscation.

As usual I have cleaned up my notes into an article: I hope it will save some a few hours and maybe inspire others to read more source code and become better engineers.

The Demo

Before starting with the code, here is a link to a HD video capture (by Michael Huth) of the legendary demo. Nowadays, it is the only way to fully experiment the marvel without graphic glitches (even DOSBox cannot run it properly).





First contact with the code

The source code is hosted on GitHub. It is one git command away :



git clone git@github.com:mtuomi/SecondReality.git

At first the content is confusing: 32 folders and a mysterious U2.EXE that won't run under DosBox.







The working title of the demo was "Unreal 2" (the first "Unreal" was Future Crew's previous demo, released for the first Assembly in 1992). Only later during development the name was changed to "Second Reality". This explains "U2.EXE" but not why it doesn't work...





Runnig CLOC provides interesting metrics:



------------------------------------------------------------------------------- Language files blank comment code ------------------------------------------------------------------------------- Assembly 99 3029 1947 33350 C++ 121 1977 915 24551 C/C++ Header 8 86 240 654 make 17 159 25 294 DOS Batch 71 3 1 253 ------------------------------------------------------------------------------- SUM: 316 5254 3128 59102 -------------------------------------------------------------------------------





The codebase is "only" 50% assembly.

The codebase is almost twice as big as Doom engine.

Seventeen makefiles are there. Why not only one ?

Running the demo

It is hard to figure it out but it is possible to run what was released via DosBox: You have to rename U2.EXE and run it form the right place. Once I learned about the internals of the code, it made a lot of sense:





CD MAIN MOVE U2.EXE DATA/SECOND.EXE CD DATA SECOND.EXE



Et voila !





Architecture

Back in the 90s, the demo was mostly distributed via floppy disks. After unzipping, two big files would be installed : SECOND.EXE and REALITY.FC :



. <DIR> 01-08-2013 16:40 .. <DIR> 01-08-2013 16:40 FCINFO10 TXT 48,462 04-10-1993 11:48 FILE_ID DIZ 378 04-10-1993 11:30 README 1ST 4,222 04-10-1993 12:59 REALITY FC 992,188 07-10-1993 12:59 SECOND EXE 1,451,093 07-10-1993 13:35 5 Files(s) 2.496,343 Bytes. 2 Dir(s) 262,111,744 Bytes free.

Coming from a game developer background I always expected the big picture to be :



SECOND.EXE : The engine with all effects in the executable.

: The engine with all effects in the executable. REALITY.FC : The assets (music, sound effects, images) in a proprietary/encrypted format ala Doom WAD .

But after reading MAIN/PACK.C I discovered that I was completely wrong : The "Second Reality" engine is just a Loader and an Interrupt server (called DIS). Each scene (also called "PART") during the demo is a full self-contained DOS executable. Each parts are loaded by the Loader and started one after an other. The PARTs are stored in encrypted form at the end of SECOND.EXE :







REALITY.FC contains the two musics played during the demo (with padding and a marker at the beginning for obfuscation).

contains the two musics played during the demo (with padding and a marker at the beginning for obfuscation). SECOND.EXE contains a loader and a Demo Interrupt Server (DIS).

contains a loader and a Demo Interrupt Server (DIS). Appended after the end of SECOND.EXE , the 32 PARTs of the demo as DOS executables (encrypted).

This architecture offers many advantages :

Improved collaborative process in the team: Each member of the team can work independently on its PART as long as it produces an executable with a _start symbol and remains within the memory budget (450K).

symbol and remains within the memory budget (450K). Encrypting the EXEs at the end of SECOND.EXE and having them loaded at runtime made reverse engineering harder.

and having them loaded at runtime made reverse engineering harder. Fast startup : The Loader and DIS are only 20K. DOS loads them super fast.

Easy memory management: Once a PART is done, the loader overrides it with the next PART and free ALL the allocated memory.

No asset manager/loader needed: As seen later each PART had its assets (mostly images) compiled within the code: Loading an EXE also loaded all the images it needed like a nice package.

Any language could be used to program a PART: C, Assembly ... and Pascal can be found in the code.

Recommended readings

The three pillars to understand Second Reality source code are VGA, Assembly and PC architecture (PIC&PIT Programming). Here are incredibly helpful links :



Next

Second Reality Engine

@