The Dark Mod 2.05 arrives with new "Modular Content" and optimizations...

Posted by nbohr1more on Feb 16th, 2017

Broken Glass Studios are very pleased to announce the release of The Dark Mod 2.05!



This update includes literally hundreds of new high-quality models & other assets, along with an update to the editor that will change the way TDM missions look going forward. Please see the latest official trailer above for a quick teaser.







New Modular Models

Springheel has created over two hundred modular models that can be combined to create multiple types of environments. Tavern interiors, city streets, industrial warehouses and more. By placing modules together, high fidelity scenes can be created quicker and more easily than ever before.

New Official Mission Many of the above new assets are showcased in a new "introductory mission" that is included in this update. This mission is the first of a three mission story which serves as our official introduction to TDM's game-play and setting.





Better Performance



You should notice an increase in performance speed in TDM 2.05, thanks to rendering improvements made by duzenko and nbohr1more.



The main improvement to TDM 2.05's performance is Lightgem Rendering.

To accomplish the Lightgem functionality in The Dark Mod we originally had to dig around in the game SDK for places where we could communicate with the renderer. This was primarily done via the in-game camera systems and tricks with piping screenshot files to a RAM workspace.

(some background info) Modwiki.xnet.fi

The way the system works is:

1) If it is time to create a Lightgem image, initialize a new "View" (aka camera)

2) The player model, held objects, and other non-essential objects are hidden

3) The player model is replaced with a low-poly representation (octahedron)

(Octahedron with eyes representing cameras)

4) The camera is positioned facing down at the octahedron (below)

5) The scene is rendered and captured to memory





6) The camera is positioned facing upward looking at the bottom of the octahedron (below)

7) The scene is rendered and the image is captured to memory





8) The images are combined and then analyzed for the brightest values

9) A final light level value is return to the game code to be adjusted further by other visibility factors

With the above screenshots, the process certainly looks wasteful but those are only hand-made representations I created by spawning the octahedron model in-game. The real Lightgem render would crop away everything but the boundaries of the octahedron leaving you with a square made of 4 shaded triangles.

Still, id Tech 4 (Doom 3 engine) is very nervous about throwing away scene data during a render. If there is anything that could cast a shadow or show-up in a mirror, transparency, or postprocess the engine will tend to do a substantial amount of setup before deciding to discard the data just before sending it to the video card for render (or will actually render the invisible stuff if all the culling heuristics fail).

( Edit: Hah! It's even worse than I thought. According to Serpentine (one of the old core coders) the lightgem camera is actually inside the octahedron (an optimization I was planning on doing). This means that the visibility culling heuristics are really falling down or else these latest changes wouldn't have been so effective. )

In TDM 2.03, SteveL was struggling with the particle system and (as part of his cleanup work) he ensured that particles no longer make-up the above workload. To everyone's surprise, this was a pretty significant performance boost to scenes with lots of particles.

In 2.04 SteveL added the ability to omit Lights from the Lightgem calculation so that mappers could place decorative "bounce lights" around their maps without having to struggle against the Lightgem (because the brightest part of those lights was typically floating in mid-air and invisible to the player 'till they stepped into it).

This work laid the groundwork for the 2.05 Lightgem changes.





In 2.05 the octahedron was given a new material type "islightgemsurf" to allow the following change:

Skip all surfaces that can't create shadows except "islightgemsurf"

This, along with skipping all lights that the player isn't "inside of" drastically cut the amount of work for the Lightgem render phase.

But we were not done yet!





A new coder "Duzenko" (and veteran of the Russian "Darkfate" Thief community) arrived to complain about the cost of the Lightgem render prior to the above changes.

Parallel to the above development, he looked into another expensive aspect of the Lightgem:

"Pixel reading".

Even though our Lightgem image capture was given a superhighway to native memory space in 1.08 the process of feeding the render to RAM was still pretty slow. Duzenko studied the problem and found that an ancient OpenGL 2.1 feature called Pixel Packed Buffers would do a better job than our very legacy OpenGL code. This lead to some drastic performance improvements in fillrate heavy scenes like the cave in "No Honor Among Thieves" mission 1.

A little tangent on tweaking...

Part of the genesis of the above Lightgem optimization was my own pet project to improve our documentation on performance tweaks and cvars:

In the forums, there had been at least a few confusing\inconclusive discussions about one tweak that seemed to offer the biggest performance boost but came with some pretty bad caveats "tdm_lg_interleave"...

tdm_lg_interleave

This console variable allows you to specify how often to capture Lightgem renders. By default, this is done in every frame. Users were finding that setting it to 2 or 3 would nearly double FPS is some scenes but it was also causing terrible stutter, especially where the average FPS was below 30. It occurred to me that automatically enabling or disabling the larger skip values depending on your FPS was the ideal solution, so I created a new cvar tdm_lg_interleave_min that allows you to say "if my FPS is below X then set tdm_lg_interleave to 1 to avoid stutter". The default value is 40 (FPS).

tdm_lg_split

This console variable was created shortly after tdm_lg_interleave at the dawn of TDM to address the same stutter concern above. It was posited that the biggest problem with tdm_lg_interleave was that the frame load was so uneven with the huge wallop of frame cost during the full Lightgem scenes. One of the developers suggested calculating only the top or bottom of the Lightgem capture per pass so that the render load would be much closer to the non-Lightgem passes. This worked great but was never enabled by default in The Dark Mod because it caused particle effects to flicker. With SteveL's exclusion of particles in 2.03 the flicker is gone so this feature has been enabled by default. Further, it can be paired with tdm_lg_interleave so that the interleave frames calculate half the scene every x number of frames.

With the above knowledge, feel free to try setting tdm_lg_interleave to higher values now. I have been playing with tdm_lg_interleave 3 but I've tested it as high as 8 without seeing any stutter or anomalies. For most missions, I can set tdm_lg_interleave_min to 1 along with tdm_lg_interleave 3 and gain FPS (even at sub 30FPS areas ) with no consequence for frame stutter but Penny Dreadful 3 required me to increase the min to 50 to avoid stutter at the start area. Your mileage may vary.

image_downSize

The old image_downSize cvar settings that allow you to shrink and degrade texture quality to reduce video RAM consumption (and bandwidth consumption), no longer cause menus and GUI's (maps, signs, etc) to become blurry.

If you have limited video memory, try reducing these settings now that they no longer affect play-ability.

New AI

Grayman has added a working fire elemental to our list of available. AI opponents that will no doubt appear in future missions.

Cubemap Lights

Shortly before the horrific death of the legendary Doom3world.org forums (due to a Spammer invasion), Sikkpin (the author of Sikkmod) went on a tangent and solved a longstanding riddle about Doom 3 lighting...



Doom 3's flexible material system allows you all sorts of image manipulation options to feed into what lights project on surfaces. You can have anything as simple as a flat circle to a stained-glass window pattern or even a movie projector film. During the wild experiments in the early Doom 3 mod scene, some folks found that lights could accept "Cubemaps" (six images representing all six degrees of projection) but the result was completely wrong. This was because the sampler was just taking data from the final 2D transform of the image and using it as the projection image. Sikkpin worked around this problem by creating new vertex projection matrices in the vertex shader and changed the sampler behavior in the pixel shader.

He also did a little trick to tell the shader how to behave if the light entity was set as Omni or Projected and finally he implemented an ambient component based on Half Life 2's ambient cubemap system.

All packed into one shader, phew!

The benefit of cubemap lights is that you can more realistically represent what comes out of complex light sources such as our desk lamps.

Crude example (Before):

Notice how the floor has the same projection image as the ceiling. This is due to the 1D + 2D image setup for normal Doom 3 lights.

Crude Example (After)

Now only the ceiling has the pattern so the behavior is more realistic. Further the image is contorted spherically the way a real light projects such an image.

(Folks who regularly use Unity "Cookie Lights" should be well aware of how cubemap based light sources work.) You can also project 6 different lights for the price of rendering one light (as long as you are willing to accept that these lights must all be oriented in a cube pattern around one light_center)

HL2 in The Dark Mod

Now mission authors can use the Skybox or other cubemap images to create ambient lighting and environment reflections similar to HL2's ambient cubemap system.

Even though these features were created before Doom 3 went Open Source, we could not integrate them because we would've had to replace all existing lights with Cubemap lights.

Thanks to Rebb's work to setup a dedicated Ambient light shader pass, I was able to add branches to the renderer so the above shaders can occupy the same scene as the standard Doom 3 lights.

Right now we only have a few cube materials so if you know of any resources to generate Cubemap \ Cookie light textures or know of open licensed asset collections, please let us know.

Thank you Sikkpin (wherever you are)!

EXPERIMENTAL: UNCAPPED FPS!

Setting com_fixedTic 1 will now allow The Dark Mod to render over 60FPS without causing everything to speed-up. Duzenko ported some of the BFG code around this functionality to reduce the CPU impact of frame sync (for lowend users) but this feature can be useful to folks experimenting with Virtual Reality HMD.

I will note that there are a few quirks.

1) FPS above 200 can paradoxically cause the game to run in slow-motion or cause other strange pacing issues

2) The breath potion exhausts more quickly the higher the FPS you have

To prevent FPS from going over 144 or 90 you can create an autoexec.cfg file and place seta r_displayRefresh "90" in it to force your screen refresh then either use r_swapinterval 1 (enable vsync in-game) or enable vsync via your driver. Setting r_displayRefresh can also smooth out frame behavior on digital displays (HDMI, DVI, DisplayPort) where refresh detection can be problematic for Doom 3 so it's a good idea to set this via an autoexec.cfg regardless.

Bug fixes



Other changes include a fix to stop sleepers from falling through the floor, new beggar entities, new combination safe entities, and fixes to stop the player or AI from getting trapped by moving elevators. A more detailed list of changes can be seen in the change log and on the Wiki

Special thanks go to Bob Necro for his large texture collection contributions, Airship Ballet for prefab containers and paintings, Epifire for Steampunk models, Spooks for his example cubicLight textures, Outsider One for his Turkish menu translation, and Hamlet & Nightstalker who worked to resolve Linux compile issues and distribution problems.

Darkradiant updated

Greebo has also just release the latest version of Dark Radiant 2.2.1

Of significance in this version, you can now scale models in the editor and save the scaled model as a new model. This adds substantial flexibility to the modular model based approach described by Springheel above.

Also, support for grouping selected items for manipulation has been added. You can group huge collections of models and rotate or translate the entire collection at once, saving substantial amounts of time during mission creation.

Additional info



You can update your copy of TDM by running tdm_update.exe and as always, older saved games will be incompatible with TDM 2.05, so be sure to complete any missions you're in the middle of before updating!

The Dark Mod on GREENLIGHT!





At the (unexpected) twilight of the GreenLight offering, The Dark Mod team has finally gone though all the legal, financial, and social issues to enter into the GreenLight process.

Hopefully we will make it onto Steam before this process is ended. Although votes aren't the only metric to qualify for GreenLight approval, I feel that I must mention (boast?) that we were in the Top 10 ranking just recently!

Please visit our GreenLight page and cast your vote.