Programming War Stories

MAME & Emulation

In 1996, I first discovered emulation: the means of simulating the behavior of older hardware via carefully designed software. This discovery soon led to what would be one of the most rewarding projects of my career: MAME.

Almost as soon I had found out about MAME, I joined the team and began learning how to emulate the arcade games of my youth. As the project grew, I began to see patterns emerge that compelled me to step up and reinvent the architecture of the system to exploit them. Eventually, I got the chance to lead the project and start a move from C to C++.

And then, after 15 years of intense work, I had to set it aside after the birth of my son. Along the way, I did also explore some other arcade emulation approaches, meet a lot of cool people at gatherings, and even give some talks.

Most of my MAME work was done in the bright spotlight, and so there is a lot of documentation and information about what I worked on:

some of my early efforts were written up in arcticles called “MAME Memories” in 2001

a good summary of my MAME contributions

I kept fairly detailed logs until 2007, listing all my work

I had a relatively active blog with lots of good bits & pieces of info

Discovering Emulation

I grew up in the golden age of the arcade, spending many hours of my youth playing games at the local 7-Eleven, or at the Putt Putt Fun Center. Ultimately it was arcade games that motivated me to teach myself how to write computer software so that I could program my own games.

But you couldn’t play the actual arcade games at home. Not really. Sure, there were numerous console and computer versions of the most popular games, but they invariably fell short of the mark and always managed to disappoint.

That is, until I laid my hands on Digital Eclipse’s version of Joust, which was released for the Mac in 1994. This was the first time I had run an arcade game on a home system and really thought it was perfectly recreated. It turns out the reason it was so good was because the original code and hardware was being emulated, though I didn’t realize it at the time.

My next positive home arcade experience was when Namco released the first few entries in their Namco Museum series for the Sony PlayStation in 1995. Here were some of the true classics: Pac-Man, Ms. Pac-Man, Dig Dug, and my all-time favorite Mappy. To my untrained eye, the games all appeared to be arcade accurate, though (as it turns out) they were not actually emulated.

Around the same time, I came across the Atari 2600 emulator Stella, which was looking for someone to update its Mac version. Since I had been working furiously at the time porting LucasArts games from DOS to the Mac, I figured why not do the same for this emulator?

After coming on board, I got the chance to see the Stella source code and began wrapping my mind around it. My gaming experience at LucasArts helped me understand the gross structure of the code, and my past assembly language experience helped me quickly understand how a CPU emulator worked.

A year later I discovered the MAME project, which was not only an up-and-coming arcade emulator for PCs and Macs, but also had the source code freely available. Armed with my knowledge of Stella internals, I was able to make the leap from emulating a home console to emulating an arcade game system, and my fate was sealed.

Joining the Team

Over the following months, I dug into what an arcade game looks like from a hardware perspective, and began to learn what it takes to reverse engineer the correct behavior from examining the game code. I quickly emulated a host of my favorite games, including Mappy, Spy Hunter, Gauntlet, Marble Madness, and more.

After that, I just kept going, digging deep into the catalogs of Atari, Midway, Sega, and other companies, including less well-known ones like Sente, Art & Magic, Leland, Strata (which became Incredible Technologies), Exidy, and others.

I basically discovered that emulating anything was an exciting challenge, even if I didn’t know of or care much about the games themselves. And the more systems I emulated, the more I wanted to expand MAME to support more and more different types of systems.

One of the key early improvements I made was to introduce a formal timing and event system. Previously, all events in the system were tied to the 60Hz refresh rate of the games, but it was clear that much greater precision was required, especially when multiple CPUs and other devices were involved.

Another big early challenge was the Atari “slapstic” chip, a copy protection device that scrambled memory on most 16-bit Atari games, including the System 1 and System 2 games. With the help of Frank Palazzolo, we managed to reverse engineer the chip’s behavior and bring these games to a working state in MAME.

One of the neatest things I got to work on was an emulator for the custom Atari ASAP CPU. This was an in-house RISC processor that Atari engineers had designed. It only ever ended up powering a single prototype game called Beat Head, but investigating this piece of history was fun. I even got to talk to one of the original designers about it.

Reinventing the Architecture

Alongside all of this work emulating systems, I also spent a lot of time working on MacMAME, which was the official Mac version of MAME. This involved improving the UI, optimizing for new systems, and keeping up with architectural changes in the DOS version.

When I switched over to a Windows machine as my main system in 2000, I applied my porting experience and created an official command-line Windows version. At the time, the main version of MAME was developed for DOS, but once the Windows version was stable it quickly became the official target.

In 2002, I added the ability to incorporate artwork into the emulation. This allowed people to view not just the contents of the game monitor, but also the surrounding bezel area, complete with any LEDs, lamps, and other visual effects that were often used to enhance the game experience (especially common on early games).

The following year I added support for games that used hard disks. Up to that point, the code and data for all games in MAME came from ROM images, which were dumped to a computer file using an EPROM programmer. These files were loaded into memory and then processed by the emulator.

For a hard disk, this way of doing things doesn’t make sense: hard disks are much too large to load into memory, and they can be written to, so you need to store the changed data somehow. To address this, I created a new file format called CHD (originally Compressed Hard Disk, but eventually changed to Compressed Hunks of Data).

This new file format supported both reading and writing to a file, and random access of individual sectors. It also supported the concept of a differencing file, a separate file which held all modifications in order to keep the original CHD file intact.

The reason for adding hard disk support was to enable MAME to run the Atari/Midway “Seattle” games, which ran off of a hard disk and also rendered their graphics using the 3dfx Voodoo Graphics chip. This became my next project.

After discovering the low-level programming manual for the 3dfx chip, I set to work writing a software emulator that implemented nearly all of its functionality. In the end it was quite complete, enough so that DOSBox adopted my implementation as the basis for their 3dfx graphics support.

Leading the Project

In 2005 I got the opportunity to take over as leader of the MAME project. The previous project leader had, on several occasions, quit in frustration only to return several months later after I had covered for him. Eventually I got tired of the back and forth and refused to relinquish the reigns.

One of the first things I did as lead was to introduce a domain that we actually owned (mamedev.org), rather than relying on some other advertising-laden emulation site to host our home. I set up an integrated wiki for documentation, hosted a source code browser, and offered access to all old revisions.

Since the new web server was a full virtual host, I was also able to install Subversion and begin hosting a proper source code repository. Believe it or not, 8 years into the project, it was still being maintained and merged by hand! I figured this had to end.

As project lead, I also felt even more at liberty to introduce direction for the project. One thing I did pretty quickly was to enable the MAME debugger on all builds. Since MAME’s purpose was to document the inner workings of arcade systems, it made sense that even the release build should allow people to see what was happening under the hood.

In 2006, I introduced a new way of outputting graphics to the screen. Previously, MAME would draw everything to a single bitmap, which would be copied to the screen at the game’s refresh rate. With the new system, there could be multiple bitmaps that were composited by the video hardware.

This new mechanism had three key benefits. First, it allowed MAME to cleanly support games with multiple screens. Second, it allowed MAME to include game artwork, LEDs, lamps, and other visual effects that were often used to enhance the game experience. And third, it allowed MAME to take some advantage of accelerated graphics.

Around this time I had also begun to be very comfortable reading game schematics, and realized that a lot of game drivers were sorely lacking in accuracy. I began a campaign to fix some of these games, often rewriting a game driver from scratch in order to be more accurate to how the system actually functioned.

The Journey to C++

Certainly the biggest change I made in MAME during my reign as project leader was to move the system from C to C++. Over the years, MAME had gradually grown more complex, but overall retained a fairly object-oriented design underneath, and so the move to C++ was a pretty natural fit.

The first step in the process was to create the concept of a device class, which contained all the code necessary to implement, say, a CPU or a sound chip or a GPU or any other bit of functionality. Devices used multiple inheritance to identify themselves as CPUs or sound generators, and this allowed for a more logical method of composing systems.

In order to make C++ work, I had to create a delegate class, which is a bit of template trickery that enables calling a method attached to an object. Delegates had existed before, but due to MAME’s architecture, our delegates needed the ability to reference an object by name and bind to them later, and so I had to craft my own classes to do so.

Over the course of the following years I spent much time converting various systems in MAME into C++ classes, starting with the most naturally fitting cases. Eventually many systems in MAME became devices, and in doing so we got rid of most of the global variables and hard-coded limitations that had existed previously.

In 2008 I extended the CHD format to support laserdiscs, in order to enable MAME to support the handful of laserdisc-based games that came out in the 1980s. During this process I investigated how laserdiscs worked and developed tools to extract the frame metadata that was encoded in the vertical blanking interval.

Around the same time I introduced UML, or “unified machine language”, which was essentially an intermediate representation that could be used as the interface between the front end and back end of a dynamic recompiler. The goal was to enable recompilation of MIPS and PowerPC code in a generic way to improve performance.

I wrote the initial implementation with the MIPS front-end and both x86 and x64 back-ends in mind. I soon added a PowerPC front end to the mix, and others added support for other architectures. Eventually I envisioned there to be at least an ARM back-end, and perhaps front-ends for x86 and ARM, but it hasn’t happened yet.

Once my son was born in 2011, my involvement as MAME’s project leader was quickly put in jeopardy. There just wasn’t enough time to juggle everything. So in 2012 I relinquished leadership, and began to reduce my involvement in the project.

These days I occasionally advise the project, but generally don’t actually write much code. Most of my long-term projects, such as the conversion from C to C++ and creating more device classes, have taken on a life of their own and have been largely completed by others.

Other Arcade Emulation

In 2004, I decided to write a standalone emulator for the 3D game Radikal Bikers as an experiment, since MAME at the time was not up to running it at full speed. This was the first and only emulator that I worked on that featured static binary translation and that rasterized the 3D graphics in hardware.

I have also regularly attended California Extreme, an awesome arcade game show that takes place in Silicon Valley each year. Collectors from all around the area bring in their classic arcade games and set them all on free play so you can play them all day for a fixed cost. It’s also a great place to meet friends and video game luminaries.

In 2008 and 2009 I was invited to give talks at the conference. The first (and better) talk is actually posted on YouTube. You can view it below: