Posted by kgh on October 19, 2004 at 11:43 AM EDT

The question came up recently of "should we make Swing truly multithreaded?" My personal answer would be "no", and here's why...

The Failed Dream

There are certain ideas in Computer Science that I think of as the "Failed Dreams" (borrowing a term from Vernor Vinge). The Failed Dreams seem like obvious good ideas. So they get periodically reinvented, and people put a lot of time and thought into them. They typically work well on a research scale and they have the intriguing attribute of almost working on a production scale. Except you can never quite get all the kinks ironed out...

For me, multithreaded GUI toolkits seem to be one of the Failed Dreams. It seems like the obvious right thing to do in a multithreaded environment. Any random thread should be able to update the GUI state of buttons, text fields, etc, etc. Damned straight. It's just a matter of having a few locks, what can be so hard? OK, there are some bugs, but we can fix them, right? Unfortunately it turns out not to be so simple...

From observation, there seems to be an amazing tendency towards deadlocks and race conditions in multithreaded GUIs. I first heard about this issue anecdotally from people who had worked with the Cedar GUI libraries at Xerox PARC in the early 80's. That was a community of extremely smart people who really understood threading, so the assertion that they were having regular deadlock issues within GUI code was intriguing. But maybe that was flawed data or an exceptional situation.

Unfortunately that general pattern has been repeated regularly down the years. People often start off trying for multithreading and then slowly move to an event queue model. "It's best to let the event thread do the GUI work."

We went through this with AWT. AWT was initially exposed as a normal multi-threaded Java library. But as the Java team looked at the experience with AWT and with the deadlocks and races that people had encountered, we began to realize that we were making a promise we couldn't keep.

This analysis culminated in one of the design reviews for Swing in 1997, when we reviewed the state of play in AWT, and the overall industry experience, and we accepted the Swing team's recommendation that Swing should support only very limited multi-threading. With a few narrow exceptions all GUI toolkit work should occur on the event processing thread. Random threads should not try to directly manipulate the GUI state.

Why is this so hard?

John Ousterhout gave a great Usenix talk on Events versus Threads in 1995 that explores some of the tradeoffs between thread-driven and event-driven programming and he correctly points out many reasons why multi-threaded programming is hard and why event driven programming can be simpler. I don't necessarily agree with his analysis for all kinds of programs, but I do agree for GUI programs.

The particular threading problems of GUI toolkits seem to me to arise from the combination of input event processing and abstraction.

The problem of input event processing is that it tends to run in the opposite direction to most GUI activity. In general, GUI operations start at the top of a stack of library abstractions and go "down". I am operating on an abstract idea in my application that is expressed by some GUI objects, so I start off in my application and call into high-level GUI abstractions, that call into lower level GUI abstractions, that call into the ugly guts of the toolkit, and thence into the OS. In contrast, input events start of at the OS layer and are progressively dispatched "up" the abstraction layers, until they arrive in my application code.

Now, since we are using abstractions, we will naturally be doing locking separately within each abstraction. And unfortunately we have the classic lock ordering nightmare: we have two different kinds of activities going on that want to acquire locks in opposite orders. So deadlock is almost inevitable.

This problem will initially surface as a series of specific threading bugs. And people's first reaction is to try to adjust the locking behavior to resolve the specific bugs. Let's release that lock there and then lets use more clever locking over here. Well, that is kind of a fun activity, but it is trying to fight back an oceanic tidal force. The cleverer locking typically turns into a combination of subtle races (due to lack of locking) or clever and intricate deadlocks (due to the clever and intricate locking). We went through a bunch of that in 95-97.

Notice that the problems extends beyond the GUI toolkit layers and also appears between the toolkit layer and the application level. With great difficult one might try to adopt a single lock for all activity within the GUI layer, but the same problem then resurfaces a level up.

So what's the answer? Well, at some point you have to step back and observe that there is a fundamental conflict here between a thread wanting to go "up" and other threads wanting to go "down", and while you can fix individual point bugs, you can't fix the overall situation.

This lead to the solution that the Swing team adopted and which is used by most leading GUI toolkits: run all GUI activity on a single event thread. This means that in some sense all GUI activity becomes event driven, and the "down" threads become just a new kind of event.

This demonstrably works. It is possible to write complex GUI apps that work reliably. Hurrah! But it does make managing long running activities tougher. I wrote a smallish Swing program that I use periodically to selectively zap large boring attachments from my email archives. I don't want to hang the GUI while it reads tens of megabytes of emails, and I also want to display a progress monitor, so I ended up having to carefully balance handing off big activities to worker threads and handing GUI activities back to the event thread. It is probably more complicated than it would be if I had a magic multi-threaded library, but it has the significant saving grace that it actually seems to work reliably.

Subtleties

Are things really so black and white? Surely there have been people who have used multi-threaded toolkits successfully? Yes, but I think this demonstrates one of the characteristics of the Failed Dreams.

I believe you can program successfully with multi-threaded GUI toolkits if the toolkit is very carefully designed; if the toolkit exposes its locking methodology in gory detail; if you are very smart, very careful, and have a global understanding of the whole structure of the toolkit. If you get one of these things slightly wrong, things will mostly work, but you will get occasional hangs (due to deadlocks) or glitches (due to races). This multithreaded approach works best for people who have been intimately involved in the design of the toolkit.

Unfortunately I don't think this set of characteristics scale to widespread commercial use. What you tend to end up with is normal smart programmers building apps that don't quite work reliably for reasons that are not at all obvious. So the authors get very disgruntled and frustrated and use bad words on the poor innocent toolkit. (Like me when I first started using AWT. Sorry!)

Another wrinkle: it is possible to have multiple simultaneous GUI activities within a Java VM by using multiple event threads. That works provided the different activities are almost entirely isolated, have their own distinct GUIs (no shared components or mixed hierarchies) and provided the very lowest toolkit level can correctly dispatch events to the right event thread with minimal locking. This is useful in (for example) running multiple applets within one JVM. But it isn't a very general solution - most applications need to live within the constraint of only a single event thread.

In this note I've most been covering why Swing and other toolkits are essentially single-threaded. Chet recently blogged on some related topics around why multi-threading complicates user programs and normally won't help raw graphics performance.

Also, before I forget, some people are probably remembering that "processes and monitors are duals". Well, yes, it's true. In some sense we are using the event thread to implement a global lock. We could invert things, and create a global lock that is equivalent to the event queue. This would be fairly ugly and would require wide coordination and undermine a lot of abstractions. But the larger problem is that Java developers tend to use multiple locks and if they are to preserve the equivalence with an event queue model, they will need to follow various non-obvious rules about how they interact with these other locks. The event queue model makes the central single lock much more visible and explicit, and on the whole that seems to help people to more reliably follow the model and thus construct GUI programs that work reliably.

Conclusion

I guess the bottom line is that like many others I would really like to see a flexible, powerful, truly multi-threaded GUI toolkit. But I don't know how to get there - at this point there is fairly strong experience that the obvious approaches for multi-threading don't work. Maybe in future years people will come up with a radically new and better approach, but for now the answer seems to be that events are our friends.

- Graham