sjbrown's Writing Games Tutorial

By Shandy Brown. Please send comments / corrections via email to tutorial@ezide.com

Last Update: March 2011

We will start by trying to create a program where a little man moves around a grid of nine squares. This is a overly simple example, but easily extensible so we won't get tied up in the game rules, instead we can focus on the structure of the code.

We haven't even got to the Model yet, and already we have a difficulty. If you are familiar with using PyGame, you are probably used to seeing a main loop like this:

#stolen from the ChimpLineByLine example at pygame.org main(): ... while 1: #Handle Input Events for event in pygame.event.get(): if event.type == QUIT: return elif event.type == MOUSEBUTTONDOWN: fist.punch() elif event.type == MOUSEBUTTONUP: fist.unpunch() #Draw Everything allsprites.update() screen.blit(background, (0, 0)) allsprites.draw(screen) pygame.display.flip()

ControllerTick(): #Handle Input Events for event in pygame.event.get(): if event.type == QUIT: return False elif event.type == MOUSEBUTTONDOWN: fist.punch() elif event.type == MOUSEBUTTONUP: fist.unpunch() return True ViewTick(): #Draw Everything ... main(): ... while 1: if not ControllerTick(): return ViewTick()

Rationale Readers with some experience writing games may be balking at this point, thinking MVC is more complex than necessary, and that it will add unneeded overhead, especially when the goal is to create a simple, arcade-style game. Now historically, arcade games were just that, games written for arcade machines. The code ran "close to the metal", and would squeeze all the resources of the machine just to get a 3-color ghost to flash blue every other frame. In the 21st century, we have resource-rich personal computers where applications run a couple layers above the metal. Hence, organizing your code into a pattern has a small relative cost. For that small cost, you get the following advantages: more easily add networking, easily add new views (file loggers, radars, HUDs, multiple zoom levels, ...), keep the Model code "cleaner" by decoupling it from the view and the controller, and I contend, more readable code. Readers with some experience writing games may be balking at this point, thinking MVC is more complex than necessary, and that it will add unneeded overhead, especially when the goal is to create a simple, arcade-style game. Now historically, arcade games were just that, games written for arcade machines. The code ran "close to the metal", and would squeeze all the resources of the machine just to get a 3-color ghost to flash blue every other frame. In the 21st century, we have resource-rich personal computers where applications run a couple layers above the metal. Hence, organizing your code into a pattern has a small relative cost. For that small cost, you get the following advantages: more easily add networking, easily add new views (file loggers, radars, HUDs, multiple zoom levels, ...), keep the Model code "cleaner" by decoupling it from the view and the controller, and I contend, more readable code.

Let's examine the infinite while loop in the last bit of code. What is its job? It basically sends the Tick() message out to the View and the Controller as fast as the CPU can manage. In that sense it can be viewed as a piece of hardware sending messages into the program, just like the keyboard; it can be considered another Controller.

Perhaps if "wall clock" time affects our game there will be even another Controller that sends messages every second, or perhaps there will be another View that spits text out to a log file. We now need to consider how we are going to handle multiple Views and Controllers. This leads us to the next pattern in our architecture, the Mediator.

We implement the Mediator pattern by creating an EventManager object. This middleman will allow multiple listeners to be notified when some other object changes state. Furthermore, that changing object doesn't need to know how many listeners there are, they can even be added and removed dynamically. All the changing object needs to do is send an Event to the EventManager when it changes.

If an object wants to listen for events, it must first register itself with the EventManager. We'll use the weakref WeakKeyDictionary so that listeners don't have to explicitly unregister themselves. [TODO: more weakref rationale. gc, etc]

We will also create an Event class to encapsulate the events that can be sent via the EventManager.

class Event: """this is a superclass for any events that might be generated by an object and sent to the EventManager """ def __init__(self): self.name = "Generic Event" class EventManager: """this object is responsible for coordinating most communication between the Model, View, and Controller. """ def __init__(self ): from weakref import WeakKeyDictionary self.listeners = WeakKeyDictionary() #---------------------------------------------------------------------- def RegisterListener( self, listener ): self.listeners[ listener ] = 1 #---------------------------------------------------------------------- def UnregisterListener( self, listener ): if listener in self.listeners.keys(): del self.listeners[ listener ] #---------------------------------------------------------------------- def Post( self, event ): """Post a new event. It will be broadcast to all listeners""" for listener in self.listeners.keys(): #NOTE: If the weakref has died, it will be #automatically removed, so we don't have #to worry about it. listener.Notify( event )

class KeyboardController: ... def Notify(self, event): if isinstance( event, TickEvent ): #Handle Input Events ... class CPUSpinnerController: ... def Run(self): while self.keepGoing: event = TickEvent() self.evManager.Post( event ) def Notify(self, event): if isinstance( event, QuitEvent ): self.keepGoing = False ... class PygameView: ... def Notify(self, event): if isinstance( event, TickEvent ): #Draw Everything ... main(): ... evManager = EventManager() keybd = KeyboardController() spinner = CPUSpinnerController() pygameView = PygameView() evManager.RegisterListener( keybd ) evManager.RegisterListener( spinner ) evManager.RegisterListener( pygameView ) spinner.Run()

Diversion: Event Types and Selective Listeners As we get more and more listeners, we may find that it's inefficient to spam every listener with every event. Perhaps some listeners only care about certain events. One way to make things more efficient is to classify the events into different groups. As we get more and more listeners, we may find that it's inefficient to spam every listener with every event. Perhaps some listeners only care about certain events. One way to make things more efficient is to classify the events into different groups. For the purpose of this guide, we'll just use one kind of event, so every listener gets spammed with every event.

Advanced Event Managers If you try to use this particular Event Manager class for your own project, you might notice it has some shortcomings. In particular, if a block of code generates events A and B sequentially, and a listener catches event A and generates event C, the above Event Manager class will process the events in the order A,C,B, instead of the desired order of A,B,C. In the later examples, you can see an example of a more advanced Event Manager that always delivers events in the desired order. If you try to use this particular Event Manager class for your own project, you might notice it has some shortcomings. In particular, if a block of code generates events A and B sequentially, and a listener catches event A and generates event C, the above Event Manager class will process the events in the order A,C,B, instead of the desired order of A,B,C. In the later examples, you can see an example of a more advanced Event Manager that always delivers events in the desired order.

Here is a Model that has worked for me and is general enough to adapt to many types of games:

In our example, "little man" will be our sole Charactor.

In our example, the Map will be a discrete Map having a simple list of nine sectors.

In our example, we will allow no diagonal moves, only up, down, left and right. Each allowable move will be defined by the list of neighbors for a particular Sector, with the middle Sector having all four.

This example makes use of everything covered so far. It starts out with a list of possible events, then we define our middleman, EventManager, with all the methods we showed earlier.

Next we have our Controllers, KeyboardController and CPUSpinnerController. You'll notice keypresses no longer directly control some game object, instead they just generate events that are sent to the EventManager. Thus we have separated the Controller from the Model.

Next we have the parts of our PyGame View, SectorSprite, CharactorSprite, and PygameView. You'll notice that SectorSprite does keep a reference to a Sector object, part of our model. However we don't want to access any methods of this Sector object directly, we're just using it to identify which Sector object the SectorSprite object corresponds to. If we wanted to make this limitation more explicit we could use the id() function.

The Pygame View has a background group of green square sprites that represent the Sector objects, and a foreground group containing our "little man" or "red dot". It is updated on every TickEvent.

Finally we have the Model objects as discussed above and ultimately the main() function.

Here is a diagram of the major incoming and outgoing events.

The code in the following sections is written incrementally, so don't expect to just take the code from the first section and write a game with it. Subsequent sections sometimes address problems with the previously shown code and explain how to overcome those problems.

Rapid Development One of the goals of this tutorial is to show how game development can be done rapidly. Usually anything involving networking is anathema to "rapid" because once you introduce networking, you introduce multiprocessing, latency, error handling, and general hair-pulling. A principle of the code examples in this tutorial is to make sure that the game can be run without even turning on networking. The networking feature should have zero impact on the code in example.py -- running the game in single-player mode should not execute any networking-related code paths. By being strict about this separation, we hope to make it possible to develop the game rapidly, no matter what snags the networking code may introduce. One of the goals of this tutorial is to show how game development can be done. Usually anything involving networking is anathema to "rapid" because once you introduce networking, you introduce multiprocessing, latency, error handling, and general hair-pulling. A principle of the code examples in this tutorial is to make sure that the game can be run without even turning on networking. The networking feature should have zero impact on the code in example.py -- running the game in single-player mode should not execute any networking-related code paths. By being strict about this separation, we hope to make it possible to develop therapidly, no matter what snags the networking code may introduce.

In the "Strict" Client-Server structure, there is one "3rd party" server that all the clients connect to. Any change to the authoritative game model must happen at the server. A client can predict the authoritative state, but it must not put faith in game state until it hears from the server that that is, in fact the case. An example game would be World of Warcraft.

Client-Server @ Wikipedia

In the "Servent" Client-Server structure, one of the players, usually the one that starts the game, acts as the server as well. This suffers from the drawback that other players trust the game state as much as they trust that particular player. However no 3rd party is needed. Examples can be found in many first person shooter games. This structure is often paired with a 3rd party "matching" server that connects players with each other and then hands off to the Servent host.

Servent @ Wikipedia

In the Peer to Peer structure, all hosts have identical roles. The great benefit of a Peer to Peer structure is that it robustly deals with network disconnects from individual hosts. However trust is compromised. Trust can be bolstered by adopting token passing strategy such that the host holding the token acts as a Servent.

Peer to Peer @ Wikipedia

For our examples, we will examine the "Strict" Client-Server structure.

This asynchronous quality is fundamental to network-related code. Luckily designing our code such that there is an independent EventManager and well-defined events will make dealing with asynchronous messages from the network fairly painless.

This tutorial will use the Twisted framework for network-related code. I recommend reading the Twisted documentation, though it should not be necessary to get through this tutorial. (note, a lot of the Twisted documentation focuses on writing servers where the client implementation is unknown. I recommend skipping forward to the sections on Perspective Brokers) The ideas presented here should be independent from the choice of Twisted; the examples could just as well be implemented with raw sockets or carrier pigeons.

Twisted is a framework that hides the queue from us, it expects the programmer to call reactor.run(), which is a mainloop that consumes the queue and fires off callbacks. The callbacks are provided by the programmer.

Normally a server is something that runs as a daemon or in a text console; it does not have a graphical display. We can do this simply by replacing PygameView with a TextLogView as follows:

#------------------------------------------------------------------------------ class TextLogView: """...""" def __init__(self, evManager): self.evManager = evManager self.evManager.RegisterListener( self ) #---------------------------------------------------------------------- def Notify(self, event): if isinstance( event, CharactorPlaceEvent ): print event.name, " at ", event.charactor.sector elif isinstance( event, CharactorMoveEvent ): print event.name, " to ", event.charactor.sector elif not isinstance( event, TickEvent ): print event.name

Another thing we don't need in a server is keyboard input, so we can remove the KeyboardController. Where do input messages come from instead? They come from the network, so we'll need a Controller object for the messages sent by the clients, NetworkClientController.

from twisted.spread import pb #------------------------------------------------------------------------------ class NetworkClientController(pb.Root): """...""" def __init__(self, evManager): self.evManager = evManager self.evManager.RegisterListener( self ) #---------------------------------------------------------------------- def remote_GameStartRequest(self): ev = GameStartRequest( ) self.evManager.Post( ev ) return 1 #---------------------------------------------------------------------- def remote_CharactorMoveRequest(self, direction): ev = CharactorMoveRequest( direction ) self.evManager.Post( ev ) return 1 #---------------------------------------------------------------------- def Notify(self, event): pass

Caveat It could be tempting to make all of the objects remotely referenceable. (ie, inherit from pb.Referenceable) The problem with that approach is that it tightly couples the networking code with the rest of the code. It's preferable to separate the networking code so that the other objects just use the event passing strategy described by the Mediator pattern. It could be tempting to make all of the objects remotely referenceable. (ie, inherit from pb.Referenceable) The problem with that approach is that it tightly couples the networking code with the rest of the code. It's preferable to separate the networking code so that the other objects just use the event passing strategy described by the Mediator pattern. In our examples, we're only going to have one class in the server that is referenceable, and also only one class in the client. [TODO: expand on this]

We also don't need the CPUSpinnerController in the server, so we've removed that, and replaced it with Twisted's reactor, which similarly provides a run() method.

def main(): evManager = EventManager() log = TextLogView( evManager ) clientController = NetworkClientController( evManager ) game = Game( evManager ) from twisted.internet import reactor reactor.listenTCP( 8000, pb.PBServerFactory(clientController) ) reactor.run()

Previously, we used the Tick event to start the Game, now we'll need to explicitly start the game with our new GameStartRequest event.

class GameStartRequest(Event): def __init__(self): self.name = "Game Start Request"

If we play some dirty tricks, we can see what our server does without writing a client. Instead, we will just connect to it using the Python interactive interpreter. Now, reactor.run() is a blocking call that does not return until the reactor is shut down, so in order to get back to the interactive prompt, we have to crash the reactor and then call reactor.iterate() in order to communicate with it. It should go without saying that this is not a recommended practice. Also, if you replicate the session below, you may have to call iterate() multiple times before you see any result.

$ python Python 2.5.2 (r252:60911, Apr 21 2008, 11:17:30) [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from twisted.spread import pb >>> from twisted.internet import reactor >>> factory = pb.PBClientFactory() >>> server = None >>> def gotServer(serv): ... global server ... server = serv ... >>> connection = reactor.connectTCP('localhost', 8000, factory) >>> reactor.callLater( 4, reactor.crash ) <twisted.internet.base.DelayedCall instance at 0xac5638> >>> reactor.run() >>> d = factory.getRootObject() >>> d.addCallback(gotServer) <Deferred at 0xb1f440 current result: None> >>> reactor.iterate() >>> server.callRemote('GameStartRequest') <Deferred at 0xac5638> >>> reactor.iterate() >>> up, right, down, left = 0,1,2,3 >>> server.callRemote('CharactorMoveRequest', up) <Deferred at 0xb1f4d0> >>> reactor.iterate() >>> server.callRemote('CharactorMoveRequest', right) <Deferred at 0xac5638> >>> reactor.iterate() >>> server.callRemote('CharactorMoveRequest', down) <Deferred at 0xb1f4d0> >>> reactor.iterate() >>> server.callRemote('CharactorMoveRequest', left) <Deferred at 0xac5638> >>> reactor.iterate() Example of using the Python console as a fake client Example of using the Python console as a fake client

$ python server.py Game Start Request Map Finished Building Event Game Started Event Charactor Placement Event at <__main__.Sector instance at 0xc9b290> Charactor Move Request Charactor Move Request Charactor Move Event to <__main__.Sector instance at 0xc9b320> Charactor Move Request Charactor Move Event to <__main__.Sector instance at 0xc9b290> Charactor Move Request Charactor Move Event to <__main__.Sector instance at 0xc9b3b0> Running server.py Running server.py Note that the request to move up did not result in a Move event.

We can fake the client in a more proper way by using a tool that comes with Twisted, twisted.conch.stdio. We just start a python interpreter with this module and then we can omit the reactor abuse:

$ python -m twisted.conch.stdio >>> from twisted.spread import pb >>> from twisted.internet import reactor >>> >>> factory = pb.PBClientFactory() >>> server = None >>> >>> def gotServer(serv): ... global server ... server = serv ... >>> connection = reactor.connectTCP('localhost', 8000, factory) >>> d = factory.getRootObject() >>> d.addCallback(gotServer) <Deferred at 0xc227a0 current result: None> >>> server.callRemote('GameStartRequest') <Deferred #0> Deferred #0 called back: 1 >>> up, right, down, left = 0,1,2,3 >>> server.callRemote('CharactorMoveRequest', up) <Deferred #1> Deferred #1 called back: 1 >>> server.callRemote('CharactorMoveRequest', right) <Deferred #2> Deferred #2 called back: 1 >>> server.callRemote('CharactorMoveRequest', down) <Deferred #3> Deferred #3 called back: 1 >>> server.callRemote('CharactorMoveRequest', left) <Deferred #4> Deferred #4 called back: 1 Using twisted.conch.stdio as a fake client Using twisted.conch.stdio as a fake client

King of the Castle

# Example of a class that pumps a Twisted reactor class ReactorSlaveController(object): def __init__(self): ... factory = pb.PBClientFactory() self.reactor = SelectReactor() installReactor(self.reactor) connection = self.reactor.connectTCP('localhost', 8000, factory) self.reactor.startRunning() ... def PumpReactor(self): self.reactor.runUntilCurrent() self.reactor.doIteration(0) def Stop(self): self.reactor.addSystemEventTrigger('after', 'shutdown', self.onReactorStop) self.reactor.stop() self.reactor.run() #excrete anything left in the reactor def onReactorStop(self): '''This gets called when the reactor is absolutely finished''' self.reactor = None

# Example of using LoopingCall to fire the Tick event from twisted.internet.task import LoopingCall ... def FireTick(evManager): evManager.Post( TickEvent() ) loopingCall = LoopingCall(FireTick, evManager) interval = 1.0 / FRAMES_PER_SECOND loopingCall.start(interval)

The previous example of a server gave a good introduction to the basic networking technique, but it's a little too simple for our purposes. We don't really want to write a new function for every message the server can possibly receive. Instead, we'd like to leverage our already existing Event classes.

This brings us to one of the most important parts, but possibly the most tedious part of implementing networking. We need to go through all the possible events and answer these questions about each:

Do we need to send it from the client to the server? Do we need to send it from the server to the client? Are there security issues with sending this data over the network? Is the data formatted in a way that it can be sent over the network? If we must, how do we reformat the data so that it can be sent?

While there are many ways of doing this with Twisted, I will outline a strategy that tries to minimize the amount of code written (to combat the tediousness of this task) and to maintain the separation of the networking requirements from the remainder of the code.

Using Twisted, we must do three things to a class to make it possible to send instances of it over the network: make it inherit from twisted.spread.pb.Copyable, make it inherit from twisted.spread.pb.RemoteCopy, and call twisted.spread.pb.setUnjellyableForClass() on it [TODO: ask someone who knows Twisted if that's really necessary]. Things can become even more complicated when we consider questions 4 and 5 from our list above -- does the data require special formatting to send it over the network? The only data that doesn't require special formatting are the literal types: string, int, float, etc., None, and containers (lists, tuples, dicts) thereof.

While examining the Events, two cases will occur, either it will not require reformatting, and we can just mix-in pb.Copyable and pb.RemoteCopy, or it will require reformatting and we will have to create a new class that has a routine to change the original data into something that can be sent over the network. [TODO: link to explain Mixins somewhere]

In this next example, we've split the code into multiple files. All the events are in events.py. In network.py, we try to answer all of the above questions for each event in events.py. If a message can go from the client to the server, we append it to the clientToServerEvents list, and likewise for the serverToClientEvents list. If the data in the event is simple, like integers and strings, then we can just mix-in the pb.Copyable and pb.RemoteCopy classes and call pb.setUnjellyableForClass() on the event.

# from network.py #------------------------------------------------------------------------------ # GameStartRequest # Direction: Client to Server only MixInCopyClasses( GameStartRequest ) pb.setUnjellyableForClass(GameStartRequest, GameStartRequest) clientToServerEvents.append( GameStartRequest ) #------------------------------------------------------------------------------ # CharactorMoveRequest # Direction: Client to Server only # this has an additional attribute, direction. it is an int, so it's safe MixInCopyClasses( CharactorMoveRequest ) pb.setUnjellyableForClass(CharactorMoveRequest, CharactorMoveRequest) clientToServerEvents.append( CharactorMoveRequest )

On the other hand, if an event contains data that is not network-friendly, like an object, we need to make a replacement event to send over the wire instead of the original. The simplest way to make a replacement is just to change any event attributes that were objects to unique integers using the id() function. This strategy requires us to keep a registry of objects and their ID numbers, so that when we receive an event from the network referencing an object by its ID number, we can find the actual object.

# from network.py #------------------------------------------------------------------------------ # GameStartedEvent # Direction: Server to Client only class CopyableGameStartedEvent(pb.Copyable, pb.RemoteCopy): def __init__(self, event, registry): self.name = "Game Started Event" self.gameID = id(event.game) registry[self.gameID] = event.game pb.setUnjellyableForClass(CopyableGameStartedEvent, CopyableGameStartedEvent) serverToClientEvents.append( CopyableGameStartedEvent ) #------------------------------------------------------------------------------ # CharactorMoveEvent # Direction: Server to Client only class CopyableCharactorMoveEvent( pb.Copyable, pb.RemoteCopy): def __init__(self, event, registry ): self.name = "Charactor Move Event" self.charactorID = id( event.charactor ) registry[self.charactorID] = event.charactor pb.setUnjellyableForClass(CopyableCharactorMoveEvent, CopyableCharactorMoveEvent) serverToClientEvents.append( CopyableCharactorMoveEvent )

From the server, changes need to be sent out, so we need to create a new View on the server.

# from server.py #------------------------------------------------------------------------------ class NetworkClientView(object): """We SEND events to the CLIENT through this object""" def __init__(self, evManager, sharedObjectRegistry): self.evManager = evManager self.evManager.RegisterListener( self ) self.clients = [] self.sharedObjs = sharedObjectRegistry #---------------------------------------------------------------------- def Notify(self, event): if isinstance( event, ClientConnectEvent ): self.clients.append( event.client ) ev = event #don't broadcast events that aren't Copyable if not isinstance( ev, pb.Copyable ): evName = ev.__class__.__name__ copyableClsName = "Copyable"+evName if not hasattr( network, copyableClsName ): return copyableClass = getattr( network, copyableClsName ) ev = copyableClass( ev, self.sharedObjs ) if ev.__class__ not in network.serverToClientEvents: #print "SERVER NOT SENDING: " +str(ev) return #NOTE: this is very "chatty". We could restrict # the number of clients notified in the future for client in self.clients: print "=====server sending: ", str(ev) remoteCall = client.callRemote("ServerEvent", ev)

NetworkClientView.Notify() is primarily interested in Copyable events. The event passed in to Notify() might already be Copyable, due to the mixing in of pb.Copyable in network.py. In that case, isinstance( ev, pb.Copyable ) returns True. If it's not Copyable, there still might be a replacement class in the network module, and we can check by prepending "Copyable" to the event's class name because we used that naming convention for the replacement classes in network.py.

As can be seen in NetworkClientView.Notify(), the server expects the client to send it a remotely accessible object (like one that inherits from Twisted's pb.Root) when the client connects. Thereafter, the server can use that object to notify the client of events.

Now we'll (finally) get started on the client. From the point of view of the client, the incoming messages from the server represent a Controller, so we've got a NetworkServerController class in client.py. As you might be expecting, the client will also send events to the server through a View, the NetworkServerView.

# from client.py #------------------------------------------------------------------------------ class NetworkServerView(pb.Root): """We SEND events to the server through this object""" ... #---------------------------------------------------------------------- def Connected(self, server): self.server = server self.state = NetworkServerView.STATE_CONNECTED ev = ServerConnectEvent( server ) self.evManager.Post( ev ) ... #---------------------------------------------------------------------- def AttemptConnection(self): ... connection = self.reactor.connectTCP(serverHost, serverPort, self.pbClientFactory) deferred = self.pbClientFactory.getRootObject() deferred.addCallback(self.Connected) deferred.addErrback(self.ConnectFailed) self.reactor.startRunning() ... #---------------------------------------------------------------------- def Notify(self, event): ev = event if isinstance( event, TickEvent ): if self.state == NetworkServerView.STATE_PREPARING: self.AttemptConnection() ...

# from client.py #------------------------------------------------------------------------------ class NetworkServerController(pb.Referenceable): """We RECEIVE events from the server through this object""" def __init__(self, evManager, twistedReactor): self.evManager = evManager self.evManager.RegisterListener( self ) #---------------------------------------------------------------------- def remote_ServerEvent(self, event): self.evManager.Post( event ) return 1 #---------------------------------------------------------------------- def Notify(self, event): if isinstance( event, ServerConnectEvent ): #tell the server that we're listening to it and #it can access this object event.server.callRemote("ClientConnect", self)

We will create a PhonyModel on the client side whose state we will keep in sync with the authoritative model on the server. This PhonyModel provides the same interface as the server's model, but it has a special role - to ensure that the local game objects do not change the game state when they don't have the authority to do so. In our example, this is accomplished by keeping two EventManager objects, one called phonyEventManager, which just discards events that it receives, effectively silencing all events coming from the local game objects, and one called realEventManager, which propogates events received from the server. Events posted to the realEventManager will show up in the View objects, events posted to the phonyEventManager will not.

Because our example is very simple, we can get away with this simple implementation. One can imagine situations where we might want to allow a local game object to change the local state. This could be accomplished by making PhonyEventManager propogate these special events. Another approach could be to not have a local Model on the client, only a View object on which incoming events from the server had a direct effect.

Here's the tricky part: how do we send complex objects like Players or Charactors over the channel we've created? This is called serialization. To serialize our objects, we need to do two things.

Create a registry that maps unique IDs to game objects

For each class, have a way to change all of its internal data into numbers and strings, and a way to change those back into useful objects

When events referencing complex objects get to the NetworkClientView on the server, the objects are serialized starting in the constructor of the Copyable event.

# from server.py class NetworkClientView: ... def Notify(self, event): ... ev = event if not isinstance( ev, pb.Copyable ): evName = ev.__class__.__name__ copyableClsName = "Copyable"+evName if not hasattr( network, copyableClsName ) return copyableClass = getattr( network, copyableClsName ) #It is here that serialization starts ev = copyableClass( ev, self.sharedObjs ) elif ev.__class__ not in serverToClientEvents: return for client in self.clients: self.RemoteCall( client, "ServerEvent", ev )

# from network.py class CopyableCharactorMoveEvent( pb.Copyable, pb.RemoteCopy): def __init__( self, event, registry ): self.name = "Copyable " + event.name self.charactorID = id( event.charactor ) registry[self.charactorID] = event.charactor

When the client is sent the CopyableCharactorMoveEvent, the PhonyModel picks it up (the PhonyModel is the only object interested in events that start with "Copyable").

#from client.py class PhonyModel ... #---------------------------------------------------------------------- def Notify(self, event): ... if isinstance( event, CopyableCharactorMoveEvent ): charactorID = event.charactorID if not self.sharedObjs.has_key(charactorID): charactor = self.game.players[0].charactors[0] self.sharedObjs[charactorID] = charactor remoteResponse = self.server.callRemote("GetObjectState", charactorID) remoteResponse.addCallback(self.StateReturned) remoteResponse.addCallback(self.CharactorMoveCallback, charactorID)

This is a very generic approach to solving the problem.

Server: Hey, dude X just moved!

Hey, dude X just moved! Client: Oh really? Tell me everything you now now about that guy.

Oh really? Tell me everything you now now about that guy. Server: Well, he's in the "ACTIVE" state, and he's in the sector with the unique ID 123567.

Well, he's in the "ACTIVE" state, and he's in the sector with the unique ID 123567. Client (to self): Ah, excellent. I already know about that sector, so I'll just change my little model of the universe and move that dude into that sector.

Server: Hey, dude X just moved! To sector 123567!

Hey, dude X just moved! To sector 123567! Client (to self): Ah, excellent. I already know about that sector, so I'll just change my little model of the universe and move that dude into that sector.

Back to the code snippet, if the client has already received that object from the server, self.sharedObjs.has_key() will return true, and it can grab a reference to the object from the registry and carry on as normal. If it hasn't received that object yet (as is the case the first time this event is received), it must first create a placeholder object, and then copy the state of the object on the server into this new placeholder object. It does this by calling GetObjectState() with the unique ID of the needed object.

GetObjectState() basically just finds that object on the server (in this example, the Charactor that has moved), and serializes it's data with a call to getStateToCopy(). GetObjectState() returns the dict and the object ID that was requested.

# from network.py #------------------------------------------------------------------------------ class CopyableCharactor: def getStateToCopy(self, registry): d = self.__dict__.copy() del d['evManager'] sID = id( self.sector ) d['sector'] = sID registry[sID] = self.sector return d def setCopyableState(self, stateDict, registry): neededObjIDs = [] success = 1 if stateDict['sector'] not in registry: registry[stateDict['sector']] = Sector(self.evManager) neededObjIDs.append( stateDict['sector'] ) success = 0 else: self.sector = registry[stateDict['sector']] return [success, neededObjIDs]

The client receives this information in the StateReturned() function, which is probably the most difficult function to follow in this whole tutorial. I'll try to go through it step-by-step.

The client first requests the Object state. When the response comes, the callbacks StateReturned and CharactorMoveCallback are queued to be called in sequence.

# from client.py def Notify(self, event): ... remoteResponse = self.server.callRemote("GetObjectState", charactorID) remoteResponse.addCallback(self.StateReturned) remoteResponse.addCallback(self.CharactorMoveCallback)

# from server.py def remote_GetObjectState(self, objectID): ... return [objectID, objDict]

# from client.py #---------------------------------------------------------------------- def StateReturned(self, response): """this is a callback that is called in response to invoking GetObjectState on the server""" objID, objDict = response if objID == 0: print "GOT ZERO -- better error handler here" return None obj = self.sharedObjs[objID] success, neededObjIDs =\ obj.setCopyableState(objDict, self.sharedObjs) if success: #we successfully set the state and no further objects #are needed to complete the current object if objID in self.neededObjects: self.neededObjects.remove(objID) else: #to complete the current object, we need to grab the #state from some more objects on the server. The IDs #for those needed objects were passed back #in neededObjIDs for neededObjID in neededObjIDs: if neededObjID not in self.neededObjects: self.neededObjects.append(neededObjID) self.waitingObjectStack.append( (obj, objDict) ) retval = self.GetAllNeededObjects() if retval: # retval is a Deferred - returning it causes a chain # to be formed. return retval

However, if "success" was False, that means more data is needed to complete the originally requested object's state. The PhonyModel keeps a list of neededObjects that must be requested from the server before the originally requested object is complete. Each of these needed objects may also append to the neededObjects list for subsequent objects they need. So when we call GetAllNeededObjects() the recursive behaviour begins.

# from client.py #---------------------------------------------------------------------- def GetAllNeededObjects(self): if len(self.neededObjects) == 0: #this is the recursion-ending condition. If there are #no more objects needed to be grabbed from the server #then we can try to setCopyableState on them again and #we should now have all the needed objects, ensuring #that setCopyableState succeeds return self.ConsumeWaitingObjectStack() #still in the recursion step. Try to get the object state for #the objectID on the top of the stack. Note that the recursion #is done via a deferred, which may be confusing nextID = self.neededObjects[-1] remoteResponse = self.server.callRemote("GetObjectState",nextID) remoteResponse.addCallback(self.StateReturned) return remoteResponse

As you can see, another call is made to GetObjectState on the server that will result in StateReturned being called. Notice that this isn't truly recursive. GetAllNeededObjects doesn't block. It returns immediately. But it returns a Deferred object, remoteResponse. So the original Deferred had it's first callback called, and that returned a new Deferred object. This is called Chaining Deferreds and it causes the first callback to block until the second Deferred's callbacks are finished. Hence we get recursion over the network.

Here is a flowchart that summarizes the actions taken when the client gets an event containing a complex object.

Notice that we must make sure that the event we send over the network has enough information to update the client with any relevant changes to the state of the server. The client may already have a local version of an object, but if that object has changed, the client still has to call GetObjectState(), as is demonstrated with the CharactorMoveEvent.

With that in mind, a question is raised: where do we put the intelligence do determine what object states we need to retrieve? Right now, we've put all this logic in PhonyModel.Notify() [TODO: is this the best place? what about inside Copyable events?]

The previous discussion is a good start and provides some useful code. I encourage you to play around with it and see if you can get your game sending objects back and forth. As your code becomes more complex, you will run into some more problems:

What if we don't have enough information to call __init__ for some attributes in setCopyableState() ? What if we don't know the specific subclass for an attribute in setCopyableState() ?

To clarify, here's an example of when an issue like this might come up. Lets say we write a game where two Penguins fight each other. Each Penguin has a weapon, and every weapon is initialized with a name, like "Deathbringer" or "Destroy-o-Matic", or "Daffodil".

#------------------------------------------------------------------------------ class Weapon: def __init__( self, evManager, name ) self.evManager = evManager self.name = name

CopyablePenguin would thus look something like this:

#------------------------------------------------------------------------------ class CopyablePenguin: def getStateToCopy(self, registry): d = self.__dict__.copy() del d['evManager'] wID = id( self.weapon ) registry[wID] = self.weapon d['weapon'] = wID return d

def setCopyableState(self, stateDict, registry): neededObjIDs = [] success = 1 wID = stateDict['weapon'] if not registry.has_key( wID ): #registry didn't have the object, so create a new one self.weapon = Weapon( self.evManager, #WELL CRAP! I don't yet know what its name is, so how am I going to initialize it?

... wID = stateDict['weapon'] if not registry.has_key( wID ): #registry didn't have the object, so create a new one self.weapon = ??? #MORE CRAP! I don't even know what class of object it should be!

We can solve this problem with a Placeholder object that is very similar to the Lazy Proxy design pattern.

... [TODO: finish this section]

We will add a couple new events, PlayerJoinRequest, PlayerJoinEvent (the Player object is no longer created by when the Game is constructed), and CharactorPlaceRequest. The KeyboardController is also modified to detect new keypresses, p and c to fire off those request events, and the o key to switch between active players. (see screenshot above).

You can try this out by running python example.py from the example4.tar.gz archive below. When it starts, press p twice to request 2 PlayerJoin events, then press space bar to start the game, then press c to place one character, o to switch to the other player, then c again to place the second character. Direction keys move the charactor around, as per usual.

We want to ensure that Player One's client cannot control Player Two's charactor. We want the server to reject any request where the player instance contained in the request is not an instance the sender is allowed to control. As a first step, we need to be able to uniquely identify clients. Then we need to map clients to a set of Player objects (or more commonly, just one) that they are allowed to control. Then we need to filter out any events that should not be allowed based on that map.

Luckily, Twisted provides a rich set of tools to identify clients, aka "authentication". Most of this is explained in [TODO]Authentication with Perspective Broker in the Twisted docs. I'll go over the specific usage in our example, but you should also review those docs.

Our first change will be to change the server's NetworkClientController from a pb.Root object into a pb.Avatar object:

# from server.py class NetworkClientController(pb.Avatar): """We RECEIVE events from the CLIENT through this object There is an instance of NetworkClientController for each connected client. """ def __init__(self, evManager, avatarID, realm): self.evManager = evManager self.evManager.RegisterListener( self ) self.avatarID = avatarID self.realm = realm ... #---------------------------------------------------------------------- def perspective_GetGameSync(self): ... #---------------------------------------------------------------------- def perspective_GetObjectState(self, objectID): ... #---------------------------------------------------------------------- def perspective_EventOverNetwork(self, event): ...

As you can see, the class now inherits from pb.Avatar, and the methods that were previously named remote_BlahBlah are now named perspective_BlahBlah. Also, the NetworkClientController objects will need to keep track of their realm and their avatarID. The realm is basically a factory on the server that gets requests for new client connections, and creates new NetworkServerViews and NetworkServerControllers for each successful connection.

# from server.py class MyRealm: implements(portal.IRealm) def __init__(self, evManager): self.evManager = evManager # keep track of avatars that have been given out self.claimedAvatarIDs = [] # we need to hold onto views so they don't get garbage collected self.clientViews = [] # maps avatars to player(s) they control self.playersControlledByAvatar = {} #---------------------------------------------------------------------- def requestAvatar(self, avatarID, mind, *interfaces): if pb.IPerspective not in interfaces: raise NotImplementedError if avatarID in self.claimedAvatarIDs: # someone already has this avatar. raise Exception( 'Another client is already connected' ' to this avatar' ) self.claimedAvatarIDs.append(avatarID) ev = ClientConnectEvent( mind, avatarID ) self.evManager.Post( ev ) self.playersControlledByAvatar[avatarID] = [] view = NetworkClientView( self.evManager, avatarID, mind ) controller = NetworkClientController(self.evManager, avatarID, self) self.clientViews.append(view) return pb.IPerspective, controller, controller.clientDisconnect #---------------------------------------------------------------------- def knownPlayers(self): ... #---------------------------------------------------------------------- def Notify(self, event): if isinstance(event, ClientDisconnectEvent): self.claimedAvatarIDs.remove(event.avatarID) removee = None for view in self.clientViews: if view.avatarID == event.avatarID: removee = view if removee: self.clientViews.remove(removee)

If you look at the body of the requestAvatar method, you see where the network views and controllers get created. The requestAvatar method is also where the avatarID comes into play. It is created internally to Twisted, and passed to our code. It is an identifier guaranteed to be unique for each client. Effectively, it is a "username".

requestAvatar gets called as a result of calling login() during the AttemptConnection method of the client:

# from client.py avatarID = None def main(): global avatarID if len(sys.argv) > 1: avatarID = sys.argv[1] else: avatarID = 'user1' class NetworkServerView(pb.Root): """We SEND events to the server through this object""" ... #---------------------------------------------------------------------- def __init__(self, evManager, sharedObjectRegistry): self.evManager = evManager self.evManager.RegisterListener( self ) self.pbClientFactory = pb.PBClientFactory() self.state = NetworkServerView.STATE_PREPARING self.reactor = None self.server = None self.sharedObjs = sharedObjectRegistry #---------------------------------------------------------------------- def AttemptConnection(self): self.state = NetworkServerView.STATE_CONNECTING if self.reactor: self.reactor.stop() self.PumpReactor() else: self.reactor = SelectReactor() installReactor(self.reactor) connection = self.reactor.connectTCP(serverHost, serverPort, self.pbClientFactory) userCred = credentials.UsernamePassword(avatarID, 'pass1') controller = NetworkServerController( self.evManager ) deferred = self.pbClientFactory.login(userCred, client=controller) deferred.addCallback(self.Connected) deferred.addErrback(self.ConnectFailed) self.reactor.startRunning() #---------------------------------------------------------------------- def Disconnect(self): if not self.reactor: return self.reactor.stop() self.PumpReactor() self.state = NetworkServerView.STATE_DISCONNECTING #---------------------------------------------------------------------- def Connected(self, server): self.server = server self.state = NetworkServerView.STATE_CONNECTED ev = ServerConnectEvent( server ) self.evManager.Post( ev ) #---------------------------------------------------------------------- def ConnectFailed(self, server): self.state = NetworkServerView.STATE_DISCONNECTED

Now that we've got these usernames dictated by Twisted, we might as well use the information in our Model. [TODO: expand...]

All that remains is changing the KeyboardController. The KeyboardController keeps track of which player is "active" and controls only that player, switching when the "o" key is pressed. That works fine when running as a single process, but now that there are a couple clients and which player a client controls is regulated by the server, we need to adjust the KeyboardController.

First we'll give the constructor an optional argument, "playerName". By making the default value None, we can check to see if it's set, and anywhere it's not set, we keep the single-process behaviour. [TODO: paste in code] The only change to make is the reaction to a PlayerJoinEvent. In single-process mode, it makes sense to always control the new player, but with multiple clients, that new player could have come from a remote host and the server won't let this local host control it. So only try to control players that match the playerName. The next question may be "where does the playerName get set then". It is simply during the main() function of the client code. [TODO: paste in code]

[TODO: I need a section here on why to chain deferreds when the client receives events from the server. On a received event, the client starts getting new state information from the server. Because of the asynchronous nature of networked programs and the choice we made to not send ALL the needed information at once, there are points in time when we've collected incomplete information from the server. If we populated our phony Model with that incomplete information and then the User Interface got Ticked, it will likely result in a crash or at least a UI bug. So we chain deferreds, get all the state information we need, and once it's all collected, then we update our phony Model and post events. ]

[TODO: I need a section here talking about how to refine the client code so that you don't need a waitingObjects queue. Basically, with a better Placeholder class, and some Python self.__class__ = foo magic, we don't have to keep a queue and solidify the Placeholders after everything has been downloaded. ]

A GameSync is a request from the client to pull sufficient information about the game to recreate its current state from nothing. In our example, we just send the Game object from the authoritative model. Start by creating a new event, GameSyncEvent:

# from events.py class GameSyncEvent(Event): def __init__(self, game): self.name = "Game Synched to Authoritative State" self.game = game

Add another remotely callable method on the server, and the server-side code is done:

# from server.py class NetworkClientController(pb.Avatar): ... def perspective_GetGameSync(self): """this is usually called when a client first connects or when they reconnect after a drop """ game = sharedObjectRegistry.getGame() if game == None: raise Exception('Game should be set by this point') gameID = id( game ) gameDict = game.getStateToCopy( sharedObjectRegistry ) return [gameID, gameDict]

Next we need to hook up the client side. When should a GameSync be requested? It needs to be done at a time when the client has a connection to the server, but the client-side model (PhonyModel) has not yet been populated. A good place is the ServerConnectEvent handler in PhonyModel itself.

# from client.py class PhonyModel: ... def Notify(self, event): if isinstance( event, ServerConnectEvent ): self.server = event.server #when we reconnect to the server, we should get the #entire game state. if not self.game: self.game = Game( self.phonyEvManager ) gameID = id(self.game) self.sharedObjs[gameID] = self.game remoteResponse = self.server.callRemote("GetGameSync") remoteResponse.addCallback(self.GameSyncReturned) remoteResponse.addCallback(self.GameSyncCallback, gameID) remoteResponse.addErrback(self.ServerErrorHandler, 'ServerConnect') ...

These two functions are straightforward; they just populate the client-side sharedObjs and send the GameSyncEvent to the client-side event manager.

# from client.py class PhonyModel: ... def GameSyncReturned(self, response): gameID, gameDict = response print "GameSyncReturned : ", gameID self.sharedObjs[gameID] = self.game # StateReturned returns a deferred, pass it on to keep the # chain going. return self.StateReturned( response ) ... def GameSyncCallback(self, deferredResult, gameID): game = self.sharedObjs[gameID] ev = GameSyncEvent( game ) self.realEvManager.Post( ev )

The last detail of the reconnection involves example.py. After a client reconnects, it will have a fresh KeyboardController object. This KeyboardController won't receive a PlayerJoinEvent to set up it's activePlayer because the game is already underway (in the authoritative model, both players had already joined). So we add some new code to example.py (compromising the principles set out in the Rapid Development aside, but it'll be harmless, I promise. (can anyone suggest a better way to do this?) ) to take a GameSyncEvent and figure out which player the KeyboardController should control.

# from example.py class KeyboardController: ... def Notify(self, event): ... if isinstance( event, GameSyncEvent ): game = event.game self.players = game.players[:] # copy the list if self.playerName and self.players: self.activePlayer = [p for p in self.players if p.name == self.playerName][0] ...

You can get as complicated as you like when creating your GUI engine, but this tutorial will focus only on some simple widgets. Here are the ones we will implement:

label

button

text-entry field

[TODO ... scrollbox?]

#------------------------------------------------------------------------------ class Widget(pygame.sprite.Sprite): def __init__(self, evManager, container=None): pygame.sprite.Sprite.__init__(self) self.evManager = evManager self.evManager.RegisterListener( self ) self.container = container self.focused = 0 self.dirty = 1 #---------------------------------------------------------------------- def SetFocus(self, val): self.focused = val self.dirty = 1 #---------------------------------------------------------------------- def kill(self): self.container = None del self.container pygame.sprite.Sprite.kill(self) #---------------------------------------------------------------------- def Notify(self, event): if isinstance( event, GUIFocusThisWidgetEvent ) \ and event.widget is self: self.SetFocus(1) elif isinstance( event, GUIFocusThisWidgetEvent ) \ and self.focused: self.SetFocus(0)

#------------------------------------------------------------------------------ class LabelSprite(Widget): def __init__(self, evManager, text, container=None): Widget.__init__( self, evManager, container) self.color = (200,200,200) self.font = pygame.font.Font(None, 30) self.__text = text self.image = self.font.render( self.__text, 1, self.color) self.rect = self.image.get_rect() #---------------------------------------------------------------------- def SetText(self, text): self.__text = text self.dirty = 1 #---------------------------------------------------------------------- def update(self): if not self.dirty: return self.image = self.font.render( self.__text, 1, self.color ) self.dirty = 0

#------------------------------------------------------------------------------ class ButtonSprite(Widget): def __init__(self, evManager, text, container=None, onClickEvent=None ): Widget.__init__( self, evManager, container) self.font = pygame.font.Font(None, 30) self.text = text self.image = self.font.render( self.text, 1, (255,0,0)) self.rect = self.image.get_rect() self.onClickEvent = onClickEvent #---------------------------------------------------------------------- def update(self): if not self.dirty: return if self.focused: color = (255,255,0) else: color = (255,0,0) self.image = self.font.render( self.text, 1, color) #self.rect = self.image.get_rect() self.dirty = 0 #---------------------------------------------------------------------- def Connect(self, eventDict): for key,event in eventDict.iteritems(): try: self.__setattr__( key, event ) except AttributeError: print "Couldn't connect the ", key pass #---------------------------------------------------------------------- def Click(self): self.dirty = 1 if self.onClickEvent: self.evManager.Post( self.onClickEvent ) #---------------------------------------------------------------------- def Notify(self, event): if isinstance( event, GUIPressEvent ) and self.focused: self.Click() elif isinstance( event, GUIClickEvent ) \ and self.rect.collidepoint( event.pos ): self.Click() elif isinstance( event, GUIMouseMoveEvent ) \ and self.rect.collidepoint( event.pos ): ev = GUIFocusThisWidgetEvent(self) self.evManager.Post( ev ) Widget.Notify(self,event)

#------------------------------------------------------------------------------ class TextBoxSprite(Widget): def __init__(self, evManager, width, container=None ): Widget.__init__( self, evManager, container) self.font = pygame.font.Font(None, 30) linesize = self.font.get_linesize() self.rect = pygame.Rect( (0,0,width, linesize +4) ) boxImg = pygame.Surface( self.rect.size ).convert_alpha() color = (0,0,100) pygame.draw.rect( boxImg, color, self.rect, 4 ) self.emptyImg = boxImg.convert_alpha() self.image = boxImg self.text = '' self.textPos = (22, 2) #---------------------------------------------------------------------- def update(self): if not self.dirty: return text = self.text if self.focused: text += '|' textColor = (255,0,0) textImg = self.font.render( text, 1, textColor ) self.image.blit( self.emptyImg, (0,0) ) self.image.blit( textImg, self.textPos ) self.dirty = 0 #---------------------------------------------------------------------- def Click(self): self.focused = 1 self.dirty = 1 #---------------------------------------------------------------------- def SetText(self, newText): self.text = newText self.dirty = 1 #---------------------------------------------------------------------- def Notify(self, event): if isinstance( event, GUIPressEvent ) and self.focused: self.Click() elif isinstance( event, GUIClickEvent ) \ and self.rect.collidepoint( event.pos ): self.Click() elif isinstance( event, GUIClickEvent ) \ and self.focused: self.SetFocus(0) elif isinstance( event, GUIMouseMoveEvent ) \ and self.rect.collidepoint( event.pos ): ev = GUIFocusThisWidgetEvent(self) self.evManager.Post( ev ) elif isinstance( event, GUIKeyEvent ) \ and self.focused: newText = self.text + event.key self.SetText( newText ) elif isinstance( event, GUIControlKeyEvent ) \ and self.focused and event.key == K_BACKSPACE: #strip of last character newText = self.text[:( len(self.text) - 1 )] self.SetText( newText ) Widget.Notify(self,event)

Above is a diagram showing some common uses of a Graphical User Interface in games. In each of the above screens, there is a blue section representing buttons or other widgets. It will also serve as the idea for our next example application, "Fool The Bar".

What license is all this code under?

Unless otherwise stated, everything here is in the Public Domain.

Why do you use from module import * ? Don't you know that's bad coding style

? Don't you know that's bad coding style Yeah, my bad. I promise i'll clean all those up before i declare this "finished".

Why don't you use Twisted's Cacheable? It seems like you're just reimplementing it.

I am a bit of a newbie when it comes to Twisted. I read a bit about the Cacheable stuff, but it seemed to me I'd have to make the classes of my Model be Cacheable, and I didn't want the network code to be coupled with my model code. And seeing as I would have to write serialization code even if I used Cacheable, I just bulldozed ahead. Would it be possible to use Cacheable without coupling it with the Model code? Could someone who is familiar with Twisted point me in the right direction?

Translations