Stateless and Proud in the Realtime World

There is one question that comes up with Flask (and I can only suspect Python web applications in general) more than anything else: and that is when will we be able to do realtime applications in Flask. The only proper answer to this from my point of view has been: since forever. But not in the way you might think.

I'm going to argue that the overall stateless design of modern web frameworks is perfectly adequate even for more realtime environments.

What is Realtime? Now this is where the whole problem starts. What actually does realtime mean. If we keep the marketing side of things away the only difference between realtime and not realtime in web applications is that realtime is pushing, non realtime is polling. Let's take a concrete example where these two different paradigms are both applicable to the same problem but in different ways. The most basic example here is writing a chat application where different users can talk to each other. When the transport method is push a client sits there and waits for messages to arrive. If we would have a bidirectional communication channel like TCP that's how applications used to work for a really long time. IRC is fundamentally just that: client A sends a message into the TCP socket, the server receives that and forwards that message to all other clients that share a channel with client A. The latency for that transmission is basically network latency + processing latency on the server. On the other side there is HTTP. HTTP by itself is still a two way communication protocol but you can only send one request and get one response. Each request and response cycle is independent and in theory also very stateless. Based on those restrictions people implemented chat systems on the web in the past by making periodic HTTP requests (for instance once per second) to figure out if something on the server changed. That's the inverse of push and often referred to as polling. I still send a message to the chat server but that one then does not directly send it to you but stores it in a queue. Next time the recipient polls for new messages the queue contents are delivered and the queue is emptied.

Avoiding Polling Now let's ignore for a time being the problem of how we build such systems on the server and focus on the more pressing question: how can we avoid polling in browsers? Push based HTTP is not exactly new. Long before web sockets people were able to push information to browsers. Netscape added a server push system to their browsers a long time ago. If you ever used Bugzilla and did a search you will have seen this functionality in action. Instead of the browser sitting there on the loading circle for a long time it instead showed you the Mozilla dragon munching on bugs until the results were ready and then the loading splash disappeared and the results came up. This functionality was not well supported by anything but Mozilla browsers so it never became widespread. Another way to get information to the client in realtime is commonly named “long polling”. The idea is that instead of repeatedly polling the server for results you poll it once and the server just does not reply with “no new messages” but it waits there until a message comes up. This has the advantage over regular polling that you get the message with a lower latency assuming you have a connection open (basically just the network latency) but you can only get one message that way. Once the result is there you have to start the long polling process again. There are however many more ways to push information to browsers. A common trick was opening an iframe in the background and let the server stream pieces of javascript that invoked callback functions with provided JSON data. Neat, but obviously a huge hack. And then as of lately we have websockets. The idea behind those is that you use HTTP only to establish a connection and then use the upgrade functionality of HTTP to convert away from HTTP into a different TCP based protocol. Thanks to all the badly written HTTP proxying middleware around this works best if you do an end to end encryption via SSL on top so that nobody can break your communication channel.

Push is a Solved Problem Pushing information from the server to the client is as of now a solved problem. Depending on what browsers you want to support you might have to provide more than one solution but at the end of the day people already spend time on that and provided helper libraries that chose different transport formats for you. At first there was socket.io, now there is SockJS and other projects that help you getting push going. Now however when you ask around in the Python web development community you will hear voices that realtime web requires new paradigms and frameworks and someone has to replace WSGI with a new protocol. People are trying to revive Twisted and cling to Tornado because it is so asynchronous, fast and realtime.

Persistent Connections Alright. The plan is to do a push based web application. It means we will have to deal with persistent connections of clients. They want to keep the connection open for as long as possible, ideally they never disconnect at all unless they want to. If that standing connection is directly exposed to your application internally I bet with you that you will associate state with that standing connection. What do I mean by that? I mean you will write code like this: def chat_application ( connection ): username = None while not connection . closed : event = connection . poll () if event [ 'type' ] == 'whoami' : username = event [ 'data' ][ 'username' ] ... Basically when the connection is established the client sends something into it which is then kept around until the connection is closed. The direct consequence of that is that even if you have a proxy for the connection around, a code update means disconnecting all clients. Also one of your workers now has a standing connection to a client, you can't relocate that connection to another worker. The more direct consequence of that code however is that it mixes badly with traditional request/response based HTTP. HTTP is kinda stateless. While this paradigm is violated to some degree with the concept of cookies you can still easily build applications where the same client can be dispatched to different workers without breaking your application. If I browse a large website my client can be routed to different workers for each independent request. Many applications (and all Flask applications by default) will even used cookies with a MAC to store your session information so that you can pass state between independent workers without requiring a single database operation. It seems wrong to give up on these design ideas that make HTTP powerful just because we want to go from poll to push. But we don't have to!

Realtime in a Nutshell To make realtime work this way you end up with four separate components: Flask (or any other WSGI app if you want) would continue to work as it does. However there is probably also a separate way to feed requests to that application. In our case we're putting requests on a queue.

(or any other WSGI app if you want) would continue to work as it does. However there is probably also a separate way to feed requests to that application. In our case we're putting requests on a queue. redis for pubsub. If you want biredirectional, you can also use it as a message queue from the realtime server to the flask app.

for pubsub. If you want biredirectional, you can also use it as a message queue from the realtime server to the flask app. a realtime server that maintains the standing connections and subscribes to redis. The messages that come from redis are then sent to the connected clients that want them. If you want bidirectional communication you can also accept messages from the clients and put them on the queue for Flask to pick up.

that maintains the standing connections and subscribes to redis. The messages that come from redis are then sent to the connected clients that want them. If you want bidirectional communication you can also accept messages from the clients and put them on the queue for Flask to pick up. a regular webserver for exposing your WSGI app to the web. This could be nginx, Apache or whatever floats your boat. As far as the framework is concerned nothing changes. The only difference is that you have a way to publish information.

Pubsub at the Core Now we did not come up with that concept. In one form or another it was already implemented by people before. Juggernaut, pubnub, pusher, beaconpush and many others provide that as a webservice really. And at the end of the day it's really just message passing. However I have seen people respond to such suggestions in general very negatively. These kind of solutions are commonly seen as workarounds or hacks. I strongly disagree with that sentiment. I believe that a stateless communication protocol is what makes the web great and it can be extended to the realtime web without removing functionality. As a side effect it gives you a more decentralized infrastructure and a better experience for your or connected clients when upgrading the worker code.