Tornado is a great open source, Python based web framework. It is designed to be light-weight, easily scalable, non-blocking and it supports websockets and some other nice features. What’s more, it runs smooth and quick on the Raspberry Pi.

Thus far, my experiments with the Raspberry Pi have involved running Node.js on it to serve web pages and to interact over serial with an Arduino. Here are some posts on how to do that:

1. Raspberry Pi Setup

2. Raspberry Pi + Arduino: Getting Started

3. Raspberry Pi + Arduino + Node = Home Automation

I wanted to try accomplishing something similar using Python. Specifically, a Python program which can simultaneously handle web requests and communicate with an Arduino. I had always heard great things about Tornado and noticed some of the Raspberry Pi community had success with it.

Tornado Setup

First things first, install Tornado on your Raspberry Pi:

1. Good thing to do before every install – update agt-get and installed packages sudo apt-get update sudo apt-get upgrade sudo apt-get update sudo apt-get upgrade 2. Install easy_install and pip (which are Python package managers) sudo apt-get install python-pip python2.7-dev sudo apt-get install python-pip python2.7-dev 3. Update your easy_install (and pip) package definitions sudo easy_install -U distribute sudo easy_install -U distribute 4. Now install Tornado using pip sudo pip install tornado sudo pip install tornado 5. Follow the steps in this previous post to install Arduino on your Raspberry Pi sudo apt-get install arduino sudo usermod -a -G tty pi sudo usermod -a -G dialout pi sudo apt-get install arduino sudo usermod -a -G tty pi sudo usermod -a -G dialout pi 6. At this point might as well also install GPIO package as well, in case you need it later sudo pip install RPi.GPIO sudo pip install RPi.GPIO 7. Install another Python package to serial communication sudo apt-get install python-serial python3-serial sudo apt-get install python-serial python3-serial

To make sure everything works, create a file named “server.py” containing the following code (taken from Tornado’s site), and run it from terminal using the command “python server.py”

1 2 3 4 5 6 7 8 9 10 11 12 13 14 import tornado. ioloop import tornado. web class MainHandler ( tornado. web . RequestHandler ) : def get ( self ) : self . write ( "Hello, world" ) application = tornado. web . Application ( [ ( r "/" , MainHandler ) , ] ) if __name__ == "__main__" : application. listen ( 8080 ) tornado. ioloop . IOLoop . instance ( ) . start ( ) import tornado.ioloop import tornado.web class MainHandler(tornado.web.RequestHandler): def get(self): self.write("Hello, world") application = tornado.web.Application([ (r"/", MainHandler), ]) if __name__ == "__main__": application.listen(8080) tornado.ioloop.IOLoop.instance().start()

Now on a separate computer on the same network, navigate your browser to http://raspberrypi:8080 (or use the IP address for example 192.168.1.7:8080). You should see “Hello, world” if everything goes right.

Adding Websockets

Adding websockets to your Tornado app is easy. Below is the modified version of server.py. It also includes support for command line arguments on line 8 (so you can run the program using “python server.py –port=8080”).

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 import tornado. httpserver import tornado. ioloop import tornado. options import tornado. web import tornado. websocket from tornado. options import define , options define ( "port" , default = 8080 , help = "run on the given port" , type = int ) class IndexHandler ( tornado. web . RequestHandler ) : def get ( self ) : self . render ( 'index.html' ) class WebSocketHandler ( tornado. websocket . WebSocketHandler ) : def open ( self ) : print 'new connection' self . write_message ( "connected" ) def on_message ( self , message ) : print 'message received %s' % message self . write_message ( 'message received %s' % message ) def on_close ( self ) : print 'connection closed' if __name__ == "__main__" : tornado. options . parse_command_line ( ) app = tornado. web . Application ( handlers = [ ( r "/" , IndexHandler ) , ( r "/ws" , WebSocketHandler ) ] ) httpServer = tornado. httpserver . HTTPServer ( app ) httpServer. listen ( options. port ) print "Listening on port:" , options. port tornado. ioloop . IOLoop . instance ( ) . start ( ) import tornado.httpserver import tornado.ioloop import tornado.options import tornado.web import tornado.websocket from tornado.options import define, options define("port", default=8080, help="run on the given port", type=int) class IndexHandler(tornado.web.RequestHandler): def get(self): self.render('index.html') class WebSocketHandler(tornado.websocket.WebSocketHandler): def open(self): print 'new connection' self.write_message("connected") def on_message(self, message): print 'message received %s' % message self.write_message('message received %s' % message) def on_close(self): print 'connection closed' if __name__ == "__main__": tornado.options.parse_command_line() app = tornado.web.Application( handlers=[ (r"/", IndexHandler), (r"/ws", WebSocketHandler) ] ) httpServer = tornado.httpserver.HTTPServer(app) httpServer.listen(options.port) print "Listening on port:", options.port tornado.ioloop.IOLoop.instance().start()

And here is the corresponding index.html, which connects to the websocket server. I used KineticJS to quickly create a simple red circle on a canvas which listens from click events. Be sure that your “ws://” url is correct.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 <! DOCTYPE HTML > < html > < head > < style > body { margin : 0px ; padding : 0px ; } canvas { border : 1px solid #9C9898 ; } </ style > < script src = "http://www.html5canvastutorials.com/libraries/kinetic-v4.2.0.js" ></ script > < script > var socket = new WebSocket ( "ws://raspberrypi:8080/ws" ) ; socket. onopen = function ( ) { console. log ( "connected" ) ; } ; socket. onmessage = function ( message ) { console. log ( "receiving: " + message. data ) ; } ; socket. onclose = function ( ) { console. log ( "disconnected" ) ; } ; sendMessage = function ( message ) { socket. send ( message ) ; } ; var value = 0 ; window. onload = function ( ) { var stage = new Kinetic. Stage ( { container : "container" , width : 610 , height : 400 } ) ; var layer = new Kinetic. Layer ( ) ; var circle = new Kinetic. Circle ( { x : 100 , y : 100 , radius : 10 , fill : "red" , stroke : "black" , strokeWidth : 4 , draggable : true } ) ; circle. on ( "mousedown" , function ( ) { value = value + 20 ; console. log ( 'sending: ' + value ) ; sendMessage ( "{value:" + value + "}" ) ; } ) ; layer. add ( circle ) ; stage. add ( layer ) ; } ; </ script > </ head > < body > < div id = "container" ></ div > </ body > </ html > <!DOCTYPE HTML> <html> <head> <style> body { margin: 0px; padding: 0px; } canvas { border: 1px solid #9C9898; } </style> <script src="http://www.html5canvastutorials.com/libraries/kinetic-v4.2.0.js"></script> <script> var socket = new WebSocket("ws://raspberrypi:8080/ws"); socket.onopen = function(){ console.log("connected"); }; socket.onmessage = function (message) { console.log("receiving: " + message.data); }; socket.onclose = function(){ console.log("disconnected"); }; sendMessage = function(message) { socket.send(message); }; var value = 0; window.onload = function() { var stage = new Kinetic.Stage({ container: "container", width: 610, height: 400 }); var layer = new Kinetic.Layer(); var circle = new Kinetic.Circle({ x: 100, y: 100, radius: 10, fill: "red", stroke: "black", strokeWidth: 4, draggable: true }); circle.on("mousedown", function() { value = value + 20; console.log('sending: ' + value); sendMessage("{value:" + value + "}"); }); layer.add(circle); stage.add(layer); }; </script> </head> <body> <div id="container"></div> </body> </html>

Now to test again start up your python script by executing “python server.py” in your terminal. Next, direct your browser to http://raspberrypi:8080. A couple of lines will show up in your terminal showing a connection is made. In the browser you should see a small red circle and when you click it look for the following output in both your browsers console and your Raspberry Pi terminal (see screenshots).

Adding Multiprocessing

One key difference between Node and Python, however is that Node is by nature highly asynchronous and event-driven, while Python is generally not. With Node, you can easily set up events to be fired for both incoming web requests from clients as well as incoming serial data from the Arduino. This is demonstrated in the previous post. In a Python + Tornado setup, your program is in a single continuous IO loop waiting for web requests, which is initiated by the last line of every Tornado app:

37 tornado. ioloop . IOLoop . instance ( ) . start ( ) tornado.ioloop.IOLoop.instance().start()

Any code placed after it won’t execute. You’re allowed to interrupt this IO loop (see PeriodicCallback) to perform other tasks, for example talk to an Arduino over serial, but web requests will be blocked while other processing is going on. Another option is to use a component of Tornado called Gen which can help you set up the asynchronous execution of a separate loop. However, what I found to be the cleanest solution was to use Python’s own native “multiprocessing” module to spawn a separate process to perform the needed serial communication.

You’re probably thinking “That Node stuff was way easier. Isn’t multiprocessing a huge pain in the rear cause you have to deal with locks and concurrency and stuff like that?” Well yes, but for what we’re trying to do it doesn’t get that complex, as you’ll see. Also, Node’s asynchronicity is a double edged sword. As your Node (JavaScript) code base grows, its easy to fall into the fiery pit known as callback hell. Seasoned JavaScript developers have ways to get around this, but I found it to be a nuisance. Personally I find a larger Python program to be easier to trace through and maintain. Ultimately, which language/programming style you go with will come down to your personal preference. Try out both and pick your poison.

There’s an excellent tutorial about Python multiprocessing by Doug Hellmann. It’s all I needed to get up and running with the module and I would urge anyone interested in the topic to check it out.

Inter-process communication is accomplished using the Queue data structure which comes with the multiprocessing module. Two queue’s are created, one for outgoing messages (taskQ) and one for incoming messages (resultQ). The queue’s are then passed to the spawned SerialProcess which sends anything in the taskQ to the Arduino over serial and puts any incoming data from the Arduino onto the resultQ. Tornado monitors the resultQ using PeriodicCallback.

A complete round trip looks like this:

1. Client sends a message to Tornado

2. Tornado sends it to SerialProcess using taskQ

3. SerialProcess sends it to the Arduino over serial

4. Arduino sends something back to SerialProcess over serial

5. SerialProcess sends it to Tornado using resultQ

6. Tornado sends it to all Clients

Below are all the code files (except index.html which remains the same as above). Here’s the final version of server.py:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 #!/usr/bin/python27 import tornado. httpserver import tornado. ioloop import tornado. web import tornado. websocket import tornado. gen from tornado. options import define , options import time import multiprocessing import serialProcess define ( "port" , default = 8080 , help = "run on the given port" , type = int ) clients = [ ] class IndexHandler ( tornado. web . RequestHandler ) : def get ( self ) : self . render ( 'index.html' ) class WebSocketHandler ( tornado. websocket . WebSocketHandler ) : def open ( self ) : print 'new connection' clients. append ( self ) self . write_message ( "connected" ) def on_message ( self , message ) : print 'tornado received from client: %s' % message self . write_message ( 'got it!' ) q = self . application . settings . get ( 'queue' ) q. put ( message ) def on_close ( self ) : print 'connection closed' clients. remove ( self ) ################################ MAIN ################################ def main ( ) : taskQ = multiprocessing. Queue ( ) resultQ = multiprocessing. Queue ( ) sp = serialProcess. SerialProcess ( taskQ , resultQ ) sp. daemon = True sp. start ( ) # wait a second before sending first task time . sleep ( 1 ) taskQ. put ( "first task" ) tornado. options . parse_command_line ( ) app = tornado. web . Application ( handlers = [ ( r "/" , IndexHandler ) , ( r "/ws" , WebSocketHandler ) ] , queue = taskQ ) httpServer = tornado. httpserver . HTTPServer ( app ) httpServer. listen ( options. port ) print "Listening on port:" , options. port #tornado.ioloop.IOLoop.instance().start() def checkResults ( ) : if not resultQ. empty ( ) : result = resultQ. get ( ) print "tornado received from arduino: " + result for c in clients: c. write_message ( result ) mainLoop = tornado. ioloop . IOLoop . instance ( ) scheduler = tornado. ioloop . PeriodicCallback ( checkResults , 10 , io_loop = mainLoop ) scheduler. start ( ) mainLoop. start ( ) if __name__ == "__main__" : main ( ) #!/usr/bin/python27 import tornado.httpserver import tornado.ioloop import tornado.web import tornado.websocket import tornado.gen from tornado.options import define, options import time import multiprocessing import serialProcess define("port", default=8080, help="run on the given port", type=int) clients = [] class IndexHandler(tornado.web.RequestHandler): def get(self): self.render('index.html') class WebSocketHandler(tornado.websocket.WebSocketHandler): def open(self): print 'new connection' clients.append(self) self.write_message("connected") def on_message(self, message): print 'tornado received from client: %s' % message self.write_message('got it!') q = self.application.settings.get('queue') q.put(message) def on_close(self): print 'connection closed' clients.remove(self) ################################ MAIN ################################ def main(): taskQ = multiprocessing.Queue() resultQ = multiprocessing.Queue() sp = serialProcess.SerialProcess(taskQ, resultQ) sp.daemon = True sp.start() # wait a second before sending first task time.sleep(1) taskQ.put("first task") tornado.options.parse_command_line() app = tornado.web.Application( handlers=[ (r"/", IndexHandler), (r"/ws", WebSocketHandler) ], queue=taskQ ) httpServer = tornado.httpserver.HTTPServer(app) httpServer.listen(options.port) print "Listening on port:", options.port #tornado.ioloop.IOLoop.instance().start() def checkResults(): if not resultQ.empty(): result = resultQ.get() print "tornado received from arduino: " + result for c in clients: c.write_message(result) mainLoop = tornado.ioloop.IOLoop.instance() scheduler = tornado.ioloop.PeriodicCallback(checkResults, 10, io_loop = mainLoop) scheduler.start() mainLoop.start() if __name__ == "__main__": main()

A new file called serialProcess.py is as follows:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 import serial import time import multiprocessing class SerialProcess ( multiprocessing. Process ) : def __init__ ( self , taskQ , resultQ ) : multiprocessing. Process . __init__ ( self ) self . taskQ = taskQ self . resultQ = resultQ self . usbPort = '/dev/ttyACM0' self . sp = serial. Serial ( self . usbPort , 115200 , timeout = 1 ) def close ( self ) : self . sp . close ( ) def sendData ( self , data ) : print "sendData start..." self . sp . write ( data ) time . sleep ( 3 ) print "sendData done: " + data def run ( self ) : self . sp . flushInput ( ) while True : # look for incoming tornado request if not self . taskQ . empty ( ) : task = self . taskQ . get ( ) # send it to the arduino self . sp . write ( task + "

" ) ; print "arduino received from tornado: " + task # look for incoming serial data if ( self . sp . inWaiting ( ) > 0 ) : result = self . sp . readline ( ) . replace ( "

" , "" ) # send it back to tornado self . resultQ . put ( result ) import serial import time import multiprocessing class SerialProcess(multiprocessing.Process): def __init__(self, taskQ, resultQ): multiprocessing.Process.__init__(self) self.taskQ = taskQ self.resultQ = resultQ self.usbPort = '/dev/ttyACM0' self.sp = serial.Serial(self.usbPort, 115200, timeout=1) def close(self): self.sp.close() def sendData(self, data): print "sendData start..." self.sp.write(data) time.sleep(3) print "sendData done: " + data def run(self): self.sp.flushInput() while True: # look for incoming tornado request if not self.taskQ.empty(): task = self.taskQ.get() # send it to the arduino self.sp.write(task + "

"); print "arduino received from tornado: " + task # look for incoming serial data if (self.sp.inWaiting() > 0): result = self.sp.readline().replace("

", "") # send it back to tornado self.resultQ.put(result)

Finally, here’s the Arduino program which simply listens for incoming data and returns it:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 #define LED_PIN 13 int msgByte = - 1 ; // incoming byte const int msgSize = 50 ; // max message size char msgArray [ msgSize ] ; // array for incoming data int msgPos = 0 ; // current position void setup ( ) { Serial. begin ( 115200 ) ; pinMode ( LED_PIN , OUTPUT ) ; } void loop ( ) { handleSerial ( ) ; } void handleSerial ( ) { if ( Serial. available ( ) > 0 ) { digitalWrite ( LED_PIN , HIGH ) ; msgByte = Serial. read ( ) ; if ( msgByte != '

' ) { // add incoming byte to array msgArray [ msgPos ] = msgByte ; msgPos ++; } else { // reached end of line msgArray [ msgPos ] = 0 ; // here the message is processed // for now just send it back over serial Serial. println ( msgArray ) ; // reset byte array for ( int c = 0 ; c < msgSize ; c ++ ) msgArray [ c ] = ' ' ; msgPos = 0 ; digitalWrite ( LED_PIN , LOW ) ; } } } #define LED_PIN 13 int msgByte= -1; // incoming byte const int msgSize = 50; // max message size char msgArray[msgSize]; // array for incoming data int msgPos = 0; // current position void setup() { Serial.begin(115200); pinMode(LED_PIN, OUTPUT); } void loop() { handleSerial(); } void handleSerial() { if (Serial.available() > 0) { digitalWrite(LED_PIN, HIGH); msgByte = Serial.read(); if (msgByte != '

') { // add incoming byte to array msgArray[msgPos] = msgByte; msgPos++; } else { // reached end of line msgArray[msgPos] = 0; // here the message is processed // for now just send it back over serial Serial.println(msgArray); // reset byte array for (int c = 0; c < msgSize; c++) msgArray[c] = ' '; msgPos = 0; digitalWrite(LED_PIN, LOW); } } }

Test it out as shown above and you should now see the following output (click to enlarge):