Some preliminaries If you've been following this blog recently, you must have noticed that many of the posts in these past few weeks are about using Python to communicate via the serial port. I specifically decided to write them as separate posts and not as part of a series, because I think that each post is interesting in itself . But just in case you got confused, here's the logical order: Setting up Python to work with the serial port A “live” data monitor with Python, PyQt and PySerial Framing in serial communications In this post I want to present some useful Python code to implement the ideas of (3). Additionally, I'll introduce a very useful library for constructing frames from various forms of data.

Code The code for this post is available here. It contains the modules discussed, the sample code shown and even some unit tests.

Arrays of data in Python When we think about a sequence of bytes in Python, two approaches come to mind: an array of integers in the range 0-255, or a 'packed' string. Here's some Python terminal action that displays the difference: >>> arr = [ 0 x45, 0 xAB, 0 xC3, 0 x16] >>> arr [ 69 , 171 , 195 , 22 ] >>> str = '\x45\xAB\xC3\x16' >>> str 'E\xab\xc3\x16' >>> '' .join( chr (b) for b in arr) 'E\xab\xc3\x16' >>> [ ord (b) for b in str ] [ 69 , 171 , 195 , 22 ] This shows that the two formats are essentially interchangeable, and also that it's very easy to convert between the two. The format we're going to use is the packed string, because this is what the pyserial module uses to send and receive data.

Serializing data So, to send data over the serial port we first have to turn it into a packed string - this is called serialization . Python has a couple of built-in ways to do that - with the array and struct modules. However, both are suitable for fairly simple and unsophisticated data. To serialize arbitrarily sophisticated data formats, it's much better to use the powerful and flexible construct library . Here's a sample message format defined with construct (from sampleformat.py in this article's code archive): from construct import * message_crc = Struct( 'message_crc' , ULInt32( 'crc' )) message_format = Struct( 'message_format' , ULInt16( 'msg_id' ), ULInt16( 'dest_addr' ), Enum(Byte( 'command_type' ), RESTART = 0 x40, RESTART_ACK = 0 x80, SIGNAL = 0 x22, _default_ = Pass ), BitStruct( 'flags' , Flag( 'on' ), BitField( 'status' , 3 ), Flag( 'cache' ), Padding( 3 ) ), Byte( 'datalen' ), Array( lambda ctx: ctx[ 'datalen' ], Byte( 'data' )), Embed(message_crc) ) It shows off a few interesting features of construct : Explicit specification of endianness for multi-byte fields

Enumerations

Support for byte-oriented and bit-oriented fields

Arrays of data with specified length

Embedded structs The message should look roughly familiar for anyone designing and using binary protocols. It's very typical of how real formats look - some ID fields, flags, data, CRC . Here's how this message format can be used to pack and unpack a message: >>> from sampleformat import message_format >>> from construct import * >>> raw = message_format.build(Container( ... msg_id= 0 x1234, ... dest_addr= 0 xacba, ... command_type= 'RESTART' , ... flags=Container(on= 1 , cache= 0 , status= 4 ), ... datalen= 4 , ... data=[ 0 x1, 0 xff, 0 xff, 0 xdd], ... crc= 0 x12345678)) >>> raw.encode( 'hex' ) '3412baac40c00401ffffdd78563412' >>> c = message_format.parse(raw) >>> print c Container: msg_id = 4660 dest_addr = 44218 command_type = 'RESTART' flags = Container: on = True status = 4 cache = False datalen = 4 data = [ 1 255 255 221 ] crc = 305419896 A few things to note here: message_format is an object with two useful methods: build for packing data into a string, and parse for unpacking it back from a string.

is an object with two useful methods: for packing data into a string, and for unpacking it back from a string. Container is a class taken from construct . It's just a simple data container holding its data items in attributes. Any compatible object would do here (duck typing!) - for example a namedtuple . I chose Container because it comes with construct anyway and is simple and useful.

is a class taken from . It's just a simple data container holding its data items in attributes. Any compatible object would do here (duck typing!) - for example a . I chose because it comes with anyway and is simple and useful. raw is a packed string. The encode string method is used here to show the hex values of the string's bytes.

Framing (protocol wrapping and unwrapping) protocolwrapper.py in the code archive is a faithful Python implementation of the Framing in serial communications article. Not much more to say about it here - the code is commented and should be simple to understand if you're familiar with the theory.