Python Programming, news on the Voidspace Python Projects and all things techie.

Testing Tests

At Resolver we have some tests for our test framework.

For example:

FunctionalTestTest is a test that tests tests. In the test setup it sets-up the test that it is going to test.

My sincere apologies to anyone who actually read this...

Further Polish Adventures: Academic Computer Science Festival

Thanks to Jan Szumiec (whom I mentioned in the previous entry) is a student at the Technical University of Cracow. He is organising this year's Academic Computer Science Festival from the 8th to the 10th of March.

I've been asked to speak there on IronPython.

I was his second choice to speak, but I'm not too insulted as I'm really looking forward to the opportunity to see Cracow which is said to be a very beautiful city.

As the only Polish folk I know both program natively in Ruby, it's a little daunting. I'll be presenting to people who natively both speak and program in a different language.

Interesting Things

Here are three interesting things I've stumbled across recently. They've all been linked to elsewhere, so my apologies.

Compile C# From IronPython Some example code from a gentleman allegedly called drifter46and2. It shows how to compile C# code from IronPython and get a reference to the assembly. I haven't played with this yet, so I don't know how slow it is. It also appears to save the generated assemblies to disk, keeping them in memory would be more interesting to me. The reason this looks interesting to me is that there are two limitations with IronPython that this technique potentially overcomes . IronPython is a Python compiler - this means it compiles Python source code to assemblies . However the .NET garbage collector doesn't collect class definitions, so IronPython reuses a single class for all user defined classes. This means you can't create an assembly that can be consumed from C#. Instead you have to create a stub C# class yourself, and hook up the methods. By generating and compiling C# dynamically you could automate this. The second limitation this causes is that you can't use .NET attributes, and again have to use stub C# classes.

IronPython now Included in Mono 1.2.3 As well as several fixes to the Windows Forms implementation, this version of Mono now includes the IronPython Community Edition. W00t!

The Problem With Ruby I have two Polish friends who are Ruby developers. One of them (Jan rather than Andrzej) was of the opinion that Ruby was more object oriented than Python because you are able tack methods onto the built-in classes. Of course the 'Python Philosophy' (tm) is that allowing this is a crazy recipe for disaster. Generally the opinion in the Ruby world is that disaster doesn't happen very often and it is a technique that can be very useful. Well, disaster can happen, and the article on method_missing is a tale of how one Ruby programmer came at least temporarily unstuck. I like Simon Willison's comment: My least favourite thing about Ruby is the cultural tendency towards introducing weird new bugs in other peopleâ€™s code. Do I sound smug ?

Wing at Resolver

We've finally browbeaten Giles sufficiently and he's bought three of us at Resolver shiny new Wing IDE licenses.

One of us is using Eclipse with the free PyDev extensions and two of the developers here still use Textpad.

Eclipse has some really nice features, but also some annoying ones, and the ease with which you can script Wing was a big win for me.

A gentleman called Brandon Corfman has published an interesting few blog entries on scripting the Komodo IDE, mainly using Bicycle Repair Man which is a refactoring tool for Python.

Refactoring is something that Eclipse is particularly good at (coming from the Java world) and the couple of times I've used its Extract Method it behaved very intelligently.

There are two of Brandon's Komodo scripts that I'd like to port (or see ported) to Wing:

Rename Occurrences allows you to rename all occurrences of a variable or method name wherever it appears in your code.

Extract Method allows you to highlight a section of existing Python code in Komodo and refactor it into its own separate method.

Porting both should be well possible. We also have a custom PyLint setup that checks our whole codebase, we run this before checking in (it checks for unused imports and unused variables amongst other things). Being able to run this on a single file from the IDE will be very handy.

Programming Language Metrics

Ok, so programming language metrics aren't worth the paper they're printed on, but they're still fun.

I've just stumbled across a metric I'd not seen before:

It charts various languages by the number of sourceforge projects, and goes from 2001 to the end of 2006 when sourceforge stopped providing this information.

The results (unsurprisingly) show Java, C++ and C way out front, followed by PHP (a rising but flattening curve) and a gracefully declining Perl.

Python comes next, at 4.88%, and is a full point and a half above C# and then Javascript.

As this charts Open Source projects it is most closely a metric of language popularity rather than 'industrial' usage. It reflects what programmers like to do in their spare time. The fact that Python leads C# probably also reflects the fact that Python is much more prominent in the open source world.

There is a surprise, Ruby is shown as a relatively flat 0.44%. This is possibly because Ruby developers tend to use the popular RubyForge rather than sourceforge (which seems to be declining in importance itself).

The results are not dissimilar from the TIOBE Programming Community Index. By gathering data from search engines it is also a metric of language popularity. It too shows Python ahead of C#, but puts Visual basic much higher than the sourceforge figures.

TIOBE shows Ruby having grown significantly since mid 2006, and catching up on Python, but there are signs that the growth is slowing. The growth curve for Ruby on TIOBE has turned into a 'knee', the curve is flattening off.

There is a third metric. The Jobtrends by Indeed.com. I've only compared a few of the languages and it only shows comparisons going back around two years, but it tells a different story. By gathering data from job postings it is much more a metric of industrial usage than popularity.

It shows C# as rising, now reaching around the same point as the declining Perl, and well ahead of Python. It shows a gentle rise for Python, which is well above Ruby.

If you're obsessed with metrics like this, then you might want to look at google search trends. This shows a similar picture to TIOBE (although it allows a maximum of five terms), except it puts C# ahead of Python.

Interesting.

Akismet 0.1.5

Python Akismet 0.1.5 is now available.

Fixed a typo/bug in submit_ham . Thanks to Ian Ozsvald for pointing this out.

Python Akismet is a Python interface to the Akismet, spam blocking web-service. It is aimed at trapping spam comments.

The Python interface comes with an example CGI.

ConfigObj 4.4.0 and Validate 0.2.3

Updated versions of both ConfigObj and Validate are now available.

ConfigObj is a Python module for the simple reading and writing of config files. It has many features, whilst remaining easy to use.

With the assistance of Validate it can validate a config file against a specification, and convert members to the expected type.

Eggs for Python 2.4 & 2.5 are available from the cheeseshop.

Thanks to Nicola Larosa who implemented most of the fixes in this release.

What is New in ConfigObj 4.4.0?

Made the import of compiler conditional so that ConfigObj can be used with IronPython.

Fix for Python 2.5 compatibility.

String interpolation will now check the current section before checking DEFAULT sections. Based on a patch by Robin Munn.

Added Template-style interpolation, with tests, based on a patch by Robin Munn.

Allowed arbitrary indentation in the indent_type parameter.

parameter. Fixed Sourceforge bug #1523975 by adding the missing self

What is New in Validate 0.2.3?

Fixed validate doc to talk of boolean instead of bool; changed the is_bool function to is_boolean (Sourceforge bug #1531525).

Firedrop2: Blog Statistics

Davy Mitchell has made a nice addition to the Firedrop2 blogging tool I use.

It generates a page of statistics about your blog entries, number of words per post, number of posts per category and the like.

Formatting aside, it's a nifty little addition. Unsurprisingly 455 of my 606 blog entries are in the Python category.

The code hasn't yet been checked in, but you can download the changes as a zip file: BlogStats.zip

PyCamp UK ?

Jeff Rush has just announced that the Dallas and Houston Python User Groups are looking to arrange a regional Python 'unconference': PyCamp.

This sounds like a great idea, and Michael Sparks has picked up on the idea for the UK.

Anyone up for a PyCamp UK ? I certainly am.

Michael is suggesting a venue in Manchester and looking for volunteers to see if anyone is interested...

Book Progress

I've just finished the first draft of chapter two, the Python tutorial. I've wanted to write a Python tutorial for a long time, but that thirst is pretty much quenched right now.

It will still need a fair amount of polishing I guess, but the basic content is pretty good. I'd love to share it, and although I'm enjoying having my work assessed by a professional editor I would prefer to be able to make it available to everyone. Surprisingly (to me) the hardest parts are the introductions and the summaries.

At some point in the long distant future I'll write another Python tutorial which I'll publish on my site. As you might expect, writing is pretty much taking up all my spare time at the moment. It will be a while after the book is out before I do much more writing, first I'll be taking a good break...

Speaking of breaks, Andrzej and I now have our flights and hotel booked for PyCon. I'm really looking forward to it. We're also working on the presentation and a simple example app. to go along with it. Before we leave for the US we'll put the talk notes and the app online.

The Event Pattern

Since I started work at Resolver Systems nearly a year ago, Resolver the application has grown quite considerably. It is now just under twenty thousand lines of production Python code, plus a few hundred lines of C# (some of which is autogenerated by the Visual Studio designer ).

There are also around seventy thousand lines of test code, this is our test framework plus unit and functional tests.

The amazing thing is that we have created a highly functional application in only twenty thousand lines of code.

Resolver is structured using a fairly straightforward Model-View-Controller structure. The view is our main form class, we have several controllers managing the interaction of the model and the view. The model are our various classes that represent the data-set being edited.

We use various other patterns to good effect in Resolver: the observer pattern and the command pattern amongst others.

Some of our classes, like the controllers and the commands are data-set observers. When the data-set is changed (like a new one is loaded) they need to be informed. As the main form is responsible for loading new data-sets it sets the new ones onto all the observers.

The whole point of the MVC structure (and this post has a point too, I promise) is to decouple the model classes from the view and controller classes. The model doesn't need to know anything about these classes. Changes to the view and controllers don't (in theory anyway) affect the model at all, and vice versa.

Note Thanks to Jonathan for a correction. This should read "the View-Controller pairs are allowed to know about the model, though not vice-versa. This means changes to the model can impact the View-Controller pair.". This is why you need to define a clear public API on your model, your View-Controllers need to know how to access the model in order to present a view on the data.

The difficulty is that the controllers often need to be informed when the data-set changes, but we still want to maintain the separation. That means the data-set shouldn't need to have a reference to the controllers.

At Resolver we use a solution I haven't seen before. It may in fact be a part of the classic observer pattern, but it is certainly a natural extension of it.

IronPython has a nice way of interacting with .NET events. You add and remove event handlers to events using add and remove in place syntax.

button = Button ( Text = "Button Text" )

def onClick ( sender , event ) :

print 'You clicked me'







button . Click += onClick





button . Click -= onClick

At Resolver we have created our own event hook class which is used in the same way. I actually introduced this class in my Background Worker Threads article on IronPython. I'd like to briefly show how it can be used to communicate between model and controllers, in concert with the observer pattern.

class EventHook ( object ) :

def __init__ ( self ) :

self . __handlers = [ ]



def __iadd__ ( self , handler ) :

self . handlers . append ( handler )

return self



def __isub__ ( self , handler ) :

self . handlers . remove ( handler )

return self



def fire ( self , * args , ** keywargs ) :

for handler in self . __handlers :

handler ( * args , ** keywargs )

Small and cute don't you think ?

Lets imagine we have a DataSet class that needs to inform (some of) its observers when it has changed, perhaps to show a modified flag on the GUI to give a trivial example.

We have a MainForm class which manages the observers. When a new data-set is loaded it sets the 'dataset' property on all the observers.

class MainForm ( Form ) :



def __init__ ( self ) :

self . _controller = Controller ( )

self . observers = [ self . _controller ]



def loadDataset ( self ) :

dataset = self . magicallyGetNewDataset ( )



for observer in self . observers :

observer . dataset = dataset

We also have a Controller class (initialised in MainForm above). In its handler for the 'dataset' property it registers itself to listen to the 'modified' event on the new dataset. If necessary it unregisters itself from any previous events it may have been listening to.

class Controller ( object ) :



def __init__ ( self ) :

self . __dataset = None



def setDataSet ( self , dataset ) :

if self . __dataset is not None :





self . __dataset . modified -= self . onModified



self . __dataset = dataset







self . __dataset . modified += self . onModified



dataset = property ( lambda self : self . __dataset , setDataSet )



def onModified ( self ) :







When the MainForm sets the dataset on the controller, setDataSet is called.

And so the final piece of the jigsaw, the DataSet class.

class DataSet ( object ) :

def __init__ ( self ) :

self . modified = EventHook ( )



def doSomething ( self ) :



if somethingHasChanged :

self . modified . fire ( )

In the initialiser it creates an event hook; the 'modified' attribute. When the dataset is set on the observers, any that are interested in the modified flag can register with the modified event.

When the data-set is modified, it calls self.modified.fire() and all interested parties are notified. The observers can then use the public API of the dataset (which they hold a reference to) to work out how they need to respond. The dataset itself doesn't need to hold a reference to any of the observers.

The advantage of this kind of approach (and I'm sure there are many alternatives) is that you can then re-use your core classes in another situation, hopefully without having to rewrite a line.

It does require careful managing of dependencies, and a sensible public API. In the example above you could take DataSet and use it in a tool that doesn't have a front-end at all (like a data converter) or embed it in a web application.

Archives