Python Programming, news on the Voidspace Python Projects and all things techie.

Silverlight Article Updates and PyCrypto for AMD64

I've updated a couple of the Silverlight articles on my IronPython and Silverlight Pages:

From Silverlight to Javascript and Back Again An article on using the Scriptable attributes to call from IronPython into Javascript and from Javascript into IronPython. The update makes a couple of parts less confusing and corrects a mistake. Thanks to Jimmy Schementi for pointing these out. The download is unchanged.

Embedding IronPython in a C# Silverlight Application Although the minimal example I had in this article worked, it had problems as soon as you tried to do anything non-trivial (like import Python modules for example!). This has now been corrected; thanks to Richard Klafter for pointing this out and providing the correction. The example project for download has been extended with the new code.

I've also posted online a binary distribution of PyCrypto 2.0.1 for Windows, for Python 2.6, for AMD64 processors! This was provided, by Jeffrey Foran. Many thanks. He says (by the way):

It was built with the free MSVC compiler and some minor hackery (minimal hackery to the pycrypto files, lots of hackery to the python and microsoft build tools). It passes all the tests fine, but it was not fun to actually get together. I tried very hard to build 2.5 also, but it was much more difficult for various reasons.

Resolverhacks: Unit Testing Spreadsheets and Importing CSV Files with Cell.Formula

I've created two new spreadsheets for the Resolver Hacks Website (with more in the works). I haven't yet written the articles that accompany them, but I've linked to them here in case it takes me a while to get round to it... Both of these articles use new features in Resolver One 1.4. Resolver One is a .NET programmable spreadsheet for developing applications and spreadsheet systems. Spreadsheets created with Resolver One are represented as IronPython programs which you can add code to using Python and .NET libraries: the dreamweaver of spreadsheets.

Both of these spreadsheets are hosted on the Resolver Exchange:

Importing CSV Files Using the Cell Formula Property This spreadsheet is both a tool and an example. Resolver One doesn't currently support importing from CSV files out of the box . You can easily import CSVs from user code, but the import is repeated every recalc and you can't edit the values. This spreadsheet allows you to import a comma or tab delimited file into a worksheet. It uses a .NET CSV Parser shipped with Resolver One and so has no external dependencies. Click the button on the 'Import' spreadsheet and select the CSV file to import. After importing, the contents of the file will be in the 'Data' worksheet and you can delete the user code and save the spreadsheet with a new name. Resolver One version 1.4 allows you to set data in the grid using the Formula property on cells. As well as being a useful tool this spreadsheet provides examples of using the cell Formula property plus using the OpenFileDialog and MessageBox (Windows Forms) dialogs from user code.

Test Driven Development of Spreadsheets One of the coolest features of Resolver One is RunWorkbook. It can be used to create spreadsheet systems where the data is stored in a single spreadsheet and used in multiple places, or for turning spreadsheets into functions that can be called with new data to run calculations on. RunWorkbook is an IronPython function that exposes the Resolver One calculation engine - and we can actually use it from IronPython to Access Resolver One spreadsheets from Python programs. One of the interesting possibilities this opens up is doing Test Driven Development of spreadsheets. You can write unit tests (using the Python unittest module) for the spreadsheets passing in known data to RunWorkbook and introspecting the returned workbook object to make assertions. Although it will probably make more sense when the article is written, the zip file for download contains the spreadsheet and a unit test file that incrementally tests features in the spreadsheet. See the note below about the setup needed to run the tests (you need Resolver One 1.4 installed for starters).

Note Both of these spreadsheets require Resolver One version 1.4. You can download it from here. In order to run the tests for the TDD example you will need to follow the instructions in Using Resolver One Spreadsheets from IronPython. This means copying ipy.exe from IronPython 2.0.1 into the bin directory of your Resolver One install.

We also have a new Resolver One ad on The Daily WTF, this time promoting the Resolver One Spreadsheet Challenge. You might have to refresh the page a few times to see it!

Resolver One 1.4: IronPython 2, Numpy, and R

Resolver One is the Python and .NET programmable spreadsheet created by Resolver Systems After a lot of work, Resolver One 1.4 is now released! Version 1.4 took us a long time; the main feature is that Resolver One is now built on IronPython 2. We worked closely with the IronPython team on improving the performance of IronPython 2 - Resolver One exposed performance issues that only show up when you run a large Python application and IronPython 2.0.1 is largely the result of the work of Glenn (Resolver Systems) and Dino Veihland (Microsoft IronPython team) working together on it.

Tangible benefits this brings to Resolver One users include: Python 2.5 ( with statement - yay!) and improved compatibility with the standard library (especially urllib2 which many users have requested). Due to the new compilation technique startup is also slightly faster.

The three other major features in Resolver One 1.4 are:

The first step towards model side scripting - button handler code can now modify values and formulae in the spreadsheet (see Cell.Formula documentation)

Support for Numpy arrays in the spreadsheet grid - through Ironclad (see this screencast for an illustration of what is possible)

24 new statistical functions now supported by Resolver One (see this list of all Spreadsheet functions in Resolver One)

You can read all about the release and a full list of all changes:

As always you can download a 31 days free trial, or if you are sharing the spreadsheets you create you can request an open source license: Download Resolver One.

The Resolver Challenge is still running and we've just announced the winner of round two. Marjan Ghahremani has worked out how to get Resolver One to work with R, an advanced statistical system.

Marjan is a student at UC Davis, double majoring in psychology and anthropology. Her interest is in doing research on social and developmental psychology, and for data analysis she needs a very strong statistical tool. R fits the bill perfectly, and with her integration code she can call into it from the Resolver One graphical interface.

Many congratulations to Marjan, and a huge "thank you" to all the other entrants! You can download the R integration code, along with an explanatory PDF file, from here.

In order to run complicated statistical analysis/modeling I use R (an open source clone of the powerful commercial language S-Plus). R is the most successful statistical library currently existing on the market. R has different powerful libraries for data mining, genetics, portfolio optimization, asset allocation, etc. Although R is the strongest tool for statistical analysis, unlike SPSS it lacks an easy to use yet a powerful spreadsheet. On the other hand building a bridge between R and Resolver one is extremely difficult and involved a number of trial and errors and using Interop objects. By using the provided framework one can enjoy the power of all of the R libraries with the comfort and the power of Resolver One. The accompanying examples will help the reader to use similar strategies to solve complicated statistical problems and to develop exciting results and 2D/3D graphs using OpenGL capabilities and drawing. Users are able to develop fly-through 3D visualizations from Resolver One by using RGL library in R (see screenshot). I have made an effort to provide a complete framework with directions for how to write a program in this framework and how to do error handling between R and Resolver One. All the complex libraries are provided in binary format and users only need to call them from within their Resolver One. It will then enable them to use all the functionality of R inside their Resolver One spreadsheets and send them information successfully back and forth between these two programs. The same component can be used to call SciLab functions (a clone of MATLAB, a scientific computing environment), using the same procedure provided in the article, users are able to use SciLab similar to the way they use R inside Resolver One.

If you weren't able to enter this round of the competition, or want to try again, there are three more rounds to go - with a $2,000 prize for each - before the final "best of the best" $15,000 decision in May

Interface design, Usability and Kitchen Hobs

I'm very interested in application usability. I'm aware that for those who aren't computer experts they can be complex and bewildering. As someone who has been using using computers inside and out for years this is something that is easy to forget; that the metaphors on which a modern OS is built are not 'intuitive' to the average human. To use modern computers fluently requires a huge amount of background experience that only comes from time, and even better from taking apart machines and programs to understand what is really going on underneath the metaphors.

Poorly designed applications and user interfaces on top of this make computers painfully difficult. On the other hand computers and applications can be usable, and if the things we create don't make people's lives less painful then we're wasting our time. I'm also interested in programming language design, and I like this quote from Guido van Rossum: "a programming language is an HCI" (Human Computer Interface). Programming languages are compromises between the language that computers speak (an electromagnetic dance where even 1s and 0s are an abstraction for the sake of humans) and a language that we're capable of reading and writing.

User interface design isn't restricted to the world of computers. My Dad is a health and safety consultant, specialising in health and safety as it applies to computer controlled systems. This includes industrial plant and machinery, power stations, oil rigs and the like. One of the most common causes of accidents, including fatal accidents, is operator error. Behind the operator error is often a poorly designed user interface, making the system or machinery much harder to operate correctly under pressure: bad UIs kill!

Because of this my Dad has also been interested in interface design for many years. He tells me that some of the best studies in the field are based on child psychology and there are aspects of interface design that can be categorised as objectively good or bad. One thing that every child learns early in its development is the principle of one-to-one correspondence. In its basic form you can see this in young children learning to use their limbs; that signals they send from their brain result in a corresponding movement in their arms and legs is a fascinating voyage of discovery for young babies.

This principle of one-to-one correspondence is baked into us and applies to the HCI. When we move the mouse physically with our hands the pointer moves in exact correspondence; which makes it easy to learn.

One example of an astonishingly badly designed user interface is the kitchen hob. The most common design has four cooking rings arranged in a square pattern. If it had four control knobs arranged in the same pattern then everyone would know which knob corresponded to which ring intuitively. The most common design pattern for hobs lays the four knobs out in a straight line; violating one-to-one correspondence with the consequence that you have to look and work out which one you need to use (and if you're anything like me probably getting it wrong anyway). If the knobs were arranged in the same pattern as the rings then everyone would find them straightforward to operate; one-to-one correspondence at work.

Real World IronPython at QCon

Next week I'll be at QCon in London, speaking on Real World IronPython:

IronPython is a port of Python, a popular open source dynamic language, to the .NET framework. .NET has traditionally been programmed with statically typed languages like C# and VB.NET. Resolver Systems has been developing a full application with IronPython for the last three years. In this session we'll see how a dynamic language integrates with the .NET framework, learn what advantages it brings and look at several real-world use cases for IronPython; including adding user scripting to .NET applications.

This will be a new talk for me, and I'm looking forward to it. I was invited to QCon by Ola Bini who is running the Emerging Languages in the Enterprise track. Other speakers in this track include Martin Fowler, so I'm in illustrious company!

If you're at QCon come and say hi, see you there.

Archives