Python Programming, news on the Voidspace Python Projects and all things techie.

IronPython and Unicode (or another reason to use IronPython - or why wait until Python 3 ?)

It's kind of obvious now I think about it, but I'd never seen this :

>> > type ( u'hello' )

< type 'str' >

>> > unicode

< type 'str' >

>> > 'Andr\202'

u'Andr\x82'

>> > type ( u'Andr\x82' )

< type 'str' >

How did I not notice this before ?

Creating .NET Classes Dynamically from IronPython

There are two restrictions in IronPython that I would like an automated approach to overcome. They can currently both be solved by writing stub classes in C#; but I would rather not have to write C# (by hand) if I don't have to.

You can't use .NET attributes in IronPython

You can't create classes in IronPython and consume them from other .NET languages

Additionally there is a problem with some types of data bound controls that require real dotnet classes rather than instances of IronPython classes (even when they inherit from dotnet classes it seems). One of the approaches mentioned here may be appropriate for solving this problem as well.

The stub C# classes you need to create are simple 'proxy' classes, and I would like to automate their creation from the IronPython side. The proxy instances need to call back into the IronPython engine, and call the appropriate method on a corresponding instance of an IronPython class. In order to keep the CLR happy when we create a proxy method we need to specify the types for the call signature and the return value. If we want to use dotnet attributes we will need some way of specifying these for methods as well.

I'm not looking to create a fully generic solution and obviously with this approach we won't easily be able to add methods at runtime or change the types accepted by methods, but this seems like a reasonable restriction for the moment.

This entry is the start of my exploration of dynamic / semi-dynamic generation of assemblies and classes from IronPython.

The standard .NET API for programmatic generation of classes is called System.Reflection.Emit . It's a bit fiddly.

Obviously if I get a working solution I can build a nice API on top of it. All I've managed to do so far is translate an example in the MSDN docs from C# into IronPython. It is interesting enough that some of you might want to browse though.

Note There is a tool that I've found invaluable when translating C# examples into IronPython. This is Instant Python by Tangible Software. The demo version will translate up to one hundred lines at a time. It doesn't get everything right, but not only are they very responsive to bug reports it can be darn useful.

The example below creates an assembly (which is later saved to disk as "DynamicAsm.dll"). It contains a class called "DynamicType", with a single static method called 'test'. This method takes four integers and adds them together.

The nasty part is that the body of the 'test' method is created by emitting the appropriate IL : .NET bytecode .

from System import *

from System . Threading import *

from System . Reflection import *

from System . Reflection . Emit import *



def addMethodDynamically ( typeBuilder , methodName , methodParams , returnType ) :

methodBuilder = typeBuilder . DefineMethod (

methodName ,

MethodAttributes . Public | MethodAttributes . Static ,

returnType ,

methodParams

)

ILout = methodBuilder . GetILGenerator ( )

numParams = len ( methodParams )

x = 0

while x < numParams :

ILout . Emit ( OpCodes . Ldarg_S , x )

x += 1

y = 0

while y < ( numParams - 1 ) :

ILout . Emit ( OpCodes . Add )

y += 1

ILout . Emit ( OpCodes . Ret )





def main ( ) :

domain = Thread . GetDomain ( )

asmName = AssemblyName ( )

asmName . Name = "DynamicAssembly"



asmBuilder = domain . DefineDynamicAssembly ( asmName , AssemblyBuilderAccess . RunAndSave )

module = asmBuilder . DefineDynamicModule ( "DynamicModule" , "DynamicAsm.dll" )

typeBuilder = module . DefineType ( "DynamicType" , TypeAttributes . Public )



methodName = "test"

inputNumsList = ( 2 , 3 , 4 , 5 )



methodParams = Array [ Type ] ( ( Int32 , Int32 , Int32 , Int32 ) )

inputValsList = Array [ object ] ( inputNumsList )



returnType = Int32





addMethodDynamically ( typeBuilder , methodName , methodParams , returnType )

myType = typeBuilder . CreateType ( )

result = myType . InvokeMember ( methodName ,

BindingFlags . InvokeMethod | BindingFlags . Public | BindingFlags . Static ,

None , None ,

inputValsList

)

print "The result of adding the inputted values is: %s" % result

print "---"

















asmBuilder . Save ( "DynamicAsm.dll" )

methodInfo = myType . GetMethod ( methodName )

print "Your Dynamic Method: %r;" % methodInfo



main ( )

The nice thing is that this saves "DynamicAsm.dll" to disk. You can then start an IronPython interactive interpreter session and do the following :

>> > import clr

>> > clr . AddReference ( 'DynamicAsm.dll' )

>> > import DynamicType

>> > DynamicType . test ( 2 , 3 , 4 , 5 )

14

Wow, we've dynamically generated a dotnet assembly containing a class, and then imported it into IronPython and called a generated static method on the class.

So, in order to create my proxy classes, they would somehow need a reference to the IronPython engine to be used - and a way of calling back into it. I guess the best approach would depend on the situation. You could create an initialisation class that creates and configures an IronPython engine, executing Python code to create the classes and create the interface layer on the Python side. In the constructors of the proxy classes (which would need to mimic the __init__ method of their corresponding Python classes), when the first instance is created it could do the initialisation.

If I wanted to follow this path I could write a proxy class by hand (not very difficult) and then disassemble the bytecode. I still don't fancy having to learn enough about IL to be able to do this , although the semantics of the Call opcode don't seem too tricky. Autogenerating C# would be much nicer, and if the target is a compiled assembly then it seems appropriate.

Luckily there is an example of doing exactly this by drifter46and2. It works and it doesn't seem too slow. For generating assemblies that provide proxy classes this would be a good approach.

For runtime (dynamic) generation of classes, then the MethodBuilder technique would be much better, especially if all you want is access to .NET attributes and you don't want to have to recompile an assembly every time (even if the code that does it is Python code). The way to generate attributes is with CustomAttributeBuilder. Hmmm....

Parsing Fun

I've had some fun playing with grammars using the excellent PLY.

Parsers can be a PITA, but it seems that the Python lexer ought to be able to handle the following without falling over :

>> > 3. __long__ ( )

File "<stdin>" , line 1

3. __long__ ( )

^

SyntaxError : invalid syntax

>> > 3 . __long__ ( )

3L

>> >

In case you can't see it, the second example has a space between the integer literal and the method call.

Archives