0x00 Introduction

Classes in Python are existed as objects, so you can create classes at runtime dynamically, which also shows the flexibility of Python.

I'll introduce how to use type to create classes dynamically as well as some related usage and techniques in this article.

0x01 The class

What is a class? A class is an abstraction for a kind of things which have common characteristics in real life, and it describes the properties and methods that are common to the objects being created. In common compiled languages ​​(such as C++), classes are defined at compile time and cannot be created at runtime dynamically. So how does Python do it?

Look at the following code:

class A(object): pass print(A) print(A.__class__)

The execution results in Python 2 are as follows:

<class '__main__.A'> <type 'type'>

The execution results in Python 3 are as follows:

<class '__main__.A'> <class 'type'>

It can be seen that the type of the class A is type , which means that the type will be instantiated as the class, and the class will be instantiated as the object.

0x02 Use type to create a class dynamically

The definition for the parameters of type are as follows:

Type(name, bases, dict)

name : the generated name of the class

bases : a list of generated base classes, and its type is tuple

dict : the properties or methods contained in the generated class

Let's say you want to create the class A , you can use the following method.

cls = type('A', (object,), {'__doc__': 'class created by type'}) print(cls) print(cls.__doc__)

The output is as follows:

<class '__main__.A'> class created by type

It can be seen that the class created in this way is nearly the same as the statically defined class, and the former is even more flexible in use.

One of the usage scenarios for the method is:

You may need to pass a class as a parameter to some places, and there are variables which can be affected by the outside to be used in the class; of course, you can use global variables to solve the problem, but it looks ugly. So at this point you can use the method of creating a class dynamically.

Here is an example:

import socket try: import SocketServer except ImportError: # python3 import socketserver as SocketServer class PortForwardingRequestHandler(SocketServer.BaseRequestHandler): '''process the request of port forwarding ''' def handle(self): sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.connect(self.server) # self.server is passed in during the time of the classes being created dynamically # connect to the target server and forward the data # The following code is omitted... def gen_cls(server): '''create the subclasses dynamically ''' return type('%s_%s' % (ProxyRequestHandler.__name__, server), (PortForwardingRequestHandler, object), {'server': server}) server = SocketServer.ThreadingTCPServer(('127.0.0.1', 8080), gen_cls(('www.tutorialdocs.com', 80))) server.serve_forever()

In the above example, since the target server address is passed in by the user, and the instantiation of the PortForwardingRequestHandler class is implemented in the ThreadingTCPServer , we can't control them. Therefore, using the method of dynamically creating a class can solve the problem very well.

0x03 Use the metaclass

A class is a template for an instance, and a metaclass is a template for a class. Classes can be created by metaclasses, and the default metaclass of a class is type , so all metaclasses must be the subclasses of type .

Here is an example for a metaclass:

import struct class MetaClass(type): def __init__(cls, name, bases, attrd): super(MetaClass, cls).__init__(name, bases, attrd) def __mul__(self, num): return type('%s_Array_%d' % (self.__name__, num), (ArrayTypeBase,), {'obj_type': self, 'array_size': num, 'size': self.size * num}) class IntTypeBase(object): '''a type base class ''' __metaclass__ = MetaClass size = 0 format = '' # strcut format def __init__(self, val=0): if isinstance(val, str): val = int(val) if not isinstance(val, int): raise TypeError('type error：%s' % type(val)) self._net_order = True # store network order data by default self.value = val self._num = 1 def __str__(self): return '%d(%s)' % (self._val, self.__class__.__name__) def __cmp__(self, val): if isinstance(val, IntTypeBase): return cmp(self.value, val.value) elif isinstance(val, (int, long)): return cmp(self.value, val) elif isinstance(val, type(None)): return cmp(int(self.value), None) else: raise TypeError('type error：%s' % type(val)) def __int__(self): return int(self.value) def __hex__(self): return hex(self.value) def __index__(self): return self.value def __add__(self, val): return int(self.value + val) def __radd__(self, val): return int(val + self.value) def __sub__(self, val): return self.value - val def __rsub__(self, val): return val - self.value def __mul__(self, val): return self.value * val def __div__(self, val): return self.value / val def __mod__(self, val): return self.value % val def __rshift__(self, val): return self.value >> val def __and__(self, val): return self.value & val @property def net_order(self): return self._net_order @net_order.setter def net_order(self, _net_order): self._net_order = _net_order @property def value(self): return self._val @value.setter def value(self, val): if not isinstance(val, int): raise TypeError('type error：%s' % type(val)) if val < 0: raise ValueError(val) max_val = 256 ** (self.size) - 1 if val > max_val: raise ValueError('%d is more than the maximum size %d' % (val, max_val)) self._val = val def unpack(self, buff, net_order=True): '''extract data from buffer ''' if len(buff) < self.size: raise ValueError(repr(buff)) buff = buff[:self.size] fmt = self.format if not net_order: fmt = '<' + fmt[1] self._val = struct.unpack(fmt, buff)[0] return self._val def pack(self, net_order=True): '''return the memory data ''' fmt = self.format if not net_order: fmt = '<' + fmt[1] return struct.pack(fmt, self._val) @staticmethod def cls_from_size(size): '''return the corresponding class from the integer size ''' if size == 1: return c_uint8 elif size == 2: return c_uint16 elif size == 4: return c_uint32 elif size == 8: return c_uint64 else: raise RuntimeError('Unsupported integer data length：%d' % size) @classmethod def unpack_from(cls, str, net_order=True): obj = cls() obj.unpack(str, net_order) return int(obj) class ArrayTypeBase(object): '''array type base class ''' def __init__(self, val=''): init_val = 0 if isinstance(val, int): init_val = val else: val = str(val) self._obj_array = [self.obj_type(init_val) for _ in range(self.array_size)] # initialization self.value = val def __str__(self): return str(self.value) def __repr__(self): return repr(self.value) def __getitem__(self, idx): return self._obj_array[idx].value def __setitem__(self, idx, val): self._obj_array[idx].value = val def __getslice__(self, i, j): result = [obj.value for obj in self._obj_array[i:j]] if self.obj_type == c_ubyte: result = [chr(val) for val in result] result = ''.join(result) return result def __add__(self, oval): if not isinstance(oval, str): raise NotImplementedError('%s is not supported by type %s' % (self.__class__.__name__, type(oval))) return self.value + oval def __radd__(self, oval): return oval + self.value def __iter__(self): '''iterator ''' for i in range(self.length): yield self[i] @property def value(self): result = [obj.value for obj in self._obj_array] if self.obj_type == c_ubyte: result = [chr(val) for val in result] result = ''.join(result) return result @value.setter def value(self, val): if isinstance(val, list): raise NotImplementedError('ArrayType is not supported type list') elif isinstance(val, str): self.unpack(val) def unpack(self, buff, net_order=True): ''' ''' if len(buff) == 0: return if len(buff) < self.size: raise ValueError('unpack length error：%d %d' % (len(buff), self.size)) for i in range(self.array_size): self._obj_array[i].unpack(buff[i * self.obj_type.size:], net_order) def pack(self, net_order=True): ''' ''' result = '' for i in range(self.array_size): result += self._obj_array[i].pack() return result class c_uint8(IntTypeBase): '''unsigned char ''' size = 1 format = '!B' class c_ubyte(c_uint8): pass class c_uint16(IntTypeBase): '''unsigned short ''' size = 2 format = '!H' class c_ushort(c_uint16): pass class c_uint32(IntTypeBase): '''unsigned int32 ''' size = 4 format = '!I' class c_ulong(c_uint32): pass class c_uint64(IntTypeBase): '''unsigned int64 ''' size = 8 format = '!Q' class c_ulonglong(c_uint64): pass cls = c_ubyte * 5 print(cls) val = cls(65) print(val)

The outputs for the above code in Python 2.7 are as follows:

<class '__main__.c_ubyte_Array_5'> AAAAA

The definition of the metaclass has been modified in Python 3:

class IntTypeBase(object, metaclass=MetaClass): pass

You can use the methods in the six base for compatibility:

import six @six.add_metaclass(MetaClass) class IntTypeBase(object): pass

The advantage of using metaclasses is that you can create classes in a more elegant way, such as the c_ubyte * 5 in the above code, which can improve code readability.

0x04 Rewrite the __new__ method

Each class that inherits from object will have a __new__ method, which will be called earlier than __init__ when the class is being instantiated. And the type it returns determines the type of the object that is created ultimately.

Let's look at the following code:

class A(object): def __new__(self, *args, **kwargs): return B() class B(object): pass a = A() print(a)

The output is as follows:

<__main__.B object at 0x023576D0>

As you can see, though it instantiates A in the code, the returned object type is B . which is mainly attributed to the __new__ method.

The following example shows how to create a class dynamically in the __new__ ：

class B(object): def __init__(self, var): self._var = var def test(self): print(self._var) class A(object): def __new__(self, *args, **kwargs): if len(args) == 1 and isinstance(args[0], type): return type('%s_%s' % (self.__name__, args[0].__name__), (self, args[0]), {}) else: return object.__new__(self, *args, **kwargs) def output(self): print('output from new class %s' % self.__class__.__name__) obj = A(B)('Hello World') obj.test() obj.output()

The output is as follows:

Hello World output from new class A_B

The example implements the process of creating subclasses for the two classes dynamically, and it's suitable for the scenes where you need to generate many subclasses from arranging and combining classes, which can avoid the pain of writing a bunch of subclass code.

0x05 Summary

You must use the type implementation to create classes dynamically. However, you can choose different methods depending on the scenes.

Actually, it's not friendly to static analysis tools, because the type has changed during the run time. Moreover, it will also reduce the readability of the code, so generally, it's not recommended to use such skilled code.