tl;dr

pip install fast-enum

What are enums

# /path/to/package/static.py: INITIAL = 0 PROCESSING = 1 PROCESSED = 2 DECLINED = 3 RETURNED = 4 ...

class MyModelStates: INITIAL = 0 PROCESSING = 1 PROCESSED = 2 DECLINED = 3 RETURNED = 4

namedtuple()

MyModelStates = namedtuple('MyModelStates', ('INITIAL', 'PROCESSING', 'PROCESSED', 'DECLINED', 'RETURNED')) EntityStates = MyModelStates(0, 1, 2, 3, 4)

namedtuple

INITIAL

gettext

INITIAL = (0, 'My_MODEL_INITIAL_STATE')

class MyModelStates: INITIAL = (0, 'MY_MODEL_INITIAL_STATE')

namedtuple

EntityStates = MyModelStates((0, 'MY_MODEL_INITIAL_STATE'), ...)

my_entity.state = INITIAL[0]

my_entity.state = MyModelStates.INITIAL[0]

my_entity.state = EntityStates.INITIAL[0]

And then Enums come at the stage

class MyEntityStates(Enum): def __init__(self, val, description): self.val = val self.description = description INITIAL = (0, 'MY_MODEL_INITIAL_STATE') PROCESSING = (1, 'MY_MODEL_BEING_PROCESSED_STATE') PROCESSED = (2, 'MY_MODEL_PROCESSED_STATE') DECLINED = (3, 'MY_MODEL_DECLINED_STATE') RETURNED = (4, 'MY_MODEL_RETURNED_STATE')

{% for state in MyEntityState %} <option value=”{{ state.val }}”>{{ _(state.description) }}</option> {% endfor %}

my_entity.state = MyEntityStates.INITIAL.val

Why is it faster?

3 times faster on member access

~8.5 times faster on attribute ( name , value ) access

, ) access 3 times faster on enum access by value (call on enum's class MyEnum(value) )

) 1.5 times faster on enum access by name (dict-like MyEnum[name] )

__slots__

Slots

__slots__

__slots__

__slots__

Descriptors

value = my_obj.attribute # this is a direct access to the attribute value by the pointer that the object holds for that attribute

__get__

obj_attribute = my_obj.attribute obj_attribute_value = obj_attribute.__get__(my_obj)

Enums in Standard Library

name

value

types.DynamicClassAttribute

name

value

one_value = StdEnum.ONE.value # that is what you write in your code one_value_attribute = StdEnum.ONE.value one_value = one_value_attribute.__get__(StdEnum.ONE)

# and this is what really __get__ does (python 3.7 implementation): def __get__(self, instance, ownerclass=None): if instance is None: if self.__isabstractmethod__: return self raise AttributeError() elif self.fget is None: raise AttributeError("unreadable attribute") return self.fget(instance)

# since DynamicClassAttribute is a decorator on Enum methods `name` and `value` the final row of __get__() ends up with: @DynamicClassAttribute def name(self): """The name of the Enum member.""" return self._name_ @DynamicClassAttribute def value(self): """The value of the Enum member.""" return self._value_

def get_func(enum_member, attrname): # this is also a __dict__ lookup so hash + hashtable scan also occur return getattr(enum_member, f'_{attrnme}_') def get_name_value(enum_member): name_descriptor = get_descriptor(enum_member, 'name') if enum_member is None: if name_descriptor.__isabstractmethod__: return name_descriptor raise AttributeError() elif name_descriptor.fget is None: raise AttributeError("unreadable attribute") return get_func(enum_member, 'name')

from enum import Enum class StdEnum(Enum): def __init__(self, value, description): self.v = value self.description = description A = 1, 'One' B = 2, 'Two' def get_name(): return StdEnum.A.name from pycallgraph import PyCallGraph from pycallgraph.output import GraphvizOutput graphviz = GraphvizOutput(output_file='stdenum.png') with PyCallGraph(output=graphviz): v = get_name()

name

value

def name(self)

from fast_enum import FastEnum class MyNewEnum(metaclass=FastEnum): A = 1 B = 2 def get_name(): return MyNewEnum.A.name from pycallgraph import PyCallGraph from pycallgraph.output import GraphvizOutput graphviz = GraphvizOutput(output_file='fastenum.png') with PyCallGraph(output=graphviz): v = get_name()

name

value

name

value

Enum

_member_names_ — a list that holds all the names of enum members;

— a list that holds all the names of enum members; _member_map_ — an OrderedDict that maps a name of an enum member to the member itself;

— an OrderedDict that maps a name of an enum member to the member itself; _value2member_map_ — a reverse dictionary that maps enum member values to corresponding enum members.

StdEnum.MEMBER

Our way

an Enum must be as static as possible; by “static” we mean: If something could be calculated once and at declaration time, it should;

an Enum can not be subclassed (must be a “final” class) if a subclass defines new enum members — this is true for standard library implementation, with the exception that subclassing is prohibited even if no new members defined;

an Enum should have vast possibilities for extensions (additional attributes, methods and so on).

value

=

A = 1, 'One'

1, "One"

value

A: 'MyEnum' = 1, 'One'

1

value

__slots__

__slots__

__dict__

__slots__

__slots__

What are the additional perks?

typing

typing

__new__

auto

_ZERO_VALUED

Aliases and how they could help

package_a.some_lib_enum.MyEnum

class MyEnum(metaclass=FastEnum): ONE: 'MyEnum' TWO: 'MyEnum'

package_b.some_lib_enum.MyMovedEnum

class MyMovedEnum(MyEnum): pass

MyEnum

MyMovedEnum

MyEnum

MyEnum

MyMovedEnum

MyEnum

MyEnum

MyMovedEnum

MyMovedEnum

MyEnum

MyEnum

MyMovedEnum

MyMovedEnum

MyEnum

(If you think you know that — scroll down to the “Enums in Standard Library” section).Imagine that you need to describe a set of all possible states for the entities in your database model. You'll probably use a bunch of constants defined as module-level attributes:...or as class-level attributes defined in their own class:That helps you refer to those states by their mnemonic names, while they persist in your storage as simple integers. By this, you get rid of magic numbers scattered through your code and make it more readable and self-descriptive.But, both the module-level constant and the class with the static attributes suffer from the inherent nature of python objects: they are all mutable. You may accidentally assign a value to your constant at runtime, and that is a mess to debug and rollback your broken entities. So, you might want to make your set of constants immutable, which means both the number of constants declared and the values they are mapped to must not be modified at runtime.For this purpose you could try to organize them into named tuples with, as an example:However, this still doesn't look too understandable: in addition to that,objects aren't really extensible. Let's say you have a UI which displays all these states. You can then use your module-based constants, your class with the attributes, or named tuples to render them (the latter two are easier to render, while we're at it). But your code doesn't provide any opportunities to give the user an adequate description for each state you've defined. Furthermore, if you plan to implement multi-language support and i18n in your UI, you'll find that filling in all the translations for these descriptions becomes an unbelievably tedious task. The matching state values might not necessarily have matching descriptions which means that you cannot just map all of yourstates onto the same description in. Instead, your constant becomes this:Your class then becomes this:And finally, yourbecomes this:Well, good enough, it now makes sure both the state value and the translation stub are mapped to the languages supported by your UI. But now you may notice that the code which uses those mappings has turned into a mess. Whenever you try to assign a value to your entity, you also need not forget to extract value at index 0 from the mapping you use:ororAnd so on. Keep in mind that the first two approaches using constants and class attributes, respectively, still suffer from mutability.That’s it. Now you could easily iterate the enum in your renderer (Jinja2 syntax):Enum is immutable for both member set (you can’t define a new member at runtime, nor can you delete a member already defined) and those member values they keep (you can’t reassign any attribute values or delete an attribute).In your code you just assign values to your entities like this:Well, clear enough. Self-descriptive. Fairly extensible. That’s what we use Enums for.But the default ENUM is rather slow so we asked ourselves — could we make it faster?As it turns out, we can. Namely, it is possible to make it:Types and objects are dynamic in Python. But Python has the tools to limit the dynamic nature of the objects. With their help one can gain significant performance boost usingas well as avoid using Data Descriptors where possible without significant complexity growth or if you can get benefit in speed.For example, one could use a class declaration with— in this case, class instances would have only a restricted set of attributes: attributes declared inand allof parent classes.By default the Python interpreter returns an attribute value of an object directly:According to the Python data model, if the attribute value of an object is itself an object that implements the Data Descriptor Protocol, it means that when you try to get that value you first get the attribute as an object and then a special methodis called on that attribute-object passing the keeper object itself as an argument:At leastandattributes of standard Enum implementation are declared as. That means that when you try to get a member’s(or) the flow is following:So, the complete flow could be represented as the following pseudo-code:We’ve made a simple script that demonstrates the above conclusion:And after we’ve run the script it created this picture for us:This proves that each time you access stdlib enum’s attributesandit calls a descriptor. That descriptor in turn ends up with a call to stdlib enum’sproperty decorated with the descriptor.Well, you can compare this to our FastEnum:Which outputs this picture:That is what is really done inside standard Enum implementation each time you accessandattributes of your Enum members. And that’s why our implementation is faster.When we tried to use standard enum in our projects we’ve noticed how many descriptor protocol calls forandattributes of themembers were invoked. And because enumerations were used excessively throughout the code, the resulting performance was poor.Furthermore, the standard enum class contains a couple of helper “protected” attributes:Dictionary lookups are slow since each one leads to a hash calculation and a hash table lookup, making those dictionaries non-optimal base structures for the enum class. Even the member retrieval itself (as in) is a dictionary lookup.While developing our Enum implementation, we kept in mind those pretty C-language enumerations and the beautiful extensible Java Enums. The main features we wanted in our implementation:The only time we use dictionary lookups is in a reverse mappingto Enum member. All other calculations are done just once during the class declaration (where metaclasses hooks used to customize type creation).In contrast to the standard library implementation, we treat the first value after thesign in the class declaration as the member value:in standard library enum the whole tupleis treated asin our implementation onlyis treated asFurther speed-up is gained by usingwhenever possible. In the Python data model classes declared withdo not haveattribute that holds instance attributes (so you can’t assign any attribute that is not mentioned in). Additionally, attributes defined inaccessed at constant offsets to the C-level object pointer. That is high-speed attribute access since it avoids hash calculations and hashtable scans.FastEnum is not compatible with any version of Python prior to 3.6, since it excessively usesmodule that was introduced in Python 3.6; One could assume that installing a backportedmodule from PyPI would help. The answer is: no. The implementation uses PEP-484 for some functions and methods arguments and return value type hinting, so any version prior to Python 3.5 is not supported due to syntax incompatibility. But then again, the very first line of code inof the metaclass uses PEP-526 syntax for variable type hinting. So Python 3.5 won’t do either. It’s possible to port the implementation to older versions, though we in Qrator Labs tend to use type hinting whenever possible since it helps developing complex projects greatly. And, hey! You don't want to stick to any python prior to 3.6 since there's no backwards incompatibilities with your existing code (assuming you are not using Python 2) though a lot of work was done in asyncio compared to 3.5.That, in turn, makes special imports likeunnecessary, unlike in standard library. You type-hint all your Enum members with your Enum class name, providing no value at all — and the value would be generated for you automatically. Though python 3.6 is sufficient to work with FastEnum, be warned that the standard dictionary order of declaration guarantee was introduced only in python 3.7. We don’t know any useful appliances where auto-generated value order is important (since we assume the value generated itself is not the value a programmer does care about). Nevertheless, consider yourself warned if you still stick with python 3.6;Those who need their enum start from 0 (zero) instead of default 1 can do this with a special enum declaration attribute, that attribute is «erased» from the resulting Enum class;There are some limitations though: all enum member names must be CAPITALIZED or they won’t be picked up by the metaclass and won’t be treated as enum members;However, you could declare a base class for your enums (keep in mind that base class can use enum metaclass itself, so you don't need to provide metaclass to all subclasses): you may define common logic (attributes and methods) in this class, but may not define enum members (so that class won't be «finalized»). You could then subclass that class in as many enum declarations as you want and that would provide you with all the common logic;Aliases. We’ll explain them in a separate topic (implemented in 1.2.5)Suppose you have code that uses:And that MyEnum is declared like this:Now, you decided to make some refactoring and want to move your enum into another package. You create something like this:Where MyMovedEnum is declared like this:Now. You are ready to begin the «deprecation» stage for all the code that uses your enums. You divert direct usages ofto use(the latter has all its members be proxied into). You state within your project docs thatis deprecated and will be removed from the code at some point in the future. For example, in the next release. Consider your code saves your objects with enum attributes using pickle. At this point, you usein your code, but internally all your enum members are still theinstances. Your next step would be to swap declarations ofandso thatwill now not be a subclass ofand declare all its members itself;, on the other hand, would not declare any members but become just an alias (subclass) ofAnd that concludes it. On the restart of your runtimes on unpickle stage all your enum values will be re-routed intoand become re-bound to that new class. The moment you are sure all your pickled objects have been un(re)pickled with this class organization structure, you are free to make a new release, where previously marked as deprecated yourcan be declared obsolete and obliterated from your codebase.We encourage you to try it! github.com/QratorLabs/fastenum pypi.org/project/fast-enum . All credits go to the FastEnum author santjagocorkez