What’s New In Python 3.3¶

This article explains the new features in Python 3.3, compared to 3.2. Python 3.3 was released on September 29, 2012. For full details, see the changelog.

See also PEP 398 - Python 3.3 Release Schedule

PEP 405: Virtual Environments¶ Virtual environments help create separate Python setups while sharing a system-wide base install, for ease of maintenance. Virtual environments have their own set of private site packages (i.e. locally-installed libraries), and are optionally segregated from the system-wide site packages. Their concept and implementation are inspired by the popular virtualenv third-party package, but benefit from tighter integration with the interpreter core. This PEP adds the venv module for programmatic access, and the pyvenv script for command-line access and administration. The Python interpreter checks for a pyvenv.cfg , file whose existence signals the base of a virtual environment’s directory tree. See also PEP 405 - Python Virtual Environments PEP written by Carl Meyer; implementation by Carl Meyer and Vinay Sajip

PEP 420: Implicit Namespace Packages¶ Native support for package directories that don’t require __init__.py marker files and can automatically span multiple path segments (inspired by various third party approaches to namespace packages, as described in PEP 420) See also PEP 420 - Implicit Namespace Packages PEP written by Eric V. Smith; implementation by Eric V. Smith and Barry Warsaw

PEP 3118: New memoryview implementation and buffer protocol documentation¶ The implementation of PEP 3118 has been significantly improved. The new memoryview implementation comprehensively fixes all ownership and lifetime issues of dynamically allocated fields in the Py_buffer struct that led to multiple crash reports. Additionally, several functions that crashed or returned incorrect results for non-contiguous or multi-dimensional input have been fixed. The memoryview object now has a PEP-3118 compliant getbufferproc() that checks the consumer’s request type. Many new features have been added, most of them work in full generality for non-contiguous arrays and arrays with suboffsets. The documentation has been updated, clearly spelling out responsibilities for both exporters and consumers. Buffer request flags are grouped into basic and compound flags. The memory layout of non-contiguous and multi-dimensional NumPy-style arrays is explained. Features¶ All native single character format specifiers in struct module syntax (optionally prefixed with ‘@’) are now supported.

With some restrictions, the cast() method allows changing of format and shape of C-contiguous arrays.

Multi-dimensional list representations are supported for any array type.

Multi-dimensional comparisons are supported for any array type.

One-dimensional memoryviews of hashable (read-only) types with formats B, b or c are now hashable. (Contributed by Antoine Pitrou in issue 13411)

Arbitrary slicing of any 1-D arrays type is supported. For example, it is now possible to reverse a memoryview in O(1) by using a negative step. API changes¶ The maximum number of dimensions is officially limited to 64.

The representation of empty shape, strides and suboffsets is now an empty tuple instead of None.

Accessing a memoryview element with format ‘B’ (unsigned bytes) now returns an integer (in accordance with the struct module syntax). For returning a bytes object the view must be cast to ‘c’ first.

memoryview comparisons now use the logical structure of the operands and compare all array elements by value. All format strings in struct module syntax are supported. Views with unrecognised format strings are still permitted, but will always compare as unequal, regardless of view contents.

For further changes see Build and C API Changes and Porting C code. (Contributed by Stefan Krah in issue 10181) See also PEP 3118 - Revising the Buffer Protocol

PEP 393: Flexible String Representation¶ The Unicode string type is changed to support multiple internal representations, depending on the character with the largest Unicode ordinal (1, 2, or 4 bytes) in the represented string. This allows a space-efficient representation in common cases, but gives access to full UCS-4 on all systems. For compatibility with existing APIs, several representations may exist in parallel; over time, this compatibility should be phased out. On the Python side, there should be no downside to this change. On the C API side, PEP 393 is fully backward compatible. The legacy API should remain available at least five years. Applications using the legacy API will not fully benefit of the memory reduction, or - worse - may use a bit more memory, because Python may have to maintain two versions of each string (in the legacy format and in the new efficient storage). Functionality¶ Changes introduced by PEP 393 are the following: Python now always supports the full range of Unicode codepoints, including non-BMP ones (i.e. from U+0000 to U+10FFFF ). The distinction between narrow and wide builds no longer exists and Python now behaves like a wide build, even under Windows.

to ). The distinction between narrow and wide builds no longer exists and Python now behaves like a wide build, even under Windows. With the death of narrow builds, the problems specific to narrow builds have also been fixed, for example: len() now always returns 1 for non-BMP characters, so len('\U0010FFFF') == 1 ; surrogate pairs are not recombined in string literals, so '\uDBFF\uDFFF' != '\U0010FFFF' ; indexing or slicing non-BMP characters returns the expected value, so '\U0010FFFF'[0] now returns '\U0010FFFF' and not '\uDBFF' ; all other functions in the standard library now correctly handle non-BMP codepoints.

The value of sys.maxunicode is now always 1114111 ( 0x10FFFF in hexadecimal). The PyUnicode_GetMax() function still returns either 0xFFFF or 0x10FFFF for backward compatibility, and it should not be used with the new Unicode API (see issue 13054).

is now always ( in hexadecimal). The function still returns either or for backward compatibility, and it should not be used with the new Unicode API (see issue 13054). The ./configure flag --with-wide-unicode has been removed. Performance and resource usage¶ The storage of Unicode strings now depends on the highest codepoint in the string: pure ASCII and Latin1 strings ( U+0000-U+00FF ) use 1 byte per codepoint;

) use 1 byte per codepoint; BMP strings ( U+0000-U+FFFF ) use 2 bytes per codepoint;

) use 2 bytes per codepoint; non-BMP strings ( U+10000-U+10FFFF ) use 4 bytes per codepoint. The net effect is that for most applications, memory usage of string storage should decrease significantly - especially compared to former wide unicode builds - as, in many cases, strings will be pure ASCII even in international contexts (because many strings store non-human language data, such as XML fragments, HTTP headers, JSON-encoded data, etc.). We also hope that it will, for the same reasons, increase CPU cache efficiency on non-trivial applications. The memory usage of Python 3.3 is two to three times smaller than Python 3.2, and a little bit better than Python 2.7, on a Django benchmark (see the PEP for details). See also PEP 393 - Flexible String Representation PEP written by Martin von Löwis; implementation by Torsten Becker and Martin von Löwis.

PEP 397: Python Launcher for Windows¶ The Python 3.3 Windows installer now includes a py launcher application that can be used to launch Python applications in a version independent fashion. This launcher is invoked implicitly when double-clicking *.py files. If only a single Python version is installed on the system, that version will be used to run the file. If multiple versions are installed, the most recent version is used by default, but this can be overridden by including a Unix-style “shebang line” in the Python script. The launcher can also be used explicitly from the command line as the py application. Running py follows the same version selection rules as implicitly launching scripts, but a more specific version can be selected by passing appropriate arguments (such as -3 to request Python 3 when Python 2 is also installed, or -2.6 to specifclly request an earlier Python version when a more recent version is installed). In addition to the launcher, the Windows installer now includes an option to add the newly installed Python to the system PATH (contributed by Brian Curtin in issue 3561). See also PEP 397 - Python Launcher for Windows PEP written by Mark Hammond and Martin v. Löwis; implementation by Vinay Sajip. Launcher documentation: Python Launcher for Windows Installer PATH modification: Finding the Python executable

PEP 3151: Reworking the OS and IO exception hierarchy¶ The hierarchy of exceptions raised by operating system errors is now both simplified and finer-grained. You don’t have to worry anymore about choosing the appropriate exception type between OSError , IOError , EnvironmentError , WindowsError , mmap.error , socket.error or select.error . All these exception types are now only one: OSError . The other names are kept as aliases for compatibility reasons. Also, it is now easier to catch a specific error condition. Instead of inspecting the errno attribute (or args[0] ) for a particular constant from the errno module, you can catch the adequate OSError subclass. The available subclasses are the following: BlockingIOError

ChildProcessError

ConnectionError

FileExistsError

FileNotFoundError

InterruptedError

IsADirectoryError

NotADirectoryError

PermissionError

ProcessLookupError

TimeoutError And the ConnectionError itself has finer-grained subclasses: BrokenPipeError

ConnectionAbortedError

ConnectionRefusedError

ConnectionResetError Thanks to the new exceptions, common usages of the errno can now be avoided. For example, the following code written for Python 3.2: from errno import ENOENT , EACCES , EPERM try : with open ( "document.txt" ) as f : content = f . read () except IOError as err : if err . errno == ENOENT : print ( "document.txt file is missing" ) elif err . errno in ( EACCES , EPERM ): print ( "You are not allowed to read document.txt" ) else : raise can now be written without the errno import and without manual inspection of exception attributes: try : with open ( "document.txt" ) as f : content = f . read () except FileNotFoundError : print ( "document.txt file is missing" ) except PermissionError : print ( "You are not allowed to read document.txt" ) See also PEP 3151 - Reworking the OS and IO Exception Hierarchy PEP written and implemented by Antoine Pitrou

PEP 380: Syntax for Delegating to a Subgenerator¶ PEP 380 adds the yield from expression, allowing a generator to delegate part of its operations to another generator. This allows a section of code containing yield to be factored out and placed in another generator. Additionally, the subgenerator is allowed to return with a value, and the value is made available to the delegating generator. While designed primarily for use in delegating to a subgenerator, the yield from expression actually allows delegation to arbitrary subiterators. For simple iterators, yield from iterable is essentially just a shortened form of for item in iterable: yield item : >>> def g ( x ): ... yield from range ( x , 0 , - 1 ) ... yield from range ( x ) ... >>> list ( g ( 5 )) [5, 4, 3, 2, 1, 0, 1, 2, 3, 4] However, unlike an ordinary loop, yield from allows subgenerators to receive sent and thrown values directly from the calling scope, and return a final value to the outer generator: >>> def accumulate (): ... tally = 0 ... while 1 : ... next = yield ... if next is None : ... return tally ... tally += next ... >>> def gather_tallies ( tallies ): ... while 1 : ... tally = yield from accumulate () ... tallies . append ( tally ) ... >>> tallies = [] >>> acc = gather_tallies ( tallies ) >>> next ( acc ) # Ensure the accumulator is ready to accept values >>> for i in range ( 4 ): ... acc . send ( i ) ... >>> acc . send ( None ) # Finish the first tally >>> for i in range ( 5 ): ... acc . send ( i ) ... >>> acc . send ( None ) # Finish the second tally >>> tallies [6, 10] The main principle driving this change is to allow even generators that are designed to be used with the send and throw methods to be split into multiple subgenerators as easily as a single large function can be split into multiple subfunctions. See also PEP 380 - Syntax for Delegating to a Subgenerator PEP written by Greg Ewing; implementation by Greg Ewing, integrated into 3.3 by Renaud Blanch, Ryan Kelly and Nick Coghlan; documentation by Zbigniew Jędrzejewski-Szmek and Nick Coghlan

PEP 409: Suppressing exception context¶ PEP 409 introduces new syntax that allows the display of the chained exception context to be disabled. This allows cleaner error messages in applications that convert between exception types: >>> class D : ... def __init__ ( self , extra ): ... self . _extra_attributes = extra ... def __getattr__ ( self , attr ): ... try : ... return self . _extra_attributes [ attr ] ... except KeyError : ... raise AttributeError ( attr ) from None ... >>> D ({}) . x Traceback (most recent call last): File "<stdin>" , line 1 , in <module> File "<stdin>" , line 8 , in __getattr__ AttributeError : x Without the from None suffix to suppress the cause, the original exception would be displayed by default: >>> class C : ... def __init__ ( self , extra ): ... self . _extra_attributes = extra ... def __getattr__ ( self , attr ): ... try : ... return self . _extra_attributes [ attr ] ... except KeyError : ... raise AttributeError ( attr ) ... >>> C ({}) . x Traceback (most recent call last): File "<stdin>" , line 6 , in __getattr__ KeyError : 'x' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "<stdin>" , line 1 , in <module> File "<stdin>" , line 8 , in __getattr__ AttributeError : x No debugging capability is lost, as the original exception context remains available if needed (for example, if an intervening library has incorrectly suppressed valuable underlying details): >>> try : ... D ({}) . x ... except AttributeError as exc : ... print ( repr ( exc . __context__ )) ... KeyError('x',) See also PEP 409 - Suppressing exception context PEP written by Ethan Furman; implemented by Ethan Furman and Nick Coghlan.

PEP 414: Explicit Unicode literals¶ To ease the transition from Python 2 for Unicode aware Python applications that make heavy use of Unicode literals, Python 3.3 once again supports the “ u ” prefix for string literals. This prefix has no semantic significance in Python 3, it is provided solely to reduce the number of purely mechanical changes in migrating to Python 3, making it easier for developers to focus on the more significant semantic changes (such as the stricter default separation of binary and text data). See also PEP 414 - Explicit Unicode literals PEP written by Armin Ronacher.

PEP 3155: Qualified name for classes and functions¶ Functions and class objects have a new __qualname__ attribute representing the “path” from the module top-level to their definition. For global functions and classes, this is the same as __name__ . For other functions and classes, it provides better information about where they were actually defined, and how they might be accessible from the global scope. Example with (non-bound) methods: >>> class C : ... def meth ( self ): ... pass >>> C . meth . __name__ 'meth' >>> C . meth . __qualname__ 'C.meth' Example with nested classes: >>> class C : ... class D : ... def meth ( self ): ... pass ... >>> C . D . __name__ 'D' >>> C . D . __qualname__ 'C.D' >>> C . D . meth . __name__ 'meth' >>> C . D . meth . __qualname__ 'C.D.meth' Example with nested functions: >>> def outer (): ... def inner (): ... pass ... return inner ... >>> outer () . __name__ 'inner' >>> outer () . __qualname__ 'outer.<locals>.inner' The string representation of those objects is also changed to include the new, more precise information: >>> str ( C . D ) "<class '__main__.C.D'>" >>> str ( C . D . meth ) '<function C.D.meth at 0x7f46b9fe31e0>' See also PEP 3155 - Qualified name for classes and functions PEP written and implemented by Antoine Pitrou.

PEP 412: Key-Sharing Dictionary¶ Dictionaries used for the storage of objects’ attributes are now able to share part of their internal storage between each other (namely, the part which stores the keys and their respective hashes). This reduces the memory consumption of programs creating many instances of non-builtin types. See also PEP 412 - Key-Sharing Dictionary PEP written and implemented by Mark Shannon.

PEP 362: Function Signature Object¶ A new function inspect.signature() makes introspection of python callables easy and straightforward. A broad range of callables is supported: python functions, decorated or not, classes, and functools.partial() objects. New classes inspect.Signature , inspect.Parameter and inspect.BoundArguments hold information about the call signatures, such as, annotations, default values, parameters kinds, and bound arguments, which considerably simplifies writing decorators and any code that validates or amends calling signatures or arguments. See also PEP 362: - Function Signature Object PEP written by Brett Cannon, Yury Selivanov, Larry Hastings, Jiwon Seo; implemented by Yury Selivanov.

PEP 421: Adding sys.implementation¶ A new attribute on the sys module exposes details specific to the implementation of the currently running interpreter. The initial set of attributes on sys.implementation are name , version , hexversion , and cache_tag . The intention of sys.implementation is to consolidate into one namespace the implementation-specific data used by the standard library. This allows different Python implementations to share a single standard library code base much more easily. In its initial state, sys.implementation holds only a small portion of the implementation-specific data. Over time that ratio will shift in order to make the standard library more portable. One example of improved standard library portability is cache_tag . As of Python 3.3, sys.implementation.cache_tag is used by importlib to support PEP 3147 compliance. Any Python implementation that uses importlib for its built-in import system may use cache_tag to control the caching behavior for modules. SimpleNamespace¶ The implementation of sys.implementation also introduces a new type to Python: types.SimpleNamespace . In contrast to a mapping-based namespace, like dict , SimpleNamespace is attribute-based, like object . However, unlike object , SimpleNamespace instances are writable. This means that you can add, remove, and modify the namespace through normal attribute access. See also PEP 421 - Adding sys.implementation PEP written and implemented by Eric Snow.

A Finer-Grained Import Lock¶ Previous versions of CPython have always relied on a global import lock. This led to unexpected annoyances, such as deadlocks when importing a module would trigger code execution in a different thread as a side-effect. Clumsy workarounds were sometimes employed, such as the PyImport_ImportModuleNoBlock() C API function. In Python 3.3, importing a module takes a per-module lock. This correctly serializes importation of a given module from multiple threads (preventing the exposure of incompletely initialized modules), while eliminating the aforementioned annoyances. (Contributed by Antoine Pitrou in issue 9260.)

Builtin functions and types¶ open() gets a new opener parameter: the underlying file descriptor for the file object is then obtained by calling opener with (file, flags). It can be used to use custom flags like os.O_CLOEXEC for example. The 'x' mode was added: open for exclusive creation, failing if the file already exists.

gets a new opener parameter: the underlying file descriptor for the file object is then obtained by calling opener with (file, flags). It can be used to use custom flags like for example. The mode was added: open for exclusive creation, failing if the file already exists. print() : added the flush keyword argument. If the flush keyword argument is true, the stream is forcibly flushed.

: added the flush keyword argument. If the flush keyword argument is true, the stream is forcibly flushed. hash() : hash randomization is enabled by default, see object.__hash__() and PYTHONHASHSEED .

: hash randomization is enabled by default, see and . The str type gets a new casefold() method: return a casefolded copy of the string, casefolded strings may be used for caseless matching. For example, 'ß'.casefold() returns 'ss' .

type gets a new method: return a casefolded copy of the string, casefolded strings may be used for caseless matching. For example, returns . The sequence documentation has been substantially rewritten to better explain the binary/text sequence distinction and to provide specific documentation sections for the individual builtin sequence types (issue 4966)

New Modules¶ faulthandler¶ This new debug module faulthandler contains functions to dump Python tracebacks explicitly, on a fault (a crash like a segmentation fault), after a timeout, or on a user signal. Call faulthandler.enable() to install fault handlers for the SIGSEGV , SIGFPE , SIGABRT , SIGBUS , and SIGILL signals. You can also enable them at startup by setting the PYTHONFAULTHANDLER environment variable or by using -X faulthandler command line option. Example of a segmentation fault on Linux: $ python -q -X faulthandler >>> import ctypes >>> ctypes.string_at(0) Fatal Python error: Segmentation fault Current thread 0x00007fb899f39700: File "/home/python/cpython/Lib/ctypes/__init__.py", line 486 in string_at File "<stdin>", line 1 in <module> Segmentation fault ipaddress¶ The new ipaddress module provides tools for creating and manipulating objects representing IPv4 and IPv6 addresses, networks and interfaces (i.e. an IP address associated with a specific IP subnet). (Contributed by Google and Peter Moody in PEP 3144) lzma¶ The newly-added lzma module provides data compression and decompression using the LZMA algorithm, including support for the .xz and .lzma file formats. (Contributed by Nadeem Vawda and Per Øyvind Karlsen in issue 6715)