PEP 362 -- Function Signature Object

PEP: 362 Title: Function Signature Object Author: Brett Cannon <brett at python.org>, Jiwon Seo <seojiwon at gmail.com>, Yury Selivanov <yury at edgedb.com>, Larry Hastings <larry at hastings.org> Status: Final Type: Standards Track Created: 21-Aug-2006 Python-Version: 3.3 Post-History: 04-Jun-2012 Resolution: https://mail.python.org/pipermail/python-dev/2012-June/120682.html

Abstract Python has always supported powerful introspection capabilities, including introspecting functions and methods (for the rest of this PEP, "function" refers to both functions and methods). By examining a function object you can fully reconstruct the function's signature. Unfortunately this information is stored in an inconvenient manner, and is spread across a half-dozen deeply nested attributes. This PEP proposes a new representation for function signatures. The new representation contains all necessary information about a function and its parameters, and makes introspection easy and straightforward. However, this object does not replace the existing function metadata, which is used by Python itself to execute those functions. The new metadata object is intended solely to make function introspection easier for Python programmers.

Signature Object A Signature object represents the call signature of a function and its return annotation. For each parameter accepted by the function it stores a Parameter object in its parameters collection. A Signature object has the following public attributes and methods: return_annotation : object The "return" annotation for the function. If the function has no "return" annotation, this attribute is set to Signature.empty .

parameters : OrderedDict An ordered mapping of parameters' names to the corresponding Parameter objects.

bind(*args, **kwargs) -> BoundArguments Creates a mapping from positional and keyword arguments to parameters. Raises a TypeError if the passed arguments do not match the signature.

bind_partial(*args, **kwargs) -> BoundArguments Works the same way as bind() , but allows the omission of some required arguments (mimics functools.partial behavior.) Raises a TypeError if the passed arguments do not match the signature.

replace(parameters=<optional>, *, return_annotation=<optional>) -> Signature Creates a new Signature instance based on the instance replace was invoked on. It is possible to pass different parameters and/or return_annotation to override the corresponding properties of the base signature. To remove return_annotation from the copied Signature , pass in Signature.empty . Note that the '=<optional>' notation, means that the argument is optional. This notation applies to the rest of this PEP. Signature objects are immutable. Use Signature.replace() to make a modified copy: >>> def foo() -> None: ... pass >>> sig = signature(foo) >>> new_sig = sig.replace(return_annotation="new return annotation") >>> new_sig is not sig True >>> new_sig.return_annotation != sig.return_annotation True >>> new_sig.parameters == sig.parameters True >>> new_sig = new_sig.replace(return_annotation=new_sig.empty) >>> new_sig.return_annotation is Signature.empty True There are two ways to instantiate a Signature class: Signature(parameters=<optional>, *, return_annotation=Signature.empty) Default Signature constructor. Accepts an optional sequence of Parameter objects, and an optional return_annotation . Parameters sequence is validated to check that there are no parameters with duplicate names, and that the parameters are in the right order, i.e. positional-only first, then positional-or-keyword, etc.

Signature.from_function(function) Returns a Signature object reflecting the signature of the function passed in. It's possible to test Signatures for equality. Two signatures are equal when their parameters are equal, their positional and positional-only parameters appear in the same order, and they have equal return annotations. Changes to the Signature object, or to any of its data members, do not affect the function itself. Signature also implements __str__ : >>> str(Signature.from_function((lambda *args: None))) '(*args)' >>> str(Signature()) '()'

Parameter Object Python's expressive syntax means functions can accept many different kinds of parameters with many subtle semantic differences. We propose a rich Parameter object designed to represent any possible function parameter. A Parameter object has the following public attributes and methods: name : str The name of the parameter as a string. Must be a valid python identifier name (with the exception of POSITIONAL_ONLY parameters, which can have it set to None .)

default : object The default value for the parameter. If the parameter has no default value, this attribute is set to Parameter.empty .

annotation : object The annotation for the parameter. If the parameter has no annotation, this attribute is set to Parameter.empty .

kind Describes how argument values are bound to the parameter. Possible values: Parameter.POSITIONAL_ONLY - value must be supplied as a positional argument. Python has no explicit syntax for defining positional-only parameters, but many built-in and extension module functions (especially those that accept only one or two parameters) accept them. Parameter.POSITIONAL_OR_KEYWORD - value may be supplied as either a keyword or positional argument (this is the standard binding behaviour for functions implemented in Python.) Parameter.KEYWORD_ONLY - value must be supplied as a keyword argument. Keyword only parameters are those which appear after a "*" or "*args" entry in a Python function definition. Parameter.VAR_POSITIONAL - a tuple of positional arguments that aren't bound to any other parameter. This corresponds to a "*args" parameter in a Python function definition. Parameter.VAR_KEYWORD - a dict of keyword arguments that aren't bound to any other parameter. This corresponds to a "**kwargs" parameter in a Python function definition. Always use Parameter.* constants for setting and checking value of the kind attribute.

replace(*, name=<optional>, kind=<optional>, default=<optional>, annotation=<optional>) -> Parameter Creates a new Parameter instance based on the instance replaced was invoked on. To override a Parameter attribute, pass the corresponding argument. To remove an attribute from a Parameter , pass Parameter.empty . Parameter constructor: Parameter(name, kind, *, annotation=Parameter.empty, default=Parameter.empty) Instantiates a Parameter object. name and kind are required, while annotation and default are optional. Two parameters are equal when they have equal names, kinds, defaults, and annotations. Parameter objects are immutable. Instead of modifying a Parameter object, you can use Parameter.replace() to create a modified copy like so: >>> param = Parameter('foo', Parameter.KEYWORD_ONLY, default=42) >>> str(param) 'foo=42' >>> str(param.replace()) 'foo=42' >>> str(param.replace(default=Parameter.empty, annotation='spam')) "foo:'spam'"

BoundArguments Object Result of a Signature.bind call. Holds the mapping of arguments to the function's parameters. Has the following public attributes: arguments : OrderedDict An ordered, mutable mapping of parameters' names to arguments' values. Contains only explicitly bound arguments. Arguments for which bind() relied on a default value are skipped.

args : tuple Tuple of positional arguments values. Dynamically computed from the 'arguments' attribute.

kwargs : dict Dict of keyword arguments values. Dynamically computed from the 'arguments' attribute. The arguments attribute should be used in conjunction with Signature.parameters for any arguments processing purposes. args and kwargs properties can be used to invoke functions: def test(a, *, b): ... sig = signature(test) ba = sig.bind(10, b=20) test(*ba.args, **ba.kwargs) Arguments which could be passed as part of either *args or **kwargs will be included only in the BoundArguments.args attribute. Consider the following example: def test(a=1, b=2, c=3): pass sig = signature(test) ba = sig.bind(a=10, c=13) >>> ba.args (10,) >>> ba.kwargs: {'c': 13}

Implementation The implementation adds a new function signature() to the inspect module. The function is the preferred way of getting a Signature for a callable object. The function implements the following algorithm: If the object is not callable - raise a TypeError

If the object has a __signature__ attribute and if it is not None - return it

attribute and if it is not - return it If it has a __wrapped__ attribute, return signature(object.__wrapped__)

attribute, return If the object is an instance of FunctionType , construct and return a new Signature for it

, construct and return a new for it If the object is a bound method, construct and return a new Signature object, with its first parameter (usually self or cls ) removed. ( classmethod and staticmethod are supported too. Since both are descriptors, the former returns a bound method, and the latter returns its wrapped function.)

object, with its first parameter (usually or ) removed. ( and are supported too. Since both are descriptors, the former returns a bound method, and the latter returns its wrapped function.) If the object is an instance of functools.partial , construct a new Signature from its partial.func attribute, and account for already bound partial.args and partial.kwargs

, construct a new from its attribute, and account for already bound and If the object is a class or metaclass: If the object's type has a __call__ method defined in its MRO, return a Signature for it If the object has a __new__ method defined in its MRO, return a Signature object for it If the object has a __init__ method defined in its MRO, return a Signature object for it

Return signature(object.__call__) Note that the Signature object is created in a lazy manner, and is not automatically cached. However, the user can manually cache a Signature by storing it in the __signature__ attribute. An implementation for Python 3.3 can be found at . The python issue tracking the patch is .

Design Considerations No implicit caching of Signature objects The first PEP design had a provision for implicit caching of Signature objects in the inspect.signature() function. However, this has the following downsides: If the Signature object is cached then any changes to the function it describes will not be reflected in it. However, If the caching is needed, it can be always done manually and explicitly

object is cached then any changes to the function it describes will not be reflected in it. However, If the caching is needed, it can be always done manually and explicitly It is better to reserve the __signature__ attribute for the cases when there is a need to explicitly set to a Signature object that is different from the actual one Some functions may not be introspectable Some functions may not be introspectable in certain implementations of Python. For example, in CPython, built-in functions defined in C provide no metadata about their arguments. Adding support for them is out of scope for this PEP. Signature and Parameter equivalence We assume that parameter names have semantic significance--two signatures are equal only when their corresponding parameters are equal and have the exact same names. Users who want looser equivalence tests, perhaps ignoring names of VAR_KEYWORD or VAR_POSITIONAL parameters, will need to implement those themselves.

Examples Visualizing Callable Objects' Signature Let's define some classes and functions: from inspect import signature from functools import partial, wraps class FooMeta(type): def __new__(mcls, name, bases, dct, *, bar:bool=False): return super().__new__(mcls, name, bases, dct) def __init__(cls, name, bases, dct, **kwargs): return super().__init__(name, bases, dct) class Foo(metaclass=FooMeta): def __init__(self, spam:int=42): self.spam = spam def __call__(self, a, b, *, c) -> tuple: return a, b, c @classmethod def spam(cls, a): return a def shared_vars(*shared_args): """Decorator factory that defines shared variables that are passed to every invocation of the function""" def decorator(f): @wraps(f) def wrapper(*args, **kwargs): full_args = shared_args + args return f(*full_args, **kwargs) # Override signature sig = signature(f) sig = sig.replace(tuple(sig.parameters.values())[1:]) wrapper.__signature__ = sig return wrapper return decorator @shared_vars({}) def example(_state, a, b, c): return _state, a, b, c def format_signature(obj): return str(signature(obj)) Now, in the python REPL: >>> format_signature(FooMeta) '(name, bases, dct, *, bar:bool=False)' >>> format_signature(Foo) '(spam:int=42)' >>> format_signature(Foo.__call__) '(self, a, b, *, c) -> tuple' >>> format_signature(Foo().__call__) '(a, b, *, c) -> tuple' >>> format_signature(Foo.spam) '(a)' >>> format_signature(partial(Foo().__call__, 1, c=3)) '(b, *, c=3) -> tuple' >>> format_signature(partial(partial(Foo().__call__, 1, c=3), 2, c=20)) '(*, c=20) -> tuple' >>> format_signature(example) '(a, b, c)' >>> format_signature(partial(example, 1, 2)) '(c)' >>> format_signature(partial(partial(example, 1, b=2), c=3)) '(b=2, c=3)' Annotation Checker import inspect import functools def checktypes(func): '''Decorator to verify arguments and return types Example: >>> @checktypes ... def test(a:int, b:str) -> int: ... return int(a * b) >>> test(10, '1') 1111111111 >>> test(10, 1) Traceback (most recent call last): ... ValueError: foo: wrong type of 'b' argument, 'str' expected, got 'int' ''' sig = inspect.signature(func) types = {} for param in sig.parameters.values(): # Iterate through function's parameters and build the list of # arguments types type_ = param.annotation if type_ is param.empty or not inspect.isclass(type_): # Missing annotation or not a type, skip it continue types[param.name] = type_ # If the argument has a type specified, let's check that its # default value (if present) conforms with the type. if param.default is not param.empty and not isinstance(param.default, type_): raise ValueError("{func}: wrong type of a default value for {arg!r}". \ format(func=func.__qualname__, arg=param.name)) def check_type(sig, arg_name, arg_type, arg_value): # Internal function that encapsulates arguments type checking if not isinstance(arg_value, arg_type): raise ValueError("{func}: wrong type of {arg!r} argument, " \ "{exp!r} expected, got {got!r}". \ format(func=func.__qualname__, arg=arg_name, exp=arg_type.__name__, got=type(arg_value).__name__)) @functools.wraps(func) def wrapper(*args, **kwargs): # Let's bind the arguments ba = sig.bind(*args, **kwargs) for arg_name, arg in ba.arguments.items(): # And iterate through the bound arguments try: type_ = types[arg_name] except KeyError: continue else: # OK, we have a type for the argument, lets get the corresponding # parameter description from the signature object param = sig.parameters[arg_name] if param.kind == param.VAR_POSITIONAL: # If this parameter is a variable-argument parameter, # then we need to check each of its values for value in arg: check_type(sig, arg_name, type_, value) elif param.kind == param.VAR_KEYWORD: # If this parameter is a variable-keyword-argument parameter: for subname, value in arg.items(): check_type(sig, arg_name + ':' + subname, type_, value) else: # And, finally, if this parameter a regular one: check_type(sig, arg_name, type_, arg) result = func(*ba.args, **ba.kwargs) # The last bit - let's check that the result is correct return_type = sig.return_annotation if (return_type is not sig._empty and isinstance(return_type, type) and not isinstance(result, return_type)): raise ValueError('{func}: wrong return type, {exp} expected, got {got}'. \ format(func=func.__qualname__, exp=return_type.__name__, got=type(result).__name__)) return result return wrapper

Acceptance PEP 362 was accepted by Guido, Friday, June 22, 2012 . The reference implementation was committed to trunk later that day.