[Python-Dev] Status of PEP 484 and the typing module

Hi Mark, We're down to the last few items here. I'm CC'ing python-dev so folks can see how close we are. I'll answer point by point. On Thu, May 21, 2015 at 6:24 AM, Mark Shannon <mark at hotpy.org> wrote: > Hi, > > The PEP itself is looking fairly good. > I hope you'll accept it at least provisionally so we can iterate over the finer points while a prototype of typing.py in in beta 1. > However, I don't think that typing.py is ready yet, for a number of > reasons: > > 1. > As I've said before, there needs to be a distinction between classes and > types. > They is no need for Any, Generic, Generic's subtypes, or Union to subclass > builtins.type. > I strongly disagree. They can appear in many positions where real classes are acceptable, in particular annotations can have classes (e.g. int) or types (e.g. Union[int, str]). > Playing around with typing.py, it has also become clear to me that it > is also important to distinguish type constructors from types. > > What do I mean by a type constructor? > A type constructor makes types. > "List" is an example of a type constructor. It constructs types such as > List[T] and List[int]. > Saying that something is a List (as opposed to a list) should be rejected. > The PEP actually says that plain List (etc.) is equivalent to List[Any]. (Well, at least that's the intention; it's implied by the section about the equivalence between Node() and Node[Any](). > 2. > Usability of typing as it stands: > > Let's try to make a class that implements a mutable mapping. > > >>> import typing as tp > #Make some variables. > >>> T = tp.TypeVar('T') > >>> K = tp.TypeVar('K') > >>> V = tp.TypeVar('V') > > #Then make our class: > > >>> class MM(tp.MutableMapping): pass > ... > #Oh that worked, but it shouldn't. MutableMapping is a type constructor. > It means MutableMapping[Any]. > #Let's make one > >>> MM() > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > File "/home/mark/repositories/typehinting/prototyping/typing.py", line > 1095, in __new__ > if _gorg(c) is Generic: > File "/home/mark/repositories/typehinting/prototyping/typing.py", line > 887, in _gorg > while a.__origin__ is not None: > AttributeError: type object 'Sized' has no attribute '__origin__' > > # ??? > Sorry, that's a bug I introduced in literally the last change to typing.py. I will fix it. The expected behavior is TypeError: Can't instantiate abstract class MM with abstract methods __len__ > #Well let's try using type variables. > class MM2(tp.MutableMapping[K, V]): pass > ... > >>> MM2() > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > File "/home/mark/repositories/typehinting/prototyping/typing.py", line > 1095, in __new__ > if _gorg(c) is Generic: > File "/home/mark/repositories/typehinting/prototyping/typing.py", line > 887, in _gorg > while a.__origin__ is not None: > AttributeError: type object 'Sized' has no attribute '__origin__' > Ditto, and sorry. > > At this point, we have to resort to using 'Dict', which forces us to > subclass 'dict' which may not be what we want as it may cause metaclass > conflicts. > > 3. > Memory consumption is also a worry. There is no caching, which means every > time I use "List[int]" as an annotation, a new class object is created. > Each class may only be a few KB, but collectively this could easily add up > to several MBs. > This should be easy to fix. > I can work on this after the beta-1 release. Until then, type aliases can be used to avoid redundant type creation (and often they are clearer anyway :-). > 4. > PY2, etc. really need to go. > Assuming that this code type checks OK: > > if typing.PY2: > type_safe_under_py2_only() > else: > type_safe_under_py3_only() > > Is the checker supposed to pass this: > > if sys.hexversion < 0x03000000: > type_safe_under_py2_only() > else: > type_safe_under_py3_only() > > If it should pass, then why have PY2, etc. at all. > If it should fail, well that is just stupid and annoying. > > Pylint already understands version checks, as does our (Semmle's) checker. > I suspect most IDEs do as well. > I have to negotiate this with Jukka but I think he'll agree. > 5. > Removing isinstance() support: > > As I said before, this is the job of a checker not typing.py. > > It also introduces some strange situations: > D = tp.Dict[str,int] > d = {} > assert isinstance(d, D) > d["x"] = None > assert isinstance(d, D) > > In the above case the first check passes, and the second fails. > But d is either of type D or it isn't. It can't be both, as types > are static properties of programs, unlike classes. > Well, isinstance() is a dynamic function. The type checker has no authority over its behavior beyond its signature. > And it's broken anyway: > >>> D = tp.Dict[str,'D'] > >>> d = {"x": {}} > >>> isinstance(d, D) > False > That's because _ForwardRef doesn't implement __instancheck__ or __subclasscheck__. It's easily fixed. > > Realistically, I don't see typing.py being ready in time for 3.5. > I'd be happy to be proved wrong. > > Cheers, > Mark. > > > P.S. > I am worried by the lack of formal specification. It all seems a bit > hand-waving. A formal spec reduces the likelihood of some unforeseen corner > case being a permanent wart. > Formal specs are not my cup of tea. :-( (I'm not proud of this, but it just is a fact -- see how terrible a job I've done of the Python reference manual.) The best I could come up with is PEP 483. > Take the recursive type above. There is no mention of recursive types in > the PEP and they are clearly possible. Are they allowed? > They should be allowed. I imagine you could create one for which a naive isinstance() imeplementation ends up in an infinite loop. That can be fixed too (we fixed this for printing self-referential lists and dicts). > I'm guessing that Jukka's thesis should cover a lot of this. > Has it been published yet? > Hopefully Jukka can answer that. :-) -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/python-dev/attachments/20150521/409b5458/attachment.html>