Project description

################################################################################

REVISION DATE: 20081019



AUTHOR: kai zhu - kaizhu@hpc.usc.edu



HOMEPAGE: http://www-rcf.usc.edu/~kaizhu/work/py3to2



ABSTRACT:

py3to2 is a patch / module that backports py3k opcode instructions

to python 2.x (py2x). this allows py2x to run py3k code natively @ the

bytecode-level.



the motivation is that py3k lacks many useful 2.x modules. the lack

of 3rd-party modules in particular (eg scipy, PIL, ...) is a hindrance to py3k

development/testing. py3to2 addresses these issues by allowing py3k code to b

developed/integrated/run natively under the robust, software-rich, py2x

environment.



MECHANISM:

1. upon init, py3to2 starts up a py3k server w/ pipe io.



2. -> received via pipe io from py3to2, the py3k server 1st

natively compiles the src code into a py3k codeobj, then converts it

to py2x format w/ the addition of the backported opcodes.



3. <- the serialized py2x codeobj is piped back to py3to2,

which unserializes it & eval/exec it as normal.



in theory, this mechanism should transparently implement any bytecode-level py3k

language feature. performance and robustness should b minimally impacted, since

the compiled code is directly run as native 2.x++ bytecode.



BACKWARD COMPATIBILITY:

make test (patched python 2.6 on redhat opteron x86_64)

...

324 tests OK.

36 tests skipped:

test_aepack test_al test_applesingle test_bsddb185 test_bsddb3

test_cd test_cl test_codecmaps_cn test_codecmaps_hk

test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses

test_dl test_gdbm test_gl test_imageop test_imgfile test_kqueue

test_linuxaudiodev test_macos test_macostools test_normalization

test_ossaudiodev test_pep277 test_py3kwarn test_scriptpackages

test_socketserver test_startfile test_sunaudiodev test_timeout

test_urllib2net test_urllibnet test_winreg test_winsound

test_zipfile64

1 skip unexpected on linux2:

test_gdbm



################################################################################



PY3K FEATURES TESTED TO WORK:

pep3102 Keyword-Only Arguments

pep3104 Access to Names in Outer Scopes

pep3105 Make print a function

pep3107 Function Annotations

pep3111 Simple input built-in in Python 3000

pep3112 Bytes literals in Python 3000

pep3113 Removal of Tuple Parameter Unpacking

pep3114 Renaming iterator.next() to iterator.__next__()

pep3115 Metaclasses in Python 3000

pep3127 Integer Literal Support and Syntax

pep3129 Class Decorators

pep3132 Extended Iterable Unpacking

pep3135 New Super



PY3K FEATURES NOT WORKING:



pep3101 Advanced String Formatting

pep3106 Revamping dict.keys(), .values() & .items()

pep3108 Standard Library Reorganization

pep3109/3110 Exceptions in Python 3000

pep3116 New I/O

pep3118 Revising the buffer protocol

pep3119 Introducing Abstract Base Classes

pep3120 Using UTF-8 as the default source encoding

pep3121 Extension Module Initialization & Finalization

pep3123 Making PyObject_HEAD conform to standard C

pep3131 Supporting Non-ASCII Identifiers

pep3134 Exception Chaining and Embedded Tracebacks

pep3137 Immutable Bytes and Mutable Buffer

pep3141 A Type Hierarchy for Numbers

pep3138 String representation in Python 3000



REQUIREMENTS:

* linux / unix (windows has io problems w/ py3k server)

* python-2.6 src (to b patched)

* python-3.0rc1 executable (as py3k server)



py3to2 files: (download from http://www-rcf.usc.edu/~kaizhu/work/py3to2)

* ceval.c - patched file

* py3to2.py - python script



BUILD / INSTALL:

* get python-2.6 src. backup & replace python-2.6/Python/ceval.c

* build & install python-2.6

(eg "./configure --prefix=/installdir; make clean; make; make install")



* copy py3to2.py to <python-2.6>/installdir/.../site-packages/

* install vanilla python-3.0 & make sure $PATH env contains "python3.0" executable



USAGE: simply add the magic line "from __future__ import py3k_syntax" to ur script



EXAMPLE: pep3132 Extended Iterable Unpacking

################################################################################

# save to file pep3132.py

from __future__ import py3k_syntax

a, b, *c = 1, 2, 3, 4

print(a, b, c)

################################################################################



Python 2.6 (r26:66714, Oct 19 2008, 02:49:13)

[GCC 3.4.6 20060404 (Red Hat 3.4.6-10)] on linux2

Type "help", "copyright", "credits" or "license" for more information.

>>>

>>> import py3to2

created read/write pipes: (4, 5)

py3k server starting with pipes in/out/err: 7 5 -2

py3k server: Python 3.0rc1 (r30rc1:66499, Oct 15 2008, 15:43:09)

py3k server: [GCC 3.4.6 20060404 (Red Hat 3.4.6-10)] on linux2

py3k server: Type "help", "copyright", "credits" or "license" for more

information.

py3k server: >>> ''

>>>

>>> import pep3132

1 2 [3, 4]



################################################################################



CHANGELOG:

20081019

ported to python-2.6

consolidate & simplify patches to 1 file: ceval.c

created extension module builtins_py3k

revamped import hook again

removed unicode support & restrict source code to ascii-only

20080727

revampled import hook

20080911

consolidate patches to 2 files: bltinmodule.c & ceval.c

20080828

add kwonlyargcount 'attr' to codeobj

add __annotations__ & __kwdefaults__ attr to funcobj

add __pseudomethod__ feature to baseobj

20080819

pure python import hook - removed magic comment & use magic path instead

revamped str & bytes handling

revamped py3k .pyc file handling

20080802

pep3135 New Super

20080717

pep3107 Function Annotations

pep3120 Using UTF-8 as the default source encoding

pep3131 Supporting Non-ASCII Identifiers

20080713

import / reload works transparently on py3k scripts using a magic comment

added pep3102 Keyword-Only Arguments

20080709 added a py3k preparser

20080702

rewrote py3k server's pipe io. implemented partial bytearray & bytes class.

wrote a few simple tests

20080630

__build_class__ function to bltmodule.c. tested class decorators to b working.

################################################################################