Py25Lite

Easy Extend


	Author: Kay Schluehr LastModified: 17.10.2006 Version: 0.5 - requires Python 2.4 and EasyExtend v 2.0

Introduction

You might think this is a hoax or a parody but unfortunetaly I'm deadly serious about this fiber and the whole idea is just too simple: most new features of Python25 can be backported to a Python24 easily by means of a fiber extension. This work was inspired by the PEP 343 description of Pythons with-statement in terms of a protocoll defined in pure Python. This fiber can be used by all those who won't or can't upgrade to Python 2.5 but want to use Python 2.5 features such as (accessible ) conditional expressions or enhanced generators. The Py25Lite fiber is an example for upwards compatible programming and opens a challenge to rectify large parts of Python to a simpler version of itself. Other than an axiomatically founded kernel language approach using rectification might not lead to a unique result. Moreover it can change over time as new non-trivial features are added. So a simpler version of the language itself does not mean that the final version has any historical meaning. In case of Py25Lite this meaning is of course intended.

1. conditional expression

Translating expressions of the kind A if B else C.

2. try...except...finally statement

Unification of try...except and try...finally.

3. Enhanced Generators

Yield expressions and coroutines.

4. with statement

Translating the with-statement.

Used Py25Lite Grammar for Pytthon 2.4

1. conditional expression

Before Python25 writing conditional expressions required some tricky use of boolean operators and and or. Given two expressions E1, E2 the following rules applied:

if
E1:

   assert E2 == (E1 and E2)

else:

   assert E1 == (E1 and E2)

      if E1:

   assert E1 == (E1 or E2)

else:

   assert E2 == (E1 or E2)

Combining these rules we get:

if
E1:

   if E2:

       assert E2 == E1 and E2 or E3

   else:

       assert E3 == E1
and E2 or E3 

      else:

   assert E3 == E1 and E2 or E3

For getting always (E2 if E1) we slightly modify our statement:

if
E1:

   assert E2 == (E1 and (E2,True) or
(E3,True))[0]

      else:

   assert E3 == (E1 and (E2,True) or
(E3,True))[0]

Finally we translate

E1 if E2 else E3

==>

(E2 and (E1,True) or (E3,True))[0]

2. Try ... Finally

The try...finally clause is important for the following implementations of the generator enhancements and the with-statement. Until Python 2.5 it has not been possibl to write a try...except...finally statement using both an except and a finally clause. This restriction is dropped now.

Lets first restate the semantics of the try...finally clause from Pythons tutorial:

A finally clause is always executed before leaving the try statement, whether an exception has occurred or not. When an exception has occurred in the try clause and has not been handled by an except clause (or it has occurred in a except or else clause), it is re-raised after the finally clause has been executed. The finally clause is also executed ``on the way out'' when any other clause of the try statement is left via a break, continue or return statement.

We need to take great care for the requirement of correct treatment of flow-statements such as return, continue and break. We use following rules for a try...except...else...finally block:

1) Call finally block before each return statement that is not defined within a closure.
2) Inspect the toplevel statements of try...except...else suites and call finally block before each break and continue statement. Additionally inspect all other toplevel statements for containig a break or continue that is not defined within a while or for loop and not within a closure or class.
3) Call a finally block at the end of each try...except...else block.

We take additional care for uncaught exceptions raised in the finally block itself since we may not call this same block for cleanup again. Our general model for a finally block is:

FINALLY_BLOCK

==>

try:
    FINALLY_BLOCK
except Exception:   # uncaught exception in FINALLY_BLOCK
    raise FinallyException(sys.exc_info())

More generally:

try:
    TRY_BLOCK
except SomeException:
    EXCEPT_BLOCK
finally:
    FINALLY_BLOCK

==>

try:
    try:
        TRY_BLOCK
        FINALLY_BLOCK
    except SomeException:
        EXCEPT_BLOCK
        FINALLY_BLOCK
except Exception:
    exc_cls, exc_value, exc_tb = sys.exc_info()
    # stems from Exception clause of FINALLY_BLOCK
    if exc_cls == FinallyException:
        fin_exc_cls, fin_exc_value, fin_exc_tb = exc_value
        raise fin_exc_cls, fin_exc_value, fin_exc_tb
    else:
        FINALLY_BLOCK
    raise exc_cls, exc_instance, exc_tb

If an additional else-clause is present no FINALLY_BLOCK will be generated within the inner the try-clause but in the else clause instead.

According to the rules stated above this translation scheme may be modified according to the structure of the TRY_BLOCK or EXCEPT_BLOCK. For example:

for i in range(10):
   try:
       break
   finally:
       print "Hello"

==>

for i in range(10):
    try:
        FINALLY_BLOCK
        break
    except Exception:
        ...

3. Enhanced Generators

It suffices to just read the introductory words of What's new in Python 2.5 to get the basic requirements:

Python 2.5 adds a simple way to pass values into a generator. As introduced in Python 2.3, generators only produce output; once a generator's code was
invoked to create an iterator, there was no way to pass any new information into the function when its execution is resumed. Sometimes the ability to pass in some information would be useful.

and it closes with a nice guideline:

Hackish solutions to this include making the generator's code look at a global variable and then changing the global variable's value, or passing in some mutable object that callers then modify.

BTW there is nothing hackish by some kind of continuation-passing-style where the calling context is passed into the called function.

Following the guideline we have to pass some variable into a generator when it gets initialized. This variable is shared between the generator and its caller.

def gen():
   a = 0
   for i in range(10):
   a = yield i+a

==>

def gen(**kwd):            # - modified signature if kwds are not already declared
    gvar = kwd["gvar"]     # - read gvar from kwd
    a = 0
    for i in range(10):
      yield i+a          # - break down yield expression into a yield statement and
      a = gvar["value"] #   a variable assignment

Now we add some framework code used to pass gvar automatically and trigger send() events:

class EnhancedGenerator(object):
    def __init__(self, gen):
        self.gen_func = gen

    def __call__(self, *args, **kwd):
        self.gvar = {"value":None, "throw":None, "close":None}
        kwd["gvar"] = self.gvar
        self.gen = self.gen_func(*args, **kwd)
        return self

    def next(self):
        return self.gen.next()

    def send(self, value):
self.gvar["value"] = value
return self.next()

def enhance_generator(func):
    eg = EnhancedGenerator(func)
    fn = lambda *args, **kwd: eg(*args, **kwd)
    fn.__name__ = func.__name__
    return fn

Now we can put the pieces together:

def gen():
   a = 0
   for i in range(10):
   a = yield i+a

==>

@enhanced_generator
def gen(**kwd):            # - modified signature if kwds are not already declared
    gvar = kwd["gvar"]     # - read gvar from kwd
    a = 0
    for i in range(10):
      yield i+a          # - break down yield expression into a yield statement and
      a = gvar["value"] #   a variable assignment

Now we add some framework code used to pass gvar automatically and trigger send() events: All aformentioned code transformations can be performed on CST level. In either case we just have to inspect functions for a yield expression, decompose the yield expression, add the throw/close clauses, making a lookup on gvar when the generator is started and add the enhanced_generator decorator.

3.1 Unimplemented features

Py25Lite does not support expressions containing multiple yield expressions such as

(yield x) + (yield y)

f((yield x), (yield y))

etc.

The reason for this restriction is the complexity of the transformations in these cases and the low urgency of provisioning.

4. With-statement

The most obvious translation is that of the with-statement which is defined in PEP 343 in terms of a Python code template.

with <EXPR> as <VAR>:
    <BLOCK>

===>

                                                                       mgr   = (<EXPR>)
exit = mgr.__exit__     # Not calling it yet
value = mgr.__enter__()
exc = True
try:
    try:
        <VAR> = value    # Only if "as VAR" is present
        <BLOCK>
    except:
        # The exceptional case is handled here
        exc = False
        if not exit(*sys.exc_info()):
            raise
        # The exception is swallowed if exit() returns true
finally:
    # The normal and non-local-goto cases are handled here
    if exc:
        exit(None, None, None)
    SUITE1
else:
    SUITE2

The right hand side translation block can be passed unchanged into a macro transformer. Additionally we have to

d = {2:5, 3:9}
x = 0
repeat:
    on m = d.get(x):
        x = m
        break
    x+=1
until: x == 3

assert x == 5