Lonely Python is Intelligent Design and how to liberate from it

Python is doomed. Well, of course it is not yet doomed. Python is an evolutionary dead end living lonely on its own fitness plateau. It is clear that Python won’t have offspring because of significant whitespace aka indentation sensitivity. No children, no future.

You might ask where are the news? There are none and we knew this for ever and a decade but sometimes it is in the news again. Read this post by the author of a programming language called Reia. He struggles endlessly with indentation delimited blocks and finally abandons them for Ruby style `end` terminators ( not before a popularity poll in the Web 2.0 age and consulting the author of the failed Logix project who is now a Ruby advocate). Finally his language looks like Ruby which demonstrates the evolutionary fitness of Ruby. Python on the other hand is a piece of Intelligent Design. Guido van Rossum has managed a ” beautiful set of compromises” when he designed the Python grammar. So it ends up in overspecialization whereas languages like JavaScript are flowing more with the nature. To illustrate this nature you might examine the file jQuery.js that implements the popular jQuery library. The impatients might immediately scroll to the last line. It’s sweet.

Some people have argued that Haskell style layouts may save the soul of significant whitespace. Transplanted to Python this would imply mandatory explicit block delimiters that can also be omitted for the now classical indentation style. If you are not willing to decide just give multiple choice style freedom to anyone.

In the rest of the article I’m discussing optional explicit delimiters for Python built into the grammar and making the statement/expression distinction weightless. I promise not to cringe when it becomes “ugly”. Not behaving like little girls seems to be an evolutionary dead end among male, adult programmers as well but I take it with stoicism.

Grammar Extension

atom: ( '(' [embedded_stmt|yield_expr] ')' |'[' [listmaker] ']' |
        '{' [dictmaker] '}' |
         '`' testlist1 '`' |
         NAME |  NUMBER | STRING+)
trailer: '(' [arglist|embedded_suite] ')' | '[' subscriptlist ']' | '.' NAME
embedded_stmt: ( embedded_funcdef |
    embedded_if_stmt |
    embedded_while_stmt |
    embedded_for_stmt |
    embedded_try_stmt |
    embedded_with_stmt |
    embedded_classdef |
    embedded_small )
embedded_small: small_stmt (';' small_stmt)*
embedded_suite: embedded_stmt+
embedded_funcdef: [decorators] 'def' NAME parameters ':' embedded_suite
embedded_if_stmt: ( 'if' test ':' embedded_suite ('elif' test ':' embedded_suite)*
                   ['else' ':' embedded_suite])
embedded_while_stmt: 'while' test ':' embedded_suite ['else' ':' embedded_suite]
embedded_for_stmt: ( 'for' exprlist 'in' testlist ':' embedded_suite
                     ['else' ':' embedded_suite] )
embedded_try_stmt: ('try' ':' suite ((except_clause ':' embedded_suite)+
                    ['else' ':' embedded_suite] ['finally' ':' embedded_suite] |
                   'finally' ':' embedded_suite))
embedded_with_stmt: 'with' test [ with_var ] ':' embedded_suite
embedded_classdef: 'class' NAME ['(' [testlist] ')'] ':' embedded_suite

This is a conservative grammar extension of the official Python 2.5 Grammar. Conservative means that all expressions / statements written in Python 2.5 remain valid. Those non-terminals that substitute existing ones are `atom` and `trailer`. In detail the `testlist_gexp` non-terminal in `atom` is replaced by the newly defined `embedded_stmt` non-terminal and instead of an `arglist` an `embedded_suite`can be used . The `embedded_stmt` non-terminal is the gateway for all kinds of statements embedded into expressions. The statement-expression distinction becomes de facto obsolete.
What happens to significant whitespace? Post-processing the token stream filters whitespace inside of parentheses. Hence the token stream won’t contain `NEWLINE`, `INDENT`and `DEDENT` token within expressions. All those `embedded_xxx` non-terminals are only adjustments of existing ones reflecting the absence of the mentioned whitespace token in expressions. Notice that you can’t mix. Once you are living in a grammatical expression whitespace becomes definitely insignificant.

The grammar is not LL(1) and you need a stronger parser than Pythons own one to handle it. There is also no behavior associated with the new definitions. Semantics shall be entirely neglected and only the future shape of things shall be highlighted. Here are a few examples:

Multiline Lambda

lambda x,y: (
    z = x+y;              # semicolons always separate small statements
    print z;
    return z
)(3, 2)

Long Assertions

assert ( if x>0:
    x == 2
        x == -2)

The Return of the Return

def foo(x):
    return ( return )

My Property

x = property(
    def fget(self):
        return self._x
    def fset(self, value):
        self._x = value

Function in the middle

x + (def fact(n):
     if n<=1:
          return 1
          return fact(n-1))(7) + z
This entry was posted in DSL, Grammars, Python. Bookmark the permalink.

2 Responses to Lonely Python is Intelligent Design and how to liberate from it

  1. tav says:

    Nice idea Kay, but I fear that the Python-dev community will not be too keen on ideas like this — even if it would ensure the longevity of the language =(

    After spending quite some effort — you’ve already seen the Ruby block one — there’s also this: http://tav.espians.com/paving-the-way-to-securing-the-python-interpreter.html — I’m tending towards abandoning improving Python.

    Instead my thoughts are to create a whole new Python-esque language =)

    Would you be interested in collaborating on such an effort?

    — All the best, tav@espians.com

  2. kay says:

    Hi Tav, I won’t move into the effort of creating a new runtime. This is a no-go area for me. I’m also not much inclined in participating on PyPy or similar projects.

    Instead I’m shaping and re-shaping EasyExtend which exists on a little higher level than creating a language frontend – like Lex/Yacc or ANTLR but integrated with Python and following a very beautiful and almost axiomatic method that still has to be fleshed out and understood in its implications.

    I’m right now somewhere in the middle between EE 3 and EE 4 and will blog about my findings occasionally. As I see it EE is deeper and more radical than programming language frontend design but of course closely connected to it. One can breed languages and sub-projects like rabbits but the day has 24 hours only…

    So it’s not really resistance when I say participation in creating Python dialects lets me cold because I’m doing it anyway when being “inside the system”. Just like writing Lisp macros is a regular activity for anyone dealing with Lisp.


Leave a Reply

Your email address will not be published. Required fields are marked *