Lonely Python is Intelligent Design and how to liberate from it

Python is doomed. Well, of course it is not yet doomed. Python is an evolutionary dead end living lonely on its own fitness plateau. It is clear that Python won’t have offspring because of significant whitespace aka indentation sensitivity. No children, no future.

You might ask where are the news? There are none and we knew this for ever and a decade but sometimes it is in the news again. Read this post by the author of a programming language called Reia. He struggles endlessly with indentation delimited blocks and finally abandons them for Ruby style `end` terminators ( not before a popularity poll in the Web 2.0 age and consulting the author of the failed Logix project who is now a Ruby advocate). Finally his language looks like Ruby which demonstrates the evolutionary fitness of Ruby. Python on the other hand is a piece of Intelligent Design. Guido van Rossum has managed a ” beautiful set of compromises” when he designed the Python grammar. So it ends up in overspecialization whereas languages like JavaScript are flowing more with the nature. To illustrate this nature you might examine the file jQuery.js that implements the popular jQuery library. The impatients might immediately scroll to the last line. It’s sweet.

Some people have argued that Haskell style layouts may save the soul of significant whitespace. Transplanted to Python this would imply mandatory explicit block delimiters that can also be omitted for the now classical indentation style. If you are not willing to decide just give multiple choice style freedom to anyone.

In the rest of the article I’m discussing optional explicit delimiters for Python built into the grammar and making the statement/expression distinction weightless. I promise not to cringe when it becomes “ugly”. Not behaving like little girls seems to be an evolutionary dead end among male, adult programmers as well but I take it with stoicism.

Grammar Extension

atom: ( '(' [embedded_stmt|yield_expr] ')' |'[' [listmaker] ']' |
        '{' [dictmaker] '}' |
         '`' testlist1 '`' |
         NAME |  NUMBER | STRING+)
trailer: '(' [arglist|embedded_suite] ')' | '[' subscriptlist ']' | '.' NAME
 
embedded_stmt: ( embedded_funcdef |
    embedded_if_stmt |
    embedded_while_stmt |
    embedded_for_stmt |
    embedded_try_stmt |
    embedded_with_stmt |
    embedded_classdef |
    embedded_small )
 
embedded_small: small_stmt (';' small_stmt)*
embedded_suite: embedded_stmt+
embedded_funcdef: [decorators] 'def' NAME parameters ':' embedded_suite
embedded_if_stmt: ( 'if' test ':' embedded_suite ('elif' test ':' embedded_suite)*
                   ['else' ':' embedded_suite])
embedded_while_stmt: 'while' test ':' embedded_suite ['else' ':' embedded_suite]
embedded_for_stmt: ( 'for' exprlist 'in' testlist ':' embedded_suite
                     ['else' ':' embedded_suite] )
embedded_try_stmt: ('try' ':' suite ((except_clause ':' embedded_suite)+
                    ['else' ':' embedded_suite] ['finally' ':' embedded_suite] |
                   'finally' ':' embedded_suite))
embedded_with_stmt: 'with' test [ with_var ] ':' embedded_suite
embedded_classdef: 'class' NAME ['(' [testlist] ')'] ':' embedded_suite

This is a conservative grammar extension of the official Python 2.5 Grammar. Conservative means that all expressions / statements written in Python 2.5 remain valid. Those non-terminals that substitute existing ones are `atom` and `trailer`. In detail the `testlist_gexp` non-terminal in `atom` is replaced by the newly defined `embedded_stmt` non-terminal and instead of an `arglist` an `embedded_suite`can be used . The `embedded_stmt` non-terminal is the gateway for all kinds of statements embedded into expressions. The statement-expression distinction becomes de facto obsolete.
What happens to significant whitespace? Post-processing the token stream filters whitespace inside of parentheses. Hence the token stream won’t contain `NEWLINE`, `INDENT`and `DEDENT` token within expressions. All those `embedded_xxx` non-terminals are only adjustments of existing ones reflecting the absence of the mentioned whitespace token in expressions. Notice that you can’t mix. Once you are living in a grammatical expression whitespace becomes definitely insignificant.

The grammar is not LL(1) and you need a stronger parser than Pythons own one to handle it. There is also no behavior associated with the new definitions. Semantics shall be entirely neglected and only the future shape of things shall be highlighted. Here are a few examples:

Multiline Lambda

lambda x,y: (
    z = x+y;              # semicolons always separate small statements
    print z;
    return z
)(3, 2)

Long Assertions

assert ( if x>0:
    x == 2
    else:
        x == -2)

The Return of the Return

def foo(x):
    return ( return )

My Property

x = property(
    def fget(self):
        return self._x
    def fset(self, value):
        self._x = value
)

Function in the middle

x + (def fact(n):
     if n<=1:
          return 1
     else:
          return fact(n-1))(7) + z