{"id":1040,"date":"2009-07-24T17:15:41","date_gmt":"2009-07-24T16:15:41","guid":{"rendered":"http:\/\/fiber-space.de\/wordpress\/?p=1040"},"modified":"2009-07-24T17:22:06","modified_gmt":"2009-07-24T16:22:06","slug":"redesign-of-the-codepy-and-codeoppy-modules","status":"publish","type":"post","link":"http:\/\/fiber-space.de\/wordpress\/2009\/07\/24\/redesign-of-the-codepy-and-codeoppy-modules\/","title":{"rendered":"Redesign of the code.py and codeop.py modules"},"content":{"rendered":"<p>Brett Cannon <a href=\"http:\/\/sayspy.blogspot.com\/2009\/07\/informal-poll-what-python-stdlib.html\">asks<\/a> for modules of the stdlib to be redesigned. I find the idea rather bizarre to initiate a poll for this but maybe that&#8217;s just the future of programming where the quality of an implementation is judged by democratic voting. So I immersed into the hive mind and voted for <strong>distutils<\/strong>. Seems like Tarek Ziade addresses this already but I&#8217;m not entirely sure he goes far enough. Last time I looked at the source code there were still all kinds of compiler modules in the lib which contain config information closely coupled with application code. That&#8217;s not so nice and mostly a refactoring bit.<\/p>\n<p>Some other of the stdlib modules I&#8217;d rewrite are not mentioned in the voting list. Maybe they are not sexy enough for the majority of web programmers that dominate all the discussions about Python? Among my favorites are `code.py` and `codeop.py`. Here is a brief but incomplete list of requirements and refactorings.<\/p>\n<ul>\n<li>The heuristics used to determine incomplete Python commands in <em>_maybe_compile<\/em> is pretty weak.<\/li>\n<li>Can you tell the difference between <em>Compile<\/em>, <em>CommandCompiler<\/em> and <em>compile_command<\/em> in <em>codeop.py<\/em>?<\/li>\n<li>Encapsulate  the <em>raw_input<\/em> function in <em>interact<\/em> within a method that can be overwritten.<\/li>\n<li>provide two methods <em>at_start<\/em> and <em>at_exit<\/em> in <em>InteractiveConsole<\/em> to make startup and shutdown customizable.<\/li>\n<li>Separate interactive loop from line processing and implement the line processor as a generator. It&#8217;s easier to write custom interactive loops for systems that interface with Python. The default <em>interact<\/em> method becomes<\/li>\n<\/ul>\n<pre lang=\"python\">    def interact(self):\r\n        self.at_start()\r\n        try:\r\n            gen_process = self.process_line()\r\n            line = None\r\n            while True:\r\n                try:\r\n                    prompt = gen_process.send(line)\r\n                    line   = self.user.get_input(prompt)\r\n                except StopIteration:\r\n                    break\r\n        finally:\r\n            self.at_exit()<\/pre>\n<ul>\n<li>Move the the line terminating heuristics from <em>_maybe_compile<\/em> into <em>process_line<\/em> and define a <em>try_parse<\/em> function together with a <em>try_compile<\/em> function. I&#8217;d go a little further even and define a <em>try_tokenize<\/em> function which isn&#8217;t essential though.<\/li>\n<li>Provide a subclass for interactive sessions which can be recorded and replayed and command line options accordingly. This is optional though and not part of a redesign strategy.<\/li>\n<\/ul>\n<p>There are other modules I&#8217;d like to rewrite such as `tokenizer.py`. Having a lexer in the stdlib which handles Python as a special case would be quite a big deal IMO. But it&#8217;s delicate and I struggle with writing lexers which can be both extended in a simple way ( without the trouble of running into ordered choice problems of the current regular expression engine ) and have a high performance. So far I only accomplished the first of the goals, at least partially, but not the second one.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Brett Cannon asks for modules of the stdlib to be redesigned. I find the idea rather bizarre to initiate a poll for this but maybe that&#8217;s just the future of programming where the quality of an implementation is judged by &hellip; <a href=\"http:\/\/fiber-space.de\/wordpress\/2009\/07\/24\/redesign-of-the-codepy-and-codeoppy-modules\/\">Continue reading <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[4],"tags":[],"_links":{"self":[{"href":"http:\/\/fiber-space.de\/wordpress\/wp-json\/wp\/v2\/posts\/1040"}],"collection":[{"href":"http:\/\/fiber-space.de\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/fiber-space.de\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/fiber-space.de\/wordpress\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/fiber-space.de\/wordpress\/wp-json\/wp\/v2\/comments?post=1040"}],"version-history":[{"count":6,"href":"http:\/\/fiber-space.de\/wordpress\/wp-json\/wp\/v2\/posts\/1040\/revisions"}],"predecessor-version":[{"id":1046,"href":"http:\/\/fiber-space.de\/wordpress\/wp-json\/wp\/v2\/posts\/1040\/revisions\/1046"}],"wp:attachment":[{"href":"http:\/\/fiber-space.de\/wordpress\/wp-json\/wp\/v2\/media?parent=1040"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/fiber-space.de\/wordpress\/wp-json\/wp\/v2\/categories?post=1040"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/fiber-space.de\/wordpress\/wp-json\/wp\/v2\/tags?post=1040"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}