Jython 2.5rc4 (Release_2_5rc4:6470, Jun 8 2009, 13:23:16) [Java HotSpot(TM) Client VM (Sun Microsystems Inc.)] on java1.6.0_13 Type "help", "copyright", "credits" or "license" for more information. >>> L = range(10000) >>> eval(str(L)) Traceback (most recent call last): File "", line 1, in java.lang.ClassFormatError: Invalid method Code length 99894 in class file org/python/pycode/_pyx9 at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(Unknown Source) at org.python.core.BytecodeLoader$Loader.loadClassFromBytes(BytecodeLoader.java:116) at org.python.core.BytecodeLoader.makeClass(BytecodeLoader.java:35) at org.python.core.BytecodeLoader.makeCode(BytecodeLoader.java:65) ... at org.python.util.InteractiveConsole.interact(InteractiveConsole.java:90) at org.python.util.jython.run(jython.java:293) at org.python.util.jython.main(jython.java:113) java.lang.ClassFormatError: java.lang.ClassFormatError: Invalid method Code length 99894 in class file org/python/pycode/_pyx9 >>> |
64K ought to be enough for anybody.
irb(main):001:0> L = (1...10000).to_a ;nil => nil irb(main):002:0> SL = eval "["+L.join(",")+"]"; nil => nil irb(main):003:0> SL[10..15] => [11, 12, 13, 14, 15, 16] irb(main):004:0> |
Just to get this straight. It is argued for almost decade that Python cannot have some advanced features ( the whole FP/Stackless story ) because this would break Jython and Jython is implemented s.t. `eval` is broken and Jython users cannot even import Python data-structures like generated dicts or lists that exceed 64KB. What about backporting this behavior to CPython in order to preserve conformance between implementations?
Note that the Ruby example is JRuby – not that anyone blames the JVM. Ruby is morally inferior though and wildly coerces types on its own behalf but this is another story.
Excellent post. You’ve echoed some of my own thoughts.
I’ve started using Stackless for some serious work in the last few months. I shied away from it for a long time because I thought that if “Guido rejected it, it can’t be good.”.
Big mistake. Stackless is an outstanding piece of software, and works with all my Python modules, including binary modules written in C. Stackless is now my secret weapon, and from what I can see, it was rejected and stayed rejected from CPython mostly for political reasons. Stupid Stupid Stupid.
Why hold back state of the art features and cripple innovation to allow inferior runtimes to keep up with good features? Stackless provides things that .NET and JVM will probably never be able to do. Your post highlights the stupidity of this decision. In my experience, Stackless can run any Python code you throw at it, it’s completely compatible.
Take a look at all the concurrency frameworks popping up to try and emulate Stackless in regular CPython. Kaemlia, Fibra, Concurrence, Cogen. These approaches are vastly inferior (which is a hard thing for me to say, as I wrote Fibra) to what Stackless provides. See these benchmarks for proof.
http://entitycrisis.blogspot.com/2009/03/concurrent-scaling-benchmarks.html
i’d never thought i’d say this in the public, but over the years i’ve come to appreciate guido’s conservatism in language-feature related questions as overall beneficial for python the language. it has turned out somehwat less beneficial for python the library—even with some modules deprecated there are still some horrendously outdated pieces among the batteries. things like ‘writing .aiff files on sun vmx blah machines’—is that still in the standard distribution? other things that could have been very well placed into the standard library are not in there. the python package index, pypi, is a step in the right direction; but, of course, in the absence of peer review, it is also full of abandoned software. as for language features, python moves very slowly, the benefit being a general absence of bewildering creeptures of the likes that haunt ruby. BUT yes! stackless should have made it. never understood why it didn’t. i am also not sure why the python developers deem it so important to make only 75% of the features you need for 32bit unicode available in the standard ‘narrow’ builds when there IS a working ‘wide’ python option. sigh. that being said, you really should advocate abandoning support for lists and dicts over 64KB. history has shown that overly long data structures hamper logical thinking and further obesity in people, which is why windows 3.1 notepad and jython chose to be restrictive in this regard. i also think that 20 LET STR A$ = ‘foo’ is the correct way to assign a string value to a variable. and we need line numbers, no? just joking.
Note that I generally agree about careful integration of language features. I just wonder if the “it does not fit with Jython” – or any other arbitrary Python implementation on the planet that has elementary flaws which are not even considered is just an excuse.
Zed Shaw complained about double standards concerning the stdlib. However the PEP process is the right way to go and the stdlib wasn’t under supervision of the PEP process for about half of the lifetime of the language. So the double standard is also mostly something historical.
I’m also a strict supporter of the idea of the stdlib and that it is not that well managed doesn’t mean it should be replaced by entropic package repositories. I know it sounds like a smartass comment but the stdlib could require a little more software engineering discipline and also deprecation policies.
+1+1 on that. i rejoice on counting a whopping 118 modules marked as deprecated in the py2.7 stdlib docs, mostly mac and irix-specific stuff. most of it probably does or did at some point make sense, but is too specific for a ‘standard’ library at any rate. for some odd reason, however, things like sunau, aifc, uu are still in. it is pretty hard to really get an overview of the stdlib with its three to four hundred items, many of which are a small countinent of complexity in their own right. after nearly a decade of working with python i still have occasional first encounters down in those trenches.
i recently realised my python install had developed a pretty sluggish startup behavior; after throwing out over 100 packages, i gained several seconds per startup. this tells me that the sheer size of the stdlib may very well be an impediment to startup or even overall performance of python, so maybe it would be worthwhile to do a conservative dependency check on projects, and maybe also using a dedicated python installation for each major piece that has only exactly those batteries aboard as are actually needed. you can still install turtle if you ever want to do to logo turtle graphics on tk, you know; otherwise, the mere presence of that module will impede every single import statement (ahh, the import system… so complex, so full of pitfalls, such a black hole for your spare time when trying to extend / mend / fix it…).
on a more constructive side, why not write up some of your experiences with stackless? i shied away from using it coz it’s not ‘standard’, but maybe i and many others are missing out on something. it’s easy to write a round-robin scheduler using coroutines, and there are many packages that offer turn key solutions for this, so what are the specific advantages of stackless?
(ahh, the import system… so complex, so full of pitfalls, such a black hole for your spare time when trying to extend / mend / fix it…).
You remember that I blogged about it…
About tk. I think one can get rid of it once one has a proper replacement for idle. Once again I like the rationale behind idle being in the stdlib – I just wonder if one can’t do better and provide a web based IDE ( which might connect to localhost as well of course ) that can be used by various Python implementations. I’m right now in the midst of writing a command shell in Flash and it will become a very cool piece of software if not the technical problems with Flash remoting will kill it before it reaches infancy.
on a more constructive side, why not write up some of your experiences with stackless?
I’d put it the other way round. The absence of stackless caused me to write generator_tools which is hackish and will never reach a complete coverage of generators unless I’m attacking Python 2.5 + X generators using source code transformation decorators implemented in EasyExtend. Stackless was years ahead and its feature set is slowly approximated by “standard” Python that still lags behind.
kay@”””The absence of stackless caused me to write generator_tools which is hackish […] Stackless was years ahead and its feature set is slowly approximated by “standard” Python that still lags behind”””—so maybe instead of writing one’s own toolkit rather make a distribution, add some discussion on a web page, thereby making stackless more available and more accessible for people? they do provide windows binary installers, i haven’t tried the thing on linuxes though (part of the reason i shied away from always using it per default—is it difficult? apt-get-wash-and-go?).
kay@”””About tk. I think one can get rid of it once one has a proper replacement for idle.”””—my replacement is textpad. argh. i’d love to have something better. i work a lot with unicode and SciTE is one of the few pieces of software that works more or less correctly and with (very) good performance on large files, even tho there are a few bumps (no regexes over astral plane characters etc). maybe http://www.pnotepad.org/ is promising, too. also i don’t think idle needs turtle graphics. i am really keen to see how many of the 300…400 stdlib names must remain visible for some of my projects.
kay@”””I just wonder if one can’t do better and provide a web based IDE ( which might connect to localhost as well of course ) […] writing a command shell in Flash”””—YESYESYES! web-based IDEs are coming! sadly https://bespin.mozilla.com/ doesn’t look like it’s moving anywhere at this time.
btw i don’t think one has to or should use flash to implement such a thing—what does flash have except for graphics, sound and multiple file upload that you can’t do with javascript+html+css (+canvas, if it’s the graphics really)? just wondering. (also, interfacing flash with the rest of the web page seems to be less than stellar; http://www.azarask.in/blog/post/flash-canvas/ points out he had to do “a very minimum 8 separate levels of indirection” to get javascript messages through to flash, which slowed down his canvas emulator to an unbelievable 0.4s…1.2s per call.)
to summarize, ►i for one do not use idle for programming or tk for guis; when i need a gui i do it the webbish way or stick with the command line; ►i plan to slim down my python and move to stackless, probably when migrating to py3k (which may take some time, depending in part on extension availability); ►i want to have a web-based ide that i can run both locally and remotely without big differences in functionality. ►for me, the walking legs of an ide are 1) very sound and code-centric text-editing; 2) an integrated search engine—programming, after all, is all about finding le ‘mot/module/procedure juste’; 3) ability to deal with all the file types that are current on the web—that should be self-evident in a web-based ide. i tried ever so often to get warm with eclipse, but wouldn’t. it is too heavy and the editor just does not feel right.
i already have a bunch of modules that deal with ssh and mercurial support. never came around to quite finish them tho. the central functionality is to check hg versions on the local and the remote machine, then upload and apply hg update bundles. parts of it work; if you’re interested i can send you a *.zip.
kay@”””You remember that I blogged about it…”””—you probably refer to “Broken beyond repair – Pythons import system”. makes for a fine tabloid headline. “Attempts To Fix Import Considered Harmful”. “Obama, Berlusconi Agree To Ban Python Import”. “Python Import Repair Taskforce: Seven Reported Lost In Code”. “NASA reports blame Python import statement for satellite loss”.—the last one is not so unlikely.
I used Flash/Flex/AS3 because it is the most complete and mature solution on the market right now for the kind of stuff I intend to do. Next year or in 2011 it might be Canvas + HTML 5 – who knows? I’m also not a dedicated web application programmer and have even less experience with JS than with AS3. The shell implementation so far is not huge but delicate and it is not just a clone of an Xterm or a DOS-box for the web. Both of them cover text input well but are poor when it comes to output which can be arbitrary stuff: images, movie-clips, controls etc. JavaFX might even be more appropriate because it is multi-threaded but a fairly new technology otherwise and I’m a bit scared about the weight of the JRE. It would be a good target for a migration and for learning JavaFX but I prototype in AS3.
You probably refer to “Broken beyond repair – Pythons import system”. makes for a fine tabloid headline.
Sure. No one reads my technical articles anyway. So why not give at least a well crafted polemics a juicy headline 😉
I don’t see it happening too often except for generated code, but Jython will in a future release relax the 64K limit for code bodies, including the top-level code body of a module. This would include such initializations as a list or dict. No guarantees, but this relaxation likely will be part of 2.5.1. It’s simply waiting for our a variant on our compiler that will emit Python bytecode instead of Jython bytecode, and useful for other circumstances like Android and applets where it’s not possible to dynamically generate Java bytecode; we have the corresponding VM released as part of 2.5.0.
Incidentally, with a little work such long methods can be even run now, by using CPython, if installed, to create a pyc file, which can then be imported with the import hook implemented by pycimport.
Hi Jim, I didn’t know about pycimport. That’s very cool and removes the major obstacle for running EasyExtend langlets on Jython. EasyExtend generates parse tables as Python dictionaries and they can grow easily beyond 1 MB.