It often happens that language features we have once distasted become comfortable after a while. We learn to live with them, take their advantages and forget about their annoyance. For many Python developers things like the explicit `self` in the argument list of a method is among them or `len` as a function. Experienced Python programmers are usually weakly excited about alternative proposals and the longer this quirk is discussed their mood turns towards -0 which is encoded as “why bothering?”. <\/p>\n
Other features are the cause of continuous struggle and we never get used to them but don’t talk much about them because they aren’t low hanging fruits. They might even turn from bad to worse in the course of improving them in some respects. Those features feel like being broken beyond repair<\/span> which means that it’s not easy to patch them but they need a fundamental redesign. In my opinion Pythons import machinery has this quality and it was regrettably forgotten in the many “cleanups” of Python 3.<\/p>\n Pythons import machinery is complicated to understand but this alone isn’t an argument against it. Many systems are far more complex but they are well thought out. The import machinery seems to be designed by implementation and cobbled together. This has finally lead to accidental design artifacts like the module\/script distinction which makes sense only in the light of Pythons import implementation.<\/p>\n Python always suffered from the problem that two modules `M1.py` and `M2.py` which import a third module `M3.py` from different import paths also receive different module objects ` Not only are module objects path dependent but the success of a module import may depend on the liveliness of other modules. Suppose `M1.py` defines the following relative import: `from . import M2`. This import fails if the package `P` containing both `M1.py` and `M2.py` has not been imported already. When running `M1.py` from the command line it’s not relevant that we declared `P` as a package by placing an `__init__.py` file into it we still get the increasingly famous error message:<\/p>\n ValueError: Attempted relative import in non-package<\/span><\/p>\n This is not a Monty Python skit. The package exists and exists not. The interpreter doesn’t make any use of the availability of package `P` as long as it hasn’t been imported explicitly. But why does it has to be imported only to find `M2.py`? There is no obvious answer. It’s just the way it is implemented.<\/p>\n “We should<\/em> forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. (Donald E. Knuth)“<\/strong><\/p>\n Back to the first problem. Once `M1.py` has imported `M3.py` the ` This lookup with a module-path string is fast and it is usually faster than ordinary attribute lookup in objects which has to jump through a nested dictionary. I do think that’s basically it<\/i>. If we implement a proper module cache providing the moderate performance of attribute lookups in objects we can get rid of the tedious import machinery that Python has right now. But I’d like to go even a little further and claim that a whole import system can be designed directly from a tiny set of objects and their properties. No Finders, Loaders, Importers and all the clunky stuff which pretends to be objects but is actually a service which serves the void.<\/p>\n I’ll flesh this idea out in a series of articles on this blog soon. This can be understood then as the constructive part of my criticism.<\/p>\n Newsflash: chances are that higher Python officers will succeed in confusing the situation even more. PEP 382<\/a> is on the way. Good luck with explaining why Python needs two notions of packages whereas it is difficult enough to justify even one. It’s not a coincidence though that this is placed in the context of setuptools. When Pythons import system is a tragedy than distutils and the layers above are the accompanying farce<\/a>.<\/span><\/p>\n","protected":false},"excerpt":{"rendered":" It often happens that language features we have once distasted become comfortable after a while. We learn to live with them, take their advantages and forget about their annoyance. For many Python developers things like the explicit `self` in the … Continue reading