Speeding up the python "import" loader
python
Solution
zipping up as many `pyc` files as feasible (with proper directory structure for packages), and putting that zipfile as the very first entry in sys.path (on the best available local disk, ideally) can speed up startup times a lot.
Problem
I'm getting seriously frustrated at how slow python startup is. Just importing more or less basic modules takes a second, since python runs down the sys.path looking for matching files (and generating 4 `stat()` calls - ["foo", "foo.py", "foo.pyc", "foo.so"] - for each check). For a complicated project environment, with tons of different directories, this can take around 5 seconds -- all to run a script that might fail instantly. Do folks have suggestions for how to speed up this process? For instance, one hack I've seen is to set the `LD_PRELOAD_32` environment variable to a library that caches the result of `ENOENT` calls (e.g. failed `stat()` calls) between runs. Of course, this has all sorts of problems (potentially confusing non-python programs, negative caching, etc.).