python-startup | Python start-up script implementing | Interpreter library

I have figured this out – the answer turns out to be simultaneously illuminating and embarrassing – and my solution may help others out when faced with similar circumstances.

In a nutshell: the aggravatingly long ~12s pauses I experienced while the Python interpreter loaded were being caused by having an excessively large amount of Python extension modules installed. It was not an issue with Python 2.7’s bundled xml.parsers.expat module, nor with its C-API pyexpat extension.

To wit: my use of the gnomon tool, which furnished what appeared to be direct and straightforward evidence pointing to these modules, ended up misleading me in my conclusions as to where the problematic code was to be found.

After posting my question, I did a bit of additional forensic poking-around. By varying the Python code I was passing to the interpreter while invoking the command-line speed checks, I found that the gnomon report would show the same twelve-plus-second halt, but at the incidence of different import statements. Further, I found that some command variants (e.g. those executed using the pythonpy CLT) weren’t plagued by the halting behavior at all.

I was able to pinpoint the lines of code responsible for the issues’ manifestation when I stumbled upon it by accident – while running my tests, the interminably long halts were no less annoying, and I ended up control-C’ing a number of tests in mid-halt. Those aborted test runs terminated with KeyboardInterrupt exceptions, and the accompanying stacktrace output revealed the function in which things were dragging:

… the pkg_resources module, when imported, walks each of the extensions directories named in sys.path, enumerating each package in each extension, and subsequently reading in and then parsing all the associated metadata for all of those. Using any part of pkg_resources (which itself is part of the essential setuptools module) triggers this time-consuming action (which is then cached, at least, for the duration of that particular interpreter invocation’s lifetime). Depending on how your Python install is set up, and how you invoke your interpreter, you may or may not end up doing something to trigger the use of pkg_resources, but it is in pretty wide use across Python extension packages, so chances are good that it’ll get triggered by something.

The actual function responsible for the actual loop that actually enumerates the packages is _initialize_master_working_set() – it’s the one I’ve highlighted in the screenshot above. This is what all my KeyboardInterrupt stacktraces revealed. From there, it was immediately evident that the frustrating halts were a steeply linear function of the number of Cheese Shop packages present (something I had been reckless with after upgrading my laptop).

I immediately proceeded to pip-uninstall roughly 50% of the extensions I had gratuitously installed, and then pared down another 40% or so by hoisting most of my actively-developed Python stuff into isolated virtualenv project directories.

I felt pretty dumb afterward, as I had managed to cleverly mislead myself with my use of fancy analysis tools, and then found the actual solution by accident – one to a problem resulting from my own careless inattention, no less. Regardless, it is still something that could bite other Pythonic developers out there, and thus worth the writeup. You are hereby invited to learn from my circuitous adventures in issue-triage and diagnosis, indeed!