Drop into an Interpreter anytime in Python

debugging, python

Solution

You can use the signal module to setup a handler that will launch the debugger when you hit control-C or control-Z or whatever.. SIGINTR, SIGSUSP.

For example, define a module `instant_debug.py` that overrides SIGQUIT,

import signal
import pdb

def handler(signum, frame):
  pdb.set_trace()

signal.signal(signal.SIGQUIT, handler)

Then make a script

import instant_debug
import time

for i in xrange(1000000):
  print i
  time.sleep(0.1)

At any point during execution, you can jump into the code by typing `CTRL+\`, examine the stack with `u` and `d` as in normal `pdb`, then continue with `c` as if nothing ever happened. Note that you will only jump in at the end of the next "atomic" operation -- that means no stopping in the middle of a giant C module.

Problem

I know how to drop into an interpreter with `pdb` and `IPython`, but this requires me knowing beforehand exactly where I want to stop. However, I often run number crunching scripts that take minutes to hours, and I would like to know exactly what it's progress is. One solution is to simply put lots of logging statements everywhere, but then I either inundate myself with too much information or fail to log exactly what I want to know. Is there a way to initialize a listener loop that under some key combination will drop me into the code wherever it currently is? Think CTRL+Z but leaving me in Python rather than Bash.

Original source