Use Python's `timeit` from a program but functioning the same way as the command line?

python, timeit

Solution

When you call `timeit` from the command line like this:

python -mtimeit -s'import test' 'test.foo()'

The `timeit` module is called as a script. In particular, the `main` function is called:

if __name__ == "__main__":
    sys.exit(main())

If you look at the source code, you'll see that the `main` function can take an `args` argument:

def main(args=None):    
    if args is None:
        args = sys.argv[1:]

So indeed it is possible to run `timeit` from within a program with exactly the same behavior as you see when run from the CLI. Just supply your own `args` instead of allowing it to be set to `sys.argv[1:]`:

import timeit
import shlex

def foo():
    total = 0
    for i in range(10000):
        total += i**3
    return total

timeit.main(args=shlex.split("""-s'from __main__ import foo' 'foo()'"""))

will print something like

100 loops, best of 3: 7.9 msec per loop

Unfortunately, `main` prints to the console, instead of returning the time per loop. So if you want to programmatically use the result, perhaps the easiest way would be to start by copying the `main` function and then modifying it -- changing the printing code to instead return `usec`.

Example by OP: If you place this in `utils_timeit.py`:

import timeit
def timeit_auto(stmt="pass", setup="pass", repeat=3):
    """
    http://stackoverflow.com/q/19062202/190597 (endolith)
    Imitate default behavior when timeit is run as a script.

    Runs enough loops so that total execution time is greater than 0.2 sec,
    and then repeats that 3 times and keeps the lowest value.

    Returns the number of loops and the time for each loop in microseconds
    """
    t = timeit.Timer(stmt, setup)

    # determine number so that 0.2 <= total time < 2.0
    for i in range(1, 10):
        number = 10**i
        x = t.timeit(number) # seconds
        if x >= 0.2:
            break
    r = t.repeat(repeat, number)
    best = min(r)
    usec = best * 1e6 / number
    return number, usec

you can use it in scripts like this:

import timeit
import utils_timeit as UT

def foo():
    total = 0
    for i in range(10000):
        total += i**3
    return total

num, timing = UT.timeit_auto(setup='from __main__ import foo', stmt='foo()')
print(num, timing)

Problem

For instance, documentation says: Note however that timeit will automatically determine the number of repetitions only when the command-line interface is used. Is there a way to call it from within a Python script and have the number of repetitions be determined automatically, with only the shortest number returned?

Original source