List comprehension, map, and numpy.vectorize performance
list-comprehension, numpy, performance, python
Solution
First comment: don't mix usage of `xrange(`) or `range()` in your samples... doing so invalidates your question as you're comparing apples and oranges.
I second @Gabe's notion that if you have many large data structures, numpy should win overall... just keep in mind most of the time C is faster than Python, but then again, most of the time, PyPy is faster than CPython. `:-)`
As far as listcomps vs. `map()` calls go... one makes 101 function calls while the other one makes 102. meaning you won't see a significant difference in timing, as shown below using the timeit module as @Mike has suggested:
List Comprehension
`$ python -m timeit "def foo(x):pass; [foo(i) for i in range(100)]"` `1000000 loops, best of 3: 0.216 usec per loop` `$ python -m timeit "def foo(x):pass; [foo(i) for i in range(100)]"` `1000000 loops, best of 3: 0.21 usec per loop` `$ python -m timeit "def foo(x):pass; [foo(i) for i in range(100)]"` `1000000 loops, best of 3: 0.212 usec per loop`
`map()` function call
`$ python -m timeit "def foo(x):pass; map(foo, range(100))"` `1000000 loops, best of 3: 0.216 usec per loop` `$ python -m timeit "def foo(x):pass; map(foo, range(100))"` `1000000 loops, best of 3: 0.214 usec per loop` `$ python -m timeit "def foo(x):pass; map(foo, range(100))"` `1000000 loops, best of 3: 0.215 usec per loop`
With that said however, unless you are planning on using the lists that you create from either of these techniques, try avoid them (using lists) completely. IOW, if all you're doing is iterating over them, it's not worth the memory consumption (and possibly creating a potentially massive list in memory) when you only care to look at each element one at a time just discard the list as soon as you're done.
In such cases, I highly recommend the use of generator expressions instead as they don't create the entire list in memory... it is a more memory-friendly, lazy iterative way of looping through elements to process w/o creating a largish array in memory. The best part is that its syntax is nearly identical to that of listcomps:
a = (foo(i) for i in range(100))
2.x users only: along the lines of more iteration, change all the `range()` calls to `xrange()` for any older 2.x code then switch to `range()` when porting to Python 3 where `xrange()` replaces and is renamed to `range()`.
Problem
I have a function foo(i) that takes an integer and takes a significant amount of time to execute. Will there be a significant performance difference between any of the following ways of initializing a: ``` a = [foo(i) for i in xrange(100)] a = map(foo, range(100)) vfoo = numpy.vectorize(foo) a = vfoo(range(100)) ``` (I don't care whether the output is a list or a numpy array.) Is there a better way?