Column wise sum V row wise sum: Why don't I see a difference using NumPy?

Question

I've tested an example demonstrated in this talk [pytables] using numpy (page 20/57). It is stated, that `a[:,1].sum()` takes 9.3 ms, whereas `a[1,:].sum()` takes only 72 us. I tried to reproduce it, but failed to do so. Am I measuring wrongly? Or have things changed in NumPy since 2010? ``` $ python2 -m timeit -n1000 --setup \ 'import numpy as np; a = np.random.randn(4000,4000);' 'a[:,1].sum()' 1000 loops, best of 3: 16.5 usec per loop $ python2 -m timeit -n1000 --setup \ 'import numpy as np; a = np.random.randn(4000,4000);' 'a[1,:].sum()' 1000 loops, best of 3: 13.8 usec per loop $ python2 --version Python 2.7.7 $ python2 -c 'import numpy; print numpy.version.version' 1.8.1 ``` While I can measure a benefit of the second version (supposedly fewer cache misses because numpy uses C-style row ordering), I don't see that drastic difference as stated by the pytables contributor. Also, it seems I cannot see more cache misses when using column V row summation. EDIT So far the insight for m…

Accepted Answer

I don't see anything wrong with your attempt at replication, but bear in mind that those slides are from 2010, and numpy has changed rather a lot since then. Based on the dates of numpy releases, I would guess that Francesc was probably using v1.5.

Using this script to benchmark row v column sums:

```
#!python

import numpy as np
import timeit

print "numpy version == " + str(np.__version__)

setup = "import numpy as np; a = np.random.randn(4000, 4000)"

rsum = "a[1, :].sum()"
csum = "a[:, 1].sum()"

nloop = 1000
nrep = 3

print "row sum:	%.3f us" % (
    min(timeit.repeat(rsum, setup, repeat=nrep, number=nloop)) / nloop * 1E6)
print "column sum:	%.3f us" % (
    min(timeit.repeat(csum, setup, repeat=nrep, number=nloop)) / nloop * 1E6)
```

I detect about a 50% slowdown for column sums with numpy v1.5:

```
$ python sum_benchmark.py 
numpy version == 1.5.0
row sum:        8.472 us
column sum:     12.759 us
```

Compared with about a 30% slowdown with v1.8.1, which you're using:

```
$ python sum_benchmark.py 
numpy version == 1.8.1
row sum:        12.108 us
column sum:     15.768 us
```

It's interesting to note that both types of reduction have actually gotten a bit slower in the more recent numpy versions. I would have to delve a lot deeper into numpy's source code to understand exactly why this is the case.

Update

- For the record, I'm running Ubuntu 14.04 (kernel v3.13.0-30) on a quad-core i7-2630QM CPU @ 2.0GHz. Both versions of numpy were pip-installed and compiled using GCC-4.8.1.

- I realize my original benchmarking script wasn't totally self-explanatory - you need to divide the total time by the number of loops (1000) in order to get the time per call.

- It also probably makes more sense to take the minimum across repeats rather than the average, since this is more likely to represent the lower bound on the execution time (on top of which you'd get variability due to background processes etc.).

I've updated my script and results above accordingly

We can also negate any effect of caching across calls (temporal locality) by creating a brand-new random array for every call - just set `nloop` to 1 and `nrep` to a reasonably small number (unless you really enjoy watching paint dry), say 10.

`nloop=1`, `nreps=10` on a 4000x4000 array:

```
numpy version == 1.5.0
row sum:        47.922 us
column sum:     103.235 us

numpy version == 1.8.1
row sum:        66.996 us
column sum:     125.885 us
```

That's a bit more like it, but I still can't really replicate the massive effect that Francesc's slides show. Perhaps this isn't that surprising, though - the effect may be very compiler-, architecture, and/or kernel-dependent.

Column wise sum V row wise sum: Why don't I see a difference using NumPy?

Solution

Problem

Related problems