How does parakeet differ from Numba? Because I didn't see any improvements on some NumPy expressions

numba, numexpr, numpy, parakeet, python

Solution

As of the current release of Numba (which you are using in your tests), there is incomplete support for ufuncs with the `@jit` function. On the other hand you can use `@vectorize` and it faster:

import numpy as np
from numba import jit, vectorize
import numexpr as ne

def numpy_complex_expr(A, B):
    return(A*B+4.1*A > 2.5*B)

def numexpr_complex_expr(A, B):
    return ne.evaluate('A*B+4.1*A > 2.5*B')

@jit
def numba_complex_expr(A, B):
    return A*B+4.1*A > 2.5*B

@vectorize(['u1(float64, float64)'])
def numba_vec(A,B):
    return A*B+4.1*A > 2.5*B

n = 1000
A = np.random.rand(n,n)
B = np.random.rand(n,n)

Timing results:

%timeit numba_complex_expr(A,B)
1 loops, best of 3: 49.8 ms per loop

%timeit numpy_complex_expr(A,B)
10 loops, best of 3: 43.5 ms per loop

%timeit numexpr_complex_expr(A,B)
100 loops, best of 3: 3.08 ms per loop

%timeit numba_vec(A,B)
100 loops, best of 3: 9.8 ms per loop

If you want to leverage numba to its fullest, then you want to unroll any vectorized operations:

@jit
def numba_unroll2(A, B):
    C = np.empty(A.shape, dtype=np.uint8)
    for i in xrange(A.shape[0]):
        for j in xrange(A.shape[1]):
            C[i,j] = A[i,j]*B[i,j] + 4.1*A[i,j] > 2.5*B[i,j]

    return C

%timeit numba_unroll2(A,B)
100 loops, best of 3: 5.96 ms per loop

Also note that if you set the number of threads that numexpr uses to 1, then you'll see that its main speed advantage is that it's parallelized:

ne.set_num_threads(1)
%timeit numexpr_complex_expr(A,B)
100 loops, best of 3: 8.87 ms per loop

By default numexpr uses `ne.detect_number_of_cores()` as the number of threads. For my original timing on my machine, it was using 8.

Problem

I am wondering if anyone knows some of the key differences between the parakeet and the Numba jit? I am curious, because I was comparing Numexpr to Numba and parakeet, and for this particular expression (which I expected to perform very very well on Numexpr, because it was the one that is mentioned in its documentation) So the results are and the functions I tested (via timeit - minimum of 3 repetitions and 10 loops per function) ``` import numpy as np import numexpr as ne from numba import jit as numba_jit from parakeet import jit as para_jit def numpy_complex_expr(A, B): return(A*B-4.1*A > 2.5*B) def numexpr_complex_expr(A, B): return ne.evaluate('A*B-4.1*A > 2.5*B') @numba_jit def numba_complex_expr(A, B): return A*B-4.1*A > 2.5*B @para_jit def parakeet_complex_expr(A, B): return A*B-4.1*A > 2.5*B ``` I you can also grab the IPython nb if you'd like to double-check the results on your machine. If someone is wondering if Numba is installed correctly... I think so, it performed as expected in my previous benchmark:

Original source