fast dot product on all pair of rows
arrays, numpy, python
Solution
That is obtained by doing `result = X.dot(X.T)`
When the array becomes large, it can be done be blocks, but depending on your numpy backend this should already parallelize threadwise as much as possible. It seems that this is what you are looking for.
If for some reason you don't want to rely on that, and finally do resort to multiprocessing, you can try something along the lines of
import numpy as np
X = np.random.randn(1000, 100000)
block_size = 10000
from sklearn.externals.joblib import Parallel, delayed
products = Parallel(n_jobs=10)(delayed(np.dot)(X[:, pos:pos + block_size], X.T[pos:pos + block_size]) for pos in range(0, X.shape[1], block_size))
product = np.sum(products, axis=0)
I don't think this is useful for relatively small arrays. And threading can sometimes take care of this better as well.
Problem
I have a 2d numpy array `X = (xrows, xcols)` and I want to apply dot product on each row combination of the array to obtain another array which is of the shape `P = (xrow, xrow)`. The code looks like the following: ``` P = np.zeros((xrow, xrow)) for i in range(xrow): for j in range(xrow): P[i, j] = numpy.dot(X[i], X[j]) ``` which works well if the array `X` is small but takes a lot of time for huge `X`. Is there any way to make it faster or do it more pythonically so that it is fast?