How to nest multiple parfor loops

Question

`parfor` is a convenient way to distribute independent iterations of intensive computations among several "workers". One meaningful restriction is that `parfor`-loops cannot be nested, and invariably, that is the answer to similar questions like there and there. Why parallelization across loop boundaries is so desirable Consider the following piece of code where iterations take a highly variable amount of time on a machine that allows 4 workers. Both loops iterate over 6 values, clearly hard to share among 4. ``` for row = 1:6 parfor col = 1:6 somefun(row, col); end end ``` It seems like a good idea to choose the inner loop for `parfor` because individual calls to `somefun` are more variable than iterations of the outer loop. But what if the run time for each call to `somefun` is very similar? What if there are trends in run time and we have three nested loops? These questions come up regularly, and people go to extremes. Pattern needed for combining loops Ideally, `somefun` is run fo…

Accepted Answer

MrAzzman already pointed out how to linearise nested loops. Here is a general solution to linearise n nested loops.

1) Assuming you have a simple nested loop structure like this:

```
%dummy function for demonstration purposes
f=@(a,b,c)([a,b,c]);

%three loops
X=cell(4,5,6);
for a=1:size(X,1);
    for b=1:size(X,2);
        for c=1:size(X,3);
            X{a,b,c}=f(a,b,c);
        end
    end
end
```

2) Basic linearisation using a for loop:

```
%linearized conventional loop
X=cell(4,5,6);
iterations=size(X);
for ix=1:prod(iterations)
    [a,b,c]=ind2sub(iterations,ix);
    X{a,b,c}=f(a,b,c);
end   
```

3) Linearisation using a parfor loop.

```
%linearized parfor loop
X=cell(4,5,6);
iterations=size(X);
parfor ix=1:prod(iterations)
    [a,b,c]=ind2sub(iterations,ix);
    X{ix}=f(a,b,c);
end
```

4) Using the second version with a conventional for loop, the order in which the iterations are executed is altered. If anything relies on this you have to reverse the order of the indices.

```
%linearized conventional loop
X=cell(4,5,6);
iterations=fliplr(size(X));
for ix=1:prod(iterations)
    [c,b,a]=ind2sub(iterations,ix);
    X{a,b,c}=f(a,b,c);
end
```

Reversing the order when using a `parfor` loop is irrelevant. You can not rely on the order of execution at all. If you think it makes a difference, you can not use `parfor`.

How to nest multiple parfor loops

Solution

Problem

Related problems