parRF on caret not working for more than one core

Question

parRF from the caret R package is not working for me with more than one core, which is quite ironic, given the par in parRF stands for parallel. I'm on a windows machine, if that is a relevant piece of information. I checked that I'm using the latest an greatest regarding caret and doParallel. I made a minimal example and and give the results below. Any ideas? Source code ``` library(caret) library(doParallel) trCtrl <- trainControl( method = "repeatedcv" , number = 2 , repeats = 5 , allowParallel = TRUE ) # WORKS registerDoParallel(1) train(form = Species~., data=iris, trControl = trCtrl, method="parRF") closeAllConnections() # FAILS registerDoParallel(2) train(form = Species~., data=iris, trControl = trCtrl, method="parRF") closeAllConnections() ``` Output ``` > library(caret) > library(doParallel) > > trCtrl <- trainControl( + method = "repeatedcv" + , number = 2 + , repeats = 5 + , allowParallel = TRUE + ) > > > # WORKS > registerDoParallel(1) > train(form = Species~., data=iris,…

Accepted Answer

This is clearly a bug in caret 6.0-30 that was introduced sometime after version 5.17-7. It's also another problem that is more likely to hit Windows users, since the doParallel "mclapply mode" works, while the "clusterApplyLB mode" fails.

I've run some tests, and it appears that the problem is due to the cluster workers not being properly initialized to perform nested parallel computations, so you can work-around the bug by loading the foreach package in the cluster workers before calling "train". To do this, you need to explicitly create the cluster object, rather than letting the "registerDoParallel" function create it for you (which it does on Windows). For example:

```
cl <- makePSOCKcluster(2)
clusterEvalQ(cl, library(foreach))
registerDoParallel(cl)
```

I'll contact the author of caret to discuss a solution to the problem.

parRF on caret not working for more than one core

Solution

Problem