Parallelization in R: how to "source" on every node?

parallel-processing, r

Solution

The following code serves your purpose:

library(parallel)

cl <- makeCluster(4)
clusterCall(cl, function() { source("test.R") })

## do some parallel work

stopCluster(cl)

Also you can use `clusterEvalQ()` to do the same thing:

library(parallel)

cl <- makeCluster(4)
clusterEvalQ(cl, source("test.R"))

## do some parallel work

stopCluster(cl)

However, there is subtle difference between the two methods. `clusterCall()` runs a function on each node while `clusterEvalQ()` evaluates an expression on each node. If you have a variable list of files to source, `clusterCall()` will be easier to use since `clusterEvalQ(cl,expr)` will regard any `expr` as an expression so it's not convenient to put a variable there.

Problem

I have created parallel workers (all running on the same machine) using: ``` MyCluster = makeCluster(8) ``` How can I make every of these 8 nodes source an R-file I wrote? I tried: ``` clusterCall(MyCluster, source, "myFile.R") clusterCall(MyCluster, 'source("myFile.R")') ``` And several similar versions. But none worked. Can you please help me to find the mistake? Thank you very much!

Original source