Installing Rmpi on LAM/MPI cluster

cluster-computing, mpi, r

Solution

We have a very similar setup. I haven't touched it recently enough to be able to speak to your particular situation, but I can pass along a few of the commands that got us up-and-running...

The first step for us, as Dirk mentioned, was to get OpenMPI running. Once that's setup, you'll need to be sure your `LD_LIBRARY_PATH` is properly set:

export LD_LIBRARY_PATH=/opt/openmpiv2/lib/:$LD_LIBRARY_PATH

The command that finally got Rmpi installed for us (after running the above command and starting R) was:

install.packages("Rmpi", configure.args = c("--with-Rmpi-include=/opt/openmpiv2/include/", "--with-Rmpi-libpath=/opt/openmpiv2/lib/", "--with-Rmpi-type=OPENMPI", "--with-mpi=/opt/openmpiv2/"))

Which installed the package successfully for us. Then we can run jobs using commands such as:

qsub -I -l nodes=2:ppn=12
/opt/openmpiv2/bin/mpirun --hostfile $PBS_NODEFILE -n 1 R --slave -f rmpitest.R

These commands may be so specific to our setup that they may not be useful, but hopefully something in here can save you some time!

As @Sergej proposes below, a more general solution may be:

install.packages(
  "Rmpi", 
  configure.args = c(
   "--with-Rmpi-include=/usr/include/", # This is where LAM's mpi.h is located
   "--with-Rmpi-libpath=/usr/lib/",     # This is where liblam.so is located (actually as I type it mine was located in /usr/lib64/liblam.so.0, so maybe this is not needed at all)
   "--with-Rmpi-type=LAM"               # This says that the type is LAM (there is also OPENMPI and MPICH)
))

Our setup: RHEL 5, Rmpi 0.5-9, OpenMPI 1.4.3., R 2.15.0

Problem

I'm trying to install `Rmpi` package on a LAM MPI cluster machine. Previously I had been compiling and testing some stuff (`mpi4py` and small C++ programs) so I'm sure the MPI itself works. However installing `Rmpi` package fails when linking libraries. My main suspect is a call to `gcc` instead of `mpicc` in makefile (I'm trying to find the line in configuration to change this but so far could not locate it). Does somebody have experience with installing Rmpi on LAM, and how did you manage that? Architecture - LAM MPI (or maybe PBS MPI if such exist, how do I check?). One thing for sure, I have mpicpp file wich I don't think is common to all mpi implementations - R-2.15.2 - Rmpi_0.6-1 - CentOS release 5.5 (Final), x86_64 Command called from R ``` install.packages("Rmpi") ``` Log ``` trying URL 'http://cran.at.r-project.org/src/contrib/Rmpi_0.6-1.tar.gz' Content type 'application/x-gzip' length 92977 bytes (90 Kb) opened URL ================================================== downloaded 90 Kb * installing *source* package âRmpiâ ... ** package âRmpiâ successfully unpacked and MD5 sums checked checking for gcc... gcc -std=gnu99 checking whether the C compiler works... yes checking for C compiler default output file name... a.out checking for suffix of executables... checking whether we are cross compiling... no checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc -std=gnu99 accepts -g... yes checking for gcc -std=gnu99 option to accept ISO C89... none needed I am here /usr and it is OpenMPI Trying to find mpi.h ... Found in /usr/include Trying to find libmpi.so or libmpich.a ... Found libmpi in /usr/lib checking for openpty in -lutil... yes checking for main in -lpthread... yes configure: creating ./config.status config.status: creating src/Makevars ** libs gcc -std=gnu99 -I/g/software/linux/pack/r-2.15.2/centos-5/lib64/R/include -DNDEBUG -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -I/usr/include -DMPI2 -DOPENMPI -I/usr/local/include -fpic -g -O2 -c RegQuery.c -o RegQuery.o gcc -std=gnu99 -I/g/software/linux/pack/r-2.15.2/centos-5/lib64/R/include -DNDEBUG -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -I/usr/include -DMPI2 -DOPENMPI -I/usr/local/include -fpic -g -O2 -c Rmpi.c -o Rmpi.o gcc -std=gnu99 -I/g/software/linux/pack/r-2.15.2/centos-5/lib64/R/include -DNDEBUG -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -I/usr/include -DMPI2 -DOPENMPI -I/usr/local/include -fpic -g -O2 -c conversion.c -o conversion.o gcc -std=gnu99 -I/g/software/linux/pack/r-2.15.2/centos-5/lib64/R/include -DNDEBUG -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE_URL=\"\" -I/usr/include -DMPI2 -DOPENMPI -I/usr/local/include -fpic -g -O2 -c internal.c -o internal.o gcc -std=gnu99 -shared -L/usr/local/lib64 -o Rmpi.so RegQuery.o Rmpi.o conversion.o internal.o -L/usr/lib -lmpi -lutil -lpthread -L/g/software/linux/pack/r-2.15.2/centos-5/lib64/R/lib -lR /usr/bin/ld: skipping incompatible /usr/lib/libutil.so when searching for -lutil /usr/bin/ld: skipping incompatible /usr/lib/libutil.a when searching for -lutil /usr/bin/ld: skipping incompatible /usr/lib/libpthread.so when searching for -lpthread /usr/bin/ld: skipping incompatible /usr/lib/libpthread.a when searching for -lpthread /usr/bin/ld: skipping incompatible /usr/lib/libc.so when searching for -lc /usr/bin/ld: skipping incompatible /usr/lib/libc.a when searching for -lc installing to /home/andrejev/R/x86_64-unknown-linux-gnu-library/2.15/Rmpi/libs ** R ** demo ** inst ** preparing package for lazy loading ** help *** installing help indices ** building package indices ** testing if installed package can be loaded Error : .onLoad failed in loadNamespace() for 'Rmpi', details: call: dyn.load(file, DLLpath = DLLpath, ...) error: unable to load shared object '/home/andrejev/R/x86_64-unknown-linux-gnu-library/2.15/Rmpi/libs/Rmpi.so': /usr/lib/libmpi.so.0: undefined symbol: _kio Error: loading failed Execution halted ERROR: loading failed * removing â/home/andrejev/R/x86_64-unknown-linux-gnu-library/2.15/Rmpiâ The downloaded source packages are in â/tmp/Rtmp5qzeqp/downloaded_packagesâ Warning message: In install.packages("Rmpi") : installation of package âRmpiâ had non-zero exit status ```

Original source