R and snow on amazon EC2 using starcluster

amazon-ec2, amazon-web-services, r

Solution

It turns out, after much tinkering, that all that was needed was an update to R version 2.15. The command `cl = makeCluster(c("masterIP.compute-1.amazonaws.com", "node001IP.compute-1.amazonaws.com"), type = "SOCK")` worked perfectly after that.

Problem

I'm trying to run analysis in parrallel in R on an AWS EC2 cluster. I am using starcluster to setup and manage the EC2 cluster, and am trying to use `snow` and `foreach` in R. To start off, I have 2 nodes in the cluster, 1 master and 1 worker. ``` starcluster start mycluster starcluster listinstances ----------------------------------------- mycluster (security group: @sc-mycluster) ----------------------------------------- .... Cluster nodes: master running i-xxxxxxxxx masterIP.compute-1.amazonaws.com node001 running i-xxxxxxxxx node001IP.compute-1.amazonaws.com Total nodes: 2 starcluster sshmaster mycluster ``` I then start R and load the `snow` package and try to create a cluster object. ``` R library("snow") cl = makeCluster(c("masterIP.compute-1.amazonaws.com", "node001IP.compute-1.amazonaws.com"), type = "SOCK") ``` This, however, gives me the following error message: ``` The authenticity of host 'masterIP.compute-1.amazonaws.com (xx.xxx.xx.xx)' can't be established. ECDSA key fingerprint is xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added 'masterIP.compute-1.amazonaws.com,xx.xxx.xx.xx' (ECDSA) to the list of known hosts. Permission denied (publickey). ``` So I tried copying my ssh key (`keyname.rsa` to be specific) to the .ssh file on EC2 and trying again. That still didn't work; I received the same `Permission denied (publickey).` error. It was my thought that starcluster handled the setup of ssh and communication between nodes, so I'm a little confused as to why I'm not able to set this up. I also tried to just add node001, so `cl = makeCluster(c("node001IP.compute-1.amazonaws.com"), type = "SOCK")`, but the same error occurs.

Original source