Post hook for Elastic MapReduce

amazon-web-services, emr, hadoop, hadoop-streaming

Solution

You'll want to configure the job end notification URL.

jobEnd.notificationUrl

AWS will hit this url, presumably with query variables that indicate which job has completed (job id).

You could then have this URL on your server process your email notifications, assuming you had already stored a relationship between emails and job ids.

https://issues.apache.org/jira/browse/HADOOP-1111

Problem

I wonder if there is an example of post process for EMR (Elastic MapReduce)? What I am trying to achieve is send an email to group of people right after Amazon's Hadoop finished the job.

Original source