HADOOP - Writing YARN applications

hadoop, hadoop-yarn, java

Solution

Writing a native YARN apps may be a little awkward because Hadoop YARN project doesn't have a higher level framework to help with this. However there are few projects which are doing the heavy lifting so that user would not have a need to worry about this boilerplate code.

Spring YARN as part of Spring for Apache Hadoop with YARN Samples

Apache Twill

Slider

In a link you posted, yes there are two main classes, one for running the client which deploys and starts the app on YARN, other for appmaster.

Problem

I am reading this and I am a little bit confused. Do I have to write always Client and ApplicationMaster Class to run my own code in YARN? For example if I want to run an application which just generates random numbers, Do I have to include these classes to my code or it can run it without these? Or, are they both different examples because I can see 2 main classes in there (1 main in each class)? I tried it with all of those ways but I am still getting something like an error --> `usage [input][ouput]:`

Original source