Could not parse Master URL: 'spark:http://localhost:18080'

java, spark-streaming, twitter

Solution

The problem is that you specify 2 schemas in the URL you pass to `SparkConf.setMaster()`.

The `spark` is the schema, so you don't need to add `http` after `spark`. See the javadoc of `SparkConf.setMaster()` for more examples.

So the master URL you should be using is `"spark://localhost:18080"`. Change this line:

SparkConf conf = new SparkConf().setAppName("App_Name")
    .setMaster("spark://localhost:18080").set("spark.ui.port","18080");

Problem

When I'm trying to run my code it throws this `Exception`: ``` Exception in thread "main" org.apache.spark.SparkException: Could not parse Master URL:spark:http://localhost:18080 ``` This is my code: ``` SparkConf conf = new SparkConf().setAppName("App_Name").setMaster("spark:http://localhost:18080").set("spark.ui.port","18080"); JavaStreamingContext ssc = new JavaStreamingContext(sc, new Duration(1000)); String[] filet=new String[]{"Obama","ISI"}; JavaReceiverInputDStream<Status> reciverStream=TwitterUtils.createStream(ssc,filet); JavaDStream<String> statuses = reciverStream.map(new Function<Status, String>() { public String call(Status status) { return status.getText(); } } ); ssc.start(); ssc.awaitTermination();}} ``` Any idea how can I fix this problem?

Original source