How to get SSSP actual path by apache spark graphX?

apache-spark, scala, spark-graphx

Solution

You have to modify algorithm in order to store not only shortest path length but also actual path. So instead of storing `Double` as property of vertex you should store tuple: `(Double, List[VertexId])` Maybe this code can be useful for you.

object Pregel_SSSP {
  def main(args: Array[String]) {
    val sc = new SparkContext("local", "Allen Pregel Test", System.getenv("SPARK_HOME"), SparkContext.jarOfClass(this.getClass))
    // A graph with edge attributes containing distances
    val graph: Graph[Int, Double] =
      GraphGenerators.logNormalGraph(sc, numVertices = 5).mapEdges(e => e.attr.toDouble)
    graph.edges.foreach(println)
    val sourceId: VertexId = 0 // The ultimate source

    // Initialize the graph such that all vertices except the root have distance infinity.
    val initialGraph : Graph[(Double, List[VertexId]), Double] = graph.mapVertices((id, _) => if (id == sourceId) (0.0, List[VertexId](sourceId)) else (Double.PositiveInfinity, List[VertexId]()))

    val sssp = initialGraph.pregel((Double.PositiveInfinity, List[VertexId]()), Int.MaxValue, EdgeDirection.Out)(

      // Vertex Program
      (id, dist, newDist) => if (dist._1 < newDist._1) dist else newDist, 

      // Send Message
      triplet => {
        if (triplet.srcAttr._1 < triplet.dstAttr._1 - triplet.attr ) {
          Iterator((triplet.dstId, (triplet.srcAttr._1 + triplet.attr , triplet.srcAttr._2 :+ triplet.dstId)))
        } else {
          Iterator.empty
        }
      },
      //Merge Message
      (a, b) => if (a._1 < b._1) a else b)
    println(sssp.vertices.collect.mkString("\n"))
  }
}

Problem

I have ran the single source shortest path (SSSP) example on spark site as follows: graphx-SSSP pregel example Code(scala): ``` object Pregel_SSSP { def main(args: Array[String]) { val sc = new SparkContext("local", "Allen Pregel Test", System.getenv("SPARK_HOME"), SparkContext.jarOfClass(this.getClass)) // A graph with edge attributes containing distances val graph: Graph[Int, Double] = GraphGenerators.logNormalGraph(sc, numVertices = 5).mapEdges(e => e.attr.toDouble) graph.edges.foreach(println) val sourceId: VertexId = 0 // The ultimate source // Initialize the graph such that all vertices except the root have distance infinity. val initialGraph = graph.mapVertices((id, _) => if (id == sourceId) 0.0 else Double.PositiveInfinity) val sssp = initialGraph.pregel(Double.PositiveInfinity, Int.MaxValue, EdgeDirection.Out)( // Vertex Program (id, dist, newDist) => math.min(dist, newDist), // Send Message triplet => { if (triplet.srcAttr + triplet.attr < triplet.dstAttr) { Iterator((triplet.dstId, triplet.srcAttr + triplet.attr)) } else { Iterator.empty } }, //Merge Message (a, b) => math.min(a, b)) println(sssp.vertices.collect.mkString("\n")) } } ``` sourceId: 0 Get the result: (0,0.0) (4,2.0) (2,1.0) (3,1.0) (1,2.0) But I need actual path like as follows: => 0 -> 0,0 0 -> 2,1 0 -> 3,1 0 -> 2 -> 4,2 0 -> 3 -> 1,2 How to get SSSP actual path by spark graphX? anybody give me some hint? Thanks for your help!

Original source