使用对RDD在Graphx中创建图形

Question

从RDD中提取(1)如下所示：

res2: Array[(String, String)] = Array((905067378709,905458844980))

Answer 1

有向图的解法

让我们假设您有以下RDD对保持边：

val data: RDD[(String, String)] = sc.parallelize(
  Seq(
    ("905067378709", "905458844980"),
    ("905067378709", "905458844980"),
    ("905458844980", "905067378709"),
    ("905067378709", "905458844980"),
    ("905458844982", "905458844984"),
    ("905067378709", "905458844984"),
    ("905067378712", "905067378709")))

创建以下RDD[(VertexId，VertexId)]:

val edgesRDD: RDD[(VertexId, VertexId)] = data.map { case (a, b) => (a.toLong, b.toLong) }

val graph = Graph.fromEdgeTuples(edgesRDD, 1)
// to print 
val vert: VertexRDD[Int] = graph.vertices
vert.foreach { println }

val edg: EdgeRDD[Int] = graph.edges
edg.foreach { println }

val subgraph = graph.partitionBy(PartitionStrategy.CanonicalRandomVertexCut)
.groupEdges((a, b) => a + b)

// To print

val vert2: VertexRDD[Int] = subgraph.vertices
vert2.foreach { println }

val edg2: EdgeRDD[Int] = subgraph.edges
edg2.foreach { println }

边缘（905067378709,905458844980,3）边缘出现3次

边缘(905458844980，905067378709，1)

边缘(905458844982、905458844984、1)