获得RDD[(VertexId, Array[VertexId])]中某个VertexId对应的Array[VertexId]
请问RDD的查询操作用什么?
在Spark中我计算得到一个RDD[(VertexId, Array[VertexId])], 我要获得其中某个VertexId对应的Array[VertexId]大概要怎么写代码啊?
代码:产生一个RDD[(VertexId, Array[VertexId])],并显示其中的具体内容
val neighbors:RDD[(VertexId,Array[VertexId])] =
social.collectNeighborIds(EdgeDirection.Either) //VertexId是Long的别名
neighbors.collect()
res14: Array[(org.apache.spark.graphx.VertexId, Array[org.apache.spark.graphx.VertexId])] =
Array((44000000000001232,Array(21000000000002332)),
(77000000000006009,Array(21000000000002332)),
(96000000000000101,Array(83000000000004422)),
(78000000000001982,Array(74000000000000202)),
(76000000000007234,Array(21000000000002332)),
(40000000000001381,Array(97000000000008002)),
(84000000000006934,Array(21000000000002332)),
(78000000000008000,Array(36000000000009826, 44000000000004587,
96000000000009876)), (64000000000003255,Array(38000000000001282)),
(46000000000007691,Array(77000000000008322)),
(12000000000002345,Array(21000000000002332)),
(56000000000001000,Array(23000000000002371, 54000000000002771)),
(34000000000009887,Array(21000000000002332)),
(98000000000006989,Array(64000000000003021))...
我尝试了lookup()操作,直接报错:key值太大!具体见下面的代码演示:
neighbors.lookup(44000000000001232)
<console>:1: error: integer number too large
直接定义一个变量,赋值44000000000001232,也报错说integer number too large
val id = 78000000000008000
<console>:1: error: integer number too large
val id = 78000000000008000
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
Scala、Java中表示Long数的时候后面一定要加L
代码改成: