擅长:python、mysql、java
<p>我想reduceByKey会有帮助的。在</p>
<p>请记住,当使用reduceByKey或任何其他“按键”转换或操作时,可以将键定义为元组。在</p>
<pre><code>rdd = sc.parallelize([(2,10482422,0.18),
(2,10482422,0.4),
(2,10482423,0.15),
(2,10482423,0.43),
(2,10482424,0.18),
(2,10482424,0.49),
(2,10482425,0.21),
(2,10482425,0.52),
(2,10482426,0.27),
(2,10482426,0.64),
(2,10482427,0.73)])
print rdd.map(lambda x: ((x[0], x[1]), x[2]))\
.reduceByKey(lambda x, y: x if x >= y else y)\
.collect()
</code></pre>
<p>结果:</p>
^{pr2}$