Spark two RDD加入ISU

Question

我有两个RDD。你知道吗

moviesRDD =[(1,'monster'),(2,'minions 3D'),...] #(movieID,title)
ratingsRDD =[(1,(3,4)),(2,(4,5)),.....]  #(movieID,(numbersofrating,avg_rating))

理想的结果是：

newRDD =[(3,'monster',4),(4,'minions 3D',5),....] #(numbersofrating,title,avg_rating)

我不知道如何得到新的RDD。你知道吗

Answer 1

这应该可以做到：

(moviesRDD
    .join(ratingsRDD) # Join by key
    .values() # Extract values
    .map(lambda x: (x[1][0], x[0], x[1][1]))) # Reshape