擅长:python、mysql、java
<p>你需要在stacktrace中输入一些错误,以便社区能够快速回答你的问题。
为了在Scala代码中运行Python脚本(这就是我的假设),可以通过以下方式实现:</p>
<p><strong>示例</strong>:</p>
<p><strong>Python文件:将输入数据转换为大写的代码</p>
<pre><code>#!/usr/bin/python
import sys
for line in sys.stdin:
print line.upper()
</code></pre>
<p><strong>火花代码:用于管道数据</strong></p>
<pre><code>import org.apaches.spark.{SparkConf, SparkContext}
val conf = new SparkConf().setAppName("Pipe")
val sc = new SparkContext(conf)
val distScript = "/path/on/driver/PipeScript.py"
val distScriptName = "PipeScript.py"
sc.addFile(distScript)
val ipData = sc.parallelize(List("asd","xyz","zxcz","sdfsfd","Ssdfd","Sdfsf"))
val opData = ipData.pipe(SparkFiles.get(distScriptName))
opData.foreach(println)
</code></pre>