我想把星火和卡夫卡结合起来。我让kafka消费者有json数据。我想用Spark处理卡夫卡的消费者。当我运行下面的代码错误是抛出。在
bin\spark-submit --packages org.apache.spark:spark-streaming-kafka-0-8_2.11:2.0.0 test.py localhost:9092 maktest
我的测试.py在下面
^{pr2}$我得到了下面的错误
18/12/10 16:41:40 INFO VerifiableProperties: Verifying properties
18/12/10 16:41:40 INFO VerifiableProperties: Property group.id is overridden to
18/12/10 16:41:40 INFO VerifiableProperties: Property zookeeper.connect is overridden to
<pyspark.streaming.kafka.KafkaTransformedDStream object at 0x000002A6DA9FE6A0>
18/12/10 16:41:40 ERROR StreamingContext: Error starting the context, marking it as stopped
java.lang.IllegalArgumentException: requirement failed: No output operations registered, so nothing to execute
at scala.Predef$.require(Predef.scala:224)
Traceback (most recent call last):
File "C:/Users/maws/Desktop/spark-2.2.1-bin-hadoop2.7/test.py", line 12, in <module>
ssc.start()
py4j.protocol.Py4JJavaError: An error occurred while calling o25.start.
: java.lang.IllegalArgumentException: requirement failed: No output operations registered, so nothing to execute
您没有使用受支持的Spark Streaming DStream output operation。在
对于pyspark API,应该使用:
print()
不能与pyspark一起使用,因此在检查Scala或Java的其他Spark流示例时,请确保更改为pprint()
相关问题 更多 >
编程相关推荐