<p>我创建了一个示例JSON数据集来匹配该模式:</p>
<pre><code>{"ClientNum":"abc123","Filters":[{"Op":"foo","Type":"bar","Val":"baz"}]}
select(s.col("ClientNum"),s.col("Filters").cast(StringType)).show(false)
+---------+------------------------------------------------------------------+
|ClientNum|Filters |
+---------+------------------------------------------------------------------+
|abc123 |org.apache.spark.sql.catalyst.expressions.UnsafeArrayData@60fca57e|
+---------+------------------------------------------------------------------+
</code></pre>
<p>最好使用explode()函数来解决您的问题,该函数先展平一个数组,然后使用star expand表示法:</p>
<pre><code>s.selectExpr("explode(Filters) AS structCol").selectExpr("structCol.*").show()
+---+----+---+
| Op|Type|Val|
+---+----+---+
|foo| bar|baz|
+---+----+---+
</code></pre>
<p>要使其成为由逗号分隔的单列字符串,请执行以下操作:</p>
<pre><code>s.selectExpr("explode(Filters) AS structCol").select(F.expr("concat_ws(',', structCol.*)").alias("single_col")).show()
+-----------+
| single_col|
+-----------+
|foo,bar,baz|
+-----------+
</code></pre>
<p>分解数组引用:<a href="https://stackoverflow.com/questions/32906613/flattening-rows-in-spark">Flattening Rows in Spark</a></p>
<p>“struct”类型的星扩展引用:<a href="https://stackoverflow.com/questions/38753898/how-to-flatten-a-struct-in-a-spark-dataframe#38754012">How to flatten a struct in a spark dataframe?</a></p>