擅长:python、mysql、java
<p>如果导入spark隐式,则可以使用美元<code>$</code>插值器创建对列的引用。此外,您还可以使用带有列的API来生成逻辑,它将是这样的</p>
<pre><code>val sourceSystem = "QWDS"
val ruleCode = "STD00003"
import spark.implicits._
val TransformationRules = spark.read.jdbc(jdbcUrl, "ADF.TransformationRules", connectionProperties)
.select("RuleCode","SourceSystem","PrimaryTable", "PrimaryColumn", "SecondaryColumn", "NewColumnName","CurrentFlag")
.where($"SourceSystem" === sourceSystem && $"RuleCode" === ruleCode)
val ssColumn: Column = $"SourceSystem"
</code></pre>
<p>正如您所看到的,美元将提供一个列对象,具有合作、强制转换重命名等逻辑。与<code>org.apache.spark.sql.function</code>中的函数相结合,将允许您实现几乎所有需要的功能</p>