下面看似简单的代码抛出以下错误:
Traceback (most recent call last):
File "/home/nirmal/process.py", line 165, in <module>
'time_diff': f.last(adf['time_diff']).over(window_device_rows)
TypeError: __call__() got an unexpected keyword argument 'this_campaign'
# Function to flag network timeouts
def flag_network_timeout(**kwargs):
if kwargs['this_network'] != kwargs['last_network'] \
or kwargs['this_campaign'] != kwargs['last_campaign'] \
or kwargs['this_adgroup'] != kwargs['last_adgroup'] \
or kwargs['this_creative'] != kwargs['last_creative'] \
or kwargs['time_diff'] > network_timeout:
return 1
else:
return 0
flag_network_timeout = f.udf(flag_network_timeout, IntegerType())
# Column spec to go over the device events and flag network resets
network_timeout_flag = flag_network_timeout(**{
'last_network': f.first(adf['network']).over(window_device_rows),
'last_campaign': f.first(adf['campaign']).over(window_device_rows),
'last_adgroup': f.first(adf['adgroup']).over(window_device_rows),
'last_creative': f.first(adf['creative']).over(window_device_rows),
'this_network': f.last(adf['network']).over(window_device_rows),
'this_campaign': f.last(adf['campaign']).over(window_device_rows),
'this_adgroup': f.last(adf['adgroup']).over(window_device_rows),
'this_creative': f.last(adf['creative']).over(window_device_rows),
'time_diff': f.last(adf['time_diff']).over(window_device_rows)
})
# Update dataframe with the new columns
adf = adf.select('*', network_timeout_flag.alias('network_timeout'))
请问我做错什么了?谢谢您。
出现异常是因为^{} 只支持varargs而不支持关键字args。
在更基本的级别上,UDF只能接收
Column
参数,这些参数将在运行时扩展到相应的值,而不是标准的Python对象。就我个人而言,我根本不会使用
**kwargs
来实现这一点,但忽略了通过组合SQL表达式可以实现所需的功能:相关问题 更多 >
编程相关推荐