Python hdfs3连接问题

2024-06-03 16:49:52 发布

您现在位置:Python中文网/ 问答频道 /正文

我在尝试使用python hdfs3库以特权用户的身份连接到HA kerberized HDFS集群时收到not allowed to impersonate错误。你知道吗

import hdfs3

fs = hdfs3.HDFileSystem(ticket_cache='/tmp/krb5cc_12345')

2019-08-30 00:57:12.730173, p34778, th140163658831680, ERROR Failed to invoke RPC call "getFsStats" on server "namenodehost:namenodeport":
RpcChannel.cpp: 483: HdfsRpcException: Failed to invoke RPC call "getFsStats" on server "namenodehost:namenodeport"
...
Caused by
RpcChannel.cpp: 931: HdfsRpcServerException: org.apache.hadoop.security.authorize.AuthorizationException: User: myuser@domain.com is not allowed to impersonate 1�H�M�dH3
...
2019-08-30 00:57:12.730965, p34778, th140163658831680, INFO Retry idempotent RPC call "getFsStats" on server "namenodehost:namenodeport"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/miniconda3/envs/dev/lib/python3.7/site-packages/hdfs3/core.py", line 76, in __init__
    self.connect()
  File "/miniconda3/envs/dev/lib/python3.7/site-packages/hdfs3/core.py", line 141, in connect
    raise ConnectionError('Connection Failed: {}'.format(msg))
ConnectionError: Connection Failed: HdfsRpcException: Failed to invoke RPC call "getFsStats" on server "namenodehost:namenodeport"  Caused by: HdfsRpcServerException: org.apache.hadoop.security.authorize.AuthorizationException: User: myuser@domain.com is not allowed to impersonate 1�H�M�dH3

我还尝试通过自定义的confdict覆盖从hdfs xml文件读取的默认值,如hdfs3HA文档中建议的那样:

conf = {'myhdfskeys': 'myhdfsvals'}
fs = hdfs3.HDFileSystem(pars=conf)

但这产生了与上面所示相同的not allowed to impersonate错误。你知道吗

环境信息:

os: linux
python: 3.7.4
hdfs3: 0.3.0
libhdfs3: 2.3.0

Tags: toserveronnotrpccallfileallowed