java无法解压缩snappy文件,即使在指定了Djava之后也是如此。图书馆路径=/*/*/
线程“main”中出现异常
java.lang.RuntimeException: native snappy library not available: this version of libhadoop was built without snappy support.
at org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:65)
at org.apache.hadoop.io.compress.SnappyCodec.getDecompressorType(SnappyCodec.java:193)
at org.apache.hadoop.io.compress.CodecPool.getDecompressor(CodecPool.java:178)
at org.apache.hadoop.io.compress.CompressionCodec$Util.createInputStreamWithCodecPool(CompressionCodec.java:157)
at org.apache.hadoop.io.compress.SnappyCodec.createInputStream(SnappyCodec.java:164)
我可以看到snappy库安装正确。我提到了使用-Djava的lib路径。图书馆路径,仍然导致相同的错误
有什么建议吗
hadoop checknative -a
本机库检查:
hadoop: true /opt/cloudera/parcels/CDH-5.7.1-1.cdh5.7.1.p0.11/lib/hadoop/lib/native/libhadoop.so.1.0.0
zlib: true /lib64/libz.so.1
snappy: true /opt/cloudera/parcels/CDH-5.7.1-1.cdh5.7.1.p0.11/lib/hadoop/lib/native/libsnappy.so.1
lz4: true revision:10301
bzip2: true /lib64/libbz2.so.1
openssl: true /usr/lib64/libcrypto.so
# 1 楼答案
看起来您遇到了:
如果找不到libhadoop,Snappy可能会抱怨Hadoop是在没有Snappy的情况下构建的
https://issues.apache.org/jira/browse/HADOOP-13684
您应该调试libhadoop。因此,请先看看您的LD_库路径
# 2 楼答案
问题解决了
我在类路径中使用了不正确版本的hadoop core jar文件
修改为使用当前CDH版本的hadoop core jar后,能够执行snappy压缩和解压缩