13518219792
今天就跟大家聊聊有关Mac开发中Unable to load native-hadoop library问题该怎么解决，可能很多人都不太了解，为了让大家更加了解，小编给大家总结了以下内容，希望大家根据这篇文章可以有所收获。
专注于为中小企业提供成都网站设计、做网站、成都外贸网站建设公司服务,电脑端+手机端+微信端的三站合一,更高效的管理,为中小企业牟定免费做网站提供优质的服务。我们立足成都，凝聚了一批互联网行业人才，有力地推动了近1000家企业的稳健成长，帮助中小企业通过网站建设实现规模扩充和转变。
在使用hhbase读取使用了snappy压缩的hfile时，报错如下：
java.lang.UnsatisfiedLinkError: org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z
再往上找log中还会有这样的一行提示：
Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
分析
这个问题呢，需要我们自己编译mac平台的hadoop native，但是编译了好些版本hadoop都会遇到cmake的问题:
hadoop-yarn-server-nodemanager: make failed，这个通过YARN-8622 patch可以解决
hadoop-mapreduce-client-nativetask: make failed，这个没找到解决办法
所以最终编译了hadoop 2.8.5 ，hadoop 3.1.1，hadoop 3.2.1，hadoop 3.2.1都有类似的问题，有的版本第一个问题已经解决，第二个问题都会碰到，只好放弃编译了
还好有开源，在github上热搜索hadoop native mac，真的有人共享出来，然后替换到本机${hadoop_home}/lib/native 还有一些配置如下：
$ vim .zshrc
export HADOOP_HOME=/Users/jiazz/devEnvs/hadoop-3.1.1
export HADOOP_PREFIX=/Users/jiazz/devEnvs/hadoop-3.1.1
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
export HADOOP_OPTS="-Djava.library.path=${HADOOP_HOME}/lib/native"
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${HADOOP_HOME}/lib/native
export JAVA_LIBRARY_PATH=$JAVA_LIBRARY_PATH:${HADOOP_HOME}/lib/native
$ source .zshrc是否成功其实可以通过命令检测一下：
这里需要注意一下，mac会检查你替换的文件来源，需要在mac电脑：设置->安全性与隐私->通用->允许打开
# 修复之前 $ hadoop checknative -a WARNING: HADOOP_PREFIX has been replaced by HADOOP_HOME. Using value of HADOOP_PREFIX. 2020-11-03 13:28:45,307 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable Native library checking: hadoop: false zlib: false zstd : false snappy: false lz4: false bzip2: false openssl: false ISA-L: false # 修复之后 $ hadoop checknative -a WARNING: HADOOP_PREFIX has been replaced by HADOOP_HOME. Using value of HADOOP_PREFIX. 2020-11-03 13:28:59,672 WARN bzip2.Bzip2Factory: Failed to load/initialize native-bzip2 library system-native, will use pure-Java version 2020-11-03 13:28:59,677 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library 2020-11-03 13:28:59,967 WARN erasurecode.ErasureCodeNative: Loading ISA-L failed: Failed to load libisal.2.dylib (dlopen(libisal.2.dylib, 9): image not found) 2020-11-03 13:28:59,967 WARN erasurecode.ErasureCodeNative: ISA-L support is not available in your platform... using builtin-java codec where applicable Native library checking: hadoop: true /Users/jiazz/devEnvs/hadoop-3.1.1/lib/native/libhadoop.dylib zlib: true /usr/lib/libz.1.dylib zstd : true /usr/local/Cellar/zstd/1.4.5/lib/libzstd.1.4.5.dylib snappy: true /usr/local/Cellar/snappy/1.1.8/lib/libsnappy.1.1.8.dylib lz4: true revision:10301 bzip2: false openssl: false EVP_CIPHER_CTX_reset ISA-L: false Loading ISA-L failed: Failed to load libisal.2.dylib (dlopen(libisal.2.dylib, 9): image not found) 2020-11-03 13:28:59,995 INFO util.ExitUtil: Exiting with status 1: ExitException
再次在idea中运行程序，问题解决
经验证如果是java -jar在服务器上跑还需要加参数-Djava.library.path=$HADOOP_HOME/lib/native
看完上述内容，你们对Mac开发中Unable to load native-hadoop library问题该怎么解决有进一步的了解吗？如果还想了解更多知识或者相关内容，请关注创新互联行业资讯频道，感谢大家的支持。
