Why Does Hadoop Show a 'Native-Hadoop Library' Warning, and How Can I Fix It?

Hadoop "Unable to Load Native-Hadoop Library" Warning

When configuring Hadoop 2.2.0 on a CentOS server, users may encounter the following warning upon running start-dfs.sh or stop-dfs.sh:

WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

This error stems from the native Hadoop library being compiled for a different platform (32-bit) than the current environment (64-bit CentOS).

Environment Variables

The issue persists despite adding the following environment variables to hadoop-env.sh:

export HADOOP_OPTS="$HADOOP_OPTS -Djava.library.path=/usr/local/hadoop/lib/"
export HADOOP_COMMON_LIB_NATIVE_DIR="/usr/local/hadoop/lib/native/"

Solution

While the warning does not affect Hadoop's functionality, it can be eliminated by recompiling libhadoop.so.1.0.0 on a 64-bit system. Using the Hadoop source code, follow these steps:

1. Download the Hadoop source code.
2. Recompile libhadoop.so.1.0.0 on a 64-bit system.
3. Replace the existing 32-bit libhadoop.so.1.0.0 with the recompiled version.

Specific instructions for recompiling on Ubuntu can be found at:

• http://www.ercoppa.org/Linux-Compile-Hadoop-220-fix-Unable-to-load-native-hadoop-library.htm

The above is the detailed content of Why Does Hadoop Show a 'Native-Hadoop Library' Warning, and How Can I Fix It?. For more information, please follow other related articles on the PHP Chinese website!