DeepOps小笔记-ubuntu-18.04.3环境下hadoop3.1.2的安装与配置

参考文档:

https://hadoop.apache.org/docs/r3.1.2/hadoop-project-dist/hadoop-common/SingleCluster.html

集群安装参考文档:

https://hadoop.apache.org/docs/r3.1.2/hadoop-project-dist/hadoop-common/ClusterSetup.html

主要内容

  • 环境及相关软件安装
  • 设置hadoop

1 环境及相关软件安装

1.1 安装java:

export JAVA_HOME=/home/deepops/envs/java/jdk
export JRE_HOME=/home/deepops/envs/java/jdk/jre
export CLASSPATH=.:$JAVA_HOME/lib:$JRE_HOME/lib
export PATH=$JAVA_HOME/bin:$PATH  

1.2 安装ssh软件

$ sudo apt-get install ssh   
$ sudo apt-get install pdsh 

2 设置hadoop

2.1设置JAVA_HOME

打开hadoop安装目录下etc/hadoop/hadoop-env.sh ,找到JAVA_HOME接触注释,病将自己的java安装路径填写

# set to the root of your Java installation   
export JAVA_HOME= /home/deepops/envs/java/jdk 

然后运行 bin/hadoop 能够出来hadoop的用法就ok了

2.2 运行官方教程中的示例

#在hadoop目录下
$ mkdir input
$ cp etc/hadoop/*.xml input
$ bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-3.1.2.jar grep input output 'dfs[a-z.]+'
$ cat output/*

到此hadoop单节点部署完毕。收工

Leave a Comment

Your email address will not be published.