下載
java(jdk-8u111-linux-x64.rpm)
Hive2.1.1(apache-hive-2.1.1-bin.tar.gz jdk-8u111-linux-x64.rpm)
Hadoop2.7.3(hadoop-2.7.3.tar.gz)
下載Java(JDK)
點擊打開鏈接
下載安裝在官網可下載最新版(Hadoop/Hive)點擊打開鏈接
把下載文件存放在CentOS桌面文件(Hadoop)
[root@localhost Hahoop]# yuminstall -y jdk-8u111-linux-x64.rpm查看安裝后的版本[root@localhost Hahoop]# java -versionjava version "1.8.0_111"
Java(TM) SE Runtime Environment (build1.8.0_111-b14)
Java HotSpot(TM) 64-Bit Server VM(build 25.111-b14, mixed mode)
2、 解壓hadoop和hive程序
[root@localhost Hahoop]# tar -xzfhadoop-2.7.3.tar.gz[root@localhost Hahoop]# tar -xzfapache-hive-2.1.1-bin.tar.gz查看解壓文件夾[root@localhost Hahoop]# lsapache-hive-2.1.1-bin hadoop-2.7.3 jdk-8u111-linux-x64.rpmapache-hive-2.1.1-bin.tar.gz hadoop-2.7.3.tar.gz
3、 移動解壓文件并重命名hive/Hadoop
[root@localhost Hahoop]# mvhadoop-2.7.3 /usr/Hadoop[root@localhost Hahoop]# mvapache-hive-2.1.1-bin /usr/hive4、 配置環境變量(HADOOP)
[root@localhost hadoop]# vim~/.bashrc添加:
# set hadoop/hive/jdk(java) pathexport HADOOP_HOME=/usr/hadoopexport HIVE_HOME=/usr/hiveexport JAVA_HOME=/usr/java/jdk1.8.0_111export PATH="$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$HIVE_HOME/bin:$JAVA_HOME/bin"生效:[root@localhost hadoop]# source~/.bashrc5、 創建目錄Hadoop
[root@localhost hadoop]# cd /usr/Hadoop[root@localhost hadoop]# mkdir tmp[root@localhost hadoop]# mkdir hdfs[root@localhost hadoop]# mkdir hdfs/data[root@localhost hadoop]# mkdir hdfs/name6、 設置配置文件
5.0、指定文件所在路徑
[root@localhost hadoop]# cd/usr/hadoop/etc/Hadoop5.1、配置hadoop-env.sh]
[root@localhost hadoop]# vimhadoop-env.sh--添加#export JAVA_HOME=${JAVA_HOME}export JAVA_HOME=/usr/java/jdk1.8.0_1115.2、配置yarn-env.sh
[root@localhost hadoop]# vim yarn-env.sh添加# export JAVA_HOME=/home/y/libexec/jdk1.6.0/exportJAVA_HOME=/usr/java/jdk1.8.0_1115.3、配置core-site.xml
[root@localhost hadoop]# vimcore-site.xml<configuration> <PRoperty> <name>fs.default.name</name> <value>hdfs://localhost:9000</value> <description>HSDF的URL,文件系統:namenode標識:端口號</description> </property> <property> <name>hadoop.tmp.dir</name> <value>/usr/hadoop/tmp</value> <description>本地hadoop臨時文件夾</description> </property></configuration>5.4、配置hdfs-site.xml
[root@localhost hadoop]# vim hdfs-site.xml<configuration> <property> <name>dfs.name.dir</name> <value>/usr/hadoop/hdfs/name</value> <description>namenode上存儲hdfs名字空間元數據</description> </property> <property> <name>dfs.data.dir</name> <value>/usr/hadoop/hdfs/data</value> <description>datanode上數據塊的物理存儲位置</description> </property> <!--指定HDFS副本的數量--> <property> <name>dfs.replication</name> <value>1</value> <description>副本個數,默認3應該小于datanode機器數量</description> </property></configuration>5.5、配置yarn-site.xml
[root@localhost hadoop]# vim yarn-site.xml<configuration> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property><property> <name>yarn.resourcemanager.webapp.address</name> <value>localhost:8099</value></property></configuration>5.6、配置mapred-site.xml(啟用服務時沒用到,可不用配置)
[root@localhost hadoop]# mvmapred-site.xml.template mapred-site.xml[root@localhost hadoop]# vim mapred-site.xml<configuration> <property> <name>mapreduce.framework.name</name> <value>yarn</value> <!--客戶端訪問為yarn--> </property></configuration>6、配置ssh無密碼(1.產生RSA密鑰對;2.群集內共享密鑰)啟動hadoop服務(啟動dfs/yarn時的無需密碼.注:如果home目錄沒有通過NFS共享需要其他方法共享密鑰(ssh-copy-id,復制到遠程主機調用格式:ssh-copy-id -i ~/.ssh/id_rsa.pub root@192.168.1.2)
[root@localhost hive]# ssh-keygen -t rsa-p'' -f ~/.ssh/id_rsa[root@localhost hive]# cat ~/.ssh/id_rsa.pub>> ~/.ssh/authorized_keys7、啟動Hadoop
7.1、格式化namenode
[root@localhost hadoop]# bin/hdfs namenode-format7.2、啟動hdfs(按提示輸入yes和賬號密碼,配置了(步驟6) ssh時跳過輸入密碼步驟)
[root@localhost hadoop]# sbin/start-dfs.sh7.3、啟動yarn(按提示輸入yes和賬號密碼,配置了(步驟6) ssh時跳過輸入密碼步驟)
[root@localhost hadoop]# sbin/start-yarn.sh7.4、查看進程:
[root@localhost hadoop]# jps26161 DataNode
26021 NameNode
26344 SecondaryNameNode
26890 Jps
26492 ResourceManager
26767 NodeManager
新聞熱點
疑難解答