13518219792
这期内容当中小编将会给大家带来有关如何实现Ambari大数据平台集群利器的探索与实践，文章内容丰富且以专业的角度为大家分析和叙述，阅读完这篇文章希望大家可以有所收获。
创新互联建站-专业网站定制、快速模板网站建设、高性价比阳明网站开发、企业建站全套包干低至880元,成熟完善的模板库,直接使用。一站式阳明网站制作公司更省心,省钱,快速模板网站建设找我们，业务覆盖阳明地区。费用合理售后完善，十多年实体公司更值得信赖。
Ambari是 Apache Software Foundation 中的一个顶级项目，它是用来创建、管理、监视Hadoop整个生态圈的工具。 Ambari是分布式架构，主要由Ambari Server和Ambari Agent组成。
HDP是hortonworks的软件栈，里面包含了hadoop生态系统的所有软件项目，其实就是软件包合集。
HDP-UTILS是工具类库。各软件版本对照表：
https://supportmatrix.hortonworks.com/
hadoop生态圈示图：
[root@ambari-server01 ~]# wget http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.1.0/ambari-2.7.1.0-centos7.tar.gz [root@ambari-server01 ~]# wget http://public-repo-1.hortonworks.com/HDP/centos7/3.x/updates/3.0.1.0/HDP-3.0.1.0-centos7-rpm.tar.gz [root@ambari-server01 ~]# wget http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7/HDP-UTILS-1.1.0.22-centos7.tar.gz
#!/bin/bash # #服务器初始化脚本 # 关闭防火墙 systemctl stop firewalld systemctl disable firewalld # 关闭 SeLinux setenforce 0 sed -i "s/SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config # 关闭 swap swapoff -a sed -i 's/.*swap.*/#&/' /etc/fstab # echo "1" >/proc/sys/net/bridge/bridge-nf-call-iptables # 安装必要的调试程序 yum install -y vim lrzsz telnet net-tools tar wget yum install -y java-1.8.0-openjdk* # 安装配置时间同步服务 yum install -y ntp systemctl start ntpd.service systemctl enable ntpd.service # 设置进程文件最大打开数量 echo "* soft nofile 65535" >> /etc/security/limits.conf echo "* hard nofile 65535" >> /etc/security/limits.conf # 设置系统总限制文件最大打开数量 echo 6553560 > /proc/sys/fs/file-max echo "fs.file-max = 6553560" >> /etc/sysctl.conf
# 三台主机分别设置hostname [root@ambari-server01 ~]# hostnamectl set-hostname ambari-server01.test.com [root@ambari-node01 ~]# hostnamectl set-hostname ambari-node01.test.com [root@ambari-node02 ~]# hostnamectl set-hostname ambari-node02.test.com # 添加hosts解析（所有节点） [root@ambari-server01 ~]# cat >> /etc/hosts << EOF 192.168.2.111 ambari-server01.test.com 192.168.2.112 ambari-node01.test.com 192.168.2.113 ambari-node02.test.com EOF
# 在192.168.2.111节点执行： [root@ambari-server01 ~]# ssh-keygen [root@ambari-server01 ~]# ssh-copy-id ambari-server01.test.com [root@ambari-server01 ~]# ssh-copy-id ambari-node01.test.com [root@ambari-server01 ~]# ssh-copy-id ambari-node02.test.com [root@ambari-server01 ~]# scp -r .ssh/ ambari-node01.test.com:/root [root@ambari-server01 ~]# scp -r .ssh/ ambari-node02.test.com:/root
# 下载并安装mysql
[root@ambari-server01 ~]# wget -i -c http://dev.mysql.com/get/mysql-community-release-el7-5.noarch.rpm
[root@ambari-server01 ~]# yum -y install mysql-community-release-el7-5.noarch.rpm
[root@ambari-server01 ~]# yum -y install mysql-community-server
# 启动mysql
[root@ambari-server01 ~]# systemctl start mysqld.service # 启动mysql
[root@ambari-server01 ~]# systemctl status mysqld.service # 查看mysql状态
[root@ambari-server01 ~]# systemctl enable mysqld.service # 开机自启
# 设置mysql密码
> set password for root@localhost = password('root123'); # 方法一，需要登陆mysql执行
# 创建ambari数据库及用户名和密码
mysql> create database ambari character set utf8;
mysql> CREATE USER 'ambari'@'%'IDENTIFIED BY 'Ambari123';
mysql> GRANT ALL PRIVILEGES ON ambari.* TO 'ambari'@'%';
mysql> GRANT ALL PRIVILEGES ON ambari.* TO 'ambari'@'ambari-server01.test.com'IDENTIFIED BY 'Ambari123';
mysql> FLUSH PRIVILEGES;
# 创建hive数据库及用户名和密码（如果不需要安装hive服务，则不需要配置以下数据库）
mysql> create database hive character set utf8;
mysql> CREATE USER 'hive'@'%'IDENTIFIED BY 'Hive123';
mysql> GRANT ALL PRIVILEGES ON hive.* TO 'hive'@'%';
mysql> GRANT ALL PRIVILEGES ON hive.* TO 'ambari'@'ambari-server01.test.com'IDENTIFIED BY 'Hive123';
mysql> FLUSH PRIVILEGES;
# 如果需要安装oozie、ranger服务，创建数据库方法同上。
# 下载 mysql-connector-java-5.1.46-bin.jar包
# 复制到/var/lib/ambari-server/resources/mysql-connector-java-5.1.46-bin.jar目录；
# 再复制到/usr/share/java/mysql-connector-java-5.1.46-bin.jar到这个目录一份。# 安装软件 [root@ambari-server01 ~]# yum -y install yum-utils createrepo httpd # 创建目录 [root@ambari-server01 ~]# mkdir -p /var/www/html/ambari/ [root@ambari-server01 ~]# mkdir -p /var/www/html/hdp/ [root@ambari-server01 ~]# mkdir -p /var/www/html/hdp/HDP-UTILS/ # 解压tar包 [root@ambari-server01 ~]# tar -zxvf ambari-2.7.1.0-centos7.tar.gz -C /var/www/html/ambari/ [root@ambari-server01 ~]# tar -zxvf HDP-3.0.1.0-centos7-rpm.tar.gz -C /var/www/html/hdp/ [root@ambari-server01 ~]# tar -zxvf HDP-UTILS-1.1.0.22-centos7.tar.gz -C /var/www/html/hdp/HDP-UTILS/ # 启动httpd [root@ambari-server01 ~]# systemctl start httpd [root@ambari-server01 ~]# systemctl enable httpd
**浏览器查看，如下即为正常**  # 配置本地repo [root@ambari-server01 ~]# wget -O /etc/yum.repos.d/ambari.repo http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.7.1.0/ambari.repo # 将下载的repo修改内容为如下 [root@ambari-server01 ~]# vim /etc/yum.repos.d/ambari.repo #VERSION_NUMBER=2.7.1.0-139 [ambari-2.7.1.0] #json.url = http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json name=ambari Version - ambari-2.7.1.0 baseurl=http://192.168.2.111/ambari/ambari/centos7/2.7.1.0-169 gpgcheck=1 gpgkey=http://192.168.2.111/ambari/ambari/centos7/2.7.1.0-169/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins enabled=1 priority=1 # 配置HDP和HDP-UTILS的repo [root@ambari-server01 ~]# vim /etc/yum.repos.d/HDP.repo #VERSION_NUMBER=3.0.1.0-187 [HDP-3.0.1.0] name=HDP Version - HDP-3.0.1.0 baseurl=http://192.168.2.111/hdp/HDP/centos7 gpgcheck=1 gpgkey=http://192.168.2.111/hdp/HDP/centos7/3.0.1.0-187/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins enabled=1 priority=1 [HDP-UTILS-1.1.0.22] name=HDP-UTILS Version - HDP-UTILS-1.1.0.22 baseurl=http://192.168.2.111/hdp/HDP-UTILS/centos7/1.1.0.22 gpgcheck=1 gpgkey=http://192.168.2.111/hdp/HDP-UTILS/centos7/1.1.0.22/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins enabled=1 priority=1 # 将repo拷贝到其他节点 [root@ambari-server01 ~]# cd /etc/yum.repos.d/ [root@ambari-server01 ~]# scp ambari.repo HDP.repo root@192.168.2.112:/etc/yum.repos.d/ [root@ambari-server01 ~]# scp ambari.repo HDP.repo root@192.168.2.113:/etc/yum.repos.d/ # 生成本地源 [root@ambari-server01 ~]# createrepo /var/www/html/hdp/HDP/centos7/ [root@ambari-server01 ~]# createrepo /var/www/html/hdp/HDP-UTILS/
[root@ambari-server01 ~]# yum -y install ambari-server [root@ambari-server01 ~]# ambari-server setup Using python /usr/bin/python Setup ambari-server Checking SELinux... SELinux status is 'disabled' Customize user account for ambari-server daemon [y/n] (n)? y Enter user account for ambari-server daemon (root):root # 用户 Adjusting ambari-server permissions and ownership... Checking firewall status... Checking JDK... [1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8 [2] Custom JDK ============================================================================== Enter choice (1): 2 # 选择自定义jdk WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts. WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts. Path to JAVA_HOME: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.262.b10-0.el7_8.x86_64/ #jdk安装路径 Validating JDK on Ambari Server...done. Check JDK version for Ambari Server... JDK version found: 8 Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server. Checking GPL software agreement... GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html Enable Ambari Server to download and install GPL Licensed LZO packages [y/n] (n)? y Completing setup... Configuring database... Enter advanced database configuration [y/n] (n)? y Configuring database... ============================================================================== Choose one of the following options: [1] - PostgreSQL (Embedded) [2] - Oracle [3] - MySQL / MariaDB [4] - PostgreSQL [5] - Microsoft SQL Server (Tech Preview) [6] - SQL Anywhere [7] - BDB ============================================================================== Enter choice (1): 3 # 选择安装的mysql Hostname (localhost): ambari-server01.test.com # 配置hostname Port (3306): # 默认 Database name (ambari): # 默认 Username (ambari): # 默认 Enter Database Password (bigdata): # 输入密码 Re-enter password: Configuring ambari database... Should ambari use existing default jdbc /usr/share/java/mysql-connector-java.jar [y/n] (y)? y Configuring remote database connection properties... WARNING: Before starting Ambari Server, you must run the following DDL directly from the database shell to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql # 此处需注意，启动ambari之前需要执行此句 Proceed with configuring remote database connection properties [y/n] (y)? y Extracting system views... ambari-admin-2.7.3.0.139.jar .... Ambari repo file contains latest json url http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json, updating stacks repoinfos with it... Adjusting ambari-server permissions and ownership... Ambari Server 'setup' completed successfully. # 安装成功
[root@ambari-server01 ~]# mysql -u ambari -pAmbari123 -h ambari-server01.test.com mysql> source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql; mysql> show tables;
[root@ambari-server01 ~]# ambari-server start
[root@ambari-node01 ~]# yum -y install ambari-agent
访问地址：http://192.168.2.111:8080/ 默认账号：admin 默认密码：admin
进行web页面配置时没有截图，以下截图为网上下载，仅供参考
# 执行以下命令（ambari-server01节点） [root@ambari-server01 ~]# ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java-5.1.46-bin.jar
等待启动完毕，不用担心警告，后期可以调整，搭建完成，可以在展示页面进行查看集群状态
可以查看监控界面，可以看到大数据组件中出现错误，单个组件点开处理
至此，搭建完成！
上述就是小编为大家分享的如何实现Ambari大数据平台集群利器的探索与实践了，如果刚好有类似的疑惑，不妨参照上述分析进行理解。如果想知道更多相关知识，欢迎关注创新互联行业资讯频道。
