cdh大數(shù)據(jù)實(shí)施方案

信息中心系統(tǒng)工程CDH實(shí)施方案修訂歷史版本版本日期編制人審核人批準(zhǔn)人修訂說(shuō)明V0.12023-5-30形成初稿44/73名目\l“_TOC_250052“Cdh部署實(shí)施 6\l“_TOC_250051“安裝環(huán)境預(yù)備 6\l“_TOC_250050“Cloudera安裝包下載 6\l“_TOC_250049“Cdh安裝前系統(tǒng)配置 7\l“_TOC_250048“上傳centos7.6鏡像及cdh所需安裝包 7\l“_TOC_250047“Centos離線源配置 8\l“_TOC_250046“配置cloudera離線源 9\l“_TOC_250045“配置靜態(tài)IP地址〔全部節(jié)點(diǎn)〕 9\l“_TOC_250044“及selinux〔全部節(jié)點(diǎn)〕 10\l“_TOC_250043“CM配置無(wú)密碼登錄到其他節(jié)點(diǎn) 11\l“_TOC_250042“編輯hosts文件〔全部節(jié)點(diǎn)〕 11\l“_TOC_250041“配置NTP效勞〔全部節(jié)點(diǎn)〕 12\l“_TOC_250040“禁用swap〔全部節(jié)點(diǎn)〕 14\l“_TOC_250039“安裝JDK〔全部節(jié)點(diǎn)〕 14\l“_TOC_250038“配置MySQL-JDBC驅(qū)動(dòng)包〔全部節(jié)點(diǎn)〕 14\l“_TOC_250037“禁用透亮頁(yè)〔全部節(jié)點(diǎn)〕 14\l“_TOC_250036“配置MySQL集群 15\l“_TOC_250035“安裝MySQL5.7 15\l“_TOC_250034“初始化數(shù)據(jù)庫(kù) 15\l“_TOC_250033“配置MySQL集群HA 16\l“_TOC_250032“安裝配置cloudera根底效勞 24\l“_TOC_250031“安裝規(guī)劃效勞器 24\l“_TOC_250030“安裝clouderamanager 24\l“_TOC_250029“配置cdh 24\l“_TOC_250028“建立cdh所需數(shù)據(jù)庫(kù) 24\l“_TOC_250027“同步數(shù)據(jù)庫(kù) 25\l“_TOC_250026“啟動(dòng)效勞 25\l“_TOC_250025“安裝cdh 26\l“_TOC_250024“登錄cdh安裝界面 26\l“_TOC_250023“在歡送界面點(diǎn)擊-連續(xù) 26\l“_TOC_250022“同意用戶許可-連續(xù) 27\l“_TOC_250021“上傳license 27\l“_TOC_250020“集群安裝歡送界面-連續(xù) 28\l“_TOC_250019“輸入集群名稱-連續(xù) 28\l“_TOC_250018“輸入主機(jī)名點(diǎn)擊-搜尋，看到全部主機(jī)后點(diǎn)擊連續(xù) 28\l“_TOC_250017“輸入配置好的離線源地址-連續(xù) 29\l“_TOC_250016“選擇安裝JDK-連續(xù) 29\l“_TOC_250015“輸入ssh后臺(tái)登錄密碼-連續(xù) 29\l“_TOC_250014“等待agent和parcels自動(dòng)安裝完成 30\l“_TOC_250013“完成后連續(xù) 30\l“_TOC_250012“選擇安裝組件-連續(xù) 30\l“_TOC_250011“選擇所需安裝包-連續(xù) 31\l“_TOC_250010“測(cè)試數(shù)據(jù)庫(kù)連接成功后-連續(xù) 31\l“_TOC_250009“依據(jù)實(shí)際狀況選擇安裝名目-連續(xù) 31\l“_TOC_250008“等待安裝完成-連續(xù) 32\l“_TOC_250007“點(diǎn)擊完成，完安裝 32\l“_TOC_250006“安裝成功界面預(yù)覽 32\l“_TOC_250005“Cloudera個(gè)組件HA配置 33\l“_TOC_250004“搭建Zookeeper集群 33\l“_TOC_250003“通過(guò)CM界面點(diǎn)擊添加效勞 33\l“_TOC_250002“選擇Zookeeper-連續(xù) 33\l“_TOC_250001“選擇安裝主機(jī) 33\l“_TOC_250000“配置Zookeeper數(shù)據(jù)名目 34等待安裝完成-連續(xù) 35點(diǎn)擊完成以完成安裝 35HDFS高可用配置 35通過(guò)主界面進(jìn)入HDFS 35點(diǎn)擊操作-啟用HDFS高可用 36自定義HDFS集群名稱 36選擇主機(jī) 36指定journalNode數(shù)據(jù)的存儲(chǔ)名目 37啟用HDFS高可用模式 37啟動(dòng)成功-完成 37查看HDFS的WebUI 38訪問(wèn)NameNode的WebUI 38查看HDFS實(shí)例是否啟動(dòng)成功 38更hive的metastore 38啟動(dòng)hive 39YARN高可用配置 40CM治理頁(yè)面點(diǎn)擊YARN 40選擇啟用高可用 40選擇主機(jī) 41等待安裝完成 41WebUI查看結(jié)果 42查看YARN主備狀態(tài) 42Hive高可用配置 43CM頁(yè)面點(diǎn)擊Hive進(jìn)入配置 43進(jìn)入配置頁(yè)面 43進(jìn)入“實(shí)例”,添加hive metastore角色實(shí)例 43選擇主機(jī) 44默認(rèn)端口-完成 44重啟hive效勞 44hiveserver2啟用負(fù)載均衡 44配置kerberos安全效勞 47Kerberos集群安裝配置 47KDC主效勞器安裝軟件包 47編輯主效勞器配置文件 47KDC備效勞器安裝配置 48初始化KDC主效勞器數(shù)據(jù)庫(kù) 48將主節(jié)點(diǎn)數(shù)據(jù)拷貝到備節(jié)點(diǎn) 49配置kerberos客戶端 50配置KDC主從同步 50在主備KDC啟動(dòng)效勞 50數(shù)據(jù)庫(kù)同步到備KDC數(shù)據(jù)庫(kù)中 51啟動(dòng)備KDC效勞 51KDC集群效勞測(cè)試 52使用kadmin.local添加治理員用戶 52在kerberos客戶端的進(jìn)展登陸操作 52查看主KDC的允許狀態(tài) 53查看備KDC的運(yùn)行狀態(tài) 53，觀看Kerberos客戶端是否可用 53創(chuàng)立Clouderamanager治理員賬號(hào) 54Cloudera集成配置kerberos 55在CM界面點(diǎn)擊治理-安全進(jìn)入配置 55啟用kerberos 55全部勾選 55配置KDC信息 56KRB5配置 56配置KDCAccountManager憑據(jù) 56導(dǎo)入KDCAccountManager憑據(jù) 57Configurekerberos默認(rèn)連續(xù) 57等待安裝完成 57Cloudera權(quán)限治理配置Sentry 58添加sentry效勞 58CM治理頁(yè)面添加效勞 58選擇sentry效勞-連續(xù) 58選擇主機(jī)-連續(xù) 59選擇數(shù)據(jù)庫(kù) 59將sentry添加到集群 59完成安裝 60Sentry設(shè)置 60Hive設(shè)置 60在配置界面去掉途中的勾選 61Hive效勞勾選sentry 61設(shè)置hdfs的把握列表和sentry同步 62Sentry高可用配置 62CM界面進(jìn)入sentry配置界面 62啟用sentry高可用 63選擇高可用主機(jī) 63完成安裝 63查看sentry是否安裝成功 64Impala負(fù)載均衡配置 65系統(tǒng)配置 65安裝haproxy 65編輯haproxy配置文件 65啟動(dòng)haproxy效勞 65CM配置 66CM界面點(diǎn)擊impala進(jìn)入配置界面 66配置LoadBalancer 66重啟impala效勞 67重啟過(guò)時(shí)效勞 67馬上重啟 67等等效勞完成重啟 68查看負(fù)載均衡配置是否成功 68測(cè)試impala-shell連接 68登錄效勞器 68登錄負(fù)載均衡地址測(cè)試 69添加主機(jī)到集群 70在CM治理界面點(diǎn)擊添加主機(jī) 70選擇集群-連續(xù) 70搜尋主機(jī) 70配置存儲(chǔ)庫(kù) 71選擇JDK 71輸入系統(tǒng)root密碼 72安裝agent 72安裝parcels 72檢查主機(jī)完成后-連續(xù) 73選擇安裝主機(jī)模板 73執(zhí)行安裝命令 73完成安裝 74Cdh部署實(shí)施由于內(nèi)網(wǎng)虛擬機(jī)不能連接互聯(lián)網(wǎng)，宿主操作系統(tǒng)承受centos7.6，本次安裝承受離線安裝方式進(jìn)展部署。安裝環(huán)境預(yù)備Cloudera安裝包下載安裝包下載地址：s://archive.cloudera/cm6/6.3.1/redhat7/yum/RPMS/x86_64/下載如下安裝包：CDH6.3.2安裝包地址為：s://archive.cloudera/cdh6/6.3.2/parcels/，下載如下包：Cdh安裝前系統(tǒng)配置cdh所需安裝包Xshellcentos7.6cdh所需安裝包到效勞器上傳centos7.6鏡像cdh安裝包Centos離線源配置centos7.6效勞其他節(jié)點(diǎn)通過(guò)協(xié)議進(jìn)展安裝離線源數(shù)據(jù)包。[root@cdh01~]#yuminstalld–y[root@cdh01~]#systemctlenabled&&systemctlstartd[root@cdh01~]#mkdir/var/www/html/centos[root@cdh01~]#mount–oloopCentos7.iso/var/www/html/centos[root@cdh01~]#mv/etc/yum.repo/*.repo/home[root@cdh01~]#vicentos.repo[centos]“:///centos/“baseurl=:///centos/gpgcheck=1“:///centos/RPM-GPG-KEY-CentOS-7“gpgkey=:///centos/RPM-GPG-“:///centos/RPM-GPG-KEY-CentOS-7“KEY-“:///centos/RPM-GPG-KEY-CentOS-7“CentOS-7centos.repo文件拷貝到其他節(jié)點(diǎn)的/etc/yum.repo/下[root@cdh01~]#shscp.sh離線源[root@cdh01~]#mkdir/var/www/html/cloudera[root@cdh01~]#mvcloudera/RPM/var/www/html/cloudera[root@cdh01~]#mvcloudera/xml/var/www/html/cloudera[root@cdh01~]#mv/var/www/html/cloudera/xml/var/www/html/cloudera/repodata[root@cdh01~]#vim/etc/yum.repos.d/cloudera.repocloudera.repo拷貝到其他節(jié)點(diǎn)配置靜態(tài)IP地址〔全部節(jié)點(diǎn)〕[root@cdh01network-scripts]#catifcfg-bond0#例如[root@cdh01network-scripts]#catifcfg-enp175s0f0[root@cdh01network-scripts]#catifcfg-enp176s0f0selinux〔全部節(jié)點(diǎn)〕[root@cdh01~]#systemctlstopfirewalld關(guān)閉防火墻[root@cdh01~]#systemctldisablefirewalld制止防火墻開機(jī)自啟[root@cdh01~]#vim/etc/selinux/config—>SELINUX=disabled(修改后重啟系統(tǒng))CM配置無(wú)密碼登錄到其他節(jié)點(diǎn)[root@cdh01~]#ssh-keygen-trsa #一路回車拷貝秘鑰文件到其他節(jié)點(diǎn)[root@cdh01~]#shscp.shhosts文件〔全部節(jié)點(diǎn)〕[root@cdh01~]#cathosts.sh[root@cdh01~]#shhosts.sh[root@cdh01~]#shscp.sh效勞〔全部節(jié)點(diǎn)〕效勞器端配置Cdh01cdh01效勞器做時(shí)鐘2.[root@cdh01cloudera]#vim/etc/ntp.confrestrict2nomodifynotrapnoquery 允許時(shí)鐘效勞器修改本地時(shí)間serverfudgestartum10 時(shí)鐘效勞器故障時(shí)以本地時(shí)間為準(zhǔn)[root@cdh01cloudera]#ntpdate2 同步時(shí)鐘[root@cdh01cloudera]#systemctlenablentpd[root@cdh01cloudera]#systemctlstartntpd[root@cdh01cloudera]#systemctlstatusntpd開啟定時(shí)同步時(shí)間[root@cdh01cloudera]#crontab–e*/5****/sbin/ntpdate–u2 5分鐘同步一次時(shí)間客戶端配置〔全部客戶節(jié)點(diǎn)〕[root@cdh03~]#vim/etc/ntp.conf 例如restrictnomodifynotrapnoquery 允許時(shí)鐘效勞器修改本地時(shí)間[root@cdh03~]#systemctlenablentpd[root@cdh03~]#ntpdate同步時(shí)間[root@cdh03~]#systemctlstartntpd[root@cdh03~]#crontab–e*/8****/sbin/ntpdate–u 每8分鐘同步一次時(shí)間[root@cdh03~]#hwclock–w 同步本地時(shí)間到BIOS禁用swap〔全部節(jié)點(diǎn)〕[root@cdh01~]#echo“vm.swappiness=1“>>/etc/sysctl.conf[root@cdh01~]#echo1>/proc/sys/vm/swappiness[root@cdh01~]#vim/etc/fstab修改配置文件后重啟系統(tǒng)生效安裝JDK〔全部節(jié)點(diǎn)〕--nogpgcheck–yJava環(huán)境變量，添加如下內(nèi)容到/etc/profile[root@cdh01~]#source/etc/profile[root@cdh01~]#java–versionMySQL-JDBC驅(qū)動(dòng)包〔全部節(jié)點(diǎn)〕[root@cdh01~]#mkdir-p/usr/share/java例如[root@cdh01~]#mv/opt/mysql-j/mysql-connector-java-5.1.34.jar/usr/share/java/[root@cdh01~]#mysql-connector-java-5.1.34.jarmysql-connector-java.jar禁用透亮頁(yè)〔全部節(jié)點(diǎn)〕[root@cdh01~]#echonever>/sys/kernel/mm/transparent_huge/defrag[root@cdh01~]#echonever>/sys/kernel/mm/transparent_huge/enabledMySQL集群主機(jī)主機(jī)IPVIPcdh02.irc00cdh26.irc6cdh02cdh26MySQLRPM包[root@manager~]#tar–xfmysql-5.7.26-1.el7.x86_64.rpm-bundle.tar解壓縮安裝包[root@manager~]#rpm–qa|grepmariadb 查詢mariadb，假設(shè)安裝將其卸載[root@manager~]#yumremovemariadb*-y 卸載mariadb包[root@manager~]#rpm–ivhmysql-community-common-5.7.26-1.el7.x86_64.rpm[root@manager~]#rpm-ivhmysql-community-libs-5.7.26-1.el7.x86_64.rpm[root@manager~]#rpm-ivhmysql-community-client-5.7.26-1.el7.x86_64.rpm[root@manager~]#rpm-ivhmysql-community-server-5.7.26-1.el7.x86_64.rpm[root@manager~]#rpm-ivhmysql-community-libs-compat-5.7.26-1.el7.x86_64.rpm初始化數(shù)據(jù)庫(kù)[root@cdh02~]#mysqld--initialize--user=mysql#mysqlmysql用戶[root@cdh02~]#cat/var/log/mysqld.log#最終一行將會(huì)有隨機(jī)生成的密碼[root@cdh02~]#systemctlenablemysqld#開機(jī)自啟動(dòng)數(shù)據(jù)庫(kù)[root@cdh02~]#systemctlstartmysqld#啟動(dòng)數(shù)據(jù)庫(kù)[root@cdh02~]#mysql–uroot–pMySQLmysql>setPASSWORD=PASSWORD(”BIIpass01”)修改初始密碼在另一臺(tái)主機(jī)dh26上配置一樣的操作HA效勞器cdh02.ircCdh26.irc

IP6

VIP00

安裝效勞MySQL-master/slaveMySQL-master/slaveCdh02配置[root@cdh02~]#vim/etc/f#編輯配置文件，添加紅框內(nèi)容開啟bin-log日志功能[root@cdh02~]#systemctlrestartmysqld#MySQL效勞使其生效[root@cdh02~]#mysql-uroot–p#登錄數(shù)據(jù)庫(kù)mysql>grantallon*.*toadmin@”%”identifiedby”BIIpass01”;#創(chuàng)立授權(quán)用戶mysql>showmasterstatus;#查看bin-log日志內(nèi)容Cdh26配置[root@cdh26~]#vim/etc/f#編輯配置文件，添加紅框內(nèi)容開啟bin-log日志功能[root@cdh26~]#systemctlrestartmysqld#MySQL效勞使其生效[root@cdh26~]#mysql-uroot–p#登錄數(shù)據(jù)庫(kù)mysql>grantallon*.*toadmin@”%”identifiedby”BIIpass01”;#創(chuàng)立授權(quán)用戶mysql>showmasterstatus;#查看bin-log日志內(nèi)容配置主主同步[root@cdh26~]#mysql-uroot–p#MySQLcdh26配置同步CHANGEMASTERTOMASTER_HOST=””,MASTER_USER=”admin”,MASTER_PASSWORD=”BIIpass01”,MASTER_LOG_FILE=”mysql-bin.000001”,MASTER_LOG_POS=435;mysql>startslave;mysql>showslavestatus\G;查看狀態(tài)，紅框都為yes表示成功[root@cdh02~]#mysql-uroot–pcdh02cdh26同步CHANGEMASTERTOMASTER_HOST=”6”,MASTER_USER=”admin”,MASTER_PASSWORD=”BIIpass01”,MASTER_LOG_FILE=”mysql-bin.000002”,MASTER_LOG_POS=145;mysql>startslave;mysql>showslavestatus\G;查看狀態(tài)，紅框都為yes表示成功驗(yàn)證數(shù)據(jù)庫(kù)是否同步1cdh02cdh庫(kù)mysql>createdatabasecdh;2cdh26查看是否同步成功，有則表示成功3cdh26cdh庫(kù)mysql>dropdatabasecdh;4cdh02cdh庫(kù)是否存在，存在表示同步失敗，不存在表示成功keepalived配置cdh02配置1、安裝keepalived所需軟件包[root@cdh02~]#yuminstallkeepalivedipvsadm–y2、修改配置文件[root@cdh02keepalived]#vimkeepalived.conf3、開機(jī)啟動(dòng)效勞[root@cdh02keepalived]#systemctlenablekeepalived4、啟動(dòng)效勞[root@cdh02keepalived]#systemctlstartkeepalived5、檢查啟動(dòng)結(jié)果，看到紅框內(nèi)容表示成功Cdh26配置1、安裝keepalived所需軟件包[root@cdh26~]#yuminstallkeepalivedipvsadm–y2、修改配置文件3、開機(jī)啟動(dòng)效勞[root@cdh26keepalived]#systemctlenablekeepalived4、啟動(dòng)效勞[root@cdh26keepalived]#systemctlstartkeepalived5VIP，當(dāng)主節(jié)點(diǎn)故障時(shí)備節(jié)點(diǎn)接收主節(jié)點(diǎn)，自動(dòng)啟動(dòng)VIP集群檢測(cè)腳本1、mysql_check.sh腳本內(nèi)容2run.sh當(dāng)效勞停頓后，通過(guò)效勞檢查腳本啟動(dòng)停頓效勞3、效勞自動(dòng)檢查配置[root@cdh02keepalived]#crontab–e*/1****sh/etc/keepalived/run.shcdh02cdh26一樣cloudera根底效勞安裝規(guī)劃效勞器效勞器效勞器IP安裝效勞cdh01.ircclouderamanagercdh02.ircCDH-1+MySQLcdh03.ircCDH-2cdh04.irc工具節(jié)點(diǎn)cdh05.ircDataNodecdh06.ircDataNodecdh07.ircDataNodecdh08.ircDataNodecdh25.irc5clouderamanagercdh26.irc6CDH-3+MySQL[root@cdh02]#yuminstallcloudera-manager-*.rpm–y[root@cdh25]#yuminstallcloudera-manager-*.rpm–y[root@cdh02]#mv/root/cloudera/parcels/*/opt/cloudera/parcel-repo#parcel包移動(dòng)到指定位置[root@cdh25]#mv/root/cloudera/parcels/*/opt/cloudera/parcel-repocdh所需數(shù)據(jù)庫(kù)MySQL[(none)]>createdatabasecmserverDEFAULTCHARSETutf8COLLATEutf8_general_ci;MySQL[(none)]>grantalloncmserver.*tocm@”%”identifiedby”123”;MySQL[(none)]>createdatabasemetastoredefaultcharsetutf8collateutf8_general_ci;MySQL[(none)]>grantallprivilegesonmetastore.*tohiveuser@”%”identifiedby”123”;MySQL[(none)]>createdatabaseamondefaultcharsetutf8collateutf8_general_ci;MySQL[(none)]>grantallprivilegesonamon.*toamon@”%”identifiedby”123”;MySQL[(none)]>createdatabasermandefaultcharsetutf8collateutf8_general_ci;MySQL[(none)]>grantallprivilegesonrman.*torman@”%”identifiedby”123”;MySQL[(none)]>createdatabaseooizedefaultcharsetutf8collateutf8_general_ci;MySQL[(none)]>grantallprivilegesonooize.*toooize@”%”identifiedby”123”;MySQL[(none)]>createdatabasehuedefaultcharsetutf8collateutf8_general_ci;MySQL[(none)]>grantallprivilegesonhue.*tohue@”%”identifiedby”123”;MySQL[(none)]>createdatabasemanagerdefaultcharsetutf8collateutf8_general_ci;MySQL[(none)]>grantallprivilegesonmanager.*tomanager@”%”identifiedby”123”;MySQL[(none)]>createdatabaseNvaAudidefaultcharsetutf8collateutf8_general_ci;MySQL[(none)]>grantallprivilegesonNvaAudi.*toNvaAudi@”%”identifiedby”123”;MySQL[(none)]>createdatabaseNvaMetdefaultcharsetutf8collateutf8_general_ci;MySQL[(none)]>grantallprivilegesonNvaMet.*toNvaMet@”%”identifiedby”123”;MySQL[(none)]>createdatabaseamdefaultcharsetutf8collateutf8_general_ci;MySQL[(none)]>grantallprivilegesonam.*toam@”%”identifiedby”123”;同步數(shù)據(jù)庫(kù)[root@cdh01~]#/opt/cloudera/cm/schema/scm_prepare_database.shmysql-h00cmservercm123啟動(dòng)效勞[root@cdh01~]#systemctlenablecloudera-scm-server[root@cdh01~]#systemctlstatuscloudera-scm-server44/73cdhcdh安裝界面在任意掃瞄器輸入登錄界面地址：://:7180進(jìn)入界面用戶名：admin密碼：admin在歡送界面點(diǎn)擊-連續(xù)44/73同意用戶許可-連續(xù)license1、選擇專業(yè)版2license-連續(xù)集群安裝歡送界面-連續(xù)輸入集群名稱-連續(xù)輸入主機(jī)名點(diǎn)擊-搜尋，看到全部主機(jī)后點(diǎn)擊連續(xù)44/73輸入配置好的離線源地址-連續(xù)JDK-連續(xù)ssh后臺(tái)登錄密碼-連續(xù)44/73parcels自動(dòng)安裝完成完成后連續(xù)選擇安裝組件-連續(xù)選擇所需安裝包-連續(xù)測(cè)試數(shù)據(jù)庫(kù)連接成功后-連續(xù)依據(jù)實(shí)際狀況選擇安裝名目-連續(xù)等待安裝完成-連續(xù)點(diǎn)擊完成，完安裝安裝成功界面預(yù)覽ClouderaHA配置Zookeeper集群CM界面點(diǎn)擊添加效勞Zookeeper-連續(xù)選擇安裝主機(jī)Zookeeper集群為奇數(shù)仲裁，所以至少需要三臺(tái)效勞器數(shù)據(jù)名目等待安裝完成-連續(xù)點(diǎn)擊完成以完成安裝HDFS高可用配置HDFS點(diǎn)擊操作-HDFS高可用集群名稱選擇主機(jī)數(shù)據(jù)的存儲(chǔ)名目留意：journalNode安裝名目不能與namenode安裝名目一樣高可用模式啟動(dòng)成功-完成實(shí)例是否啟動(dòng)成功hivemetastore首先停頓hivehue效勞，然后到hive更hiveYARN高可用配置YARN選擇啟用高可用選擇主機(jī)等待安裝完成查看結(jié)果主備狀態(tài)Hive高可用配置Hive進(jìn)入配置進(jìn)入配置頁(yè)面DBTokenStore，然后選擇org.apache.hadoop.hive.thrift.DBTokenStore，如圖進(jìn)入“實(shí)例”,添加hive metastore角色實(shí)例選擇主機(jī)默認(rèn)端口-完成hive效勞hiveserver2啟用負(fù)載均衡1、在一臺(tái)主節(jié)點(diǎn)上安裝haproxy[root@cdh04~]#yuminstallhaproxy2、進(jìn)入hive效勞添加多個(gè)hiveserver2實(shí)例3、配置haproxy[root@cdh04~]#vim/etc/haproxy/haproxy.cfg5、進(jìn)入配置，搜尋load,haproxysource進(jìn)展均衡的端口(hive-hue),不hue會(huì)查詢hive會(huì)消滅特別6、重啟hive效勞7、查看hiveserver2實(shí)例狀態(tài)正常kerberos安全效勞Kerberos集群安裝配置KDC主效勞器安裝軟件包[root@cdh01~]#yum-yinstallkrb5-serverkrb5-auth-dialogkrb5-workstationkrb5-develkrb5-libsopenldap-clients編輯主效勞器配置文件[root@cdh01~]#vim/etc/krb5.conf#Configurationsnippetsmaybeplacedinthisdirectoryaswellincludedir/etc/krb5.conf.d/[logging]default=FILE:/var/log/krb5libs.logkdc=FILE:/var/log/krb5kdc.logadmin_server=FILE:/var/log/kadmind.log[libdefaults]default_realm=IRCdns_lookup_realm=falseticket_lifetime=10drenew_lifetime=10dforwardable=truerdns=falsepkinit_anchors=/etc/pki/tls/certs/ca-bundle.crt[realms]IRC={kdc=cdh01.irc:88kdc=cdh25.irc:88admin_server=cdh01.irc:749default_domain=IRC}[domain_realm].irc=IRCirc=IRC[kdc]profile=/var/kerberos/krb5kdc/kdc.conf[root@cdh01~]#vim/var/kerberos/krb5kdc/kdc.conf[root@cdh01~]#vim/var/kerberos/krb5kdc/kadm5.aclKDC備效勞器安裝配置[root@cdh25~]#yuminstall-ykrb5-serveropenldap-clientskrb5-workstationkrb5-libs[root@cdh25~]#vim/var/kerberos/krb5kdc/kpropd.aclKDC主效勞器數(shù)據(jù)庫(kù)[root@cdh01~]#kdb5_utilcreate-rIRC–s[root@cdh01~]#kadmin.local-q“ank-randkey“mailto:host/cdh01.irc@IRC“host/cdh01.irc@IRC“[root@cdh01~]#kadmin.local-q“ank-randkey“mailto:host/cdh25.irc@IRC“host/cdh25.irc@IRC“[root@cdh01~]#kadmin.local-q“xst“mailto:host/cdh01.irc@IRC“host/cdh01.irc@IRC“[root@cdh01~]#kadmin.local-q“xst“mailto:host/cdh25.irc@IRC“host/cdh25.irc@IRC“[root@cdh01~]#klist-ket/etc/krb5.keytab將主節(jié)點(diǎn)數(shù)據(jù)拷貝到備節(jié)點(diǎn)[root@cdh01~]#scp/etc/krb5.conf5:/etc/[root@cdh01~]#scp/var/kerberos/krb5kdc/kdc.conf5:/var/kerberos/krb5kdc/[root@cdh01~]#scp/var/kerberos/krb5kdc/kadm5.acl5:/var/kerberos/krb5kdc/[root@cdh01~]#scp/var/kerberos/krb5kdc/.k5.IRC5:/var/kerberos/krb5kdc/[root@cdh01~]#scp/etc/krb5.keytab5:/etc/krb5.keytabkerberos客戶端[root@cdh03~]#yuminstall-ykrb5-workstationkrb5-devel #全部客戶端節(jié)點(diǎn)都要安裝[root@cdh01~]#for i in {3..29};do scp /etc/krb5.conf10.254.16.$i:/etc/krb5.conf;done拷貝krb5.conf到客戶端KDC主從同步KDC啟動(dòng)效勞KDC主效勞器[root@cdh01~]#systemctlstartkrb5kdc[root@cdh01~]#systemctlenablekrb5kdc[root@cdh01~]#systemctlstatuskrb5kdc[root@cdh01~]#systemctlstartkadmin[root@cdh01~]#systemctlenablekadmin[root@cdh01~]#systemctlstatuskadminKDC備效勞器[root@cdh25~]#systemctlstartkprop[root@cdh25~]#systemctlenablekprop[root@cdh25~]#systemctlstatuskpropKDC數(shù)據(jù)庫(kù)中[root@cdh01~]#kdb5_utildump/var/kerberos/krb5kdc/slave_datatrans[root@cdh01~]#kprop-f/var/kerberos/krb5kdc/slave_datatranscdh25.irc設(shè)置自動(dòng)同步腳本[root@cdh01~]#mkdir/var/kerberos/{shell,log}[root@cdh01~]#vim/var/kerberos/shell/dump_principal.sh[root@cdh01~]#chmod+x/var/kerberos/shell/dump_principal.sh[root@cdh01~]#crontab–e*/50 * * * * /bin/date >>/var/kerberos/log/dump.log 2>&1;/var/kerberos/shell/dump_principal.sh>>/var/kerberos/log/dump.log2>&1KDC效勞[root@cdh25~]#systemctlstartkrb5kdc[root@cdh25~]#systemctlenablekrb5kdc[root@cdh25~]#systemctlstatuskrb5kdcKDC集群效勞測(cè)試使用kadmin.local添加治理員用戶[root@cdh01~]#kadmin.local-q“addprincadmin“治理員賬號(hào)：admin密碼：admin[root@cdh01~]#kadmin.local-q“l(fā)istprincs“kerberos客戶端的進(jìn)展登陸操作[root@cdh08~]#kinitadmin[root@cdh