大數(shù)據(jù)培訓(xùn)-beh運維實操

Information

Change

t1heWorldBONC提綱—Hadoop集群架構(gòu)三Hadoop安裝部署二Hadoop日常操作與四Hadoop高可用相關(guān)Information

Change

the

WorldBONCHadoop2.3主要進程Hadoop基礎(chǔ)進程高可用相關(guān)進程NamenodeDataNodeResourceManagerNodeManagerQuorumPeerMainZKFailoverControllerJournalNodeZookeeper為ZKFC和MRZKFC提供了基礎(chǔ)服務(wù).保證了

ameNode節(jié)點服務(wù)級別的自動熱切.JournalNode保證了NameNode雙機元數(shù)據(jù)的強一致性.BONCJournalNodeJournalNodeJournalNodeJournalNodeJournalNodeNameNodeActiveNameNodeStandby寫入Active狀態(tài)的NameNode負責響應(yīng)所有的客戶端操作,并將相關(guān)操作對EditLog所做 寫入到所有的JournalNode.Standby狀況的NameNode才能 JournalNode中的變動信息,并且Edit

Log的更改應(yīng)用于本身的元數(shù)據(jù)信息.QJM的基本原理就是用2N+1臺JournalNodeEditLog,每次寫數(shù)據(jù)操作有大多數(shù)（>=N+1）返回成功時即認為該次寫成功，數(shù)據(jù)不的是最多有N臺機器掛掉，如果多于會丟失了.當然這個算法所能N臺掛掉，這個算法就失效了.QJM/Qurom

Journal

ManagerInformation

Change

t3he

WorldBONC提綱—Hadoop集群架構(gòu)三Hadoop安裝部署二Hadoop日常操作與四Hadoop高可用相關(guān)Information

Change

t4he

WorldBONC集群架構(gòu)拓撲圖hadoop001hadoop003hadoop005hadoop007hadoop002hadoop004hadoop006hadoop008hadoop009hadoop010Information

Change

t5he

WorldDataNodeNameNodeMysql交換機以10臺hadoop服務(wù)器為例來講解一下安裝實施的過程步驟.集群共有10臺服務(wù)器,分別為hadoop001至hadoop010安裝hadoop組件包括:HDFS、Mapreduce、Yarn、ZooKeeper、Hive、Sqoop等BONC集群各節(jié)點服務(wù)hadoop001和hadoop002作為NameNode互為熱備。hadoop003、hadoop004、hadoop005、hadoop006、hadoop007、hadoop008作為DataNode節(jié)點存放數(shù)據(jù)。Hive元數(shù)據(jù)庫Mysql安裝在hadoop009和hadoop010做主從

。相關(guān)ETL流程及其他應(yīng)用部署hadoop009和hadoop010。Information

Change

t6he

WorldBONC安裝準備工作修改/etc/hosts添加所有節(jié)點及相應(yīng)主機名并分發(fā).添加hadoop用戶并配置hadoop用戶之間的ssh互信（免密）.關(guān)閉 和Selinux.配置時間同步ntp服務(wù).創(chuàng)建DataNode磁盤數(shù)據(jù) 用以存放數(shù)據(jù),并修改為hadoop屬主./opt/beh 修改為hadoop屬主所有操作修改以及啟停服務(wù)在hadoop用戶下進行將 tar包beh.tar解壓至/opt

下修改相關(guān)配置文件并且分發(fā)至所有集群節(jié)點Information

Change

the

WorldInformation

Change

the

WorldBONCZooKeeper相關(guān)配置文件/opt/beh/core/zookeeper/conf/zoo.cfg#

The

number

of

milliseconds

of

each

ticktickTime=2000#

The

number

of

ticks

that

the

initial#

synchronization

phase

cantakeinitLimit=10#

The

number

of

ticks

that

can

pass

between#

sending

a

request

and

getting

an

acknowledgementsyncLimit=5#

the

directorywhere

the

snapshot

is

stored.#

do

not

use

/tmp

for

storage,

/tmp

here

is

just#

example

sakes.dataDir=/opt/beh/zookeeper/data#

the

portat

whichthe

clients

will

connectclientPort=2181server.1=hadoop001:2888:3888server.2=hadoop002:2888:3888server.3=hadoop003:2888:3888##

Be

sure

to

read

the

maintenance

section

ofthe#

administrat uide

before

turning

on

autopurge.####

The

number

o apshots

to

retain

in

dataDir#autopurge.snapRetainCount=3#

Purge

task

interval

in

hours#

Set

to

"0"

to

disable

auto

purge

feature#autopurge.purgeInterval=1配置文件中的dataDir參數(shù)對應(yīng)的各zookeeper節(jié)點的唯一性標示.在相應(yīng) 下創(chuàng)建文件myid,分別填寫相應(yīng)的邏輯id.Information

Change

the

WorldBONCHDFS配置相關(guān)文件core-site.xml<configuration><property><name>fs.defaultFS</name><value>hdfs://beh</value></property><property><name>hadoop.tmp.dir</name><value>/opt/beh/data/namenode</value></property><property><name>ha.zookeeper.quorum</name><value>hadoop001:2181,hadoop002:2181,hadoop003:2181</value></property><property><name>io.file.buffer.size</name><value>131072</value></property></configuration>/opt/beh/hadoop/etc/hadoop/core-site.xmlcore-site.xmlInformation

Change

the

WorldBONCHDFS配置相關(guān)文件hdfs-site.xml<property><name>dfs.data.dir</name><value>/data/disk1/data</value></property><property><name>dfs.block.size</name><value>134217728</value></property><name>dfs.datanode.failed.volumes.tolerated</name><value>4</value></property><property><name>dfs.replication</name><value>3</value></property>/opt/beh/core/hadoop/etc/hadoop/hdfs-site.xml/opt/beh/core/hadoop/etc/hadoop/slave填寫所有DataNode節(jié)點的主機名或者IP地址.hadoop003hadoop004hadoop005hadoop006hadoop007hadoop008hdfs-site.xmlBONCMapred配置相關(guān)文件mapred-site.xml<property><name></name><value>yarn</value></property><property><name>mapreduce.jobhistory.address</name><value>hadoop001:10020</value></property><property><name>mapreduce.jobhistory.webapp.address</name><value>hadoop001:19888</value></property><property><name>mapred.child.java.opts</name><value>-Xmx2g</value></property>/opt/beh/core/hadoop/etc/hadoop/mapred-site.xmlmapred-site.xmlInformation

Change

the

WorldBONCYARN配置相關(guān)文件yarn-site.xml<property><name>yarn.resourcemanager.cluster-id</name><value>beh</value></property><property><name>yarn.resourcemanager.ha.rm-ids</name><value>rm1,rm2</value></property><property><name>yarn.resourcemanager.ha.id</name><value>rm1</value></property>/opt/beh/core/hadoop/etc/hadoop/yarn-site.xmlarn-site.xmlInformation

Change

the

WorldBONCHbase配置相關(guān)文件hbase-site.xml<property><name>hbase.zookeeper.quorum</name><value>hadoop001,hadoop002,hadoop003</value></property><property><name>dfs.replication</name><value>3</value></property><property><name>hbase.hregion.max.filesize</name><value>2147483648</value></property>/opt/beh/core/hbase/conf/hbase-site.xml/opt/beh/core/hbase/conf/hbase-site.xml/regionservershadoop003hadoop004hadoop005hadoop006hadoop007hadoop008填寫所有需要啟動Regionserver節(jié)點的主機名或者IP地址.hbase-site.xmlInformation

Change

the

WorldBONCHive相關(guān)配置文件hive-site.xml<property><name>javax.jdo.option.ConnectionURL</name><value>jdbc:mysql://hivemysql:3306/hive?createDatabaseIfNotExist=true;char;useUnicode=true</value>acterEncoding=UTF-8</property><property><name>javax.jdo.option.ConnectionDriverName</name><value>com.mysql.jdbc.Driver</value></property><property><name>javax.jdo.option.ConnectionUserName</name><value>hadoop</value></property><property><name>javax.jdo.option.ConnectionPassword</name><value>hadoop</value></property>Information

Change

the

WorldInformation

Change

the

WorldBONC環(huán)境變量相關(guān)配置ME/bin:$SPARK_HOME/bin:$MAHOUT_HOME/bin:$PATH集群所有節(jié)點修改/etc/profile添加如下內(nèi)容exportBEH_HOME=/opt/behexportJAVA_HOME=/opt/beh/core/jdk1.7.0_55exportHBASE_HOME=/opt/beh/core/hbaseexportHIVE_HOME=/opt/beh/core/hiveexport

HADOOP_HOME=/opt/beh/core/hadoopexportMAHOUT_HOME=/opt/beh/core/mahoutexport

MON_HOME=/opt/beh/core/hadoopexportHADOOP_MAPRED_HOME=/opt/beh/core/hadoopexportSQOOP_HOME=/opt/beh/core/sqoopexportZOOKEEPER_HOME=/opt/beh/core/zookeeper

exportLD_LIBRARY_PATH=/opt/beh/core/hadoop/lib/nativeexportPUB_KEY_HOME=/home/hadoop/.ssh/id_rsaexportZOO_LOG_DIR=$BEH_HOME/logs/zookeeperexport

SCALA_HOME=/opt/beh/core/scala-2.9.3exportSPARK_HOME=/opt/beh/core/sparkexport

SPRAK_EXAMPLES_JAR=/opt/beh/core/spark/examples/lib/spark-examples_2.10-0.9.0-cdh5.0.0.jarexportCLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/jre/lib:$SQOOP_HOME/lib:$HIVE_HOME/lib:$HADOOP_HOME/lib:/opt/beh/core/hive/hcatalog/share/hcatalog:$CLASSPATHexportPATH=$SCALA_HOME/bin:$SPARK_HOME/bin:$ZOOKEEPER_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$HBASE_HOME/bin:$HIVE_HOME/bin:$SQOOP_HOBONCInformation

Change

the

World啟動并格式化ZooKeeper集群所有zookeeper節(jié)點分別執(zhí)行：zkServer.sh

start驗證：zkCli.sh

并執(zhí)行l(wèi)s/再執(zhí)行quit可推出交互.執(zhí)行jps命令檢測是否存在zookeeper服務(wù)在hadoop001上執(zhí)行：hdfs

zkfc-formatZK驗證：zkCli.sh

并執(zhí)行l(wèi)s/再執(zhí)行quit可推出交互執(zhí)行如下命令在所有zookeeper服務(wù)器查看zookeeper角色zkServer.sh

statusBONC啟動并格式化NameNode所有journalnode分別執(zhí)行：/opt/beh/core/hadoop/sbin/hadoop-daemon.sh

start

journalnodejps檢測是否存在journalnode服務(wù)任選一個,這里選擇hadoop001，執(zhí)行：hd amenode

-formathd amenode

-initializeSharedEditshadoop-daemon.sh

start

namenodejps驗證namenode服務(wù)在hadoop002執(zhí)行：hd amenode

-bootstrapStandby在hadoop002執(zhí)行：hadoop-daemon.sh

start

namenode執(zhí)行jps驗證namenode服務(wù)是否存在皆顯示此時登陸

與Standby狀態(tài)Information

Change

the

WorldBONC啟動并格式化DataNode在hadoop001執(zhí)行(這個是 全啟動)：hadoop-daemons.sh

start

datanode或者在每個datanode上執(zhí)行hadoop-daemon.sh

start

datanodejps驗證是否存在datanode服務(wù)注:對于重建的Hadoop集群,格式化NameNode時并不會將DataNode的舊文件刪除,需要手動將數(shù)據(jù)和臨時文件 下的舊文件刪除.否則造成DataNode啟動失敗.Information

Change

the

WorldBONC啟動并查看

ameNode狀態(tài)所有NameNode節(jié)點分別執(zhí)行：hadoop-daemon.sh

start

zkfc驗證：zkCli.sh

并執(zhí)行l(wèi)s/再執(zhí)行quit可推出交互登陸namenode服務(wù)器web端查看服務(wù)器狀態(tài)與其中一個為active另一個為此時登陸standby狀態(tài)Information

Change

the

WorldBONCInformation

Change

the

World啟動YARN在hadoop001,hadoop002上啟動：yarn-daemon.sh

start

resourcemanager在hadoop003,hadoop004,hadoop005.....hadoop008上啟動：yarn-daemon.sh

start

datamanagerBONCInformation

Change

the

World啟動Hbase相關(guān)服務(wù)啟動regionserver#在所有regionserver角色上執(zhí)行

hbase-daemon.sh

start

regionserver啟動hmaster#在需要登陸的客戶端上執(zhí)行

hbase-daemon.sh

start

masterBONCHive元數(shù)據(jù)Mysql數(shù)據(jù)庫設(shè)置安裝Mysql數(shù)據(jù)庫,一般選擇操作系統(tǒng)安裝包自帶的rpm包安裝即可.MySQL安裝及主從配置.txtInformation

Change

the

WorldBONCInformation

Change

the

WorldHive元數(shù)據(jù)Mysql庫字符集問題目前在Hive中使用中文注釋時,會有部分問題.desc

tablename查看注釋可通過下方法解決.但是show

create

table

tablename仍然存在問題.解決辦法:mysql

-uroot

-pmysql123usehive;show

tables;

##找到COLUMNS的那張表,表名因版本略有不同，看下數(shù)據(jù)。show

full

columnsfrom

COLUMNS_V2;

##表名是大小寫敏感的。

alter

table

COLUMNS_V2

modify

column

COMMENT

varchar(256)character

set

utf8;show

full

columns

from

TABLE_PARAMS;alter

table

TABLE_PARAMSmodify

column

PARAM_VALUEvarchar(4000)

character

set

utf8;BONC運行Hive與jar包程序使用hive使用hive客戶端時應(yīng)使用hive命令.Debug模式,如果有什么報錯可以用這個進 試查看問題在哪里./hive

-hiveconfhive.root.logger=DEBUG,console運行jar包程序在運行jar包程序時需使用/opt/beh/core/hadoop/bin-mapreduce1目

錄下hadoop相關(guān)命令I(lǐng)nformation

Change

the

WorldBONC提綱—Hadoop集群架構(gòu)三Hadoop安裝部署二Hadoop日常操作與四Hadoop高可用相關(guān)Information

Change

t2h5eWorldBONCInformation

Change

t2h6eWorld單服務(wù)啟動命令(一)啟動zookeeper服務(wù)所有zookeeper服務(wù)節(jié)點執(zhí)行:zkServer.sh

start啟動JournalNode服務(wù)所有journalenode節(jié)點執(zhí)行:hadoop-daemon.sh

start

journalnode啟動NameNode服務(wù)所有NameNode節(jié)點執(zhí)行hadoop-daemon.sh

start

namenode啟動DataNode服務(wù)在某一個節(jié)點上執(zhí)行:hadoop-daemons.sh

start

datanode或者在所有DataNode節(jié)點上執(zhí)行:(某些時候需要單獨啟動某個節(jié)點的服務(wù))hadoop-daemon.sh

start

datanodeBONCInformation

Change

t2h7eWorld單服務(wù)啟動命令(二)啟動ZooKeeperFailoverController在所有NameNode節(jié)點上執(zhí)行:

hadoop-daemon.sh

start

zkfc啟動ResourceManager在所有resoucermanager節(jié)點上執(zhí)行:

yarn-daemon.sh

start

resoucermanager啟動nodemanager在所有nodemanager節(jié)點上執(zhí)行:

yarn-daemon.sh

start

nodemanagerBONCInformation

Change

t2h8eWorld單服務(wù)停止命令(一)啟動zookeeper服務(wù)所有zookeeper服務(wù)節(jié)點執(zhí)行:zkServer.sh

stop啟動JournalNode服務(wù)所有journalenode節(jié)點執(zhí)行:hadoop-daemon.sh

stop

journalnode啟動NameNode服務(wù)所有NameNode節(jié)點執(zhí)行hadoop-daemon.sh

stop

namenode啟動DataNode服務(wù)在某一個節(jié)點上執(zhí)行:hadoop-daemons.sh

stop

datanode或者在所有DataNode節(jié)點上執(zhí)行:(某些時候需要單獨關(guān)閉某個節(jié)點的服務(wù))

hadoop-daemon.sh

stop

datanodeBONCInformation

Change

t2h9eWorld單服務(wù)停止命令(二)停止ZooKeeperFailoverController在所有NameNode節(jié)點上執(zhí)行:

hadoop-daemon.sh

stop

zkfc停止ResourceManager在所有resoucermanager節(jié)點上執(zhí)行:

yarn-daemon.sh

stop

resoucermanager停止nodemanager在所有nodemanager節(jié)點上執(zhí)行:yarn-daemon.sh

stop

nodemanagerBONC啟停hadoop集群在NameNode節(jié)點創(chuàng)建

啟動、停止[hadoop@hadoop001

~]$

mkdir

/opt/beh/host_dir[hadoop@hadoop001

~]$cd

/opt/beh/host_dirInformation

Change

t3h0eWorldBONCInformation

Change

t3h1eWorld配置各服務(wù)HOST文件[hadoop@hadoop001

host_dir]$

cat

host_zookeeperhadoop001hadoop002hadoop003[hadoop@hadoop001

host_dir]$

cat

host_journalnodehadoop001hadoop002hadoop003[hadoop@hadoop001

host_dir]$

cat

host_namenodehadoop001hadoop002[hadoop@hadoop001

host_dir]$

cat

host_datanodehadoop003hadoop004hadoop005hadoop006hadoop007hadoop008[hadoop@hadoop001

host_dir]$

cat

host_resourcemanagerhadoop001hadoop002[hadoop@hadoop001

host_dir]$

cat

host_nodemanagerhadoop003hadoop004hadoop005hadoop006hadoop007hadoop008BONC集群

啟動[hadoop@bonc1

host_dir]$

sh

start-all.shJMX

enabled

by

defaultUsing

config:

/opt/beh/zookeeper/bin/../conf/zoo.cfgStarting

zookeeper

...STARTEDJMX

enabled

by

defaultUsing

config:

/opt/beh/zookeeper/bin/../conf/zoo.cfgStarting

zookeeper

...STARTEDJMX

enabled

by

defaultUsing

config:

/opt/beh/zookeeper/bin/../conf/zoo.cfgStarting

zookeeper

...STARTEDstarting

journalnode,

logging

to

/opt/beh/hadoop/logs/hadoop-hadoop-journalnode-bonc1.outstarting

journalnode,

logging

to

/opt/beh/hadoop/logs/hadoop-hadoop-journalnode-bonc2.outstarting

journalnode,

logging

to

/opt/beh/hadoop/logs/hadoop-hadoop-journalnode-bonc3.outstarting

namenode,

logging

to

/opt/beh/hadoop/logs/hadoop-hadoop-namenode-bonc1.outstarting

zkfc,

logging

to

/opt/beh/hadoop/logs/hadoop-hadoop-zkfc-bonc1.outstarting

namenode,

logging

to

/opt/beh/hadoop/logs/hadoop-hadoop-namenode-bonc2.outstarting

zkfc,

logging

to

/opt/beh/hadoop/logs/hadoop-hadoop-zkfc-bonc2.outbonc3:

starting

datanode,

logging

to

/opt/beh/hadoop/logs/hadoop-hadoop-datanode-bonc3.outbonc5:

starting

datanode,

logging

to

/opt/beh/hadoop/logs/hadoop-hadoop-datanode-bonc5.outbonc4:

starting

datanode,

logging

to

/opt/beh/hadoop/logs/hadoop-hadoop-datanode-bonc4.outbonc6:

starting

datanode,

logging

to

/opt/beh/hadoop/logs/hadoop-hadoop-datanode-bonc6.outsafe_mode_is_offstarting

jobtrackerha,

logging

to

/opt/beh/hadoop/logs/hadoop-hadoop-jobtrackerha-bonc1.outstarting

mrzkfc,

logging

to

/opt/beh/hadoop/logs/hadoop-hadoop-mrzkfc-bonc1.outstarting

jobtrackerha,

logging

to

/opt/beh/hadoop/logs/hadoop-hadoop-jobtrackerha-bonc2.outstarting

mrzkfc,

logging

to

/opt/beh/hadoop/logs/hadoop-hadoop-mrzkfc-bonc2.outbonc5:

starting

tasktracker,

logging

to

/opt/beh/hadoop/bin-mapreduce1/../logs/hadoop-hadoop-tasktracker-bonc5.outbonc4:

starting

tasktracker,

logging

to

/opt/beh/hadoop/bin-mapreduce1/../logs/hadoop-hadoop-tasktracker-bonc4.outbonc3:

starting

tasktracker,

logging

to

/opt/beh/hadoop/bin-mapreduce1/../logs/hadoop-hadoop-tasktracker-bonc3.outbonc6:

starting

tasktracker,

logging

to

/opt/beh/hadoop/bin-mapreduce1/../logs/hadoop-hadoop-tasktracker-bonc6.outInformation

Change

t3h2eWorldBONC集群

停止[hadoop@bonc1

host_dir]$

sh

top_all.shbonc4:

stopbonc5:

stopbonc3:

stopbonc6:

stoptasktrackertasktrackertasktrackertasktrackerstopstopstopstopmrzkfcjobtrackerhamrzkfcjobtrackerhabonc5:

stopbonc3:

stopbonc4:

stopbonc6:

stopdatanodedatanodedatanodedatanodenamenodezkfcnamenodezkfcstopstopstopstopstopstopstopjournalnodejournalnodejournalnodeJMX

enabled

by

defaultUsing

config:

/opt/beh/zookeeper/bin/../conf/zoo.cfgStop zookeeper

...STOPPEDJMX

enabled

by

defaultUsing

config:

/opt/beh/zookeeper/bin/../conf/zoo.cfgStop zookeeper

...STOPPEDJMX

enabled

by

defaultUsing

config:

/opt/beh/zookeeper/bin/../conf/zoo.cfgStop zookeeper

...STOPPEDInformation

Change

t3h3eWorldBONCNameNodeweb端口為50070,登陸http://{NameNode-ip-address}:50070可以查看NameNode相關(guān)信息,包括是否為活躍節(jié)點,查看當前hdfs占用的空間占比,以及NameNode節(jié)點各種日志.Information

Change

t3h4eWorldBONChDfs空間使用情況Information

Change

t3h5eWorldBONCNameNode節(jié)點日志Information

Change

t3h6eWorldoBONCJob

(一）web端口為19888,登陸http://{NameNode-ip-address}:19888可以查看正在執(zhí)行、結(jié)束、失敗的job以及job的各種詳細信息,包括執(zhí)行job時的各種配置參數(shù)、map數(shù)、reduce數(shù)、 文件大小、寫入文件大小等等.針對失敗的job進行日志以及參數(shù)等分析,查明原因作出相應(yīng)對策.rldBONCHadoop集群常用命令（一）hdfsdfsadmin查看集群各節(jié)點狀態(tài)[hadoop@hadoop009

~]$

hdfs

dfsadmin-reportConfigured

Capacity:

63749735645184

(57.98TB)Present

Capacity:

60496306655232

(55.02

TB)DFS

Remaining:

36531939196928

(33.23

TB)DFS

Used:

23964367458304

(21.80

TB)DFS

Used%:

39.61%Under

replicated

blocks:

275Blocks

with

corrupt

replicas:

0Missing

blocks:

0Datanodes

available:

6

(6

total,

0

dead)Live

datanodes:Name:

66:50010

(hadoop008)Hostname:

hadoop008mission

Status

:

NormalConfigured

Capacity:

10624955940864

(9.66TB)DFS

Used:

3972594683904

(3.61

TB)Non

DFS

Used:

542238167040

(505.00

GB)DFS

Remaining:

6110123089920

(5.56

TB)DFS

Used%:

37.39%DFS

Remaining%:

57.51%Configured

Cache

Capacity:

0

(0

B)CacheUsed:

0

(0

B)Cache

Remaining:

0

(0

B)Cache

Used%:

100.00%CacheRemaining%:

0.00%Last

contact:

Mon

May

19

01:01:42

CST

2014Information

Change

t3h8eWorldBONCInformation

Change

t3h9eWorldHadoop集群常用命令（二）查看安全模式hdfs

dfsadmin

-safemode

enter

|

leave

|

get

|

wait[hadoop@hadoop009

~]$hdfs

dfsadmin

-safemode

getSafe

modeisOFF查看namenode是否活躍[hadoop@hadoop009

~]$hdfshaadmin

-getServiceState

nn1active[hadoop@hadoop009

~]$hdfshaadmin

-getServiceState

nn2Standby查看hdfs的參數(shù)配置[hadoop@hadoop009~]$

hdfs

getconf-confKeydfs.replication3BONCInformation

Change

t4h0eWorldHadoop集群常用命令（三）hdfs

fsck檢測文件系統(tǒng)的塊信息[hadoop@hadoop009

~]$

hdfs

fsck

/...Status:

HEALTHYTotal

size:

7917560996782

BTotal

dirs:

1047Total

files:

36203Total

symlinks:

0Total

blocks

(validated): 61095

(avg.

block

size

129594254

B)Minimally

replicated

blocks: 61095

(100.0

%)Over-replicated

blocks:Under-replicated

blocks:0

(0.0

%)275

(0.45011866

%)Mis-replicatedblocks: 0

(0.0

%)Default

replicationfactor:

3Average

block

replication:

3.01350360Corrupt

blocks:

Missing

replicas:Number

of

data-nodes:1100

(0.5939204

%)6Number

of

racks:

1FSCK

ended

at

Mon

May

19

01:13:10

CST

2014

in

889

millisecondsBONCInformation

Change

t4h1eWorldHadoop集群常用命令（四）查看jobtracker節(jié)點的活躍狀態(tài)[hadoop@hadoop009

~]$

hadoop

mrhaadmin-getServiceState

jt1active[hadoop@hadoop009

~]$

hadoop

mrhaadmin-getServiceState

jt2Standby查看執(zhí)行job的信息

[hadoop@hadoop009

~]$

mapred

job-list1

jobs

currently

runningJobId

State

StartTime

UserName

Priority

SchedulingInfojob_201405181945_0002

4

1400433843326 hadoop

NORMAL

NA[hadoop@hadoop009

~]$

mapred

job

-list1

jobs

currently

runningPriority

SchedulingInfoJobId

State

StartTime

UserNamejob_201405181945_0002

11400433843326 hadoop

NORMAL

NABONCInformation

Change

t4h2eWorldHadoop集群常用命令（五）殺死job[hadoop@hadoop009

~]$

mapred

job

-list1

jobs

currently

runningJobId

State

StartTime

UserNamePrioritySchedulingInfojob_201405181945_0004

4

1400434102248 hadoop

NORMAL

NA[hadoop@hadoop009

~]$

mapred

job

-kill

job_201405181945_0004Killed

jobjob_201405181945_0004[hadoop@hadoop009

~]$mapred

job

-list0

jobs

currently

runningJobId

State

StartTime

UserName

Priority

SchedulingInfoBONCInformation

Change

t4h3eWorldHadoop集群常用命令（六）修改job的執(zhí)行優(yōu)先級[hadoop@hadoop009

~]$

hadoop

job

-list1

jobs

currently

runningJobId

State

StartTime

UserNamePrioritySchedulingInfojob_201405181945_0005

4

1400434312199 hadoop

NORMAL

NA[hadoop@hadoop009

~]$hadoop

job

-set-priorityjob_201405181945_0005VERY_HIGHChanged

jobpriority.[hadoop@hadoop009

~]$

hadoop

job

-list1

jobs

currently

runningPriority

SchedulingInfoJobId

State

StartTime

UserNamejob_201405181945_0005

11400434312199 hadoop

VERY_HIGH

NABONCInformation

Change

t4h4eWorldHadoop集群常用命令（七）集群數(shù)據(jù)節(jié)點平衡開啟、關(guān)閉命令[hadoop@hadoop009

~]$

start-balancer.sh

或者

hdfs

balancestartingbalancer,

loggingto

/opt/beh/hadoop/logs/hadoop-hadoop-balancer-hadoop009.out[hadoop@hadoop009

~]$

more

/opt/beh/hadoop/logs/hadoop-hadoop-balancer-hadoop009.outTime

Stamp Iteration#

Bytes

Already

Moved

Bytes

Left

To

Move

BytesBeing

MovedThe

cluster

is

balanced.

Exiting...Balancing

took

1.902

seconds(blocks,

-f)

unlimited(-i)

773705pending

signals

max

locked

memorymax

memorysize(kbytes,

-l)

64(kbytes,

-m)

unlimitedopen

files(-n)

131072pipe

size (512

bytes,

-p)

8POSIX

message

queues (bytes,

-q)

819200real-time

prioritystack

size(-r)

0(kbytes,

-s)

10240cpu

time (seconds,

-t)

unlimited(-u)

unlimitedmax

user

processesvirtual

memoryfile

locks(kbytes,

-v)

unlimited(-x)

unlimitedBONCHadoop集群常用命令（八）檢測所有節(jié)點已經(jīng)開啟的服務(wù)[hadoop@hadoop009

~]$

sh

all_jps.sh--------hadoop001----------QuorumPeerMainJournalNodeNameNodeDFSZKFailoverControllerJobTrackerHADaemonMRZKFailoverController--------hadoop002----------QuorumPeerMainJournalNodeNameNodeDFSZKFailoverControllerJobTrackerHADaemonMRZKFailoverController--------hadoop003----------QuorumPeerMainJournalNodeDataNodeTaskTracker--------hadoop004----------DataNodeTaskTracker--------hadoop005----------DataNodeTaskTracker--------hadoop006----------DataNodeTaskTracker--------hadoop007----------DataNodeTaskTracker--------hadoop008----------DataNodeTaskTracker--------hadoop009----------RunJarRunJarRunJarBootstrap--------hadoop010----------RunJarBootstrapInformation

Change

t4h5eWorldBONCNameNode節(jié)點日志相關(guān)[hadoop@hadoop001logs]$

cd

/opt/beh/logs/hadoop[hadoop@hadoop001

logs]$

pwd/opt/beh/logs/hadoop[hadoop@hadoop001

logs]$

ll##namenode進程對應(yīng)日志-rw-rw-r--

1

hadoop

hadoop

188440522

Sep

12

12:51

hadoop-hadoop-namenode-hadoop001.logInformation

Change

the

WorldBONCInformation

Change

the

WorldDataNode節(jié)點日志相關(guān)[hadoop@hadoop001

logs]$

cd

/opt/beh/logs/hadoop[hadoop@hadoop001

logs]$

pwd/opt/beh/logs/hadoop[hadoop@hadoop001

logs]$

ls-lrt

|grep

log

|grep-vlog.##DataNode進程對應(yīng)日志-rw-r--r-- 1

hadoop

hadoop 259748137

May

20

17:19

hadoop-hadoop-tasktracker-hadoop004.log##TaskTracker進程對應(yīng)日志-rw-r--r-- 1

hadoop

hadoop 132643847

May

20

17:19

hadoop-hadoop-datanode-hadoop004.logBONC集群負載

(一)主要CPU以及l(fā)oad負載.合理的調(diào)整ETL流程能有效的較少Job集中對整個流程加工的時間延遲.Load值一CPU消耗過高達到95%以上時,不宜增加新的Job。一般Load值低于CPU邏輯線程數(shù)的3倍,超過

3倍時服務(wù)器間通信會變得較慢。Information

Change

t4