試驗(yàn)4熟悉常用的HBase操作

1、實(shí)驗(yàn)4熟悉常用的HBase操作姓名：包生友 專業(yè)年級(jí)：軟件143學(xué)號(hào)：6991. 實(shí)驗(yàn)?zāi)康?. 理解HBase在Hadoop體系結(jié)構(gòu)中的角色；2. 熟練使用HBase操作常用的Shell命令；3. 熟悉HBase操作常用的Java API。2. 實(shí)驗(yàn)環(huán)境操作系統(tǒng)：Lin uxHadoop版本：2.6.0或以上版本HBase版本：1.1.2或以上版本JDK版本：1.6或以上版本Java IDE: Eclipse3. 實(shí)驗(yàn)內(nèi)容和完成情況1.編程實(shí)現(xiàn)以下指定功能，并用Hadoop提供的HBase Shell命令完成相同任務(wù):(完整可執(zhí)行代碼見代碼/QuestionOne.java)(1) 列出HBa

2、se所有的表的相關(guān)信息，例如表名；Shell:ListhbaseCinatn)ListTABLE511 rowCs) tn &.seconds-【" 圖1列岀HBase所有表的相關(guān)信息編程：列出HBase所有的表的相關(guān)信息，例如表名、創(chuàng)建時(shí)間等public static void listTables() throws IOException init();建立連接HTableDescriptor hTableDescriptors = adm in.li stTables();for(HTableDescriptor hTableDescriptor :hTableDesc

3、riptors)System.out.println(” 表名:"+hTableDescriptor.getNameAsString();close();關(guān)閉連接(2) 在終端打印出指定的表的所有記錄數(shù)據(jù)；Shell:scan 's1':e> scan 1ROWCOLUMN+CEILLa row(£) tn 0.1380 seconds圖2打印指定表的所有記錄數(shù)據(jù)編程：(2)在終端打印出指定的表的所有記錄數(shù)據(jù)public static void getData(Stri ng tableName)throwsIOExcepti onini t();Ta

4、ble table = conn ecti on. getTable(TableName.valueOf(tableName);Scan sca n = new Scan();ResultSca nner sca nner = table.getSca nn er(sca n);for (Result result:sca nner)prin tRecoder(result);close();/打印一條記錄的詳情public static void printRecoder(Result result)throws IOExceptionfor(Cell cell:result.rawCell

5、s()System.out.print("行健:"+new String(CellUtil.cloneRow(cell);System.out.print("列簇:"+new String(CellUtil.cloneFamily(cell);System.out.print("列:"+new String(CellUtil.cloneQualifier(cell);System.out.print("值:"+new String(CellUtil.cloneValue(cell);System.out.print

6、ln(” 時(shí)間戳:"+cell.getTimestamp();(3) 向已經(jīng)創(chuàng)建好的表添加和刪除指定的列族或列；p.s :此題請(qǐng)先在 Shell中創(chuàng)建si作為示例表：create 's1','score'a)在si表,添加數(shù)據(jù)：Shell:put 'si','zha ngsa n','score:Math','69'hbase(nain):933pjt 1 si1 / zhangsan',1 score:Math r *T69' 0 rt>w(s) tn 9,0609

7、 secondshba5e(nein);&D4IsNaneError; undefined local varxble or nethod 'Xs' f&r jecthbase(nalft):&05：0> ListtableSI1 row(sJ i.n 6516 seconds圖3給si添加數(shù)據(jù)編程：/向表添加數(shù)據(jù)public static void insertRow(String tableName,String rowKey,String colFamily,String col,String val) throws lOException

8、ini t();Table table = conn ecti on. getTable(TableName.valueOf(tableName);Put put = new Put(rowKey.getBytes();put.addColu mn( colFamily.getBytes(), col.getBytes(), val.getBytes();table.put(put);table.close();close();in sertRow("s1",'zha ngsa n','score','Math','6

9、9')b)在si表，刪除指定的列：Shell:delete 's1','zha ngsa n','score:Math':011:6 delete H«11'2hangan't score:Math row(s) in ft.9070 seconds圖4刪除數(shù)據(jù)編程：刪除數(shù)據(jù)public static void deleteRow(Stri ng tableName,Stri ng rowKey,Stri ng colFamily,Stri ng col) throws lOExcepti on ini t();

10、Table table = conn ecti on. getTable(TableName.valueOf(tableName);Delete delete = new Delete(rowKey.getBytes();刪除指定列族delete.addFamily(Bytes.toBytes(colFamily);刪除指定列delete.addColu mn( Bytes.toBytes(colFamily),Bytes.toBytes(col);table.delete(delete);table.close();close();deleteRow("s1",'

11、zha ngsa n','score','Math')(4) 清空指定的表的所有記錄數(shù)據(jù)；Shell:trun cate 's1hbasefnatn) ：BJ8:0-> RONLisiwanghu2tn o.ezin1 si1COLUMN+CELLCQlunn=5core:Mfltht tiricstmnpul斗822弓5后 13818 value=S5 colunn-score；r*eth* ttmesta，Iipl482Z35732643, vue-75hba»(natn)19191truncate 1sl'Trunc

12、ating * si1 tabic (it may tike a while): * Disabling t曲blc*Truncdtlny table*-,0 row(s) tn 3.7870 secondssen ' si'COLUMN+CELL seconrlhbdse(main):026:O>ROWfl row(s tn圖5清空指定表的所有記錄數(shù)據(jù)編程：(4)清空指定的表的所有記錄數(shù)據(jù)public static void clearRows(String tableName)throws lOExceptionini t();TableName table name

13、 = TableName.valueOf(tableName);adm in. disableTable(table name);admi n.deleteTable(table name);HTableDescriptor hTableDescriptor = new HTableDescriptor(tableName);adm in. createTable(hTableDescriptor);close();(5 )統(tǒng)計(jì)表的行數(shù)。Shell:coun t 's1hbase(ndin):021：count *sl 炒 row(s) in 0,0300 seconds“ 0圖6統(tǒng)計(jì)

14、表的行數(shù)編程：(5)統(tǒng)計(jì)表的行數(shù)public static void countRows(String tableName)throws IOExceptionini t();Table table = conn ectio n.getTable(TableName.valueOf(tableName);Scan sca n = new Scan();ResultSca nner sca nner = table.getSca nn er(sca n);int num = 0;for (Result result = sca nner.n ext();result!=nu ll;result=

15、sca nner.n ext()nu m+;System.out.println（"行數(shù):"+ num）;sea nn er.close（）;close（）;2. 現(xiàn)有以下關(guān)系型數(shù)據(jù)庫中的表和數(shù)據(jù)，要求將其轉(zhuǎn)換為適合于HBase存儲(chǔ)的表并插入數(shù)據(jù)：學(xué)生表（Student）學(xué)號(hào)（S_No）姓名（S_Name）性別（S_Sex）年齡（S_Age）Zhangsanmale23Maryfemale22Lisimale24課程表（Course）課程號(hào)（C No）課程名（C Name）學(xué)分（C Credit）123001Math2.0123002Computer Science5.01

16、23003En glish3.0選課表（SC）學(xué)號(hào)（SC_Sno）課程號(hào)（SC_Cno）成績(jī)（SC_Score）123001861230036912300277123003991230019812300295學(xué)生Student表p.s：主鍵的列名是隨機(jī)分配的，因此無需創(chuàng)建主鍵列創(chuàng)建表：create 'Stude nt','S_No','S_Name','S_Sex','S_Age'hbase(main) : O22r0> create 1 Student' , ' S_No' J S_

17、Narne' t 1' t ' S AgeG row(s) in 1*3S60 seconds=> Hb»E；:T©blu - studenthbAsetrwiln) :o?3：0> listTABLEStudentsi2 roM(s) tfi 0.0130 seconds二Student1*, 怙丁 hba5e(main) :024:0s* scan 1 Student * noucolumn+cellG row(5) in 0.9110 seconds圖7創(chuàng)建Student表插入數(shù)據(jù)插入數(shù)據(jù)shell命令第一行數(shù)據(jù)put 'S

18、tude nt','s001','S_No',”put 'Stude nt','s001','S_Name','Zha ngsa n'put 'Stude nt','s001','S_Sex','male'put 'Stude nt','s001','S_Age','23'第二行數(shù)據(jù)put 'Stude nt','s002','

19、;S_No',”put 'Stude nt','s002','S_Name','Mary'put 'Stude nt','s002','S_Sex','female'put 'Stude nt','s002','S_Age','22'第三行數(shù)據(jù)put 'Stude nt','s003','S_No',”put 'Stude nt',&

20、#39;s003','S_Name','Lisi'put 'Stude nt','s003','S_Sex','male'put 'Stude nt','s003','S_Age','24'hbase(nain)put h Student(,1sD61'f S_Mor,b 2015961 6 row(s) in 3.6090 secondshba(niain)：o770 nw(s) tn B.61B6pur Tstude

21、nt'1saai' seccn<J$hbase(nain);0Z8;0> e row(s tn a.160hbse(natn):029:0> 0 row(s) tn 0.6080hbasetnatfi) ?9J6：0iROWsoai5601SO01sOSl1 row(s) in 3.6310put * Student'JsDOl'* p5_5ex','nale seconds put Student', '5031' /S_Age' /231 seconds scan 'student1C

22、OLUhM+CELLcclunn=S Age:t tinestanp=14fl2236406336, valur=J3 cclunn=5_hane:p tinestanp=1462230306213 value=Zhangsan colunn=s_mo:, ttnestampiBZJ36339580, value=zai5fiBi colunn=S_Sex:t tinestamp=14B22363S5641, value=nale seconds圖8添加數(shù)據(jù)并查看hbase(nin):039iscan * Student*ROMCCLUMN+CELLCOlllF± 11meSTanp

23、=l48?23640G36, VA1 J&=?soeicolunFn-S_Nare:, ti.nesta<ip-1482236366213 , vlue-Zhangsan5051colurn=rsZNo:, tinc5tanp-18223S339580, volue-20150513001colurn-S Sex:T tincstcnp=lB22363B5Q41, volue-naleS002coLumn=S_Age: ( tline5tanp=1482230551Z74p value=22SO02colurrn=s_N3re, ttniesrafip=i斗3223旨522212

24、, value=Nar/£M2coLufrn=r£_No:, tinestarip=lS22364eeS51, vaLue=2G15O02column-5 Sex:, tSc航跡斗322弓65。斗718 vdlue-fcnaLesealcolumn-S_Age：, tlmestanp-1482236672914, value-24S003匸口5阿=£-脫呢：，ttnestanp=148223C62145OJI value=Ltsts693columnS1*51 > tinestanp822366e7e7 2, value2&15，fl93S093co

25、lumn=S_Scx： , time5tanp=H822166383Sl, valje=nale3 rw(3)in G.OSZaseconds圖9添加3個(gè)學(xué)生課程Course表創(chuàng)建表：create 'Course','C_No','C_Name','C_Credit'hbaseCmain):O40iQ> create 'Course'»“C_No'11C Nane'f 'C_Credit 0 rows) in 1.3690 seconds" Hbase:Tblf -

26、 Course圖10創(chuàng)建Course表插入數(shù)據(jù):插入數(shù)據(jù)shell命令第一行數(shù)據(jù)put 'Course','c001','C_No','123001'put 'Course','c001','C_Name','Math'put 'Course','c001','C_Credit','2.0'第二行數(shù)據(jù)put 'Course','c002','C_No',&#

27、39;123002'put 'Course','c002','C_Name',' Computer'put'Course','c002','C_Credit','5.0'第三行數(shù)據(jù)put 'Course','c003','C_No','123003'put 'Course','c003','C_Name',' English 'put

28、 'Course','c003','C_Credit','3.0'=> Hbase:Tble - Coursehbase(Hain):941:0> put bourse1 #'ceoi1 (t'123001 0 row(s tn 9,Gl&0 secondshbise(matn);042;0>a rowts) tn 0,0150tn) J943: 0、0 row(s) tn &,e&80put Xourse1 /c&Ol' /C_Nane 'Math

29、 seconds put 'course 1 'cooiH /c credtt1， 2.0 seconds圖11添加數(shù)據(jù)hbase(natn)：056!o>put *toure*cobs*."c_no,HI23ae3'G row(s) in 9,610secondshbae(nain) ;t>57; 0>put 'Course' *'*1CName',1 English'fl row(s 1n a+B65DserondhbaseCnoin):O5G:0>put bourse* /C0031 /C_

30、Credit'/3.O*0 row(s) in 0»C090secondshbase(nain):O59:scan 'Course'ROMCOLUMNiCELLcOOlcolunn-C_Credtt: t tinestanp=143223<j8330Z2, value-2.0cOOlC01Dfln=C_Napie; p ttwestamp= 148236818175, value=MathC091colunn=c_No: ( ttmestamp=iJq822368e6ii49, valuc=L23eaic6B2colunnCCredit:, tinest

31、anp=14a2237671S&6, value=5,Gc002colunnC hcnc:, tinestamp=14S2237GS47S1r value=COflputerc002cclunn=C_No: , tinestaFip=1162237&424S3, value=LZ3O02C003colunn=C_CredIt: # ttnestanip=lflr322 571Sl559, valued.0c093colunn-C_Nane : t tine£tamp=14S2237167697f value=Engli.shC093colunn=C_No： . ttm

32、estaFip=H&223715e637, valuc=123e033 ruw(s) in 0*0500seconds圖12添加3個(gè)課程選課表創(chuàng)建表：create 'SC','SC_S no ','SC_C no ','SC_Score'hbase(nain):G66;0> create 1 SC ,'SC_Sno111SC_Cno','SC_Score 0 row(s) tn 1.2790 seconds=> Hbase:Table - SC圖13創(chuàng)建表SC插入數(shù)據(jù):插入數(shù)據(jù)shell

33、命令第一行數(shù)據(jù)put'SC','sc001','SC_S no',”put 'SC','sc001','SC_C no','123001'put 'SC','sc001','SC_Score','86'第二行數(shù)據(jù)put 'SC','sc002','SC_S no',”put 'SC','sc002','SC_C no',

34、9;123003'put 'SC','sc002','SC_Score','69'第三行數(shù)據(jù)put 'SC','sc003','SC_S no',”put 'SC','sc003','SC_C no','123002'put 'SC','sc003','SC_Score','77'第四行數(shù)據(jù)put 'SC','sc004

35、9;,'SC_S no',”put 'SC','sc004','SC_C no','123003'put 'SC','sc004','SC_Score','99'第五行數(shù)據(jù)put 'SC','sc005','SC_S no',”put 'SC','sc005','SC_C no','123001'put 'SC','s

36、c005','SC_Score','98'第六行數(shù)據(jù)put 'SC','sc006','SC_S no',”put 'SC','sc006','SC_C no','123002'put 'SC','sc006','SC_Score','95'hbase(natn);0OO；O> create rSC',rSC_5no',SCjCno'f *5C_Sco

37、re o raw(s tn 1 ,?7en seconris=> Hbasc:tTable - SChb»e(natn>：e61> put 15C,1 scBM' x rSC_5nor f r2fll5091T O rin 0 *©120 secondshbase(niin)r0b2: G r ow(£) in Q,6160put SC( /SC901 * i SC_tno' /12300L secondshbaie(F«in);0C3;0> put '5C' F1scOOl1('SC.Sc

38、orer,1B6 o ro»(s) In fi.0290 seconds圖14插入數(shù)據(jù)hba&e(matn)：O12:0> ROU5C001atOOl5caniSCSB2SC&02C0D2SC 003SC033sc&OaS&043C5D4SC 8 343C0&SKOOSSC0066 row(sin O.C510can 'sc1COLUMN+CELLcolunn=SC Cno:, tinestaFpl432237297903, value=l23O01 colurin=SC_5cQre; tine5tflFip=1482237313

39、394, value=BG calumn=S匚-Eno: , tlmestiinip=14H77377H5573 , vlue=7ei506 c©Iunn-SC_Cno: , tinestanp-?1492237433269, value-12593 colunn-SC Scorc:, tXF>e5torip-H82237H3270, voluc-69 colunn-SCIsno:, tinestanp-14B22374Z2O49, Vdlue-Z0150Cl coldflr)=sc_cno； F Ctnestanp=1482237492242 # vdlue=l?3O02 c

40、olumnscscore:, ttmestanp=i48223S0be37r “alu電二TF cclurtri=£C_£no： , tinestanp-14B223747S26, value=615062 colunn=SC_ChO：, tinrstanp=L482238&G2flS3, vaLu«=123963 colunn=SC_Score: t tinestdFip=1482233015244, v1ue=99 column=sc_sno: , tirie&taFp=i482237753343, value=zei5BB2 column=S

41、C_Cno: , tineitaFip=14B223SE)99e2 V3Lue=123301 column=SC core: , tir)estaFip=14S223810990gt V01ue=9B colunn-SC Who: ti.riestanp-14B22380Q159Ovalue=Z015003 c ol iimn=sc_cno：, t ti'iPStanp=14B2?814918； f wslusl? colunn=£C_Score:F tinestanp=14822281S9793r value=95 colunn-SCSno:, tinestaFip-L43

42、2238157e94 j valuc-2eiS6e3 seconds圖15數(shù)據(jù)顯示Summary:Tdbl* hbdsr.mld kNumber of regions! 1Epi邯?jiǎng)?chuàng) oik uUuiiId,3i3O7r 1 d>2233930M5 ThJf Stu c*t nr isokay.Numbnr of cgtons! 1Dvplid an-.訕11皿耳站307482鈾菊丈1&45 旳城就n otiay.itlumtef of r測(cè)qx 1Deployed on： unuiniuj3JD?P 14S?2JJ9WB46Ta Na &1 h uky.Number o

43、f ff ginn! 1IXployed OH： UHunCu333O7r)4822J3950M5 Tbl» twok Hoh 曲.Number of fDgtons: 1Wpluyedon： uUunLjlW?, 1 d<2233950U T>h甘 hhw:ru wpflf R QkyUBaseFsck command line optionsVersiar: 1.Z2Number of ir region sp(wrs- iN jmlbEr of d?ad regie n servers; DMaster： ubj nt u 44-! 31u

44、mhpr nf harkiip imarprs' aArrigp 7.0N jmocr of reqxst,: 0Number of r«giant: 7n jnbhf »f i ipgii ns m v <tns4仍q： rNumber of &mpty REGQNNFO_QLALIFIER ro1 in bitu強(qiáng):met凱 0Number af fa bl«: bNLimb&r of regtons： 1De ployed on: ubu nc u, J J ju< lj vboH45Table Course?恪 ukay

45、.Number of rtgions: 1Do ployed on: ubu nr UJ33OZ14B2233950&45 0 ncor)sH«n:ie£ detKiMl.Sw* Ok圖16 QuestionOne運(yùn)行后控制臺(tái)消息同時(shí)，請(qǐng)編程完成以下指定功能：(完整可執(zhí)行代碼見代碼/QuestionTwo.java )(1) createTable(String tableName, String fields)創(chuàng)建表，參數(shù)tableName為表的名稱，字符串?dāng)?shù)組fields為存儲(chǔ)記錄各個(gè)域名稱的數(shù)組。要求當(dāng)HBase已經(jīng)存在名為tableName的表的時(shí)候，先刪除

46、原有的表，然后再創(chuàng)建新的表。代碼: public static void createTable(String tableName,String fields) throws IOException init();TableName tablename = TableName.valueOf(tableName);if(admin.tableExists(tablename)System.out.println("table is exists!");admin.disableTable(tablename);admin.deleteTable(tablename);/ 刪

47、除原來的表HTableDescriptor hTableDescriptor = new HTableDescriptor(tablename);for(String str:fields)HColumnDescriptor hColumnDescriptor = new HColumnDescriptor(str); hTableDescriptor.addFamily(hColumnDescriptor);admin.createTable(hTableDescriptor);close();(2) addRecord(String tableName, String row, Strin

48、g fields, String values)向表tableName、行row (用S_Name表示)和字符串?dāng)?shù)組 files指定的單元格中添加對(duì)應(yīng)的數(shù)據(jù)values。其中fields中每個(gè)元素如果對(duì)應(yīng)的列族下還有相應(yīng)的列限定符的話，用“ columnFamily:column 表示”。例如，同時(shí)向“ Math”、“ Computer Science”、“ English 三列添加成績(jī)時(shí)，字符串?dāng)?shù)組 fields 為 “Score:Math ”,”Score；Computer Science”,”Score:English ” ，數(shù)組values存儲(chǔ)這三門課的成績(jī)。代碼：public sta

49、tic void addRecord(String tableName,String row,String fields,String values) throwsIOException init();Table table = connection.getTable(TableName.valueOf(tableName);for(int i = 0;i != fields.length;i+)Put put = new Put(row.getBytes();String cols = fieldsi.split(":");put.addColumn(cols0.getB

50、ytes(), cols1.getBytes(), valuesi.getBytes(); table.put(put);table.close();close();(3) scanColumn(String tableName, String column)瀏覽表 tableName 某一列的數(shù)據(jù)，如果某一行記錄中該列數(shù)據(jù)不存在，則返回null 。要求當(dāng)參數(shù) column 為某一列族名稱時(shí)，如果底下有若干個(gè)列限定符，則要列出每個(gè)列限定符 代表的列的數(shù)據(jù)；當(dāng)參數(shù) column 為某一列具體名稱(例如 “Score:Math )”時(shí)，只需要列出 該列的數(shù)據(jù)。代碼： public static v

51、oid scanColumn(String tableName,String column)throws IOExceptioninit();Table table = connection.getTable(TableName.valueOf(tableName);Scan scan = new Scan();scan.addFamily(Bytes.toBytes(column);ResultScanner scanner = table.getScanner(scan);for (Result result = scanner.next(); result != null; result

52、 = scanner.next() showCell(result);table.close();close(); /格式化輸出 public static void showCell(Result result)Cell cells = result.rawCells();for(Cell cell:cells)System.out.println("RowName:"+new String(CellUtil.cloneRow(cell)+" ");System.out.println("Timetamp:"+cell.getTim

53、estamp()+" ");System.out.println("column Family:"+new String(CellUtil.cloneFamily(cell)+" ");System.out.println("row Name:"+new String(CellUtil.cloneQualifier(cell)+" ");System.out.println("value:"+new String(CellUtil.cloneValue(cell)+"

54、; ");(4) modifyData(String tableName, String row, String column)修改表tableName，行row (可以用學(xué)生姓名 S_Name表示)，列column指定的單元格的數(shù)據(jù)。代碼：public static void modifyData(String tableName,String row,String column,String val)throwsIOExceptioninit();Table table = connection.getTable(TableName.valueOf(tableName);Put

55、put = new Put(row.getBytes();put.addColumn(column.getBytes(),null,val.getBytes();table.put(put);table.close();close();( 5) deleteRow(String tableName, String row)刪除表 tableName 中 row 指定的行的記錄。public static void deleteRow(String tableName,String row)throws IOExceptioninit();Table table = connection.get

56、Table(TableName.valueOf(tableName);Delete delete = new Delete(row.getBytes();刪除指定列族delete.addFamily(Bytes.toBytes(colFamily);刪除指定列/delete.addColu mn (Bytes.toBytes(colFamily),Bytes.toBytes(col);table.delete(delete);table.close();close();£百f了f化 ProblTasks Javd3 Cons S3° <terminated>QuestionTwo fJava Application /usr/lib/jvm/javaX曙丨園朝區(qū)畫蘭冃審旁甲 log*lj:WARN No appenders could b found for Iogger org,apache.hado lag4j:WARN Please Initialize the k)g4| system properly, log4j:WARN See http:/logging, /lDg4|/1x2/faq»hirnl#noconfi