數(shù)煉成金課程-hadoop推薦系統(tǒng)

系統(tǒng)

第8周DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師法律【】和幻燈片為煉數(shù)成金網(wǎng)絡(luò)課程的教學(xué)資料，所有資料只能在課程內(nèi)使用，丌得在課程以外范圍散播，違者將可能被法律和經(jīng)濟(jì)責(zé)任。課程詳情

煉數(shù)成金培訓(xùn)http:

DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師用Neo4J構(gòu)筑圖模型

引擎圖數(shù)據(jù)，計(jì)算速度極快天生面向圖的數(shù)據(jù)庫引擎，便于Cypher語言可以簡便地實(shí)現(xiàn)圖查詢的圖數(shù)據(jù)庫在引擎還在快速發(fā)展，封裝

的圖論算法。無法解決計(jì)算復(fù)雜度指數(shù)級增長的問題在分布式處理上還有改進(jìn)空間DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師Cypher面向圖數(shù)據(jù)查詢和操作的丏用語言，可以對圖進(jìn)行查詢或修改描述性語法，無需構(gòu)造算法，類自然語言風(fēng)格，類似SQL既適合開發(fā)者，也適合于數(shù)據(jù)庫操作

進(jìn)行即席查詢也有執(zhí)行計(jì)劃，由Cypher引擎生成靈感來自于SQL和SPARQL（一種基于RDF的查詢語言）支持基于單條語句或多條語句的事務(wù)DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師Cypher中的重要關(guān)鍵字DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師Cypher的例子DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師Cypher的算符數(shù)學(xué)算符比較算符算符字符串算符算符屬性算符DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師表達(dá)式構(gòu)成表達(dá)式的元素常數(shù)：數(shù)值，字符串，

型等標(biāo)識符屬性（collection）函數(shù)尋路模式DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師注釋DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師PatternsCypher中最常用最

的用法，熟悉Pattern寫法可以更能發(fā)揮Cypher的用于描述你要查找的數(shù)據(jù)的范圍，通常和MATCH短語結(jié)合使用。給出了Pattern，Cypher就能知道怎樣去定位你需要的數(shù)據(jù)Pattern的基本寫法深度控制復(fù)雜寫法DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師Start語句DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師Start語句DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師Match短語DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師Match短語DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師Match短語DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師Where短語DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師Where短語DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師Where短語DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師其它短語ReturnOrder

byLimitSkipWithDATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師創(chuàng)建圖Create語句Create

unique語句DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師例子DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師例子DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師修改數(shù)據(jù)Set語句Delete語句結(jié)合foreach短語DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師函數(shù)判定函數(shù)標(biāo)量函數(shù)函數(shù)數(shù)學(xué)函數(shù)字符串函數(shù)DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師SQL

vs

Cypher手冊第192頁DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師Match的SQL對應(yīng)DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師執(zhí)行比較DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師本周內(nèi)容數(shù)據(jù)模型應(yīng)用開發(fā)模式DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師et數(shù)據(jù)模型：社交數(shù)據(jù)處理DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師算例：尋找有相同

的同事START

subject=node:user(name={name})MATCH

subject-[:WORKS_FOR]-,>company<-[:WORKS_FOR]-subject-[:INTERESTED_IN]->interest<-[:INTERESTED_IN]-RETURN .name

AS

name,COUNT(interest)

AS

score,COLLECT()

AS

interestsORDER

BY

score

DESCDATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師算例：不

有相同

的人START

subject=node:user(name={name})MATCH

subject-[:INTERESTED_IN]->interest<-[:INTERESTED_IN]--[:WORKS_FOR]->companyRETURN .name

AS

name,

AS

company,COUNT(interest)AS

score,COLLECT()

AS

interestsORDER

BY

score

DESC,DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師圖解DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師一個(gè)復(fù)雜查詢START

subject=node:user(name={name})MATCH

p=subject-[:WORKED_ON]->()-[:WORKED_ON*0..2]-()<-[:WORKED_ON]-

-[:INTERESTED_IN]->interestWHEREWITHRETURN<>subject

AND

IN

{interests},

interest,

MIN(LENGTH(p))

as

pathL

AS

name,COUNT(interest)AS

score,COLLECT()

AS

interests,((pathLength

-

1)/2)

AS

distanceORDER

BY

score

DESCLIMIT

{resultLimit}DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師圖解DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師執(zhí)行原理DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師SaaSNet數(shù)據(jù)模型：實(shí)現(xiàn) 控制DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師算例：尋找用戶能

的所有資源START

admin=node:administrator(name={administratorName})MATCH

paths=admin-[:MEMBER_OF]->()-[:ALLOWED_INHERIT]->()<-[:CHILD_OF*0..3]-company<-[:WORKS_FOR]-employee-[:HAS_ACCOUNT]->accountWHERE

NOT

(admin-[:MEMBER_OF]->()-[:DENIED]->()<-[:CHILD_OF*0..3]-company)RETURN

AS

employee,

AS

accountUNIONSTART

admin=node:administrator(name={administratorName})MATCH

paths=admin-[:MEMBER_OF]->()-[:ALLOWED_DO_NOT_INHERIT]->()<-[:WORKS_FOR]-employee-[:HAS_ACCOUNT]->accountRETURN

AS

employee,

AS

accountDATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師圖解DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師算例：檢查用戶是由

特定資源STARTadmin=node:administrator(name={adminName})e={resourceName})pany(resourceNamMATCH

p=admin-[:MEMBER_OF]->()-[:ALLOWED_INHERIT]->()<-[:CHILD_OF*0..3]-companyWHERE

NOT

(admin-[:MEMBER_OF]->()-[:DENIED]->()<-[:CHILD_OF*0..3]-company)RETURN

COUNT(p)

AS

accessCountUNIONSTARTadmin=node:administrator(name={adminName})

pany(resourceName={resourceName})MATCH

p=admin-[:MEMBER_OF]->()-[:ALLOWED_DO_NOT_INHERIT]->companyRETURN

COUNT(p)

AS

accessCountDATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師算例：查找某賬戶的管理員START

resource=node:resource(name={resourceName})MATCH

p=resource-[:WORKS_FOR|HAS_ACCOUNT*1..2]-company-[:CHILD_OF*0..3]->()<-[:ALLOWED_INHERIT]-()<-[:MEMBER_OF]-adminWHERE

NOT

(admin-[:MEMBER_OF]->()-[:DENIED]->()<-[:CHILD_OF*0..3]-company)RETURN

AS

adminUNIONSTART

resource=node:resource(name={resourceName})MATCH

p=resource-[:WORKS_FOR|HAS_ACCOUNT*1..2]-company<-[:ALLOWED_DO_NOT_INHERIT]-()<-[:MEMBER_OF]-adminRETURN

AS

adminDATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師圖解DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師LogisticsNet數(shù)據(jù)模型：路由計(jì)算LogisticsNet網(wǎng)絡(luò)基本元素DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師動(dòng)態(tài)路由：時(shí)間段1DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師動(dòng)態(tài)路由：時(shí)間段2DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師動(dòng)態(tài)路由：時(shí)間段3DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師數(shù)據(jù)模型DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師細(xì)節(jié)DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師最短路徑問題DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師最短路徑問題DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師Cypher查詢START

s=node:location(name={startLocation}),e=node:location(name={endLocation})MATCH

p1

=

s<-[:DELIVERY_ROUTE*1..2]-db1WHERE

ALL(r

in

relationships(p1)WHERE

r.start_date

<=

{intervalStart}

AND

r.end_date

>=

{intervalEnd})WITH

e,

p1,

db1MATCH

p2

=

db2-[:DELIVERY_ROUTE*1..2]->eWHERE

ALL(r

in

relationships(p2)WHERE

r.start_date

<=

{intervalStart}

AND

r.end_date

>=

{intervalEnd})WITH

db1,

db2,

p1,

p2MATCH

p3

=

db1<-[:CONNECTED_TO]-()-[:CONNECTED_TO*1..3]-db2WHERE

ALL(r

in

relationships(p3)WHERE

r.start_date

<=

{intervalStart}

AND

r.end_date

>=

{intervalEnd})WITH

p1,

p2,p3,REDUCE(weight=0,

r

in

relationships(p3)

:

weight+r.cost)

AS

scoreORDER

BY

scoreASCLIMIT

1RETURN

(nodes(p1)

+

tail(nodes(p3))

+

tail(nodes(p2)))AS

nDATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師用子查詢簡化《Graph

Databases》第171頁Java實(shí)現(xiàn)其它例子：《Neo4j手冊》第7章DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師關(guān)于事務(wù)不鎖Neo4j支持ACID所有數(shù)據(jù)修改必定觸發(fā)事務(wù)缺省

級別是MITTED，通過手工加寫鎖可以獲得最高

級別SERIALIZABLE節(jié)點(diǎn)級鎖或聯(lián)系級鎖自勱死鎖偵測所有未提交修改保存于內(nèi)存，所以需要把大事務(wù)切分以免內(nèi)存不足自勱加鎖逡輯DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師Java不Neo4j的連接將Neo4j內(nèi)嵌到j(luò)ava程序，《Neo4j手冊》第4章Api列表：

/neo4j/1.9.2/apidocs/overview-

summary.htmlREST

API（《Neo4j手冊》第5章，第22章），通過REST接口使用Cypher查詢IBM

上的開發(fā)實(shí)戰(zhàn)文章：

/

/java/j-lo-neo4j/DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師一段簡單例程package

org.neo4j.examples;import

java.io.File;import

java.io.IOException;import

org.neo4j.graphdb.GraphDatabaseService;import

org.neo4j.graphdb.Node;import

org.neo4j.graphdb.Relationship;import

org.neo4j.graphdb.RelationshipType;import

org.neo4j.graphdb.Transaction;import

org.neo4j.graphdb.factory.GraphDatabaseFactory;import

org.neo4j.kernel.impl.util.FileUtils;public

class

Test

{public

enum

RelTypes

implements

RelationshipType{KNOWS}DATAGURU與業(yè)數(shù)據(jù)分析社區(qū)系統(tǒng)講師續(xù)public

static

void

main(String[]

args)

throws

IOException

{FileUtils.deleteRecursively(

new

File(

"db"

)

);GraphDatabaseService

graphdb=new

GraphDatabaseFactory().newEmbeddedDatabase("db");Transaction

tx=graphdb.beginTx();try{Node

node1=graphdb.createNode(),node2=graphdb.createNode();node1.setProperty("name",

"1");node2.setProperty("name",

"2");Relationship

relationship=node1.createRela