文本課件教案visual analytics framework for spatio temporal analysis and偉濤a modelling

1、Tankertanker DesignTankertanker DesignAuthors： Natalia Andrienko Gennady AndrienkoPublished in：Data Mining and Knowledge DiscoveryThe premier technical journal focused on the theory, techniques andpractice for extracting information from large databases.A visual analytics framework for spatio-tempor

2、alanalysis and modelingTankertanker DesignTankertanker Design1、A visual analytics framework is proposed which enables practitioners to built time-series models for large heterogeneous spatio-temporal data sets in an interactive data-driven way. 2、Building on existing time series modeling tools and c

3、lustering tools, interactive grouping, modeling, and model evaluation of time series is supported.大量數(shù)據(jù)來(lái)自不同的位置，不同的時(shí)間，如何去分析？文章采用TS分析（DS），利用交互與可視化，基于現(xiàn)有的分析工具與算法（稍有完善），進(jìn)行兩個(gè)實(shí)例的分析，包括了聚類(lèi)、建模、模型評(píng)估、用途。AbstractTankertanker DesignTankertanker Design1、 Spatio-temporal data often have or can be transformed to the

4、form of numeric time series (TS) referring to different locations in space or different geographical objects spatial time series時(shí)間序列分析：動(dòng)態(tài)數(shù)據(jù)處理的統(tǒng)計(jì)方法；側(cè)重研究數(shù)據(jù)序列的互相依賴關(guān)系。原理：承認(rèn)事物發(fā)展的延續(xù)性，應(yīng)用過(guò)去數(shù)據(jù)，就能推測(cè)事物的發(fā)展趨勢(shì)。方法：統(tǒng)計(jì)分析中加權(quán)平均法對(duì)歷史數(shù)據(jù)進(jìn)行處理。特點(diǎn)：趨勢(shì)、季節(jié)變動(dòng)、循環(huán)波動(dòng)和不規(guī)則波動(dòng)。2、 However, analyzing and modeling each spatial TS independ

5、ently from others ignores relationships and similarities that may exist among spatial locations or objects clustering and interactive grouping單獨(dú)分析忽視了關(guān)系與相似性，本文用到了聚類(lèi)技術(shù)，并鋪以可視化來(lái)優(yōu)化聚類(lèi)IntroductionTankertanker DesignTankertanker DesignPresentation of the framework （ Components of the framework & Analysi

6、s workflow ）Components of the framework：1、 兩種展示方式Cartographic map display & TS display2、聚類(lèi)方法基于現(xiàn)有的開(kāi)源資源（ Weka library & SOM Toolbox ）3、TS 建模工具來(lái)自統(tǒng)計(jì)工具包OpenForecast.4、提供交互的可視接口（用于調(diào)整參數(shù)、完善聚類(lèi)結(jié)果、評(píng)估模型）.Analysis workflow：1、Analysis of the spatio-temporal variation of a single space and time-related att

7、ribute, such as the ice cream sales;2、Analysis of dependencies between two space and time-related attributes, for example, sales of ice cream and average air temperature.Tankertanker DesignTankertanker DesignPresentation of the framework （ Components of the framework & Analysis workflow ）Tankert

8、anker DesignTankertanker DesignPresentation of the framework （ Components of the framework & Analysis workflow ）Tankertanker DesignTankertanker DesignUse of the frameworkThefirstdataset,providedbytheItaliantelecommunicationcompanyWIND,consistsofrecordsabout2,956,739mobilephonecallsmadeduring9day

9、sfrom30/10/2008till07/11/2008.asinglespaceandtimerelated（時(shí)間、電話量）Theseconddataset,providedbyComunediMilano(MunicipalityofMilan),consistsofGPStracksof17,241carsduring1weekstartingfromApril1,2007.twospaceandtime-related（時(shí)間、車(chē)流量、車(chē)速）Tankertanker DesignTankertanker DesignUse of the frameworkphone callsStep

10、0:Datapreparation把該地區(qū)分成307個(gè)區(qū)域，每個(gè)區(qū)域得到一個(gè)TS（共307條TS），每條TS按小時(shí)劃分時(shí)間判斷（便于統(tǒng)計(jì)數(shù)量），共216個(gè)時(shí)間片。（9天）Step 1: Grouping TS displayTankertanker DesignTankertanker DesignUse of the frameworkphone callsStep1:GroupingCartographicmapdisplay從中發(fā)現(xiàn)一些潛在模式，前幾個(gè)類(lèi)都聚集在比較熱鬧的地方，地鐵站，交通線交叉口等等。Tankertanker DesignTankertanker DesignUse o

11、f the frameworkphone callsStep1:GroupingprogressiveclusteringTankertanker DesignTankertanker DesignUse of the frameworkphone callsStep1:GroupinggroupingresultsStep2:AnalysisandmodelingModelingmethodstripleexponentialsmoothing（三次指數(shù)平滑法）Tankertanker DesignTankertanker DesignUse of the frameworkphone ca

12、llsStep2:Analysisandmodeling1、用戶一個(gè)個(gè)考慮先前的聚類(lèi)結(jié)果2、代表TS用均值方法獲得（藍(lán)色的）3、用戶選建模方法，開(kāi)始結(jié)束時(shí)間（綠紅線），模型會(huì)自動(dòng)變化參數(shù)，不小心就會(huì)有bestfittingmodel，黃線是預(yù)測(cè)的，最后兩個(gè)月可以檢驗(yàn)。Tankertanker DesignTankertanker DesignUse of the frameworkphone callsStep2:Analysisandmodeling沒(méi)有最合適的模型出現(xiàn)，可手動(dòng)調(diào)整參數(shù)BeforeAfterTankertanker DesignTankertanker DesignUse o

13、f the frameworkphone callsStep3:Modelevaluation1、利用殘差（現(xiàn)實(shí)數(shù)據(jù)與預(yù)測(cè)數(shù)據(jù)之間的差異）來(lái)進(jìn)行模型評(píng)估。2、誤差的絕對(duì)值不重要，不是整體誤差最小的模型最好，而是模型能夠把握特征，讓誤差隨機(jī)分布，盡量不影響原有特征。從圖中發(fā)現(xiàn)一個(gè)問(wèn)題：intheeveningsoftheworkingdaysnegativevaluesaremorefrequentthanpositivevalues.Tankertanker DesignTankertanker DesignUse of the frameworkphone callsStep3:Modele

14、valuation為了找到問(wèn)題，查看殘差TS圖的細(xì)節(jié)，為了減少工作量，聚類(lèi)分析。1、黑線表示殘差平均值，下圖明顯分布不均，對(duì)聚類(lèi)3重新劃分組，分別建模。2、評(píng)估完所有組后，保存模型參數(shù)及數(shù)據(jù)描述等信息。Tankertanker DesignTankertanker DesignUse of the frameworkphone callsUseofthemodels1、預(yù)測(cè)（若沒(méi)有周期模式，則只能預(yù)測(cè)一小段）2、探測(cè)異常（黑線：圣西羅球場(chǎng)）Tankertanker DesignTankertanker DesignUse of the frameworkGPS dataStep0:Datapr

15、eparation7天被分成168個(gè)時(shí)間間隔；將城市劃分區(qū)域，從一個(gè)區(qū)域到另一個(gè)區(qū)域稱為一個(gè)flow，共2155個(gè)flows；每一個(gè)flow有大小和速度，大小表示車(chē)流量，速度統(tǒng)計(jì)得到（文中取平均）。可視圖中沒(méi)有顯示速度量變化Tankertanker DesignTankertanker DesignUse of the frameworkGPS dataStep0:DatapreparationTS of flow magnitudes (top) TS of average speeds (bottom)Speedstendtodecreaseatthetimeswhentheflowmag

16、nitudesincrease.Tankertanker DesignTankertanker DesignUse of the frameworkGPS dataStep0:Datapreparation由于有2個(gè)空間屬性，需要對(duì)TS分析做一定的變換，提出了根據(jù)dependencyseries(DS)來(lái)進(jìn)行分析的方法。(flowmagnitude&averagespeed)1、flowmagnitude劃分為23個(gè)intervals(02,35,68見(jiàn)下右圖X軸)2、過(guò)濾maximummagnitude少于5的(因?yàn)橄掠覉D中X軸很短)Step1:GroupingTankertanke

17、r DesignTankertanker DesignUse of the frameworkGPS dataStep2:AnalysisandmodelingTS分析模型不能用，本文用多項(xiàng)式回歸，分別對(duì)每一類(lèi)進(jìn)行建模Tankertanker DesignTankertanker DesignUse of the frameworkGPS dataStep3:Modelevaluation聚類(lèi)2的圖看到模型得到值往往偏高，故分析得到圖D（morenegativeresidualvaluesthanpositiveones）將聚類(lèi)2細(xì)分為AC，然后得到殘差分析圖ETankertanker DesignTankertanker DesignUse of the frameworkGPS dataUseofthemodels基于上圖的模型，進(jìn)行預(yù)測(cè)。結(jié)果比對(duì)基本相似，但預(yù)測(cè)值會(huì)略高于實(shí)際值，因?yàn)槟Ｐ褪腔谧畲笃骄俣葮?gòu)建。最大速度模型：基于現(xiàn)有或預(yù)測(cè)的交通狀況，某一個(gè)行程至少需要時(shí)間估算。Tankertanker