系統(tǒng)建模方法與應(yīng)用ch4-curve fittingmode

1、系統(tǒng)建模方法與應(yīng)用齊臣坤機(jī)械與動(dòng)力交通大學(xué): ch Office: 機(jī)械A(chǔ)樓923、B樓214SJTU2第四章：主要內(nèi)容曲線擬合(curve fitting)： x是1維的情況SJTU3第四章：主要內(nèi)容曲面擬合(surface fitting)： x是2維的情況SJTU4第四章：主要內(nèi)容曲線擬合問(wèn)題數(shù)學(xué)描述(problem definition)曲線擬合參數(shù)化方法(parametric method)最小二乘擬合(least squares fit)模型大小、類(lèi)型、數(shù)據(jù)量、噪聲的影響(exle of fit depending onmsize, type of mfamily, data si

2、ze, noise level)參數(shù)化方法的統(tǒng)計(jì)分析(sistical asymptoticysis)偏差方差平衡(bias - variance trade off)曲線擬合非參數(shù)化方法(non-parametric method)SJTU5問(wèn)題數(shù)學(xué)描述數(shù)據(jù)：未知函數(shù)g0(x)，對(duì)于一系列x值x1,x2,xN，觀測(cè)到對(duì)應(yīng)的y值(含有一些噪聲e)y(k ) g0 (xk ) e(k )問(wèn)題：從數(shù)據(jù)y(k ), x N中構(gòu)造估計(jì)kk 1gN (x)gN (x) g0 (x)使得誤差盡可能小SJTU6的兩大類(lèi)方法Methodology從模型參數(shù)化角度分類(lèi)：參數(shù)化方法 (parametric met

3、hod)在一個(gè)參數(shù)化的候選函數(shù)集合中搜索，構(gòu)造gN (x)非參數(shù)化方法 (non-parametric method)通過(guò)在y(k)（仔細(xì)選擇的y(k)子集）上光滑，構(gòu)造gN (x)7SJTUxky(k)System g0(x)e(k)+Optimization-y(k )Mg(x)Prediction參數(shù)化方法參數(shù)化方法 (parametric method)總體描述ng(x, ) kk 1例子：fk (x, k )基函數(shù)fk (x, k ), k , k , k 1,., n參數(shù)化方法需要確定：fk (x, k ) )的類(lèi)型函數(shù)類(lèi)（基函數(shù)模型的大小(n or dimen參數(shù)值)SJTU8參

4、數(shù)化方法局部/全局基函數(shù)(local / global basis functions) (x) (x)xx局部基函數(shù)(local basis functions)全局基函數(shù)(global basis functions)SJTU9參數(shù)化方法模型大小(msize)未知System ComplexityMComplexityMComplexityMComplexity模型復(fù)雜度大于系統(tǒng)復(fù)雜度模型復(fù)雜度接近系統(tǒng)復(fù)雜度模型復(fù)雜度小于系統(tǒng)復(fù)雜度SJTU10參數(shù)化方法參數(shù)估計(jì)：總體框架y(k)System g0(x)+e(k)Opt imization-y(k )g (x, )Parametric mP

5、redictionSJTU11xk第四章：最小二乘擬合最小二乘(Least squares)y(k ) g0 (xk ) e(k )y(k ) gN (xk )過(guò)程模型優(yōu)化問(wèn)題(Optimization problem) arg minV ( )目標(biāo)函數(shù)(Objective function)NNNV ( ) 1 | y(k ) y(k ) |2N 1 | y(k ) g(x , ) |2NkNNt 1t 1SJTU12第四章：最小二乘擬合最小二乘之外的其他指標(biāo)比如：ax | y(k ) g(xk , ) |“unknown-but-bounded”mmin | y(k ) g(xk , )

6、| instensive l1 normsupport vector machines曲線擬合方法從優(yōu)化指標(biāo)角度分類(lèi)：最小二乘指標(biāo)其他指標(biāo)SJTU13第四章：最小二乘擬合為什么要選擇最小二乘指標(biāo)？于1809年他的著作天的最小二乘法的方法體運(yùn)動(dòng)論中。1829年，提供了最小二乘法的優(yōu)化效果的證明，定理。-見(jiàn)Gauss在誤差零均值，同方差，且互不相關(guān)的線性回歸模型中，回歸系數(shù)的最佳無(wú)偏線性估計(jì)（BLUE）就是最小方差估計(jì)。一般而言，任何回歸系數(shù)的線性組合之BLUE就是它的最小方差估計(jì)。在這個(gè)線性回歸模型中，誤差既不需要假定正態(tài)分布，也不需要假定獨(dú)立（但是需要不相關(guān)這個(gè)更弱的條件），更不需要假定同分布

7、(uniform distribution)。SJTU14第四章：最小二乘擬合最小二乘(Weighted least squares)y(k ) g0 (xk ) e(k )y(k ) gN (xk )過(guò)程模型優(yōu)化問(wèn)題(Optimization problem)minV ( ) argNNNV ( ) 1 N 1 | y(k ) y(k ) |2 / , ) |2 / 目標(biāo)函數(shù)| y(k ) g(xNkkkNNt 1t 1正比于kth測(cè)量數(shù)據(jù)的“可靠性(reliability)” Ee2 (k )kkSJTU15第四章：最小二乘擬合最小二乘(Weighted least squares)y(k

8、 ) g0 (xk ) e(k )y(k ) gN (xk )過(guò)程模型優(yōu)化問(wèn)題(Optimization problem) arg minV ( )NNNV ( ) 1 N 1 L(x ) | y(k ) y(k ) |2 / , ) |2 / 目標(biāo)函數(shù)L(x ) | y(k ) g(xNkkkkkNNt 1t 1L(xk )再加一個(gè)權(quán)重，表示點(diǎn) xk的“相關(guān)性(relevance)”SJTU16第四章：最小二乘擬合正則化最小二乘(Regularized least squares)y(k ) g0 (xk ) e(k )y(k ) gN (xk )過(guò)程模型優(yōu)化問(wèn)題(Optimization

9、problem) arg minV ( ) | |2 NNNV ( ) 1 | y(k ) y(k ) |2N 1 , ) |2| y(k ) g(x目標(biāo)函數(shù)NkNNt 1t 1 | |2懲罰模型的復(fù)雜度，可取很多不同的形式SJTU17第四章：最小二乘擬合線性最小二乘g(x, ) 對(duì)于是線性的，則稱為線性最小二乘如果參數(shù)化模型g(x, ) (x)T 的二次型式基本指標(biāo)變成 1NN ) ( y(k ) (x ) )| Y |T22V (Nkk 1 1 (1)2 (1)n (1) (1) y(1) y(2) (2) (2) (2) (2)Y 12n y(N ) (N ) (n) (N ) (N )

10、 12n線性最小二乘的最優(yōu)解可表達(dá)成形式 (T )1TYNSJTU18第四章：最小二乘擬合線性最小二乘：線性方程組求解Ax b已知A和b，求xSJTU19第四章：最小二乘擬合曲線擬合實(shí)例：測(cè)量數(shù)據(jù)e=5*randnData size=50SJTU20第四章：最小二乘擬合曲線擬合實(shí)例：測(cè)量數(shù)據(jù)、估計(jì)的函數(shù)估計(jì)的模型：g(x) = p1*x4 + p2*x3 + p3*x2 + p4*x + p5e=5*randn Data size=50 M size=5M type=polySJTU21第四章：最小二乘擬合曲線擬合實(shí)例：測(cè)量數(shù)據(jù)、估計(jì)的函數(shù)、真實(shí)函數(shù)估計(jì)的模型：g(x) = p1*x4 + p

11、2*x3 + p3*x2 + p4*x + p5e=5*randn Data size=50 M size=5M type=polySJTU22第四章：最小二乘擬合曲線擬合實(shí)例：估計(jì)的模型與真實(shí)模型對(duì)比估計(jì)的模型：真實(shí)的模型：Linear mPoly4:g(x) = p1*x4 + p2*x3 + p3*x2 + p4*x + p5g0(x) = p1*x4 + p2*x3 + p3*x2 + p4*x + p5CoefficientsCoefficientsp1 = p2 = p3 = p4 = p5 =0.08567-0.94260.52014.348-4.212p1 = p2 = p3

12、= p4 = p5 =0.1-1.2210SJTU23第四章：模型大小對(duì)擬合的影響曲線擬合實(shí)例3種大小的模型M size=3估計(jì)的模型：g(x) = p1*x2 + p2*x + p3M size=5估計(jì)的模型：g(x) = p1*x4 + p2*x3 + p3*x2 + p4*x + p5M size=9估計(jì)的模型：g(x) = p1*x8 + p2*x7 + p3*x6 + p4*x5 + p5*x4 + p6*x3 + p7*x2 + p8*x + p9SJTU24第四章：模型大小對(duì)擬合的影響曲線擬合實(shí)例：測(cè)量數(shù)據(jù)、估計(jì)的函數(shù)、真實(shí)函數(shù)估計(jì)的模型：g(x) = p1*x2 + p2*x

13、+ p3Coefficientsp1 = p2 = p3 =1.14-17.6821.2e=5*randn Data size=50 M size=3M type=polyCoefficientsp1 = p2 = p3 = p4 = p5 =0.1-1.2210真實(shí)的模型：g0(x) = p1*x4 + p2*x3 + p3*x2 + p4*x + p5SJTU25第四章：模型大小對(duì)擬合的影響曲線擬合實(shí)例：測(cè)量數(shù)據(jù)、估計(jì)的函數(shù)、真實(shí)函數(shù)估計(jì)的模型：g(x) = p1*x4 + p2*x3 + p3*x2 + p4*x + p5Coefficientsp1 = p2 = p3 = p4 = p

14、5 =0.08567-0.94260.52014.348-4.212e=5*randn Data size=50 M size=5M type=polyCoefficientsp1 = p2 = p3 = p4 = p5 =0.1-1.2210真實(shí)的模型：g0(x) = p1*x4 + p2*x3 + p3*x2 + p4*x + p5SJTU26第四章：模型大小對(duì)擬合的影響Coefficients曲線擬合實(shí)例：測(cè)量數(shù)據(jù)、估計(jì)的函數(shù)、真實(shí)函數(shù)p1 = p2 = p3 = p4 = p5 = p6 = p7 = p8 = p9 =-0.00050360.02027-0.33422.909-14.

15、2638.92-57.4240.27-9.046估計(jì)的模型：g(x) = p1*x8 + p2*x7 + p3*x6 + p4*x5 + p5*x4 + p6*x3 + p7*x2 + p8*x + p9e=5*randn Data size=50 M size=9M type=polyCoefficientsp1 = p2 = p3 = p4 = p5 =0.1-1.2210真實(shí)的模型：g0(x) = p1*x4 + p2*x3 + p3*x2 + p4*x + p5SJTU27第四章：基函數(shù)類(lèi)型對(duì)擬合的影響曲線擬合實(shí)例2種不同的基函數(shù)M type=poly估計(jì)的模型：g(x) = p1*x

16、4 + p2*x3 + p3*x2 + p4*x + p5M type=gauss估計(jì)的模型：g(x) = a1*exp(-(x-b1)/c1)2) + a2*exp(-(x-b2)/c2)2)SJTU28第四章：基函數(shù)類(lèi)型對(duì)擬合的影響曲線擬合實(shí)例：測(cè)量數(shù)據(jù)、估計(jì)的函數(shù)、真實(shí)函數(shù)Coefficients a1 =-69.87估計(jì)的模型：g(x) = a1*exp(-(x-b1)/c1)2) + a2*exp(-(x- b2)/c2)2)e=5*randn Data size=50 M size=2M type=gaussb1 = c1 =7.2222.488a2 = -6.971e+009 b

17、2 =-10.17c2 =2.239Coefficientsp1 = p2 = p3 = p4 = p5 =0.1-1.2210真實(shí)的模型：g0(x) = p1*x4 + p2*x3 + p3*x2 + p4*x + p5SJTU29第四章：基函數(shù)類(lèi)型對(duì)擬合的影響曲線擬合實(shí)例：測(cè)量數(shù)據(jù)、估計(jì)的函數(shù)、真實(shí)函數(shù)估計(jì)的模型：g(x) = a1*exp(-(x-b1)/c1)2) + a2*exp(-(x-b2)/c2)2) + a3*exp(-(x-b3)/c3)2) e=5*randnData size=50 M size=3M type=gaussCoefficientsp1 = p2 = p3

18、 = p4 = p5 =0.1-1.2210真實(shí)的模型：g0(x) = p1*x4 + p2*x3 + p3*x2 + p4*x + p5SJTU30第四章：基函數(shù)類(lèi)型對(duì)擬合的影響曲線擬合實(shí)例：測(cè)量數(shù)據(jù)、估計(jì)的函數(shù)、真實(shí)函數(shù)估計(jì)的模型：g(x) = a1*exp(-(x-b1)/c1)2) + a2*exp(-(x-b2)/c2)2) +a3*exp(-(x-b3)/c3)2) + a4*exp(-(x-b4)/c4)2) e=5*randnData size=50 M size=4M type=gaussCoefficientsp1 = p2 = p3 = p4 = p5 =0.1-1.22

19、10真實(shí)的模型：g0(x) = p1*x4 + p2*x3 + p3*x2 + p4*x + p5SJTU31第四章：基函數(shù)類(lèi)型對(duì)擬合的影響曲線擬合實(shí)例：測(cè)量數(shù)據(jù)、估計(jì)的函數(shù)、真實(shí)函數(shù)估計(jì)的模型：a1*exp(-(x-b1)/c1)2) + a2*exp(-(x-b2)/c2)2) +a3*exp(-(x-b3)/c3)2) + a4*exp(-(x-b4)/c4)2) +a5*exp(-(x-b5)/c5)2) + a6*exp(-(x-b6)/c6)2)e=5*randn Data size=50 M size=6M type=gaussCoefficientsp1 = p2 = p3 =

20、 p4 = p5 =0.1-1.2210真實(shí)的模型：g0(x) = p1*x4 + p2*x3 + p3*x2 + p4*x + p5SJTU32第四章：數(shù)據(jù)量對(duì)擬合的影響曲線擬合實(shí)例2種不同長(zhǎng)度的測(cè)量數(shù)據(jù)D size=50D size=500SJTU33第四章：數(shù)據(jù)量對(duì)擬合的影響曲線擬合實(shí)例：測(cè)量數(shù)據(jù)、估計(jì)的函數(shù)、真實(shí)函數(shù)估計(jì)的模型：g(x) = p1*x4 + p2*x3 + p3*x2 + p4*x + p5Coefficientsp1 = p2 = p3 = p4 = p5 =0.08567-0.94260.52014.348-4.212e=5*randn D size=50 M si

21、ze=5M type=polyCoefficientsp1 = p2 = p3 = p4 = p5 =0.1-1.2210真實(shí)的模型：g0(x) = p1*x4 + p2*x3 + p3*x2 + p4*x + p5SJTU34第四章：數(shù)據(jù)量對(duì)擬合的影響曲線擬合實(shí)例：測(cè)量數(shù)據(jù)、估計(jì)的函數(shù)、真實(shí)函數(shù)Coefficients估計(jì)的模型： p1*x4 + p2*x3 + p3*x2 + p4*x + p5p1 = p2 = p3 = p4 = p5 =0.1015-1.2422.365-0.068890.6975e=5*randn D size=500 M size=5M type=polyCoefficientsp1 = p2 = p3 = p4 = p5 =0.1-1.2210真實(shí)的模型：g0(x) = p1*x4 + p2*x3 + p3*x2 + p4*x + p5SJTU35第四章：噪聲對(duì)擬合的影響曲線擬合實(shí)例3種不同噪聲下的測(cè)量數(shù)據(jù)e=5*randne=0.5*randne=50*randnSJTU36第四章：噪聲對(duì)擬合的影響曲線擬合實(shí)例：測(cè)量數(shù)據(jù)、估計(jì)的函數(shù)、真實(shí)函數(shù)估計(jì)的模型：g(x) = p1*x4 + p2*x3 + p3*x2 + p4*x + p5Coefficientsp1 = p2 = p3 = p4 = p5 =0.08567-0.94260