類(lèi)腦計(jì)算ppt課件

1、類(lèi)腦計(jì)算,CONTENTS,3,類(lèi)腦理念與實(shí)踐,General AI 認(rèn)知 Narrow AI 感知 傳統(tǒng)思想： 認(rèn)知科學(xué)了解大腦意識(shí)的形成 模仿人腦設(shè)計(jì)算法( Empirical ) 當(dāng)前思想： 神經(jīng)生物學(xué)了解大腦結(jié)構(gòu) 結(jié)構(gòu)仿真形成一定規(guī)模的神經(jīng)元網(wǎng)絡(luò) 再思考可以做什么( Heuristic,類(lèi)腦理念與實(shí)踐,感知: Perceptron LeNet AlexNet VGG GoogLeNet ResNet 認(rèn)知： BrainScaleS Neurons 4 million Synapses 1 billion Power 10 kW TrueNorth Neurons 16 million

2、Synapses 4 billion Power 1 W SpiNNaker Neurons 460 million Synapses 460 billion Power 50 kW Human Brain Neurons 100 billion Synapses 100 Tera Power 20 W,Deeper than Deeper,Bigger than Bigger,5,上界：認(rèn)知科學(xué) 功能模擬 類(lèi)腦計(jì)算 下界：神經(jīng)科學(xué) 解析仿真 上確界：神經(jīng)科學(xué) 結(jié)構(gòu)模仿 類(lèi)腦實(shí)踐 下確界：傳統(tǒng)計(jì)算機(jī)體系結(jié)構(gòu),類(lèi)腦理念與實(shí)踐,6,神經(jīng)形態(tài)認(rèn)知計(jì)算,神經(jīng)元狀態(tài)方程： Hodgkin-Huxley方

3、程,7,神經(jīng)形態(tài)認(rèn)知計(jì)算,CMOS模擬： 憶阻器： 憶阻器是天然的突觸模擬器件，生物突觸釋放Ca、Na離子改變傳導(dǎo)性，憶阻器通過(guò)釋放氧離子改變傳導(dǎo)性。 1個(gè)憶阻器可以模擬一個(gè)突觸的行為。 當(dāng)前集成水平可以達(dá)到10GB/cm2存儲(chǔ)密度（近似人腦神經(jīng)元密度），主頻200MHz（比人腦腦電頻率快106倍），未來(lái)可達(dá)到500G/cm21000MHz,8,Spiking Neurons and Spiking Neural Network Functions and Consumption,神經(jīng)形態(tài)認(rèn)知計(jì)算,9,ANN Assumed that neurons represent information

4、 through their mean rates of action potential firing. NonLinear(Sum() SNN How Spikes convey information ? Rate : spike count within a time window Temporal : precise time = time array,神經(jīng)形態(tài)認(rèn)知計(jì)算,10,FeatureMap Spatiotemporal Patterns Classification Solved. How to Code ? Unsolved,神經(jīng)形態(tài)認(rèn)知計(jì)算,11,Implementing

5、 Instances,神經(jīng)形態(tài)認(rèn)知計(jì)算,12,Implementing Instances Convolution Max Pooling STDP (Unsupervised Learning in Feature Extraction) Supervised Learning rules to fine tuning Spiking Network,神經(jīng)形態(tài)認(rèn)知計(jì)算,13,STDP The process adjusts the connection strengths based on the relative timing of a particular neurons output

6、and input action potentials (or spikes,神經(jīng)形態(tài)認(rèn)知計(jì)算,14,Implementing Instances,神經(jīng)形態(tài)認(rèn)知計(jì)算,15,Neuron Software 分子層面精細(xì)建模 最大程度模擬神經(jīng)元與突觸行為 對(duì)視網(wǎng)膜中央?yún)^(qū)域 2mm 范圍內(nèi)所有神經(jīng)元進(jìn)行建模 天河2號(hào)超算輔助運(yùn)算,類(lèi)腦理念與實(shí)踐,16,Spiking Array 時(shí)空關(guān)系上的連續(xù)性 超高速視屏 20,000 Frame/s Line Motion Illusion 視網(wǎng)膜與V1層功能區(qū)的協(xié)作產(chǎn)生線運(yùn)動(dòng)錯(cuò)覺(jué) V1區(qū)含有方向選擇細(xì)胞 V1區(qū)腦皮層之間的互聯(lián)形成對(duì)位置方向的感知 V1區(qū)腦皮

7、層在無(wú)刺激輸入時(shí)也會(huì)產(chǎn)生自發(fā)性放電現(xiàn)象,類(lèi)腦理念與實(shí)踐,17,Neural Encoding of Motion Direction Single Neuron Neural Population,計(jì)算神經(jīng)科學(xué),CANN Continuous Attractor Neural Network,18,Dynamical and Adaptive Information Processing From Rate to Correlation Code SFA Spike frequency Adaptation Neuronal response attenuates after experien

8、cing long time firing. Slow negative feedback modulation to neuronal response. Traveling Wave in CANN A moving bump in the network without relying on external drive,計(jì)算神經(jīng)科學(xué),19,The strategy of the brain Every animal adopts to its own optimal time scale suitable for its own survival in the natural envi

9、ronment. The brain co-evolves strategies to compensate delays. Why Neural Delays? To implement temporal code. To integrate multiple sensory cues. To integrate temporal information over time for reliable responding. Why Adaptive? Responding fast to a novel object with strong independent firing-rate c

10、ode. Encoding information efficiently with low frequency but correlation code lately,計(jì)算神經(jīng)科學(xué),20,深度學(xué)習(xí)處理器,圖像處理 : GPU 信號(hào)處理 : DSP 智能處理 : DianNao Family 未來(lái)的硬件設(shè)備可能都需要一個(gè)深度學(xué)習(xí)算法的硬件處理器 目前GPU芯片每年出貨量：0.64 Billion 產(chǎn)值：50 Billion $ NameSpeed upEnergy Efficiency DianNao :1.1x GPU100 x CPU DaDianNao: 21x GPU330 x GPU

11、 PuDianNao: 1.1x GPU200 x GPU ShiDianNao: 28.94x GPU4688x GPU 320mW,21,深度學(xué)習(xí)處理器,如何加速？ 把數(shù)據(jù)從內(nèi)存搬運(yùn)到硬件運(yùn)算單元，甚至比運(yùn)算本身更耗能量。 小尺度但支持大規(guī)模神經(jīng)網(wǎng)絡(luò) 速度：把訪存帶寬用起來(lái)，盡可能提高性能 能耗：通過(guò)優(yōu)化片上存儲(chǔ)層次盡量減少訪存次數(shù),22,深度學(xué)習(xí)處理器,DianNao 16 Neurons Win 8KB Wout 8KB Weight 32KB Disadvantage: Memory Exchange ASPLOS Best Paper,23,深度學(xué)習(xí)處理器,DaDianNao 16 Cores 16 Neurons 全網(wǎng)絡(luò)模型片上存儲(chǔ) 無(wú)內(nèi)存訪問(wèn) EDRAM MICRO Best Paper,24,深度學(xué)習(xí)處理器,PuDianNao 關(guān)注數(shù)據(jù)局部性 設(shè)計(jì)專(zhuān)用功能器件 三根柱子： InstBuf 指令緩存 HotBuf ColdBuf 常見(jiàn)運(yùn)算： 向量?jī)?nèi)積