2024微軟人工智能系統(tǒng)計算圖的編譯與優(yōu)化

1 y+*bxy+*bxw算子表達式IR2.內(nèi)存優(yōu)化3.內(nèi)核優(yōu)化4.調(diào)度優(yōu)化 AddLogWhileSubMatMulMergeMulConvBroadCastDivBatchNormReduceAddLogWhileSubMatMulMergeMulConvBroadCastDivBatchNormReduceReluLossMapTanhTransposeReshapeExpConcatenateSelectFloorSigmoid….. xwxw*b+yxwxw*b+y p1/(p0+(p3-p4)) PAGE12PAGE12Batchsame-typeoperatorsleverageGPUmassiveparallelism++×??+M??×????????+ +MMMMMRf Rzht-1xtData-flowgraphofaGRUcellWzWoWfht+×??+M+×??+M??×????????+ +MMMMMRf Rzht-1xtWzWoWfht+×??×??+M????????Mht-1RWxthtData-flowgraphofaGRUcellPAGEPAGE14 1.計算圖優(yōu)化2內(nèi)存優(yōu)化3.內(nèi)核優(yōu)化4.調(diào)度優(yōu)化PAGEPAGE19 PAGE21PAGE21AutoTM:AutomaticTensorMovementinHeterogeneousMemorySystemsusingIntegerLinearProgramming AutoTM:AutomaticTensorMovementinHeterogeneousMemorySystemsusingIntegerLinearProgrammingPAGEPAGE23 1.計算圖優(yōu)化2.內(nèi)存優(yōu)化3內(nèi)核優(yōu)化4.調(diào)度優(yōu)化問題問題:每個后端平臺針對每個算子都需要單獨實現(xiàn)至少一個內(nèi)核考慮：編程模型、數(shù)據(jù)排布、線程模型、緩存大小等等y+*bxw xw *xw*b+y張量運算表達式張量運算表達式C[i]=A[i]+B[i]361.計算圖優(yōu)化2.內(nèi)存優(yōu)化3.內(nèi)核優(yōu)化4調(diào)度優(yōu)化y+*bxwy+*bxw算子表達式IRD[i]=max(C[i],0)C[i]=A[i]+B[i]

計算圖優(yōu)化XLA,TVM,nGraph計算圖優(yōu)化XLA,TVM,nGraphD[i]=max(A[i]+B[i],0)D[i]=max(A[i]+B[i],0)MatMulMatMul:C[i]=reduce_sum(A[m,i]*B[i,n])globalglobaladd_relu_kernel(args…){//..Code}voidlaunch_kernel(add_relu_kernel);算子生成AutoTVM,TC,Halide,Tiramisu,etcSMSMSMSMSMSMSMSMSMSMSM如何最終將每個算子的計算映射到硬件上？NNFusion:全局計算調(diào)度優(yōu)化xw*b+yxw*b+y算子表達式IRTask-parallelGraph算子表達式IRBlockFusionOptimizerBlockFusionOptimizer 計算調(diào)度優(yōu)化計算調(diào)度優(yōu)化通過引出新的調(diào)度原語來支持任務(wù)級調(diào)度挑戰(zhàn)：調(diào)度算子的任務(wù)到挑戰(zhàn)：調(diào)度算子的任務(wù)到GPU上的挑戰(zhàn) 錯誤依賴死鎖

block7Conv

block8Conv

block9

block10

CannotschedulebarrierCo-scheduleto

Matmulblock0ConvMatmul

Matmulblock1ConvMatmul

Matmulblock2ConvMatmul

Matmulblock3ConvMatmul

Matmulblock4ConvMatmul

block5ConvMatmul

WaitforbarrierUnabletoexecuteGPUGPUSMSMSMSMSMSM依賴關(guān)系的映射依賴關(guān)系的映射 MatMulConvBE0BE1BE2BE3BE4BE5vblock6MatMulConvBE0BE1BE2BE3BE4BE5vblock6vblock5vblock5vblock4vblock10vblock4vblock7vblock0vblock8vblock1vblock6vblock0vblock7vblock1vblock2vblock3vblock8vblock2vblock9vblock3GPUGPUSMSMSMSMSMSM并行度的映射并行度的映射 [64,512][512,128][64,512][512,128]LowerLatencyUnderutilizationofGPUresourceBE0BE1BE2LowerLatencyUnderutilizationofGPUresourceBE0BE1BE2BE3BE4BE5vblock5vblock4 MatMul ??Convvblock6vblock7vblo