2023云棲大會：PolarDB for AI -從數(shù)據(jù)管理走向數(shù)據(jù)智能基礎(chǔ)設施

PolarDBforAI:從數(shù)據(jù)管理走APSARA云棲大會83%CEOs相信GarterAPSARA云棲大會特征、模型管理難JSONAPSARA云棲大會JSON數(shù)據(jù)、特征、模型通過數(shù)據(jù)庫來統(tǒng)一管理統(tǒng)一做運維成本APSARA云棲大會一起存一起算一起存數(shù)據(jù)、特征、模型數(shù)據(jù)、特征、模型一起存流程簡單化、低代碼量、更低的開發(fā)成本&CRMCRMAPSARA云棲大會CRMCRMCRMAPSARA云棲大會數(shù)據(jù)管理+特征管理+模型管理SQL+SQLforMLOps數(shù)據(jù)特征模型…模型創(chuàng)建模型模型評估模型模型調(diào)參模型組合模型模型部署CREATEMODELairlines_gbm_copy1WITH(model_class='lightgbm’,x_cols='Airline,Flight,AirportFrom,AirportTo,DayOfWeek,Time,Length',y_cols='Delay’,model_parameter=(boosting_type='gbdt',n_estimators=100,max_depth=8,num_leaves=256))AS(SELECT*FROMairlines_train)SELECTTripID,DelayFROMPREDICT(MODELairlines_gbm_copy1,SELECT*FROMairlines_train_1000_copy1)WITH(s_cols='TripID,Delay’,x_cols='Airline,Flight,AirportFrom,AirportTo,DayOfWeek,Time,Length’,y_cols='Delay',primary_key='TripID',AIAIAIAIAPSARA云棲大會FeatureCreation,ModelCreation,ModelInference,etc.基于高速RDMA的一個系統(tǒng)：PolarDB一套語言：SQLSQLParser/OptimizerRow-SQLParser/OptimizerRow-ExeColumn-ExeRow-indexColumn-indexAPSARA云棲大會PolarProxyPolarProxydispatcherdispatcherdispatcherdispatcherdispatcherdispatcherfeature模型創(chuàng)建CREATEMODELairlines_gbmWITH(model_class='lightgbm’,x_cols='Airline,Flight,AirportFrom,AirportTo,DayOfWeek,Time,Length',y_cols='Delay’,model_parameter=(boosting_type='gbdt',n_estimators=100,max_depth=8,num_leaves=256))模型評估SELECTDelayFROMevaluate(MODELairlines_gbm,SELEWITH(x_cols='Airline,Flight,AirportFrom,AirportTo,DayOfWeek,Time,Length’,y_cols='Delay',metrics='acc');模型推理SELECTTripID,DelayFROMPREDICT(MODELairlines_SELECT*FROMairlines_train_1000_copy1)WITH(s_cols='TripID,Delay’,x_cols='Airline,Flight,AirportFrom,AirportTo,DayOfWeek,Time,Length',y_cols='Delay’,primary_key='TripID',mode='async')INTOlightgbm_v2_predict82201;等SQLParser/OptimizerRow-ExeColumn-ExeRow-SQLParser/OptimizerRow-ExeColumn-ExeRow-indexColumn-indexAPSARA云棲大會模型推理SELECTTripID,DelayFROMPREDICT(MODELairlines_SELECT*FROMairlines_train_1000_copy1)WITH(s_cols='TripID,Delay’,x_cols='Airline,Flight,AirportFrom,AirportTo,DayOfWeek,Time,Length',y_cols='Delay’,primary_key='TripID',mode='async')INTOlightgbm_v2_predict82201;PolarProxyPolarProxydispatcherdispatcherdispatcherdispatcherdispatcherdispatcher feature數(shù)據(jù)模型推理結(jié)果AIAugmentedTable（AAT）APSARA云棲大會模型創(chuàng)建CREATEMODELairlines_gbmWITH(model_class='lightgbm’,x_cols='Airline,Flight,AirportFrom,AirportTo,DayOfWeek,Time,Length',y_cols='Delay’,model_parameter=(boosting_type='gbdt',n_estimators=100,max_depth=8,num_leaves=256))模型評估SELECTDelayFROMevaluate(MODELairlines_gbm,SELEWITH(x_cols='Airline,Flight,AirportFrom,AirportTo,DayOfWeek,Time,Length’,y_cols='Delay',metrics='acc');模型推理（離線）SELECTTripID,DelayFROMPREDICT(MODELairlines_SELECT*FROMairlines_train_1000_copy1)WITH(s_cols='TripID,Delay’,x_cols='Airline,Flight,AirportFrom,AirportTo,DayOfWeek,Time,Length',y_cols='Delay’,primary_key='TripID',mode='async')INTOlightgbm_v2_predict82201;模型推理（在線）SELECTTripID,DelayFROMPREDICT(MODELairlines_SELECT*FROMairlines_train_1000_copy1)WITH(s_cols='TripID,Delay’,x_cols='Airline,Flight,AirportFrom,AirportTo,DayOfWeek,Time,Length',y_cols='Delay’,primary_key='TripID',);模型描述DESCRIBEMODELmodel_name模型刪除DROPMODELmodel_name模型上傳UPLOADMODELmodel_nameWITH(model_location=''模型部署CREATEFUNCTIONmy_lr_modelRETURNSREALSONAME"#ailib#_my_lr_model.so";特征創(chuàng)建CREATEFEATUREfeature_nameWITH(feature_class='',parameters=())AS(SELECTselect_expr[,select_expr]...FROMtable_reference)特征更新UPDATEFEATUREfeature_nameWITH(feature_class='',parameters=())AS(SELECTselect_expr[,select_expr]...FROMtable_reference)特征刪除DROPFEATUREfeature_name據(jù)到模型到應用）場景三：Out-of-the-box據(jù)到模型到應用）場景三：Out-of-the-boxsolution從數(shù)據(jù)->算法->模型->使用從方案->使用模型推理模型加密特征計算最終結(jié)果模型部署結(jié)果模型創(chuàng)建模型訓練模型評估模型推理模型推理APSARA云棲大會從模型->使用模型模型上傳分類/聚類等模型創(chuàng)建模型訓練模型模型創(chuàng)建模型訓練模型評估結(jié)果查看模型應用模型描述模型列表模型在線推理模型離線推理APSARA云棲大會模型創(chuàng)建模型開發(fā)模型開發(fā)x_cols='Airline,Flight,AirportFrom,AirportTo,DayOfWeek,Time,Length',y_cols='Delay’,模型評估WITH(x_cols='Airline,Flight,AirportFrom,AirportTo,DayOfWeek,Ti結(jié)果查看SHOWTASK`df05244e-21f7-11ed-be66-xxxx模型描述模型列表模型在線推理SELECTTripID,DelayFROMPREDICT(MODELairlines_SELECT*FROMairlines_train_1000_copy1)WITH(s_cols='TripID,Delay’,x_cols='Airline,Flight,AirportFrom,AirportTo,DayOfWeek,Time,Length',y_cols='Delay’,primary_key='TripID')INTOlightgbm_v2_predict1030;模型離線推理SELECTTripID,DelayFROMPREDICT(SELECT*FROMairlines_train_1000_copy1)WITH(s_cols='TripID,Delay’,x_cols='Airline,Flight,AirportFrom,AirportTo,DayOfWeek,Time,Length',y_cols='Delay’,primary_key='TripID')/zh/polardb/polardb-for-mysql/user-guide/polardb-for-ai訓練訓練好的模型APSARA云棲大會模型上傳(model_location='https://xxxx/model.pkl?Expires=xxxx&OSSAccessKeyId=xxxx&Signature=xxxx’,req_location='https://xxxx/requirements.txt?Expires=xxxx&OSSAccessKeyId=xxxx&Signature=xxxx')模型部署模型在線推理SELECTYFROMPREDICT(MODELmy_morequirements.txtWITH(x_cols=requirements.txtx20,x21,x22,x23,x24,x25,x26,x27,x28',y_cols='');模型部署DEPLOYMODELmodel_nameWCREATEFUNCTIONfunction_nameRETURNSreturn_valueSONAME"soname";SELECTfunction_name("content");_polar4ai_tongyi_polar4ai_tongyi_sa,正向今年暑期檔最大的變化是現(xiàn)實題材電影大受歡迎，好萊塢大片則表現(xiàn)不佳。中國電影市場中，現(xiàn)實題材電影早已成為主流，包括《我不是藥神》、《掃毒2》、《怒火·重案》等_polar4ai_tongyi_summarize,selectproduct_reviewfromreviewswhereid=3)情感分析idproduct_idproduct_review11華為最新手機Mate60Pro自開賣后銷售火2132這個東西只是看著還行，實際體驗上不太commentsididcomment3今年暑期檔最大的變化是：好萊塢大片的失敗，與現(xiàn)實效，在題材上以動作、奇幻、冒險為主，主要靠視覺轟炸。而在后疫情時代，觀眾則越來越關(guān)注現(xiàn)實的、切身注一擲》則切中人們對電信詐騙、緬甸詐騙園、荷官設《怒火·重案》《中國醫(yī)生》，再到2022年的《人生大事》……還有其他檔期的《奇跡·笨小孩》《我的姐姐》等等APSARA云棲大會AIAI開發(fā)效率提升AI相關(guān)軟硬件成本下降A(chǔ)PSARA云棲大會AComparativeStudyofin-DatabaseInferenceApproaches（ICDE2022）SmartLite:ADBMSforServingMultipleNeuralModelswithConstraintResource（PVLDB2024）APSARA云棲大會機器學習模型生命周期相關(guān)操作數(shù)據(jù)分析數(shù)據(jù)分析模型設計模型設計特征工程模型訓練模型評估模型評估模型調(diào)優(yōu)模型調(diào)優(yōu)模型上線模型上線反復迭代多個系統(tǒng)PolarDB；多個節(jié)點:RW,RO,AI;FeatureCreation,ModelCreation,ModelInference,etc.A