面向SLO的資源調(diào)度優(yōu)化技術(shù)

1、面向SLO的資源調(diào)度優(yōu)化技術(shù)SLO-Oriented Scheduling with Resource Estimation to Optimize Utilization SLO-Oriented Scheduling with Resource Estimation to Optimize Utilization OverviewBackground and Motivations:Why we need SLO-oriented scheduling for diverse workloads in the CloudDesigns and Methodologies:Resource

2、SLO enforcementsResource estimationCold page reclaimBalanced oversubscription in real timeAchievements and Pitfalls in productionFuture ExplorationsSLO-Oriented Scheduling with Resource Estimation to Optimize Utilization Alibaba Serverless Infra.Background and MotivationsA great variety of workloads

3、 surging into the cloudUnified OperatorSLO-Oriented Scheduling with Resource Estimation to Optimize Utilization Background and MotivationsKubeDL: unified Job operatorKubeDL has been open-sourced!！X-DeepLearningCRDsCommon Lib.Scheduling EnhancementJob monitoringSLO-Oriented Scheduling with Resource E

4、stimation to Optimize Utilization Background and MotivationsSLO varies, so does the corresponding resource requirement.Web services: low response time minimum interferenceData analytics: deadline-driven potentially preemptibleAI workloads: mixed.How to make the best use of resources while respecting

5、 variant SLOs?SLO-Oriented Scheduling with Resource Estimation to Optimize Utilization Designs and Methodologies FrameworkDifferent levels of resource SLOs, enforced by QoS and priority class.Oversubscription, including RAMBalanced scheduling with real-time resource estimation to avoid hotspots.SLO-

6、Oriented Scheduling with Resource Estimation to Optimize Utilization Designs and Methodologies Resource SLOsQoS class: e.g., Guaranteed, besteffort, etc.Resource guarantees at runtimeCPU core bindings, kernel resource control.Priority bands: e.g., system, production, preemptible, etc.Chances a pod g

7、ets resource allocation at timeAlign the behavior of OOM killer and eviction policyo Restrict to sensible combinations of Priority and QoS class.QoS ClassPriority ClassPreemptionEvictionSchedulingOOMcpu bindingsrequest v.s. limitSLO-Oriented Scheduling with Resource Estimation to Optimize Utilizatio

8、n Designs and Methodologies Resource estimationPredict future resource consumption based on the historic knowledgeReclaimed resources could be safely overbookedSLO-Oriented Scheduling with Resource Estimation to Optimize Utilization Designs and Methodologies Cold page reclamationCached pages are mem

9、ory eaters! (Nearly half of total memory allocation)Identify inactive/cold pages routinely to make room for oversubscription.SLO-Oriented Scheduling with Resource Estimation to Optimize Utilization Designs and Methodologies Balanced schedulingo Balance resource usage to avoid hotspots.Pod priority q

10、ueueGet podHigh SLOLow SLOCheck resource budget&KHFN EXGJHW DIWHU UHFODPDWLRQFiltering Phase%DODQFH 1RGH XVDJH%DODQFH 1RGH XVDJH(QGScoring Phase+DV IHDVLEOH 1RGHY+DV IHDVLEOH 1RGHY (QRXJK DYDLODYEOHUHVRXFHVN3UHHPSWLRQNN.LOO %HVWHRU SRGVRequeueYAchievements and Pitfalls in ProductionAchievementsOvers

11、ubscribed CPU cores by 20%.Reclaimed cold pages by over 60%.PitfallsBe compatible with existing QoS and Priority classes.Memory is the bottleneck and should be cautiously oversubscribed.Just-in-time resource estimation updates.SLO-Oriented Scheduling with Resource Estimation to Optimize Utilization Future ExplorationsEviction trigge