智算時代基于 Kubernetes 落地云原生 AI

張凱阿里云云原生應(yīng)用平臺2023年12月1日learninglearning)?度享..更快的創(chuàng)新和迭代交付.DockerKubernetesKubeflowKserveFluidMLFlow 支持TensorFlow,Pytorch,Deep任務(wù)級調(diào)度策略（Gang,Binpack,Capacity,架），Meegatron-llm云AIAI平臺/DatasetProc速Kube-queueDatasetProc速Kube-queueKubeflowArenaPipelineDeepspeed-Kserve/Tritonmer/RDM度yCPU(x86/arn)?Total/allocatedGPUKubeletgpu0gpu0gpu0gpu1gpu0gpu0gpu0gpu0gpu0gpu1gpu0gpu0ContainerContainerArgoArgo片/Ahttps/fluid-cloudnative/fluidSOSO分布式緩存Dataset源FluidvsOSSFS(20Gb/s)70000600005000040000300002000000ModelServingProgramModelHuggingFaceTGIServer/StableDiffusion/…DistributedCacheShardShardShardShardShardLLM模型加載耗時對比（單位：秒）ArenaArena--name=tf-dist-data--workers=6--gpus=2--data=tfdata:/data_dir–--env=num_batch=100--env=batch_size=80--tensorboard--image=ali-tensorflow:gpu-tf-1.6.0\"/root/hvd-distribute.sh122”ArenaArena/job:ps/task:0/job:worker/task:1/job:worker//job:ps/task:0/job:worker/task:1/job:worker/task:2/job:ps/task:0/job:worker/task:0(chief)elv3elv3 BaichuanBaichuan--image=xxx-text-generation-inference:0.8\"text-generation-launcher--disable-custom-kernels--model-idTensorflow--label/eci=true\--label/fluid-sidecar-target=eci\--annotation/eci-use-specs=ecs.gn6v-c8g1.2xlarge\--imag=xxx-bloom-finetune:v2\--data=oss-data:/mymodels\RemoteShuffleRemoteShuffle化SparkApplicationOperatorSparkDriverSparkExecutorPods天倚SparkExecutorPod倚天FluidACKSparkApplicationOperatorSparkDriverSparkExecutorPods天倚SparkExecutorPod倚天Fluid12341234低延時LB直通pod服務(wù)化運維arenaArena低延時LB直通pod服務(wù)化運維arenaArena度TensorflowTensorflowScaling