kubecon24/content/day2/04_sponsored_ai_platform.md

title: Sponsored: Build an open source platform for ai/ml weight: 4

Jorge Palma from Microsoft with a quick introduction.

Baseline

• Kubernetes is cool and all
• Challenges:
  • Containerized models
  • GPUs in the cluster (install, management)

Kubernetes AI Toolchain (KAITO)

• Kubernetes operator that interacts with
  • Node provisioner
  • Deployment
• Simple CRD that decribes a model, infra and have fun
• Creates inferance endpoint
• Models are currently 10 (Hugginface, LLMA, etc)