kubecon24/content/day2/04_sponsored_ai_platform.md

title	weight	tags
Sponsored: Build an open source platform for ai/ml	4	keynote ai

{{% button href="https://youtu.be/r7qd0ZHt4uE" style="warning" icon="video" %}}Watch talk on YouTube{{% /button %}}

Jorge Palma from Microsoft with a quick introduction.

Baseline

• Kubernetes is cool and all
• Challenges:
  • Containerized models
  • GPUs in the cluster (install, management)

Kubernetes AI Tool chain (KAITO)

• Kubernetes operator that interacts with
  • Node provisioner
  • Deployment
• Simple CRD that describes a model, infra and have fun
• Creates inference endpoint
• Models are currently 10 (Hugginface, LLMA, etc.)