---
title: AI Keynote discussion
weight: 2
tags:
  - keynote
  - ai
  - panel
---

<!-- {{% button href="https://youtu.be/VhloarnpxVo" style="warning" icon="video" %}}Watch talk on YouTube{{% /button %}} -->

A podium discussion (somewhat scripted) lead by Priyanka

## Guests

* Tim from Mistral
* Paige from Google AI
* Jeff founder of OLLAMA

## Discussion

* What do you use as the base of dev for OLLAMA
  * Jeff: The concepts from docker, git, Kubernetes
* How is the balance between AI engineer and AI ops
  * Jeff: The classic dev vs ops divide, many ML-Engineer don't think about
  * Paige: Yessir
* How does infra keep up with the fast research
  * Paige: Well, they don't - but they do their best and Cloud native is cool
  * Jeff: Well we're not google, but Kubernetes is the savior
* What are scaling constraints
  * Jeff: Currently sizing of models is still in its infancy
  * Jeff: There will be more specific hardware and someone will have to support it
  * Paige: Sizing also depends on latency needs (code autocompletion vs performance optimization)
  * Paige: Optimization of smaller models
* What technologies need to be open source licensed
  * Jeff: The model b/c access and trust
  * Tim: The models and base execution environment -> Vendor agnosticism
  * Paige: Yes and remixes are really important for development
* Anything else
  * Jeff: How do we bring our awesome tools (monitoring, logging, security) to the new AI world
  * Paige: Currently many people just use paid APIs to abstract the infra, but we need this stuff self-hostable
  * Tim: I don't want to know about the hardware, the whole infra side should be done by the cloud native teams to let ML-Engineer to just be ML-Engine
