kubecon25/content/day1/03_operator-mistakes.md

title, weight, tags

title	weight	tags
Title	3	<tag>

Common mistake

Not using a simple client but directly talk to the api server

• Problem: A
• Problem: Updates send in the whole object -> Noop updates waste apiserver resources
• Fix: Use a cache client
• Problem: Caching validation

Don't use custom caching

• Problem: Good Luck dealing with concurrency
• Hard: Controllers mus maintain a per kind cache
• Problem: Eventual consistency makes everything more complicated
• Fix: Use a framework

Predecates only apply to the current

• If you have a predecate in the for (predecate) only appy to this call, not to other watchers
• Also check if you shold be reconciling your low-level object or reconciling the higher level ones that ref to them is better

Tools

KRT

Still under development

• Operatorions in collections (kubernetes objects with state tracking)
• Fetch function that handels transformation

StateDB

• In-memory database for go with watch channels
• You can setup a table that stores all objects of a kind (provided by the client)
• Triggers hooks when changes happen in the database that you can react to

Controller-Runtime

The kubebuilder one

• Includes a chached client
• Works on the reconciler pattern -> Makes triggers simpe

Tips

• Limit the number of api server updates
  • Check for dif yourself and don't send updates if there is nothing new
  • Use patch instead of update just with changed fields -> Especially for .status
• Use a framework that handles watching, coalescing and caching (krt, statedb, controller-runtime)
• Use predecates if you're using controller-runtime, this helps you filter out no-op events by checking them against the cache and filters

Q&A

• Do you know where your reconciliations are coming from:
  • Counts: Yes the frameworks provide metrics and you can implement your own
  • But controller runtime abstracts the patch source so you have to compare before and after state yourself - but you should not do that
• What about state sharing across multiple threads?
  • Controller runtime handels each reconcile as idempotent, so you can just multithread
  • But handling consistency can still be hard