2.0 KiB
2.0 KiB
title, weight, tags
| title | weight | tags | ||||
|---|---|---|---|---|---|---|
| How Chaos-Engineering works: Implementing Failure Injection on Kubernetes with Rust | 7 |
|
{{% button href="https://github.com/ioboi/kerris" style="info" icon="code" %}}Code/Demo{{% /button %}}
A general introduction to chaos engineering with specificly showing implementations by Chaos Mesh and Litmus. After that the talk continues into the implementation of a custom chaos operator, written in rust.
Chaos Engineering
Chaos Engineering is the discipline of experimenting on a system in order to build confidence in the systems capability to withstand turbulent conditions in production ~ Chaos Community (2015)
- CNCF-Projects: Chaos Mesh, Litmus, Chaos Blade, Krkn
- Types: Pod Chaos (Delete/Terminate), Pod Network Chaos (Faults/Packet loss), Node Chaos, JVM Chaos, Infra Chaos(Reboot vms), ...
APIs
- Chaos Mesh: A Specific CRD per Chaos Type (PodChaos, NetworkChaos, ...)
- Litmus: Chaos Engine Config that defines the type in it's spec
TODO: Steal sample CRDs
DIY
- Baseline: Written as a controller in Rust (out of curiosity) with support for Pod Chaos and Network Chaos
- Entrypoint is a reconcile function that returns an action (requeue, etc) and an error
- Network Chaos uses traffic controll (
tcpart ofiproute2) to do the limiting and loss - If you're interesten in the rust implementation: Look at the code linked above
graph LRT
controller
subgraph node
daemon
containerd
daemon-->|grpc|containerd
end
controller-->daemon