3.2 KiB
3.2 KiB
title | weight | tags | ||
---|---|---|---|---|
Comparing sidecarless service mesh from cilium and istio | 11 |
|
Global field CTO at Solo.io with a hint of servicemesh background.
History
- LinkerD 1.X was the first moder servicemesh and basicly a opt-in serviceproxy
- Challenges: JVM (size), latencies, ...
Why not node-proxy?
- Per-node resource consumption is unpredictable
- Per-node proxy must ensure fairness
- Blast radius is always the entire node
- Per-node proxy is a fresh attack vector
Why sidecar?
- Transparent (ish)
- PArt of app lifecycle (up/down)
- Single tennant
- No noisy neighbor
Sidecar drawbacks
- Race conditions
- Security of certs/keys
- Difficult sizing
- Apps need to be proxy aware
- Can be circumvented
- Challenging upgrades (infra and app live side by side)
Our lord and savior
- Potential solution: eBPF
- Problem: Not quite the perfect solution
- Result: We still need a L7 proxy (but some L4 stuff can be implemented in kernel)
Why sidecarless
- Full transparency
- Optimized networking
- Lower ressource allocation
- No race conditions
- No manual pod injection
- No credentials in the app
Architecture
- Control Plane
- Data Plane
- mTLS
- Observability
- Traffic Control
Cilium
Basics
- CNI with eBPF on L3/4
- A lot of nice observability
- Kubeproxy replacement
- Ingress (via Gateway API)
- Mutual Authentication
- Specialiced CiliumNetworkPolicy
- Configure Envoy throgh Cilium
Control Plane
- Cilium-Agent on each node that reacts to scheduled workloads by programming the local dataplane
- API via Gateway API and CiliumNetworkPolicy
flowchart TD
subgraph kubeserver
kubeapi
end
subgraph node1
kubeapi<-->control1
control1-->data1
end
subgraph node2
kubeapi<-->control2
control2-->data2
end
subgraph node3
kubeapi<-->control3
control3-->data3
end
Data plane
- Configured by control plane
- Does all of the eBPF things in L4
- Does all of the envoy things in L7
- In-Kernel Wireguard for optional transparent encryption
mTLS
- Network Policies get applied at the eBPF layer (check if id a can talk to id 2)
- When mTLS is enabled there is a auth check in advance -> It it fails, proceed with agents
- Agents talk to each other for mTLS Auth and save the result to a cache -> Now ebpf can say yes
- Problems: The caches can lead to id confusion
Istio
Basiscs
- L4/7 Service mesh without it's own CNI
- Based on envoy
- mTLS
- Classicly via sidecar, nowadays
Ambient mode
- Seperate L4 and L7 -> Can run on cilium
- mTLS
- Gateway API
Control plane
flowchart TD
kubeapi-->xDS
xDS-->dataplane1
xDS-->dataplane2
subgraph node1
dataplane1
end
subgraph node2
dataplane2
end
- Central xDS Control Plane
- Per-Node Dataplane that reads updates from Control Plane
Data Plane
- L4 runs via zTunnel Daemonset that handels mTLS
- The zTunnel traffic get's handed over to the CNI
- L7 Proxy lives somewhere™ and traffic get's routed through it as an "extra hop" aka waypoint
mTLS
- The zTunnel creates a HBONE (http overlay network) tunnel with mTLS