niggl
/
kubecon24
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
kubecon24/content/day2/11_sidecarless.md

3.2 KiB
Raw Blame History

title weight tags
Comparing sidecarless service mesh from cilium and istio 11
platform
network

Global field CTO at Solo.io with a hint of servicemesh background.

History

  • LinkerD 1.X was the first moder servicemesh and basicly a opt-in serviceproxy
  • Challenges: JVM (size), latencies, ...

Why not node-proxy?

  • Per-node resource consumption is unpredictable
  • Per-node proxy must ensure fairness
  • Blast radius is always the entire node
  • Per-node proxy is a fresh attack vector

Why sidecar?

  • Transparent (ish)
  • PArt of app lifecycle (up/down)
  • Single tennant
  • No noisy neighbor

Sidecar drawbacks

  • Race conditions
  • Security of certs/keys
  • Difficult sizing
  • Apps need to be proxy aware
  • Can be circumvented
  • Challenging upgrades (infra and app live side by side)

Our lord and savior

  • Potential solution: eBPF
  • Problem: Not quite the perfect solution
  • Result: We still need a L7 proxy (but some L4 stuff can be implemented in kernel)

Why sidecarless

  • Full transparency
  • Optimized networking
  • Lower ressource allocation
  • No race conditions
  • No manual pod injection
  • No credentials in the app

Architecture

  • Control Plane
  • Data Plane
  • mTLS
  • Observability
  • Traffic Control

Cilium

Basics

  • CNI with eBPF on L3/4
  • A lot of nice observability
  • Kubeproxy replacement
  • Ingress (via Gateway API)
  • Mutual Authentication
  • Specialiced CiliumNetworkPolicy
  • Configure Envoy throgh Cilium

Control Plane

  • Cilium-Agent on each node that reacts to scheduled workloads by programming the local dataplane
  • API via Gateway API and CiliumNetworkPolicy
flowchart TD
    subgraph kubeserver
        kubeapi
    end
    subgraph node1
        kubeapi<-->control1
        control1-->data1
    end
    subgraph node2
        kubeapi<-->control2
        control2-->data2
    end
    subgraph node3
        kubeapi<-->control3
        control3-->data3
    end

Data plane

  • Configured by control plane
  • Does all of the eBPF things in L4
  • Does all of the envoy things in L7
  • In-Kernel Wireguard for optional transparent encryption

mTLS

  • Network Policies get applied at the eBPF layer (check if id a can talk to id 2)
  • When mTLS is enabled there is a auth check in advance -> It it fails, proceed with agents
  • Agents talk to each other for mTLS Auth and save the result to a cache -> Now ebpf can say yes
  • Problems: The caches can lead to id confusion

Istio

Basiscs

  • L4/7 Service mesh without it's own CNI
  • Based on envoy
  • mTLS
  • Classicly via sidecar, nowadays

Ambient mode

  • Seperate L4 and L7 -> Can run on cilium
  • mTLS
  • Gateway API

Control plane

flowchart TD
    kubeapi-->xDS

    xDS-->dataplane1
    xDS-->dataplane2

    subgraph node1
        dataplane1
    end

    subgraph node2
        dataplane2
    end
  • Central xDS Control Plane
  • Per-Node Dataplane that reads updates from Control Plane

Data Plane

  • L4 runs via zTunnel Daemonset that handels mTLS
  • The zTunnel traffic get's handed over to the CNI
  • L7 Proxy lives somewhere™ and traffic get's routed through it as an "extra hop" aka waypoint

mTLS

  • The zTunnel creates a HBONE (http overlay network) tunnel with mTLS