---
title: Comparing sidecarless service mesh from cilium and istio
weight: 11
tags:
  - platform
  - network
---

Global field CTO at Solo.io with a hint of servicemesh background.

## History

* LinkerD 1.X was the first moder servicemesh and basicly a opt-in serviceproxy
* Challenges: JVM (size), latencies, ...

### Why not node-proxy?

* Per-node resource consumption is unpredictable
* Per-node proxy must ensure fairness
* Blast radius is always the entire node
* Per-node proxy is a fresh attack vector

### Why sidecar?

* Transparent (ish)
* PArt of app lifecycle (up/down)
* Single tennant
* No noisy neighbor

### Sidecar drawbacks

* Race conditions
* Security of certs/keys
* Difficult sizing
* Apps need to be proxy aware
* Can be circumvented
* Challenging upgrades (infra and app live side by side)

## Our lord and savior

* Potential solution: eBPF
* Problem: Not quite the perfect solution
* Result: We still need a L7 proxy (but some L4 stuff can be implemented in kernel)

### Why sidecarless

* Full transparency
* Optimized networking
* Lower ressource allocation
* No race conditions
* No manual pod injection
* No credentials in the app

## Architecture

* Control Plane
* Data Plane
* mTLS
* Observability
* Traffic Control

## Cilium

### Basics

* CNI with eBPF on L3/4
* A lot of nice observability
* Kubeproxy replacement
* Ingress (via Gateway API)
* Mutual Authentication
* Specialiced CiliumNetworkPolicy
* Configure Envoy throgh Cilium

### Control Plane

* Cilium-Agent on each node that reacts to scheduled workloads by programming the local dataplane
* API via Gateway API and CiliumNetworkPolicy

```mermaid
flowchart TD
    subgraph kubeserver
        kubeapi
    end
    subgraph node1
        kubeapi<-->control1
        control1-->data1
    end
    subgraph node2
        kubeapi<-->control2
        control2-->data2
    end
    subgraph node3
        kubeapi<-->control3
        control3-->data3
    end
```

### Data plane

* Configured by control plane
* Does all of the eBPF things in L4
* Does all of the envoy things in L7
* In-Kernel Wireguard for optional transparent encryption

### mTLS

* Network Policies get applied at the eBPF layer (check if id a can talk to id 2)
* When mTLS is enabled there is a auth check in advance -> It it fails, proceed with agents
* Agents talk to each other for mTLS Auth and save the result to a cache -> Now ebpf can say yes
* Problems: The caches can lead to id confusion

## Istio

### Basiscs

* L4/7 Service mesh without it's own CNI
* Based on envoy
* mTLS
* Classicly via sidecar, nowadays

### Ambient mode

* Seperate L4 and L7 -> Can run on cilium
* mTLS
* Gateway API

### Control plane

```mermaid
flowchart TD
    kubeapi-->xDS

    xDS-->dataplane1
    xDS-->dataplane2

    subgraph node1
        dataplane1
    end

    subgraph node2
        dataplane2
    end
```

* Central xDS Control Plane
* Per-Node Dataplane that reads updates from Control Plane

### Data Plane

* L4 runs via zTunnel Daemonset that handels mTLS
* The zTunnel traffic get's handed over to the CNI
* L7 Proxy lives somewhere™ and traffic get's routed through it as an "extra hop" aka waypoint

### mTLS

* The zTunnel creates a HBONE (http overlay network) tunnel with mTLS