---
title: "Fluent Bit v3: Unified Layer for Logs, Metrics and Traces"
weight: 7
tags:
  - monitoring
---

The last talk of the conference.
Notes may be a bit unstructured due to tired note taker.

## Background

* FluentD is already graduated
* FluentBit is a daughter-project of FluentD (also graduated)

## Basics

* FluentBit is compatible with
  * Prometheus (It can replace the Prometheus scraper and node exporter)
  * OpenMetrics
  * OpenTelemetry (HTTPS input/output)
* FluentBit can export to Prometheus, Splunk, InfluxDB or others
* So pretty much it can be used to collect data from a bunch of sources and pipe it out to different backend destinations
* Fluent ecosystem: No vendor lock-in to observability

### Architectures

* The fluent agent collects data and can send it to one or multiple locations
* FluentBit can be used for aggregation from other sources

### In the Kubernetes logging ecosystem

* Pod logs to console -> Streamed stdout/err gets piped to file
* The logs in the file get encoded as JSON with metadata (date, channel)
* Labels and annotations only live in the control plane -> You have to collect it additionally -> Expensive

## New stuff

### Limitations with classic architectures

* Problem: Multiple filters slow down the main loop

```mermaid
flowchart LR
    subgraph main[Main Thread/Event loop]
        buffer
        schedule
        retry
        fitler1
        filter2
        filter3
    end
    in-->|pipe in data|main
    main-->|filter and pipe out|out
```

### Solution

* Solution: Processor - a separate thread segmented by telemetry type
* Plugins can be written in your favorite language (c, rust, go, ...)

```mermaid
flowchart LR
    subgraph in
        reader
        streamner1
        processor2
        processor3
    end
    in-->|pipe in data|main(Main Thread/Event loop)
    main-->|filter and pipe out|out
```

### General new features in v3

* Native HTTP/2 support in core
* Content modifier with multiple operations (insert, upsert, delete, rename, hash, extract, convert)
* Metrics selector (include or exclude metrics) with matcher (name, prefix, substring, regex)
* SQL processor -> Use SQL expression for selections (instead of filters)
* Better OpenTelemetry output