docs(day1): Added opensearch talk
This commit is contained in:
parent
2af34196ba
commit
a0d8562dad
62
content/day1/06_opensearch.md
Normal file
62
content/day1/06_opensearch.md
Normal file
@ -0,0 +1,62 @@
|
|||||||
|
---
|
||||||
|
title: OpenSearch - The Open source Path to Search and Observability
|
||||||
|
weight: 6
|
||||||
|
tags:
|
||||||
|
- observability
|
||||||
|
---
|
||||||
|
|
||||||
|
<!-- {{% button href="https://youtu.be/rkteV6Mzjfs" style="warning" icon="video" %}}Watch talk on YouTube{{% /button %}} -->
|
||||||
|
<!-- {{% button href="https://docs.google.com/presentation/d/1nEK0CVC_yQgIDqwsdh-PRihB6dc9RyT-" style="tip" icon="person-chalkboard" %}}Slides{{% /button %}} -->
|
||||||
|
|
||||||
|
A introduction to opensearch and "look at the cool new features in 3.o"
|
||||||
|
|
||||||
|
## History
|
||||||
|
|
||||||
|
- Background: Was born out of the elasticsearch license change as a fork by AWS
|
||||||
|
- Since Late 2024: A part of the linux foundation
|
||||||
|
|
||||||
|
## Platform
|
||||||
|
|
||||||
|
### Elements
|
||||||
|
|
||||||
|
- Core: Distributed Search Engine with Vector DB
|
||||||
|
- Dashboards: UI with Dashboards, Alerts, Reports, ...
|
||||||
|
- Data Preppers: Prepare Data for ingest and indexing
|
||||||
|
|
||||||
|
```mermaid
|
||||||
|
graph LR
|
||||||
|
DataSource-->DataPrepper-->|Ingest into|Core
|
||||||
|
subgraph Core
|
||||||
|
LogIndex
|
||||||
|
TraceIndex
|
||||||
|
TimeseriesIndex
|
||||||
|
end
|
||||||
|
```
|
||||||
|
|
||||||
|
### Use-Cases
|
||||||
|
|
||||||
|
- Search: Well - search (e.g. for Amazon's product search)
|
||||||
|
- Free text search & fuzzy search
|
||||||
|
- Faceting (Generate Attributes based on the content and search by them)
|
||||||
|
- Geospacial Search & Vector Search
|
||||||
|
- Observability: Log analytics
|
||||||
|
- Log analytics with specialized query language or natural language
|
||||||
|
- OTEL and Jaeger Support
|
||||||
|
- Query federation to prometheus for metrics
|
||||||
|
- AI/ML: It's a vector database
|
||||||
|
- Vector database that can be used for embeddings
|
||||||
|
- Multimodal search for text image and video with one model or one model per mode
|
||||||
|
- Neural sparse search and simmilarity search
|
||||||
|
- MCP and bring your own model support
|
||||||
|
- Security: Tracing, log detection and so on
|
||||||
|
|
||||||
|
### Performance
|
||||||
|
|
||||||
|
- Problem: Large Datasets are usually slow
|
||||||
|
- Solution: Specialized improvements
|
||||||
|
|
||||||
|
## News: Openstack 3.0
|
||||||
|
|
||||||
|
- Baseupgrades for Lucene, JDK and Node (yay)
|
||||||
|
- Performance: Reader/Writer-Seperation, gRPC Support, Pull-based injection in addition to pushed-based
|
||||||
|
- Improvements: Cross cluster search for traces, better nested json support
|
Loading…
x
Reference in New Issue
Block a user