1.9 KiB
1.9 KiB
title, weight, tags
| title | weight | tags | |
|---|---|---|---|
| OpenSearch - The Open source Path to Search and Observability | 6 |
|
A introduction to opensearch and "look at the cool new features in 3.o"
History
- Background: Was born out of the elasticsearch license change as a fork by AWS
- Since Late 2024: A part of the linux foundation
Platform
Elements
- Core: Distributed Search Engine with Vector DB
- Dashboards: UI with Dashboards, Alerts, Reports, ...
- Data Preppers: Prepare Data for ingest and indexing
graph LR
DataSource-->DataPrepper-->|Ingest into|Core
subgraph Core
LogIndex
TraceIndex
TimeseriesIndex
end
Use-Cases
- Search: Well - search (e.g. for Amazon's product search)
- Free text search & fuzzy search
- Faceting (Generate Attributes based on the content and search by them)
- Geospacial Search & Vector Search
- Observability: Log analytics
- Log analytics with specialized query language or natural language
- OTEL and Jaeger Support
- Query federation to prometheus for metrics
- AI/ML: It's a vector database
- Vector database that can be used for embeddings
- Multimodal search for text image and video with one model or one model per mode
- Neural sparse search and simmilarity search
- MCP and bring your own model support
- Security: Tracing, log detection and so on
Performance
- Problem: Large Datasets are usually slow
- Solution: Specialized improvements
News: Openstack 3.0
- Baseupgrades for Lucene, JDK and Node (yay)
- Performance: Reader/Writer-Seperation, gRPC Support, Pull-based injection in addition to pushed-based
- Improvements: Cross cluster search for traces, better nested json support