Understand Systems with OpenTelemetry: A Hybrid Telemetry Data Backend
May 01, 2023
16 min
Free
opentelemetry
observability
telemetry
backend-storage
cloud-native
distributed-systems
data-management
performance-optimization
huawei-cloud
Description
This talk introduces OpenGemini, a hybrid telemetry data backend designed to address the challenges of collecting and analyzing massive amounts of telemetry data in cloud-native environments. The presenters discuss key challenges faced by HUAWEI CLOUD in observability, the evolution of backend storage solutions, and present OpenGemini as a unified storage engine for metrics and logs, leveraging OpenTelemetry standards. They detail the system's architecture, including data partitioning and indexing strategies for both metrics and logs, and present evaluation results demonstrating significant improvements in storage efficiency and query performance.