Understand Systems with OpenTelemetry: A Hybrid Telemetry Data Backend

May 01, 2023 16 min Free

Description

This talk introduces OpenGemini, a hybrid telemetry data backend designed to address the challenges of collecting and analyzing massive amounts of telemetry data in cloud-native environments. The presenters discuss key challenges faced by HUAWEI CLOUD in observability, the evolution of backend storage solutions, and present OpenGemini as a unified storage engine for metrics and logs, leveraging OpenTelemetry standards. They detail the system's architecture, including data partitioning and indexing strategies for both metrics and logs, and present evaluation results demonstrating significant improvements in storage efficiency and query performance.