DOCSEARCHbyAlgolia
Submit Docs

OpenTelemetry Collector

repository·main·Indexed 27 days ago

https://github.com/open-telemetry/opentelemetry-collector

A vendor-agnostic tool for receiving, processing, and exporting telemetry data, including traces, metrics, and logs. Includes documentation for the OpenTelemetry Collector Builder (ocb) to generate custom binaries, as well as testing utilities like the sample receiver, sample processor, and sample factory receiver for verifying mdatagen output.

Tokens
74.4K
Snippets
108
Records
433
Agent score
93%

What's inside OpenTelemetry Collector

  4. Overview of the Metadata Generator

    main
    The Metadata Generator is a tool used to define, validate, and standardize component documentation for the OpenTelemetry Collector. It uses a specific schema to ensure that component metadata—such as stability levels, supported distributions, pipeline types, and emitted metrics—is complete and well-formed. The tool ingests this metadata and produces documentation in a standardized format used across the project.

  5. Overview of Pipeline data (pdata)

    main

    Pipeline data (pdata) provides the in-memory data structures used by the OpenTelemetry Collector to represent telemetry data. All telemetry received by the collector is converted into pdata format, processed through the pipeline in this format, and then converted back to wire formats (like OTLP) by exporters.

    Key characteristics:

    • Stability: The implementations for traces, metrics, and logs are considered stable.
    • Efficiency: It uses OTLP protobuf structs as underlying data structures and maintains a pointer to the OTLP protobuf in an orig member field to allow efficient translation to/from the OTLP wire protocol.
    • Safety: The API is designed to avoid mutable data sharing; a pdata instance cannot contain a reference to an object used in another pdata instance.

  6. Overview of the OpenTelemetry Collector

    main
    The OpenTelemetry Collector is a vendor-agnostic implementation for receiving, processing, and exporting telemetry data (traces, metrics, and logs). It serves as a unified codebase that can be deployed as an agent or a collector, eliminating the need to maintain multiple agents for different open-source formats (like Jaeger or Prometheus) or different back-ends.

  7. Overview of the OpenTelemetry Collector

    main
    The OpenTelemetry Collector is a vendor-agnostic data collection and processing pipeline. It is composed of a configuration parsing mechanism and a modular system of components. This architecture allows users to switch between different Collector Distributions while ensuring that components produced by the OpenTelemetry Collector SIG remain compatible with any vendor-supported Collector.

  9. Understand Authentication Configuration Types

    main

    The OpenTelemetry Collector uses two types of authenticators to secure data flow:

    • Server type authenticators: These perform authentication for incoming HTTP/gRPC requests. They are typically used within receivers to verify the identity of clients sending data to the collector.
    • Client type authenticators: These perform client-side authentication for outgoing HTTP/gRPC requests. They are typically used within exporters to authenticate the collector when it sends data to an external service.

  10. Understand Component Health Reporting in the OpenTelemetry Collector

    main

    The OpenTelemetry Collector uses a component health reporting system to allow components within pipelines to emit information about their health. This enables the collector service or external observers to make decisions when components encounter issues.

    In the current architecture (targeting 1.0), component health is managed through a finite state machine. A component.Host implementation (like service/internal/graph.Host) may implement the componentstatus.Reporter interface to allow components to report their status.

    Key status states include:

    • Starting
    • Ok
    • Stopping
    • PermanentError

    Note: The component.TelemetrySettings.ReportStatus field has been removed in favor of components checking if their component.Host implements the componentstatus.Reporter interface.

  12. Understand Data Ownership Modes

    main

    The OpenTelemetry Collector operates in two data ownership modes, which are determined at startup based on the MutatesData field reported by the processors in a pipeline via their Capabilities function.

    Exclusive Ownership

    • Trigger: Triggered if any processor in a pipeline declares an intent to modify data (MutatesData=true).
    • Behavior: Data is owned exclusively by one processor at a time. If a receiver is attached to multiple pipelines and one is in exclusive mode, the data is cloned at the fan-out connector so each pipeline has its own copy.
    • Usage: Processors can freely modify the data they own. Once a processor calls the next processor's Consume function, it must no longer read or write to that data as ownership has passed.

    Shared Ownership

    • Trigger: Occurs when no processors in the pipeline declare an intent to modify data.
    • Behavior: No cloning is performed at the fan-out connector; all pipelines see the same single shared copy of the data.
    • Constraint: Processors are strictly prohibited from modifying the original pdata.Traces, pdata.Metrics, or pdata.Logs data. They may only read it.
    • Optimization: If a processor needs to modify data but wants to avoid the cost of full data cloning, it can implement a copy-on-write approach for sub-parts of the data and declare MutatesData=false.