Grafana Beyla Documentation

Beyla uses service discovery to identify which processes to instrument. For simple use cases, you can use environment variables like BEYLA_AUTO_TARGET_EXE or BEYLA_OPEN_PORT to target a single service or group of services.

For more complex scenarios, such as running Beyla as a Kubernetes DaemonSet to instrument all services on a node, you can use the discovery YAML configuration section to define granular selection criteria. This allows you to specify exactly which services Beyla should target, which it should exclude, and how to override their reported names or namespaces.

Beyla can be deployed in several ways depending on your environment and target application. Choose the method that matches your infrastructure:

• Docker: Instrument a process running inside a container.
• Kubernetes DaemonSet (Grafana Cloud): Use Helm to deploy as a DaemonSet specifically for Grafana Cloud.
• Kubernetes DaemonSet (Kubernetes Monitoring Helm chart): Use the standard Kubernetes Monitoring Helm chart.
• Kubernetes DaemonSet (Generic Helm): Use a standard Helm chart for Kubernetes DaemonSet deployment.
• Kubernetes DaemonSet (Grafana Alloy Helm chart): Deploy as a DaemonSet using the Grafana Alloy Helm chart.
• Kubernetes DaemonSet or Sidecar: Manual deployment as either a DaemonSet or a sidecar container.
• Standalone Process: Run Beyla as a standalone process on bare metal, virtual machines, or for local development.

Beyla is designed to run alongside applications with minimal performance impact. The overhead varies depending on the features enabled.

Key factors affecting overhead include:

• Instrumentation Scope: Instrumenting all applications in a cluster increases CPU usage as Beyla processes more traffic.
• Enabled Features: Enabling application_span, application_service_graph, or network monitoring increases both memory and CPU usage due to the generation of additional metrics and graph data.
• Debug Mode: Setting log_level: debug and bpf_debug: true increases resource consumption because of increased logging and debug information generation.
• Exporter Type: Using the OpenTelemetry metrics exporter vs. generating traces affects memory patterns (e.g., tracing may cause memory bursts during batching).

The enforce_sys_caps property (or BEYLA_ENFORCE_SYS_CAPS environment variable) controls Beyla's behavior when required system capabilities are missing at startup.

• false (default): Beyla logs the missing capabilities but continues startup.
• true: Beyla aborts startup and logs the missing capabilities.

Parent-based samplers are composite samplers that adjust their behavior based on the existence of a parent span:

1. If the span has no parent: The root sampler (the specific sampler configured) is used to make the sampling decision.
2. If the span has a parent: The sampling decision follows the configuration of the parent.

Available parent-based samplers include:

• parentbased_always_on (Default)
• parentbased_always_off
• parentbased_traceidratio (Requires an arg value)

Beyla's eBPF programs run in kernel space and execute in the context of the application, which minimizes overhead by:

• Avoiding user-space context-switching.
• Utilizing Just-In-Time (JIT) compilation for optimization.

To monitor the overhead of the eBPF programs themselves, you can use Beyla's ebpf feature to collect program latency. In benchmark tests (OpenTelemetry demo running for 24 hours), the combined latency of all probes was approximately 500ns per request, which is considered negligible.

Use the patterns list to match URL paths and assign them a common http.route value. This helps group high-cardinality paths (like those containing IDs) into a single route.

Matcher Formats

• Tags: Use :name or {name} to identify segments that should be grouped. For example, /user/{id} will match /user/123 and /user/456, setting http.route to /user/{id}.
• Wildcards: Use the * character to match path prefixes. For example, /user/* matches /user, /user/123, and /user/123/basket/1.

Example:

routes:
  patterns:
    - /user/{id}
    - /user/{id}/basket/{product}

When using the Beyla webhook, be aware of the following constraints:

• Existing LD_PRELOAD: Pods that already have an LD_PRELOAD environment variable are skipped (unless the existing value is from a previous Beyla injection).
• Host Access: Requires hostPath volume access to provide instrumentation libraries to containers.
• TLS Timing: TLS certificates must be available before the server starts.
• Export Protocol: Only supports the OTLP export protocol (via HTTP or gRPC).

The wakeup_len (or BEYLA_BPF_WAKEUP_LEN) setting determines how many messages Beyla accumulates in the eBPF ringbuffer before triggering a wake-up request to user space.

• High-load services: Set this value higher to reduce CPU overhead by decreasing the frequency of wake-up requests.
• Low-load services: Use lower values to ensure metrics are submitted and become visible more quickly, as high values can introduce delays.