Telegraf

repository·master·Indexed 12 days ago

https://github.com/influxdata/telegraf

A highly extensible agent for collecting, processing, and exporting metrics and logs from various sources to different destinations. It features a plugin-based architecture including inputs, outputs, processors, aggregators (such as Histogram, Merge, and Quantile), parsers, serializers, and secret stores.

Tokens
399.4K
Snippets
869
Records
1.3K
Agent score
96%

What's inside Telegraf

  1. Overview of Telegraf

    master

    Telegraf is a plugin-driven agent designed for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.

    Key features include:

    • Over 300 plugins for system monitoring, cloud services, and messaging.
    • Support for user-defined code to transform and transmit data.
    • Deployment as a standalone static binary with no external dependencies.
    • Configuration using the TOML format.
  2. Use the Execd Processor Plugin to run long-running external programs

    master

    The execd processor plugin runs an external program as a separate, long-running process. Telegraf pipes metrics into the process's stdin and reads the processed metrics back from its stdout. Any output written to stderr is captured in the Telegraf logs.

    Key Constraints

    • Data Format: Currently, only the influx data format is supported because the communication must be serialize-parse symmetrical to avoid losing critical type data.
    • Metric Tracking: Metrics with tracking are considered "delivered" as soon as they are passed to the external process. Because processing is asynchronous, there is no way to correlate which incoming metric corresponds to which outgoing metric from the execd process.
    • Version Requirement: Requires Telegraf v1.15.0 or newer.
    [[processors.execd]]
      command = ["/path/to/your_program", "arg1", "arg2"]
      # environment = ["KEY=value"]
      # restart_delay = "10s"
      # data_format = "influx"
  3. Use the OPC UA Client Listener Input Plugin

    master

    The opcua_listener plugin is a service input that receives data from an OPC UA server by subscribing to nodes and events.

    Important Note on Service Inputs: As a service input, this plugin starts a service to listen and wait for metrics or events. Because of this behavior:

    1. The global or plugin-specific interval setting may not apply.
    2. CLI options like --test, --test-wait, and --once may not produce output.
    [[inputs.opcua_listener]]
      name = "opcua_listener"
      endpoint = "opc.tcp://localhost:4840"
  4. Use the AWS Data Firehose Input Plugin

    master

    The inputs.firehose plugin allows Telegraf to act as an HTTP listener for metrics sent via AWS Data Firehose. It follows the official AWS request-response schema.

    Key Requirements:

    • Telegraf Version: v1.34.0 or higher.
    • Data Format: Metrics are extracted from the records.[*].data field in the request body. The data in this field must be base64 encoded and must match one of Telegraf's supported input data formats.
    • Service Input Behavior: This is a service plugin. It starts a persistent listener rather than gathering metrics on a fixed interval. Note that the global interval setting may not apply, and CLI flags like --test or --once may not produce output.
    [[inputs.firehose]]
      service_address = ":8080"
      paths = ["/telegraf"]
      data_format = "value"
      data_type = "string"
  5. Use the Heartbeat Output Plugin

    master

    The Heartbeat Output Plugin sends a heartbeat signal via an HTTP POST request to a specified endpoint at a regular interval. This is primarily used to monitor the presence and health of Telegraf instances within large-scale deployments.

    Requirements:

    • Telegraf version: v1.37.0 or higher
    • Supported platforms: All
    # Example configuration snippet (conceptual)
    [[outputs.heartbeat]]
      url = "http://your-endpoint.com/heartbeat"
      id = "telegraf-instance-01"
      token = "your-auth-token"
      interval = "1m"
  6. Use the OAuth2 Secret Store plugin

    master

    The OAuth2 Secret Store plugin retrieves and maintains secrets (tokens) from OAuth2 services like Auth0, AzureAD, or custom providers. It specifically supports the 2-legged client credentials flow and automatically renews expired tokens to prevent API call failures.

    To use a secret retrieved by this store in other Telegraf plugins, reference it using the syntax @{<store-id>:<secret-key>}.

    Note: Only specific Telegraf plugins and options support secret store referencing. Check the documentation of the plugin you are using for a Secret store support section to confirm compatibility.

    # Example reference syntax
    # If your store id is 'secretstore' and your token key is 'mytoken'
    option_name = "@{secretstore:mytoken}"
  7. Use the MQTT Producer Output Plugin

    master

    The MQTT Producer Output Plugin acts as an MQTT producer, writing Telegraf metrics to an MQTT broker. It supports both MQTT protocol versions 3.1.1 and 5 (defaulting to 3.1.1).

    Important Note for Mosquitto Users

    If you are using mosquitto MQTT server version v2.0.12 or later, you must set the keep_alive option to a non-zero value. Failure to do so may result in the server returning an identifier rejected error. A common value used by the eclipse/paho.golang library is 30.

  8. Microsoft SQL Server Input Plugin Overview

    master

    The sqlserver input plugin collects lightweight metrics from Microsoft SQL Server instances using Dynamic Management Views (DMVs).

    Supported SQL Server environments:

    • Microsoft SQL Server versions currently supported by Microsoft
    • Azure SQL Databases (Single)
    • Azure SQL Managed Instances
    • Azure SQL Elastic Pools
    • Azure Arc-enabled SQL Managed Instances
  9. Use the Windows Performance Counters Input Plugin

    master

    The win_perf_counters plugin collects metrics from Windows Performance Counters. This plugin is exclusively for Windows operating systems.

    Important Note: If the specific Performance Counter Object being polled does not exist on the system at the time the Telegraf agent starts, the plugin will not gather data for that object.

    By default, the plugin provides basic CPU and Disk statistics. You can customize the collection by specifying custom counter paths or using the print_all option to match specific patterns.

  10. Use the OPC UA Client Reader Input Plugin

    master

    The opcua input plugin gathers data from an OPC UA server by subscribing to configured nodes. It supports various authentication methods, security policies, and discovery mechanisms like browse-based discovery.

    Requirements:

    • Telegraf v1.16.0 or newer
    • Access to an OPC UA server endpoint (e.g., opc.tcp://localhost:4840)
    [[inputs.opcua]]
      endpoint = "opc.tcp://localhost:4840"
      # ... other configuration