InfluxDB Client Go

repository·master·Indexed 20 days ago

https://github.com/influxdata/influxdb-client-go

A Go language library for interacting with InfluxDB 2.x and the Flux query language. It provides support for synchronous and asynchronous data writing via Line Protocol and the Point data type, Flux query execution, and management of InfluxDB resources such as buckets, users, and organizations. Requires Go 1.17 or later.

Tokens
9.2K
Snippets
28
Records
34
Agent score
69%

What's inside influxdb-client-go

  1. Core features of the InfluxDB Client Go

    master

    The InfluxDB Client Go provides comprehensive support for InfluxDB 2.x, including:

    Querying

    • Execute queries using the Flux language.
    • Retrieve data in raw formats or as Flux table representations.

    Writing Data

    • Support for Line Protocol.
    • Support for the Point data type.
    • Two writing modes:
      • Asynchronous: Using WriteAPI for non-blocking writes.
      • Synchronous: Using WriteAPIBlocking for immediate writes.

    Management API

    • Handle setup, readiness, and health checks.
    • Manage authorizations, users, and organizations.
    • Manage buckets and deletions.
  2. Identify the correct InfluxDB client for your version

    master

    Before using this library, ensure you are using InfluxDB 2.x. This client is specifically designed for InfluxDB 2.x and the Flux query language.

    Note for v2 users: If you prefer using InfluxQL for a consistent experience with older versions, consider using the v1 client library instead.

  3. Handle failed asynchronous writes

    master

    The WriteAPI automatically retries failed writes based on a random exponential strategy.

    Retry Parameters (Defaults):

    • retryInterval: 5,000ms
    • exponentialBase: 2
    • maxRetryDelay: 125,000ms
    • maxRetries: 5
    • maxRetryTime: 180,000ms

    Note: Setting retryInterval to 0 disables retries, causing failed batches to be discarded.

    Advanced Control: Use WriteFailedCallback to control batch handling. If the callback returns true, the API continues retrying; if false, the batch is discarded.

    Reading Errors: To capture and log errors occurring during async writes, use the Errors() method, which returns a channel of errors.

    // Get errors channel
    errorsCh := writeAPI.Errors()
    
    // Create go proc for reading and logging errors
    go func() {
        for err := range errorsCh {
            fmt.Printf("write error: %s\n", err.Error())
        }
    }()
  4. Use the InfluxDB Client Go with InfluxDB 1.8 API compatibility

    master

    If you are using InfluxDB 1.8.0 or later, you can use this client to interact with it via the InfluxDB 2.0 API endpoints. When using this compatibility mode, follow these mapping rules for parameters:

    1. Authentication Token: Use the format username:password. If the server does not require authentication, use an empty string ("").
    2. Organization: The organization parameter is not used in 1.8. Use an empty string ("") where an organization is required.
    3. Bucket: Use the format database/retention-policy. To use the default retention policy, provide only the database name (e.g., telegraf).

    Available compatible APIs:

    • WriteAPI / WriteAPIBlocking: Writes data to /api/v2/write.
    • QueryAPI: Queries data via /api/v2/query (requires the flux-enabled option to be enabled on the InfluxDB server).
    • Health(): Checks instance health via /health.
    // Example of compatibility mapping:
    // Token: "my-user:my-password"
    // Org: ""
    // Bucket: "telegraf/autogen"
    client := influxdb2.NewClient("http://localhost:8086", "my-user:my-password")
    writeAPI := client.WriteAPIBlocking("", "telegraf/autogen")
  5. Use the non-blocking write client

    master

    The non-blocking WriteAPI is recommended for frequent, periodic writes. It uses implicit batching to improve performance.

    Key Behaviors:

    • Batching: Data is buffered and sent when the BatchSize (default 5000) is reached or the flush interval (default 1s) expires.
    • Retries: Automatically retries on connection failures or HTTP status codes $\ge$ 429. It uses a random exponential backoff strategy.
    • Flushing: Use writeAPI.Flush() to ensure all pending writes are sent before closing.
    • Cleanup: Always call client.Close() to stop background processes.

    Configuring Batch Size:

    client := influxdb2.NewClientWithOptions("http://localhost:8086", "my-token",
        influxdb2.DefaultOptions().SetBatchSize(20))
    writeAPI := client.WriteAPI("my-org", "my-bucket")
    package main
    
    import (
        "fmt"
        "math/rand"
        "time"
    
        "github.com/influxdata/influxdb-client-go/v2"
    )
    
    func main() {
        client := influxdb2.NewClientWithOptions("http://localhost:8086", "my-token",
            influxdb2.DefaultOptions().SetBatchSize(20))
        writeAPI := client.WriteAPI("my-org","my-bucket")
    
        for i := 0; i < 100; i++ {
            p := influxdb2.NewPoint(
                "system",
                map[string]string{
                    "id":       fmt.Sprintf("rack_%v", i%10),
                    "vendor":   "AWS",
                    "hostname": fmt.Sprintf("host_%v", i%100),
                },
                map[string]interface{}{
                    "temperature": rand.Float64() * 80.0,
                    "disk_free":   rand.Float64() * 1000.0,
                    "disk_total":  (i/10 + 1) * 1000000,
                    "mem_total":   (i/100 + 1) * 10000000,
                    "mem_free":    rand.Uint64(),
                },
                time.Now())
            writeAPI.WritePoint(p)
        }
        writeAPI.Flush()
        client.Close()
    }
  6. Concurrency and Thread Safety

    master

    The InfluxDB Go Client is thread-safe and designed for concurrent environments.

    Best Practices:

    • Single Client Instance: Use a single Client instance per server URL to optimize resource usage and reuse HTTP connections.
    • Sub-client Reuse: The client ensures a single instance of each API sub-client (e.g., one WriteAPI per org/bucket pair, one QueryAPI per org). These sub-clients can be used concurrently.
    • Custom HTTP Client: For efficient resource reuse across multiple client instances, create a single http.Client and pass it to all clients using Options.SetHTTPClient().
    // Create a shared HTTP client
    httpClient := &http.Client{
        Transport: &http.Transport{
            MaxIdleConns: 100,
            MaxIdleConnsPerHost: 100,
        },
    }
    
    // Use the same client for different server connections
    client1 := influxdb2.NewClientWithOptions("https://server1:8086", "token1", influxdb2.DefaultOptions().SetHTTPClient(httpClient))
    client2 := influxdb2.NewClientWithOptions("https://server2:8086", "token2", influxdb2.DefaultOptions().SetHTTPClient(httpClient))
  7. Generate InfluxDB domain types and client

    master

    Use oapi-codegen to generate the Go source files for the domain package. You must periodically re-run these commands to maintain compatibility with new InfluxDB releases.

    Note: Ensure you are in the domain directory and have the templates folder available as specified in the commands.

    # Generate types
    oapi-codegen -generate types -exclude-tags Checks -o types.gen.go -package domain -templates ./templates oss.yml
    
    # Generate client
    oapi-codegen -generate client -exclude-tags Checks -o client.gen.go -package domain -templates ./templates oss.yml
  8. Install the InfluxDB Client Go

    master

    The client requires Go 1.17 or later.

    For Go mod projects

    Add the package to your dependencies:

    go get github.com/influxdata/influxdb-client-go/v2

    Then import github.com/influxdata/influxdb-client-go/v2 in your source code.

    For GOPATH projects

    go get github.com/influxdata/influxdb-client-go

    Note: GO111MODULE must be set to off for go get to work in GOPATH mode.

  9. Install the oapi-codegen generator

    master

    To generate the InfluxDB client and types, you must first install the oapi-codegen tool from the specific feat/template_helpers branch. Follow these steps:

    1. Clone the repository.
    2. Checkout the required feature branch.
    3. Install the binary using go install.
    git clone git@github.com:bonitoo-io/oapi-codegen.git
    cd oapi-codegen
    git checkout feat/template_helpers
    go install ./cmd/oapi-codegen/oapi-codegen.go
  10. Download the latest InfluxDB OpenAPI specification

    master

    The client generation relies on the oss.yml specification file. You must download the latest version from the InfluxData OpenAPI repository to ensure compatibility with the latest InfluxDB release.

    wget https://raw.githubusercontent.com/influxdata/openapi/master/contracts/oss.yml
    cd domain
  11. Configure Proxy and Redirects

    master

    Proxy Configuration

    You can configure a proxy in two ways:

    1. Environment Variables: Set HTTP_PROXY or HTTPS_PROXY.
    2. Custom HTTP Client: Create an http.Client with a configured Transport.Proxy and pass it via SetHTTPClient().

    Handling Redirects

    The client follows up to 10 consecutive redirects by default. However, for security, the Authorization header is not forwarded when a redirect leads to a different domain. To bypass this, provide a custom CheckRedirect handler in your http.Client that manually re-adds the token.

    token := "my-token"
    
    // Custom redirect handler to preserve Authorization header
    httpClient := &http.Client{
        CheckRedirect: func(req *http.Request, via []*http.Request) error {
            req.Header.Add("Authorization", "Token " + token)
            return nil
        },
    }
    client := influxdb2.NewClientWithOptions("http://localhost:8086", token, influxdb2.DefaultOptions().SetHTTPClient(httpClient))
  12. Basic usage: Write and Read data

    master

    This example demonstrates the full lifecycle: creating a client, writing points using both the full constructor and the fluent style, writing via Line Protocol, and querying data back using Flux.

    Key methods used:

    • influxdb2.NewClient(url, token): Initializes the client.
    • client.WriteAPIBlocking(org, bucket): Provides a synchronous write API.
    • client.QueryAPI(org): Provides the query interface.
    • client.Close(): Ensures background processes finish.
    package main
    
    import (
        "context"
        "fmt"
        "time"
    
        "github.com/influxdata/influxdb-client-go/v2"
    )
    
    func main() {
        client := influxdb2.NewClient("http://localhost:8086", "my-token")
        writeAPI := client.WriteAPIBlocking("my-org", "my-bucket")
        
        // Write using full params constructor
        p := influxdb2.NewPoint("stat",
            map[string]string{"unit": "temperature"},
            map[string]interface{}{"avg": 24.5, "max": 45.0},
            time.Now())
        writeAPI.WritePoint(context.Background(), p)
    
        // Write using fluent style
        p = influxdb2.NewPointWithMeasurement("stat").
            AddTag("unit", "temperature").
            AddField("avg", 23.2).
            AddField("max", 45.0).
            SetTime(time.Now())
        err := writeAPI.WritePoint(context.Background(), p)
        if err != nil {
            panic(err)
        }
    
        // Write directly via Line Protocol
        line := fmt.Sprintf("stat,unit=temperature avg=%f,max=%f", 23.5, 45.0)
        err = writeAPI.WriteRecord(context.Background(), line)
        if err != nil {
            panic(err)
        }
    
        queryAPI := client.QueryAPI("my-org")
        result, err := queryAPI.Query(context.Background(), `from(bucket:"my-bucket")|> range(start: -1h) |> filter(fn: (r) => r._measurement == "stat")`)
        if err == nil {
            for result.Next() {
                if result.TableChanged() {
                    fmt.Printf("table: %s\n", result.TableMetadata().String())
                }
                fmt.Printf("row: %s\n", result.Record().String())
            }
            if result.Err() != nil {
                fmt.Printf("Query error: %s\n", result.Err().Error())
            }
        } else {
            panic(err)
        }
        client.Close()
    }