DOCSEARCHbyAlgolia
Submit Docs

go-elasticsearch

repository·main·Indexed 27 days ago

https://github.com/elastic/go-elasticsearch

The official Go client for interacting with Elasticsearch. It provides low-level and typed APIs, including DSL builders, to facilitate CRUD operations, searching, and bulk indexing. The library includes the esutil.BulkIndexer for efficient parallel indexing and supports performance optimizations such as fasthttp transport and easyjson encoding.

Tokens
46.8K
Snippets
123
Records
225
Agent score
90%

What's inside go-elasticsearch

  1. Overview of go-elasticsearch features

    main

    The go-elasticsearch client provides several ways to interact with Elasticsearch:

    • Typed API: Offers strongly typed requests and decoded responses with compile-time safety.
    • esdsl builders: A fluent DSL for constructing queries, aggregations, mappings, and sort options.
    • Low-level API: Provides raw JSON control for endpoints not yet covered by the typed API.
    • Observability: Built-in OpenTelemetry instrumentation for distributed tracing.
    • Extensibility: Supports Interceptors for custom middleware (e.g., auth rotation, custom logging).
    • Reliability: Includes automatic retries, request compression, and node discovery.
    • Helpers: Convenience tools for tasks like bulk indexing and JSON encoding.

  2. Understand the go-elasticsearch architecture

    main

    The client uses a layered architecture where both the Typed API and the Low-level API share a common transport layer:

    1. Application Layer: Your Go code using either TypedClient or the low-level Client.
    2. Interceptors Layer: Optional middleware for modifying requests and responses (e.g., for dynamic authentication).
    3. Transport Layer: Handles the core networking concerns including retry logic, request compression, node selection (round-robin), and connection pooling.
    4. Elasticsearch: The remote service.

  4. Use the Low-level API

    main

    The low-level API provides a one-to-one mapping with the Elasticsearch REST API. Each endpoint accepts raw io.Reader request bodies and returns *esapi.Response objects.

    When to use the low-level API:

    • You need an endpoint not covered by the typed API.
    • You require full control over request serialization (e.g., custom JSON encoders or streaming bodies).
    • You are working with pre-baked JSON payloads and want to avoid modeling them as Go structs.

    Note: For most new code, the typed API is recommended for compile-time safety and automatic JSON handling.

  6. Perform basic Elasticsearch operations with the Typed API

    main

    The following examples demonstrate common operations using the recommended Typed API and esdsl query builders.

    Create an index

    client.Indices.Create("my_index").Do(context.TODO())

    Index a document

    document := struct {
    Name string `json:"name"`
}{
    "go-elasticsearch",
}
client.Index("my_index").
    Id("1").
    Document(document).
    Do(context.TODO())

    Get a document

    client.Get("my_index", "1").Do(context.TODO())

    Search documents

    Use esdsl builders for fluent query syntax:

    client.Search().
    Index("my_index").
    Query(esdsl.NewMatchAllQuery()).
    Do(context.TODO())

    Update a document

    client.Update("my_index", "1").
    Request(&update.Request{
        Doc: json.RawMessage(`{"language": "Go"}`),
    }).Do(context.TODO())

    Delete a document

    client.Delete("my_index", "1").Do(context.TODO())

    Delete an index

    client.Indices.Delete("my_index").Do(context.TODO())

  7. Migrate from low-level API to typed API

    main

    The Go client provides two API surfaces: the low-level API (*elasticsearch.Client) and the typed API (*elasticsearch.TypedClient). Moving to the typed API provides type-safe requests, automatically decoded responses, fluent builders via the esdsl package, and reduced boilerplate (no manual JSON parsing or manual response body closing).

    To perform a full migration, replace the low-level client constructor with elasticsearch.NewTyped using the same functional options.

    // Before (Low-level)
client, err := elasticsearch.New(
    elasticsearch.WithAddresses("https://localhost:9200"),
    elasticsearch.WithAPIKey("API_KEY"),
)

// After (Typed)
client, err := elasticsearch.NewTyped(
    elasticsearch.WithAddresses("https://localhost:9200"),
    elasticsearch.WithAPIKey("API_KEY"),
)

  9. Deploy a Go Cloud Function using the Elasticsearch client

    main

    To deploy a Google Cloud Function that uses the Elasticsearch Go client, you must vendor your dependencies and use the gcloud CLI. Ensure you set the ELASTICSEARCH_URL environment variable to point to your Elasticsearch instance.

    Deployment steps:

    1. Run go mod vendor to prepare dependencies.
    2. Use gcloud functions deploy with the following configuration:
      • --entry-point: The name of the function to execute (e.g., Health).
      • --runtime: The Go runtime version (e.g., go111).
      • --trigger-http: Enables HTTP triggers.
      • --set-env-vars: Pass the ELASTICSEARCH_URL.
    go mod vendor
gcloud functions deploy clusterstatus \
    --entry-point Health \
    --runtime go111 \
    --trigger-http \
    --memory 128MB \
    --set-env-vars ELASTICSEARCH_URL=https://...cloud.es.io:9243

  10. Set up OpenTelemetry instrumentation for distributed tracing

    main

    The Go client supports built-in OpenTelemetry integration to create spans for every Elasticsearch API call. You can enable this by passing elasticsearch.NewOpenTelemetryInstrumentation to the client via elasticsearch.WithInstrumentation.

    NewOpenTelemetryInstrumentation accepts two parameters:

    • provider (trace.TracerProvider): The OpenTelemetry tracer provider. Pass nil to use the globally registered provider via otel.GetTracerProvider().
    • captureSearchBody (bool): When true, the search query body is captured as the db.statement span attribute for supported search endpoints.

    Warning: Enabling captureSearchBody may expose sensitive data in your traces. Only enable it in development or when your trace backend is secured.

    Supported endpoints for captureSearchBody: search, async_search.submit, msearch, eql.search, terms_enum, search_template, msearch_template, and render_search_template.

    import (
    "github.com/elastic/go-elasticsearch/v9"
    "go.opentelemetry.io/otel"
)

// Use the global TracerProvider (set up elsewhere in your application)
es, err := elasticsearch.NewTyped(
    elasticsearch.WithInstrumentation(elasticsearch.NewOpenTelemetryInstrumentation(nil, false)),
)

  11. Connect to Elastic Cloud using an API key

    main

    To connect to an Elastic Cloud deployment, use elasticsearch.NewTyped with the WithCloudID and WithAPIKey options. You can find your Cloud ID and endpoint on the My deployment page in Elastic Cloud, and generate an API key under Management > Security.

    client, err := elasticsearch.NewTyped(
    elasticsearch.WithCloudID("<CloudID>"), // <1>
    elasticsearch.WithAPIKey("<ApiKey>"),   // <2>
)

  12. Run common Elasticsearch client benchmarks

    main

    The _benchmarks directory contains the source code required to execute the common Elasticsearch client benchmarks. The benchmarking suite is organized into three main components:

    • benchmarks package: Handles general configuration for the benchmark runs.
    • runner package: Responsible for executing and measuring client actions, and storing the resulting metrics in an Elasticsearch cluster.
    • actions package: Defines the specific individual client interactions being tested.

    For detailed information on the benchmark results and metrics, visit the official Elasticsearch client benchmarks site.