Elasticsearch .NET Client

repository·main·Indexed 25 days ago

https://github.com/elastic/elasticsearch-net

The official strongly-typed .NET client (Elastic.Clients.Elasticsearch) for interacting with Elasticsearch. It handles API requests and responses, delegating transport concerns to the Elastic.Transport library. The repository also includes RequestConverter.Wasm, a WebAssembly-based tool for converting Elasticsearch Dev Console requests into C# client code.

Tokens
25K
Snippets
68
Records
102
Agent score
87%

What's inside elasticsearch-net

  1. Understand the breaking changes policy

    main

    The Elasticsearch .NET API Client is generated from a formal Elasticsearch API specification. Because the client is a reflection of this specification, changes made to fix discrepancies between the specification and the actual Elasticsearch API may result in breaking changes in the client.

    Breaking changes are categorized by release type:

    • Patch releases (e.g., 7.17.0 → 7.17.1): May include breaking changes to fix incorrect property types (e.g., changing a long to a string) or incorrect requirement constraints (e.g., making a required property optional). These are permitted to ensure API stability and usability.
    • Minor releases (e.g., 8.0 → 8.1): May include breaking changes resulting from API specification refinements, such as more precise type definitions to remove ambiguities.
    • Major releases (e.g., 8.x → 9.x): May include large-scale refactorings of the API specification or the underlying client framework to unlock new features.

    To track specific breaking changes, refer to the breaking changes release notes.

  2. Understand serialization in the Elasticsearch .NET Client

    main

    The Elasticsearch .NET client uses the Microsoft System.Text.Json library by default for all serialization and deserialization tasks.

    It manages two distinct serialization responsibilities:

    1. Library Types: Serialization of request and response types owned by the Elastic.Clients.Elasticsearch library. This is handled internally and does not require user configuration.
    2. User POCO Types: Serialization and deserialization of your application's Plain Old CLR Objects (POCOs) used to represent documents stored in Elasticsearch. This responsibility is configurable, allowing you to customize how your domain models are mapped to JSON.
  3. Understand Elasticsearch .NET Client versioning and compatibility

    main

    The client's major and minor versions are tied to the Elasticsearch server version.

    Versioning Warning

    The client does not strictly follow semantic versioning. Minor or patch updates may contain breaking changes. Always check the release notes before updating.

    Compatibility Matrix

    Clients are forward compatible with the same or next higher major version of Elasticsearch. They are never backward compatible with earlier Elasticsearch major versions.

    Client VersionElasticsearch 8.xElasticsearch 9.xElasticsearch 10.x
    9.x❌ no✅ yes✅ yes
    8.x✅ yes✅ yes❌ no

    Note: Compatibility does not imply feature parity. A client version may be compatible with a newer server but won't support new features introduced in that server version.

  4. Run Elasticsearch and Kibana locally

    main

    You can quickly set up a local instance of Elasticsearch and Kibana using the following command. This will host Elasticsearch at http://localhost:9200 and Kibana at http://localhost:5601.

    curl -fsSL https://elastic.co/start-local | sh
  5. Define document types for LINQ to ES|QL

    main

    When using LINQ to ES|QL, map your POCO (Plain Old CLR Object) classes to ES|QL columns using [JsonPropertyName] attributes. This ensures that C# property names correctly correspond to the column names in your Elasticsearch indices. The provider uses System.Text.Json for serialization.

    using System.Text.Json.Serialization;
    
    public class Product
    {
        [JsonPropertyName("product_id")]
        public string Id { get; set; }
        public string Name { get; set; }
        [JsonPropertyName("price_usd")]
        public double Price { get; set; }
        [JsonPropertyName("in_stock")]
        public bool InStock { get; set; }
    }
  6. Use System.Text.Json attributes for document modeling

    main

    The built-in source serializer uses the Microsoft System.Text.Json library. You can apply standard System.Text.Json attributes to your POCO (Plain Old CLR Object) document classes to control property naming or ignore specific properties during serialization to Elasticsearch.

    Common attributes include:

    • [JsonPropertyName("name")]: Sets a specific JSON property name.
    • [JsonIgnore]: Prevents a property from being serialized.
    using System.Text.Json.Serialization;
    
    public class Person
    {
        [JsonPropertyName("forename")]
        public string FirstName { get; set; }
    
        [JsonIgnore]
        public int Age { get; set; }
    }
    
    // Usage
    var person = new Person { FirstName = "Steve", Age = 35 };
    var indexResponse = await Client.IndexAsync(person);
    // Resulting JSON: { "forename": "Steve" }
  7. Register custom System.Text.Json converters

    main

    For advanced serialization logic that attributes cannot handle (e.g., mapping an enum to a boolean property), implement a custom JsonConverter<T>. You can then apply this converter to your document class using the [JsonConverter(typeof(YourConverter))] attribute.

    This is useful for maintaining compatibility with legacy JSON structures in Elasticsearch.

    using System.Text.Json;
    using System.Text.Json.Serialization;
    
    [JsonConverter(typeof(CustomerConverter))]
    public class Customer
    {
        public string CustomerName { get; set; }
        public CustomerType CustomerType { get; set; }
    }
    
    public enum CustomerType { Standard, Enhanced }
    
    public class CustomerConverter : JsonConverter<Customer>
    {
        public override Customer Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
        {
            // Implementation for reading JSON into Customer object
        }
    
        public override void Write(Utf8JsonWriter writer, Customer value, JsonSerializerOptions options)
        {
            // Implementation for writing Customer object to JSON
        }
    }
  8. Prefer asynchronous methods for Elasticsearch operations

    main

    The Elasticsearch .NET Client provides both synchronous and asynchronous methods. You should always prefer the asynchronous methods (identified by the Async suffix).

    Because the client performs HTTP requests to Elasticsearch servers, operations can be subject to network latency or long execution times for complex queries. Using synchronous methods blocks the calling thread until the request completes, which can lead to thread exhaustion and reduced throughput in high-load applications. Using asynchronous methods allows application threads to remain available for other work while waiting for the network response.

  9. Execute server-side async ES|QL queries

    main

    For long-running queries, use SubmitAsyncQueryAsync<T> to run the query in the background on the server. You can then poll for completion and stream or consume the results.

    Key Features:

    • Use EsqlAsyncQueryOptions to configure WaitForCompletionTimeout, KeepAlive, and KeepOnCompletion.
    • The EsqlAsyncQuery<T> object is IDisposable. Disposing it sends a delete request to clean up the server-side query.
    • Results can be consumed via AsAsyncEnumerable() (streaming) or AsEnumerable() (synchronous).
    await using var asyncQuery = await client.Esql.SubmitAsyncQueryAsync<Product>(
        q => q.Where(p => p.InStock),
        asyncQueryOptions: new EsqlAsyncQueryOptions
        {
            WaitForCompletionTimeout = TimeSpan.FromSeconds(5),
            KeepAlive = TimeSpan.FromMinutes(10),
            KeepOnCompletion = true
        });
    
    await asyncQuery.WaitForCompletionAsync();
    
    await foreach (var product in asyncQuery.AsAsyncEnumerable())
        Console.WriteLine(product.Name);
  10. Perform top-level aggregations

    main

    You can perform top-level aggregations using either the Fluent API or the Object Initializer API. This allows you to calculate metrics (like Max, Min, Avg, etc.) across your entire result set. To retrieve the result, use the helper methods on the Aggregations property of the response, such as GetMax, GetAverage, or GetSum.

    ### Fluent API
    ```csharp
    var response = await client.SearchAsync<Person>(search => search
        .Indices("persons")
        .Query(query => query
            .MatchAll()
        )
        .Aggregations(aggregations => aggregations
            .Add("agg_name", aggregation => aggregation
                .Max(max => max
                    .Field(x => x.Age)
                )
            )
        )
        .Size(10)
    );

    Consume the response

    var max = response.Aggregations!.GetMax("agg_name")!;
    Console.WriteLine(max.Value);
  11. Use the ES|QL API directly

    main

    For maximum flexibility and lower-level control, you can use the Elasticsearch ES|QL query API. This allows you to specify a response format such as csv, text, or json. Note that when using this approach, you are responsible for manually parsing the raw response data.

    var response = await client.Esql.QueryAsync(r => r
        .Query("FROM index")
        .Format("csv")
    );
    
    var csvContents = Encoding.UTF8.GetString(response.Data);