Elasticsearch API Specification

repository·main·Indexed 41 days ago

https://github.com/elastic/elasticsearch-specification

The formal contract for communication between client and server components in the Elasticsearch stack, covering nearly 500 endpoints and 3000 data types. This project provides tools to generate JSON representations and OpenAPI 3.0.3 specifications, design guidelines for consistent HTTP APIs, and a validator (es-spec-validator) to ensure compliance with specification standards.

Tokens
36.1K
Snippets
99
Records
154
Agent score
99%

What's inside Elasticsearch Specification

  1. Overview of Elasticsearch API design guidelines

    main

    The Elasticsearch API design guidelines provide a set of best practices for creating HTTP APIs that are consistent and easy to consume. These guidelines are intended to support a growing number of API consumers and enable automated systems, such as automatic API documentation generation, to work effectively.

    Key Principles:

    • Guidelines, not hard rules: These are best practices distilled from experience. Deviations are permitted but should be accompanied by solid reasoning.
    • Consistency: The primary goal is to ensure the Elastic product surface is predictable for users and compatible with automated tooling.
    • Evolutionary approach: While aligning existing APIs with these principles is a goal, it is recognized as a gradual process.
  2. Convert schema.json to OpenAPI 3.0.3

    main

    The clients_schema_to_openapi utility converts the Elasticsearch API schema.json file into an OpenAPI version 3.0.3 specification. This is useful for generating OpenAPI-compliant documentation or client libraries from the core Elasticsearch API specification.

    The conversion follows this pipeline:

    1. TypeScript API generates the initial schema.json.
    2. schema.json is processed into schema-no-generics.json.
    3. schema-no-generics.json is then converted into the final openapi.json and serverless-openapi.json files.
  3. Model enumerations for portability

    main

    To ensure enumerations work reliably across different programming languages, follow these rules:

    1. Use String Values: Always represent enum members as strings.
    2. Case Sensitivity: Values must be case-sensitive and maintain consistent casing.
    3. Basic Characters: Use only basic characters in enum values.
    4. Avoid Boolean Conflation: Do not mix booleans and enumerations. If an enum evolves from a boolean, do not simply add enum values to the boolean type. Instead, create a distinct enum or add a specific value like "disabled" to the enum to maintain type safety and backward compatibility.
  4. How the compiler models the specification structure

    main

    The compiler uses a metamodel to define the structure of the JSON output. Key components of the modeling process include:

    • modelType: A utility in compiler/model/utils.ts that recursively traverses types until they are modeled according to the metamodel.
    • getNameSpace: A method used to determine the namespace of a declaration, which is particularly important when visiting extended classes.
    • getAllBehaviors: A method that traverses all ancestors of a current class to find and pass down defined behaviors.
  5. Use type aliases instead of inline unions

    main

    Avoid using unions directly within class definitions, as this is considered a code smell. Instead, create a named type alias to describe the union. This improves readability and reusability.

    Incorrect (Inline Union):

    class Foo {
      id: string | number
    }

    Correct (Type Alias):

    class Foo {
      id: Id
    }
    type Id = string | number
    // nay
    class Foo {
      id: string | number
    }
    
    // yay
    class Foo {
      id: Id
    }
    type Id = string | number
  6. Define Request and Response types

    main

    Request and Response definitions must follow the namespace and endpoint structure of the API.

    For example, the definitions for the indices.put_mapping API should be located in /specification/indices/put_mapping.

    Requirements:

    • Location: Follow the namespace/endpoint directory pattern.
    • Naming: While the file name is flexible, the type itself must be named exactly Request or Response.
  7. Understand the Elasticsearch schema structure

    main

    The Elasticsearch API specification is designed to be consumed by both dynamically and statically typed languages. The primary output is a JSON schema located at output/schema/schema.json.

    This schema is organized into three main top-level sections:

    • _info: Metadata about the specification, such as the license and title.
    • endpoints: An array containing every supported Elasticsearch endpoint.
    • types: An array containing all types used to model the various endpoints.

    Note that the specification uses custom structures and special interfaces (described in behaviors.md) to represent API behaviors that cannot be captured by standard type definitions alone.

    {
      "_info": {
        "license": {
          "name": "Apache 2.0",
          "url": "https://github.com/elastic/elasticsearch-specification/blob/main/LICENSE"
        },
        "title": "Elasticsearch Request & Response Specification"
      },
      "endpoints": [...],
      "types": [...]
    }
  8. How to implement and use Behaviors in the specification

    main

    Behaviors are used to handle APIs that require special treatment based on the language. To implement a behavior, you must define it in two locations:

    1. In the specification: Define the behavior in specification/__spec_utils/behaviors.ts.
    2. In the compiler: Add the corresponding logic in compiler/model/utils.ts.

    Note: If a behavior is defined in the specification but is missing from compiler/model/utils.ts, the compiler will throw an error.

  9. Define arrays using correct TypeScript notation

    main

    For simple types, use the short TypeScript array notation. However, if the array contains complex types (such as a union), use the full Array<T> definition to maintain clarity.

    Correct (Simple): bar: string[]

    Correct (Complex/Union): bar: Array<string | number>

    Incorrect (Complex/Union): bar: (string | number)[]

    class Foo {
      bar: string[] // yay
      bar: (string | number)[] // nay
      bar: Array<string | number> // yay
    }
  10. Implement Variants for Union Types

    main

    Variants allow language generators to understand how to build and deserialize union types (types that can be one of several different structures).

    Internal Variants

    Used when the key that distinguishes the variant is present inside the definition. Configure the parent with @variants internal tag='<field-name>'.

    typed_keys_quirk

    Used for legacy APIs where the distinguishing key is external to the definition (e.g., aggregations). Configure the parent with @variants typed_keys_quirk and the specific class with @variant name='<field-name>'.

    Container Variants

    Used for types that contain all variants inside a single definition (e.g., QueryContainer). Use @variants container. If the container has extra properties that are not part of the variants, mark them with @variant container_property.

    Untagged Variants

    Used for unions that can only be distinguished by the type of one or more fields. Warning: Use only for legacy types. Use @variants untagged.

    /** @variants internal tag='type' */
    type FooOrBar = Foo | Bar
    
    class Foo {
      type: 'foo',
      prop: string
    }
    
    class Bar {
      type: 'bar',
      prop: boolean
    }
    
    /** @variants typed_keys_quirk */
    type FooAlias = Faz | Bar
    
    /** @variant name='faz' */
    class Faz {
      prop: string
    }
    
    /** @variant name='bar' */
    class Bar {
      prop: boolean
    }
  11. Implement CommonCatQueryParameters for Cat APIs

    main

    Use the CommonCatQueryParameters behavior to implement the set of common query parameters specifically supported by all Cat APIs. Like standard query parameters, these are listed as a behavior because they can affect request structure.

    Example usage:

    class CatRequestBase extends RequestBase implements CommonCatQueryParameters {}
    class CatRequestBase extends RequestBase implements CommonCatQueryParameters {}
  12. Use dotted notation and nested objects for structures

    main

    Dots (.) can be used as separators for nesting, but they are considered an alternate way to specify a nested structure rather than part of the name itself.

    Requirement: If an API supports dotted notation, it must also allow the same structure via nested objects. Language clients typically only support nested object notation.

    Equivalency Example:

    // These are equivalent:
    {
      "index.settings.number_of_shards": 1
    }
    {
      "index": {
        "settings": {
          "number_of_shards": 1
        }
      }
    }

    Note: Dotted keys only apply to API-specified structures. User-specified names (like field names) may contain dots without requiring nesting rules.