Elasticsearch Python Client

The Elasticsearch Python client package is composed of several specialized modules:

• Core Client: The low-level client that provides access to the entire Elasticsearch API surface. It handles JSON translation, node discovery, load balancing, and persistent connections.
• Bulk Helpers: High-level functions designed to simplify ingesting large volumes of data using Python iterables.
• ES|QL Query Builder: An idiomatic interface for constructing ES|QL queries using Python expressions.
• DSL Module: A high-level client that allows manipulating documents and queries using Python classes and objects instead of primitive types like dictionaries and lists.

Beyond querying, elasticsearch-dsl provides an optional wrapper for document management. This includes:

• Defining mappings
• Retrieving and saving documents
• Wrapping document data in user-defined Python classes

When using the dsl module, it is highly recommended to use the built-in serializer (elasticsearch.dsl.serializer.serializer). This ensures objects are correctly converted to JSON.

• create_connection and configure use this serializer automatically.
• To support custom objects, define a to_dict() method on your objects; the serializer will call this method automatically during JSON conversion.

You can create ES|QL expressions in two ways:

1. String-based: Provide the expression as a raw string (e.g., .eval(height_feet="height * 3.281")).
2. Python-based: Use the E() helper function to wrap column names. This allows you to use Python operators which are automatically translated to ES|QL.

Example of Python-based expression:

from elasticsearch.esql import ESQL, E

query = ESQL.from_("employees").eval(height_feet=E("height") * 3.281)

from elasticsearch.esql import ESQL, E

query = (
    ESQL.from_("employees")
    .sort("emp_no")
    .keep("first_name", "last_name", "height")
    .eval(height_feet=E("height") * 3.281, height_cm=E("height") * 100)
)

To avoid explicitly passing a connection to every API call, you can define a global default connection using the create_connection method. This is the recommended approach for most applications. Any subsequent DSL operations will automatically use this connection.

# Standard Python
from elasticsearch.dsl import connections
connections.create_connection(hosts=['https://localhost:9200'], request_timeout=20)

To distribute work across the cluster and avoid overloading a single node, you can pass a list of node URLs to the client. The client will use these URLs as separate nodes in the connection pool. By default, nodes are selected using a round-robin strategy.

Note: If your cluster is behind a load balancer (like Elastic Cloud), you should use the load balancer's host and port instead of a list of individual nodes.

import os
from elasticsearch import Elasticsearch

NODES = [
    "https://localhost:9200",
    "https://localhost:9201",
    "https://localhost:9202",
]

ELASTIC_PASSWORD = os.environ['ELASTIC_PASSWORD']

client = Elasticsearch(
    NODES,
    ca_certs="/path/to/http_ca.crt",
    basic_auth=("elastic", ELASTIC_PASSWORD)
)

All ES|QL functions have Python wrappers available in the elasticsearch.esql.functions module. When using these wrappers in Python-based expressions, wrap field names or other expressions in the E() helper function.

Example using functions.length():

from elasticsearch.esql import ESQL, functions, E

query = (
    ESQL.from_("employees")
    .keep("first_name", "last_name", "height")
    .where(functions.length(E("first_name")) < 4)
)

For complex usage patterns involving the Elasticsearch Domain Specific Language (DSL) module, refer to the official examples directory in the repository. These examples demonstrate how to construct sophisticated queries and aggregations using the Python DSL module.

https://github.com/elastic/elasticsearch-py/tree/master/examples/dsl