FOSElasticaBundle Documentation

repository·master·Indexed 23 days ago

https://github.com/friendsofsymfony/foselasticabundle

A Symfony bundle that integrates the Elastica library and Elasticsearch, providing features for indexing, serialization via JmsSerializer or Symfony Serializer, and automatic synchronization of Doctrine entities. It supports aliased indexes for zero-downtime updates, index templates, and asynchronous updates.

Tokens
23.4K
Snippets
67
Records
124
Agent score
79%

What's inside FOSElasticaBundle

  1. Overview of FOSElasticaBundle features

    master

    FOSElasticaBundle provides integration between Symfony and Elasticsearch (or the Elastica library). Key capabilities include:

    • Elastica Integration: Integrates the Elastica library directly into the Symfony environment.
    • Serialization: Uses JmsSerializer or Symfony Serializer to handle conversion between PHP objects and Elasticsearch data.
    • Index Configuration: Supports explicit index configuration for Elasticsearch or allows for dynamic mapping by sending data without predefined configuration.
    • Automatic Indexing: Provides Doctrine event listeners to automatically index data when Doctrine entities are modified.
  2. New features in FOSElasticaBundle 5.0

    master

    Version 5.0 introduced several new capabilities:

    • Asynchronous Updates: Added an asynchronous index update option.
    • Index Management: Added the ability to close an index.
    • Security: Added support for HTTP authentication.
    • Paginator API:
      • Added PaginatedFinderInterface::createRawPaginatorAdapter.
      • Added PaginatorAdapterInterface::getMaxScore.
  3. How the Doctrine queue listener works

    master

    By default, FOSElasticaBundle subscribes to Doctrine events (insert, update, remove) to synchronize the Elasticsearch index during the HTTP request. This can increase response times or cause application failures if the Elasticsearch server is unavailable.

    The Doctrine queue listener improves performance and fault tolerance by moving the synchronization work to a background process. Instead of performing the index update immediately, the listener sends a message to a message queue (via EnqueueBundle), which a background worker then processes to perform the actual synchronization.

  4. Use event classes instead of constants for events

    master

    In version 6.0.0-BETA1, the bundle moved away from using string constants for event names. You must now use specific event classes to listen to lifecycle changes. This applies to several event types:

    Index Population Events:

    • PRE_INDEX_POPULATE $\rightarrow$ FOS\ElasticaBundle\Event\PreIndexPopulateEvent
    • POST_INDEX_POPULATE $\rightarrow$ FOS\ElasticaBundle\Event\PostIndexPopulateEvent

    Index Reset Events:

    • PRE_INDEX_RESET $\rightarrow$ FOS\ElasticaBundle\Event\PreIndexResetEvent
    • POST_INDEX_RESET $\rightarrow$ FOS\ElasticaBundle\Event\PostIndexResetEvent

    Transformation Events:

    • PRE_TRANSFORM $\rightarrow$ FOS\ElasticaBundle\Event\PreTransformEvent
    • POST_TRANSFORM $\rightarrow$ FOS\ElasticaBundle\Event\PostTransformEvent

    Persister Events:

    • ON_EXCEPTION $\rightarrow$ FOS\ElasticaBundle\Persister\Event\OnExceptionEvent
    • POST_ASYNC_INSERT_OBJECTS $\rightarrow$ FOS\ElasticaBundle\Persister\Event\PostAsyncInsertObjectsEvent
    • PRE_INSERT_OBJECTS $\rightarrow$ FOS\ElasticaBundle\Persister\Event\PreInsertObjectsEvent
    • POST_INSERT_OBJECTS $\rightarrow$ FOS\ElasticaBundle\Persister\Event\PostInsertObjectsEvent
    • PRE_PERSIST $\rightarrow$ FOS\ElasticaBundle\Persister\Event\PrePersistEvent
    • POST_PERSIST $\rightarrow$ FOS\ElasticaBundle\Persister\Event\PostPersistEvent
    • PRE_FETCH_OBJECTS $\rightarrow$ FOS\ElasticaBundle\Persister\Event\PreFetchObjectsEvent
  5. Speed up the populate command using a message queue

    master
    As projects grow, the standard fos:elastica:populate command can become slow, consume excessive memory, or fail due to resource exhaustion. To optimize performance, you can delegate indexing work to multiple workers using a message queue via EnqueueElasticaBundle. Instead of a single process, the populate command sends small chunks of work to workers (consumers) which process them in parallel. The performance gain scales with the number of workers you run.
  6. Use different connection strategies for Elastica clients

    master

    When configuring multiple hosts, you can choose a connection_strategy to manage how connections are handled. The bundle supports the following strategies (defined in the Elastic\Transport\NodePool namespace):

    • RoundRobin: Distributes requests across available hosts in a rotating fashion.
    • RoundRobinNoResurrect: A variation of Round Robin.
    • Simple: A basic connection strategy.

    These strategies are used to interact with Elasticsearch clusters effectively.

  7. Speed up the populate command using AWS SQS

    master

    You can accelerate the fos:elastica:populate command by offloading the population tasks to AWS SQS queues. This is achieved by implementing a custom pager-persister that pushes population tasks into SQS instead of executing them synchronously. This allows you to use multiple consumer processes to process the queue in parallel.

    Prerequisites

    • An AWS IAM user with access to SQS.
    • Two AWS SQS queues: one for the population tasks (e.g., acme_fos_elastica_populate) and one for the reply queue (e.g., acme_fos_elastica_populate_reply_queue).
    • The following composer packages installed:
      • enqueue/elastica-bundle
      • enqueue/sqs
    $ composer require enqueue/elastica-bundle
    $ composer require enqueue/sqs
  8. Use Index Repositories via RepositoryManager

    master

    For complex applications requiring multiple search methods, you can use dedicated repository classes. The fos_elastica.manager (a RepositoryManagerInterface) handles these repositories.

    By default, the manager handles ORM entities. If you are using MongoDB, you must specify the default manager in your configuration:

    fos_elastica:
        default_manager: mongodb

    To use a repository, inject the fos_elastica.manager and call getRepository('Bundle:Entity').