Explorer for Laravel Scout

repository·master·Indexed 19 days ago

https://github.com/jeroen-g/explorer

A next-generation Elasticsearch driver for Laravel Scout that provides a query builder for complex searches using must(), should(), and filter() methods. It supports advanced Elasticsearch features including aggregations, index aliases for zero-downtime deployments, custom query syntax via SyntaxInterface, and flexible connection configurations for Elastic Cloud, Basic Auth, and API Keys.

Tokens
14.1K
Snippets
60
Records
68
Agent score
63%

What's inside Explorer

  1. Compose complex queries using Query builder methods

    master

    Explorer allows you to build complex queries by defining the context of the query using three primary methods: must, should, and filter. These methods follow a "more-matches-is-better" approach where documents matching more criteria receive higher scores.

    • must: The query must appear in matching documents and contributes to the relevance score.
    • should: The query should appear in the matching document (improves score if present).
    • filter: The query must appear in matching documents, but the score is ignored. This is ideal for structured data like dates, flags, or categories to improve performance and avoid unnecessary scoring calculations.
    $results = Post::search('Self-steering')
        ->filter(new Term('organisation_sector', 'IT'))
        ->get();
  2. Use nested mapping for complex objects

    master

    To handle objects or related models (like an author within a post), use nested mapping. This is supported in both the configuration-based and model-based mapping methods.

    To query nested fields, use dot notation (e.g., author.name) within a Nested query object.

    // Mapping definition
    [
        'id' => 'keyword',
        'author' => [
            'name' => 'text',
        ],
    ]
    
    // Querying
    $posts = Post::search('my post')
        ->must(new Nested('author', new Matching('author.name', 'Jeroen')))
        ->get();
  3. How index aliases work for zero downtime deployments

    master

    Explorer uses index aliases to enable zero downtime deployments. When using aliases, three distinct aliases are managed:

    1. Read alias: Used for performing search queries.
    2. Write alias: Used for writing/updating data.
    3. History alias: Aggregates all old indices, allowing them to be pruned.

    The Update Lifecycle: When updating an index (e.g., via elastic:update), Explorer creates a brand new index with a unique name. The 'write' alias is immediately pointed to this new index so that all Scout updates are forwarded to it. Once all entities have been successfully imported into the new index, the 'read' alias is also pointed to the new index. This ensures that users continue to search against the old, stable index until the new one is fully populated.

  4. Enable index aliases for a model

    master

    To use index aliases, a model must satisfy two conditions:

    1. It must implement the JeroenG\Explorer\Application\Aliased interface.
    2. The index must be explicitly enabled for aliasing in your config/explorer.php configuration.

    Important Migration Note: If you already have existing indices and want to switch to using aliases, you must delete those existing indices first. In Elasticsearch, a name cannot be both an index and an alias simultaneously.

    <?php
    
    namespace App\Models;
    
    use Illuminate\Database\Eloquent\Factories\HasFactory;
    use Illuminate\Database\Eloquent\Model;
    use JeroenG\Explorer\Application\Explored;
    use JeroenG\Explorer\Application\Aliased;
    use Laravel\Scout\Searchable;
    
    class Post extends Model implements Explored, Aliased
    {
        use HasFactory;
        use Searchable;
    
        //...
    }
    // config/explorer.php
    return [
        'indexes' => [
            'posts' => [
                'aliased' => true,
                'properties' => [
                    'id' => 'keyword',
                    'title' => 'text',
                    'created_at' => 'date',
                    'published' => 'boolean',
                    'author' => 'nested',
                ],
            ],
        ],
    ];
  5. Perform advanced Elasticsearch queries with Explorer

    master

    While standard Laravel Scout is limited to simple fuzzy term searches, Explorer provides a query builder to execute complex Elasticsearch queries. You can chain methods like must(), should(), and filter() using specific query objects to build sophisticated searches.

    Common query objects include:

    • Matching: For matching specific fields.
    • Terms: For matching multiple values, with optional boosting.
    • Term: For exact term filtering.
    // Example: Complex search with must, should (with boosting), and filter
    $posts = Post::search('lorem')
        ->must(new Matching('title', 'ipsum'))
        ->should(new Terms('tags', ['featured'], 2))
        ->filter(new Term('published', true))
        ->get();
  6. Connect using an Elastic Cloud ID

    master

    If you are using Elastic Cloud, you can connect by providing an elasticCloudId instead of individual host/port settings in the connection array.

        return [
            'connection' => [
                'elasticCloudId' => 'staging:dXMtZWFzdC0xLmF3cy5mb3VuZC5pbyRjZWM2ZjI2MWE3NGJmMjRjZTMzYmI4ODExYjg0Mjk0ZiRjNmMyY2E2ZDA0MjI0OWFmMGNjN2Q3YTllOTYyNTc0Mw',
            ],
        ];
  7. Prepare searchable data using the BePrepared interface

    master

    If you need to modify your data before it is indexed in Elasticsearch (for example, to conditionally change values or perform text transformations) without altering your Elasticsearch analyzers, you can use the data preparation feature.

    To enable this, your model must implement the JeroenG\Explorer\Application\BePrepared interface and define a prepare(array $searchable) method. The $searchable array passed to this method contains the data generated by Laravel Scout's toSearchableArray() method. The method must return the modified array.

    use JeroenG\Explorer\Application\BePrepared;
    
    class YourModel extends Model implements BePrepared
    {
        public function prepare(array $searchable): array
        {
            // Modify the data before it is indexed
            $searchable['name'] = ucfirst($searchable['name'] ?? '');
    
            return $searchable;
        }
    }
  8. Sort search results using orderBy()

    master

    By default, Explorer search results are sorted by their Elasticsearch score. To apply custom sorting, use the orderBy() method. This method leverages the underlying Laravel Scout functionality. You can specify the field name and the direction (asc or desc).

    use App\Models\Post;
    
    $results = Post::search('Self-steering')
        ->orderBy('published_at', 'desc')
        ->get();
  9. Configure text analysis and synonyms in Explorer

    master

    Text analysis is configured by implementing the IndexSettings interface on your model. This allows you to define custom analyzers and filters (such as synonyms) that Elasticsearch will use during indexing and searching.

    To implement synonyms, you can use the SynonymFilter and StandardAnalyzer classes. For example, to make searching for 'Vue' also return results for 'React', you would define a synonym mapping and include a lowercase filter to ensure case-insensitive matching.

    Note: Custom analyzers and synonym filters are computationally 'expensive' for Elasticsearch. Before implementing them, consider if wildcards or fuzzy queries can achieve your goal.

    <?php
    
    namespace App\
    Models;
    
    use Illuminate\Database\Eloquent\Factories\HasFactory;
    use Illuminate\Database\Model;
    use JeroenG\Explorer\Application\Explored;
    use JeroenG\Explorer\Application\IndexSettings;
    use JeroenG\Explorer\Domain\Analysis\Analysis;
    use JeroenG\Explorer\Domain\Analysis\Analyzer\StandardAnalyzer;
    use JeroenG\Explorer\Domain\Analysis\Filter\SynonymFilter;
    use Laravel\Scout\Searchable;
    
    class Post extends Model implements Explored, IndexSettings
    {
        use HasFactory;
        use Searchable;
    
        protected $fillable = ['title', 'published'];
    
        public function mappableAs(): array
        {
            return [
                'id' => 'keyword',
                'title' => [
                    'type' => 'text',
                    'analyzer' => 'frameworks',
                ],
                'published' => 'boolean',
                'created_at' => 'date',
            ];
        }
        
        public function indexSettings(): array
        {
            $synonymFilter = new SynonymFilter();
            $synonymFilter->setSynonyms(['vue => react']);
    
            $synonymAnalyzer = new StandardAnalyzer('frameworks');
            $synonymAnalyzer->setFilters(['lowercase', $synonymFilter]);
    
            return (new Analysis())
                ->addAnalyzer($synonymAnalyzer)
                ->addFilter($synonymFilter)
                ->build();
        }
    }
  10. Add aggregations to a search query

    master

    Aggregations allow you to summarize your search data (e.g., counting occurrences of specific terms). You can add an aggregation to a search query using the aggregation(name, aggregationObject) method on a search instance.

    To use aggregations, you must pass an instance of an aggregation class (such as TermsAggregation) as the second argument. The first argument is the custom name you wish to assign to this aggregation in the results.

    $search = Cartographer::search();
    $search->aggregation('places', new TermsAggregation('place'));
    
    $results = $search->raw();
    $aggregations = $results->aggregations();