Apache Solr

The cuVS module in Apache Solr provides specialized codecs that enable GPU-based accelerated vector search capabilities. This allows for high-performance dense vector similarity searches by offloading computations to a GPU.

CAUTION

This feature is currently experimental.

Apache Solr Learning to Rank (LTR) is a module that enables feature extraction and model deployment within Solr. It allows you to:

1. Extract features directly inside Solr to be used as input for training machine learning models.
2. Deploy trained models back into Solr to rerank the top $X$ search results based on the model's predictions.

The Solr tutorials provide a progressive learning path for developers. The sequence typically includes:

1. Basic Indexing & Search: Starting Solr, creating collections, indexing documents, and performing basic searches.
2. Faceting: Working with different datasets to explore requesting facets.
3. Custom Data: Implementing plans for your own specific data implementations.
4. Advanced Features:
   • ParamSets: Managing query parameters.
   • Vector Search: Performing similarity searches.
   • OpenNLP: Using sentiment analysis.
   • SolrCloud: Managing distributed Solr.
   • AWS: Deploying Solr to Amazon Web Services.
   • Spatial Queries: Using geospatial support for distance ranges, bounding boxes, sorting by distance, or distance-based boosting.

Apache Solr is an open-source, multi-modal search platform built on Apache Lucene. It is designed to run as a standalone full-text search server and provides high-performance capabilities for various search types and data processing tasks.

Core Capabilities:

• Search Types: Full-text, vector, analytics, and geospatial search.
• Advanced Features: Streaming, highlighting, faceting, and spellchecking.
• Deployment: Native integration with Docker and Kubernetes.
• Extensibility: Extensive plugin architecture for advanced customization.
• Interface: REST-like JSON APIs, allowing integration with virtually any programming language without requiring Java-specific client code.

The Solr SQL Module allows you to execute SQL queries against Solr collections, combining SQL syntax with Solr's full-text search capabilities. It supports both MapReduce-style and JSON Facet API aggregations to handle high query volume and high cardinality data.

To use this feature, the sql module must be enabled in your Solr configuration. The interface translates SQL queries into physical query plans using the Apache Calcite engine, which are then implemented via Solr Streaming Expressions.

Solr 10 provides two primary methods for collecting performance metrics: a pull-based Prometheus-formatted API and an OTLP push exporter.

Important Notes:

• Solr 10 metrics are currently in Beta and may change in minor releases.
• Metrics are not persisted across process restarts.
• Metrics only appear if the triggering event has occurred.
• Metrics include attributes/labels for aggregation and descriptions for clarity.

Aggregation Strategy: Solr provides raw data (Counters and Histograms). You should use a monitoring backend (like Prometheus or Grafana) to calculate rates, averages, and percentiles (p50, p95, etc.) to reduce load on Solr.

A standard Apache Solr binary distribution contains the following key directories and files:

• bin/: Contains scripts used to startup, manage, and interact with Solr instances.
• server/: A self-contained Solr instance including sample configurations and documents. Use bin/solr start --help for details on starting the server.
• example/: Contains example documents and an alternative Solr home directory with various examples.
• modules/: (Not in "slim" releases) Contains modules to extend Solr functionality. Libraries are located in modules/*/lib.
• cross-dc-manager/: (Not in "slim" releases) A standalone application under bin/ for managing Cross-Datacenter indexing across SolrClouds.
• docker/: Contains a Dockerfile for building images from source or binary distributions. Scripts for managing the Docker image are in docker/scripts.

For developers working on the Solr codebase, information is distributed across several locations:

• Gradle Command Help: Detailed documentation for Gradle commands used in the build process is located in the help/ directory as .txt files.
• Dependency and Licensing Management: Instructions for adding new dependencies and managing related licensing requirements are found in /solr/licenses/README.committers.txt.
• Development Tools: A collection of tools specifically for Solr development is available in the /dev-tools directory.
• End-User Documentation: For general usage, configuration, and API references, refer to the Solr Reference Guide.