DOCSEARCHbyAlgolia
Submit Docs

Thinking Sphinx Documentation

repository·develop·Indexed 23 days ago

https://github.com/pat/thinking-sphinx

A library that connects ActiveRecord to the Sphinx full-text search tool for Rails and other Ruby web frameworks. It provides tools for defining search indices, performing queries, and managing the Sphinx or Manticore search engines via the loadsphinx CLI and various indexing commands such as index_sql, merge_and_update, and rotate.

Tokens
4.6K
Snippets
2
Records
40
Agent score
80%

What's inside Thinking Sphinx

  1. Configure Sphinx connection (Socket vs TCP)

    develop

    Thinking Sphinx supports connecting to Sphinx via a Unix socket or via TCP. The connection type is determined by the presence of the socket setting in your configuration.

    • Socket Connection: If socket is present in your settings, Thinking Sphinx configures searchd.socket using the format #{socket}:mysql41.
    • TCP Connection: If socket is absent, it uses TCP settings. You can specify address, port, or mysql41 (which acts as a fallback for the port).

  2. Configure delta indexing for an ActiveRecord index

    develop

    To enable delta indexing (incremental updates) for an ActiveRecord model, you can configure the index with the :delta? option. When :delta? is true, the index uses a specific delta_processor and suffix.

    Relevant configuration keys in the index options include:

    • :delta?: Boolean flag to enable delta indexing.
    • :delta_processor: An object/class that can be instantiated to process delta updates. It is initialized with the database adapter and :delta_options.
    • :delta_options: A hash of options passed to the delta processor.

    When delta? is enabled, the index uses the 'delta' name suffix instead of the default 'core' suffix.

  3. Run Thinking Sphinx tests

    develop

    To run the test suite, you must first create a database named thinking_sphinx in your MySQL or PostgreSQL instance.

    Database Setup:

    CREATE DATABASE thinking_sphinx;

    Running Tests:

    • Unit tests: rake spec:unit (uses test doubles)
    • Acceptance tests: rake spec:acceptance (uses the full stack with Sphinx)
    • All tests: rake

    If using PostgreSQL, set the DATABASE environment variable:

    DATABASE=postgresql rake

  5. Use the loadsphinx CLI tool to install Sphinx or Manticore

    develop

    The loadsphinx script is a CLI utility used to download and install specific versions of either the Sphinx search engine or Manticore search engine on Linux systems. It handles dependency installation (like libmysqlclient or libodbc1) and package management (via apt or dpkg).

    Usage

    Run the script with two positional arguments:

    1. version: The specific version of the engine you want to install.
    2. engine: Either sphinx or manticore.

    Supported Engines and Versions

    Sphinx

    Supported versions include:

    • 2.1.9 (deb)
    • 2.2.11 (deb)
    • 3.0.3 (tar.gz)
    • 3.1.1 (tar.gz)
    • 3.2.1 (tar.gz)
    • 3.3.1 (tar.gz)
    • 3.4.1 (tar.gz)

    Manticore

    Supported versions include:

    • 2.6.4 (deb)
    • 2.7.5 (deb)
    • 2.8.2 (deb)
    • 3.4.2 (deb)
    • 3.5.4 (deb)
    • 4.0.2 (deb)
    • 4.2.0 (deb)
    • 6.0.0 (via official repository)

    Examples

    To install Sphinx version 3.4.1:

    ./loadsphinx 3.4.1 sphinx

    To install Manticore version 4.2.0:

    ./loadsphinx 4.2.0 manticore

  6. Install Thinking Sphinx

    develop

    Thinking Sphinx is installed as a Ruby gem. Because it needs to connect to Sphinx, you must include a MySQL driver gem in your Gemfile, even if your primary application database is PostgreSQL.

    For MRI (standard Ruby), use mysql2. For JRuby, use jdbc-mysql.

    gem 'mysql2',          '~> 0.4',    :platform => :ruby
gem 'jdbc-mysql',      '~> 5.1.35', :platform => :jruby
gem 'thinking-sphinx', '~> 6.0'

  8. Configure Thinking Sphinx settings

    develop

    Thinking Sphinx uses a configuration object to manage Sphinx connection details, index locations, and framework settings. You can access the singleton instance via ThinkingSphinx.configuration.

    Key configuration attributes include:

    • configuration_file: Path to the generated Sphinx configuration file.
    • indices_location: Location where Sphinx indices are stored.
    • version: The version of Sphinx being used (defaults to '2.2.11').
    • batch_size: Number of records to process in a single batch during indexing (defaults to 1000).
    • controller: The controller used for running Sphinx commands.
    • index_set_class: The class used to manage sets of indices.
    • indexing_strategy: The strategy used for indexing data.
    • guarding_strategy: The strategy used for guarding against re-indexing.

    Settings are typically loaded from a settings provider (like a YAML file) and applied to the indexer and searchd components.

  9. Configure connection options for Thinking Sphinx

    develop

    Connection parameters are derived from the ThinkingSphinx::Configuration instance. The following keys are used to initialize a new connection:

    • :host: The address of searchd (from configuration.searchd.address).
    • :port: The MySQL 4.1 port (from configuration.searchd.mysql41).
    • :socket: The Unix socket path (from configuration.searchd.socket).
    • :reconnect: Set to true by default.

    You can provide additional custom options by setting the connection_options key within the settings hash in your configuration.

  10. Troubleshoot SphinxQL statement length errors

    develop

    If you encounter a ThinkingSphinx::QueryLengthError, the SphinxQL statement being sent is longer than the configured maximum.

    This often happens during real-time index population if the batch size is too large. You can resolve this by reducing the batch_size in your config/thinking_sphinx.yml file.

    The maximum allowed length is determined by ThinkingSphinx::Configuration.instance.settings['maximum_statement_length'].

    development:
    batch_size: 500