DOCSEARCHbyAlgolia
Submit Docs

pgvector-ruby

repository·master·Indexed 41 days ago

https://github.com/pgvector/pgvector-ruby

Ruby support for the pgvector PostgreSQL extension, enabling vector similarity search. It provides integration with the pg and Sequel database libraries, supporting dense, sparse, half-precision (16-bit float), and bit vectors. Includes tools for nearest neighbor queries, index creation (HNSW and IVFFLAT), and type casting for Ruby arrays.

Tokens
4.5K
Snippets
22
Records
25
Agent score
51%

What's inside pgvector-ruby

  1. Use pgvector with the Sequel gem

    master

    To use pgvector with Sequel, enable the extension and use the pgvector plugin on your models.

    1. Enable the extension

    DB.run("CREATE EXTENSION IF NOT EXISTS vector")

    2. Create a table

    DB.create_table :items do
  primary_key :id
  column :embedding, "vector(3)"
end

    3. Configure the Model

    Add the :pgvector plugin to your Sequel::Model and specify the column name:

    class Item < Sequel::Model
  plugin :pgvector, :embedding
end

    4. Insert and Query

    # Insert
Item.create(embedding: [1, 1, 1])

# Get nearest neighbors for a specific record
item.nearest_neighbors(:embedding, distance: "euclidean").limit(5)

# Get nearest neighbors for a specific vector
Item.nearest_neighbors(:embedding, [1, 1, 1], distance: "euclidean").limit(5)

    Supported distance values for the distance option:

    • euclidean
    • inner_product
    • cosine
    • taxicab
    • hamming
    • jaccard

    5. Create approximate indices

    DB.add_index :items, :embedding, type: "hnsw", opclass: "vector_l2_ops"

    Note: Use vector_ip_ops for inner product and vector_cosine_ops for cosine distance.

  3. Use pgvector with the pg gem

    master

    To use pgvector with the pg driver, you must enable the extension in your database and optionally register type casting to handle vector results automatically.

    1. Enable the extension

    conn.exec("CREATE EXTENSION IF NOT EXISTS vector")

    2. Enable type casting (Optional)

    To automatically cast vector results to Ruby arrays, register the types with the PG::BasicTypeRegistry:

    registry = PG::BasicTypeRegistry.new.define_default_types
Pgvector::PG.register_vector(registry)
conn.type_map_for_results = PG::BasicTypeMapForResults.new(conn, registry: registry)

    3. Create a table and insert vectors

    # Create table
conn.exec("CREATE TABLE items (id bigserial PRIMARY KEY, embedding vector(3))")

# Insert vector
embedding = [1, 2, 3]
conn.exec_params("INSERT INTO items (embedding) VALUES ($1)", [embedding])

    4. Query nearest neighbors

    Use the distance operators in your SQL queries. For example, to find the 5 nearest neighbors using Euclidean distance (<->):

    conn.exec_params("SELECT * FROM items ORDER BY embedding <-> $1 LIMIT 5", [embedding]).to_a

    5. Create approximate indices

    You can use hnsw or ivfflat index types. Specify the operator class based on the distance metric:

    • Euclidean distance: vector_l2_ops
    • Inner product: vector_ip_ops
    • Cosine distance: vector_cosine_ops
    # HNSW index
conn.exec("CREATE INDEX ON items USING hnsw (embedding vector_l2_ops)")

# IVFFLAT index
conn.exec("CREATE INDEX ON items USING ivfflat (embedding vector_l2_ops) WITH (lists = 100)")
    # Enable the extension
conn.exec("CREATE EXTENSION IF NOT EXISTS vector")

  4. Automatic vector encoding and decoding in Sequel

    master

    When the pgvector plugin is configured for a model, it overrides standard attribute accessors to handle data conversion:

    • Writing ([]=): If you assign a non-string, non-nil value to a registered vector column, the plugin automatically calls Pgvector.encode(v) before saving.
    • Reading ([]): When reading from a registered vector column, the plugin automatically calls Pgvector.decode(v) to return the vector in its original format (e.g., an array).

  5. Register pgvector types with the pg gem

    master

    To enable seamless integration between the pg gem and pgvector types (such as vector, halfvec, bit, and sparsevec), you must register the types within your PostgreSQL connection's registry. This allows the pg gem to automatically encode Ruby objects into vector formats and decode PostgreSQL vector strings/binary into Ruby objects.

    Use Pgvector::PG.register_vector(registry) where registry is the type registry from your PG::Connection object.

    require 'pg'
require 'pgvector'

conn = PG.connect(dbname: 'your_db')
Pgvector::PG.register_vector(conn.type_map_for_queries)
# Note: Depending on your pg version/usage, you may need to pass 
# the registry used by your connection.

  6. Configure the pgvector plugin for Sequel models

    master

    To use pgvector with a Sequel model, use the pgvector plugin and specify which columns contain vector data. This allows the plugin to automatically handle encoding/decoding and enables nearest neighbor query methods.

    class MyModel < Sequel::Model
  plugin :pgvector, :embedding
end
    class MyModel < Sequel::Model
  plugin :pgvector, :embedding
end

  7. Enable pgvector support in Sequel

    master

    To use pgvector features with the Sequel ORM, you must register and load the :pgvector extension on your Sequel dataset. This provides the necessary methods to interact with vector and sparse vector types in PostgreSQL.

    # Assuming you have a Sequel database connection object 'DB'
DB.extension(:pgvector)
# Or on a specific dataset
DB[:my_table].extension(:pgvector)

  8. Work with Sparse Vectors

    master

    The Pgvector::SparseVector class allows you to create and manipulate sparse vectors. You can initialize them using an array or a hash of non-zero elements (where keys are indices starting at 0).

    Initialization

    # From an array
vec = Pgvector::SparseVector.new([1, 0, 2, 0, 3, 0])

# From a hash of non-zero elements { index => value }
vec = Pgvector::SparseVector.new({0 => 1, 2 => 2, 4 => 3}, 6)

    Accessing Data

    • dimensions: Returns the total number of dimensions.
    • indices: Returns an array of the indices of non-zero elements.
    • values: Returns an array of the values of non-zero elements.
    • to_a: Converts the sparse vector to a full array representation.
    vec = Pgvector::SparseVector.new({0 => 1, 2 => 2, 4 => 3}, 6)

  9. Use Pgvector::Bit for bit-based embeddings

    master

    The Pgvector::Bit class is used to handle bit vector embeddings in Ruby. You can initialize a bit vector from an array of booleans or a string of bits ('0' and '1').

    • From Array: Pass an array of booleans. true becomes `

  10. Initialize Pgvector::Bit

    master

    You can create a Pgvector::Bit instance using the constructor or factory methods:

    • Pgvector::Bit.new(data): If data is an Array, it converts boolean values to a bit string (true -> `

  11. Initialize a SparseVector

    master

    The Pgvector::SparseVector class allows you to represent sparse embeddings for use with pgvector. You can initialize a vector using either a Hash or an Array.

    • From a Hash: Requires a dimensions argument. The hash should map indices to values. Only non-zero values are stored.
    • From an Array: The dimensions are inferred from the array size. Only non-zero values are stored.

    Note: When using a Hash, you must provide the total number of dimensions. When using an Array, the dimensions are automatically set to the array's length.

    # Using a Hash (requires dimensions)
vector = Pgvector::SparseVector.new({0 => 1.5, 2 => 3.0}, 5)

# Using an Array (dimensions inferred from size)
vector = Pgvector::SparseVector.new([1.0, 0.0, 3.0, 0.0])

  12. Create a SparseVector from text or binary strings

    master

    You can reconstruct a Pgvector::SparseVector instance from serialized strings using class methods:

    • Pgvector::SparseVector.from_text(string): Parses a string in the format {index:value,index:value}/dimensions. Indices in the input string are expected to be 1-based.
    • Pgvector::SparseVector.from_binary(string): Parses a binary string produced by #to_binary.
    # From text format
vector = Pgvector::SparseVector.from_text("{1:1.0,3:3.0}/3")

# From binary format
vector = Pgvector::SparseVector.from_binary(binary_data)