pgvector-go

When creating approximate indexes (HNSW or IVFFlat) in PostgreSQL, you must specify the correct operator class based on the distance metric you intend to use:

• L2 Distance (Euclidean): Use vector_l2_ops.
• Inner Product: Use vector_ip_ops.
• Cosine Distance: Use vector_cosine_ops.

Example SQL for HNSW index: CREATE INDEX ON items USING hnsw (embedding vector_l2_ops)

To use pgvector-go with sqlx:

1. Install: go get github.com/pgvector/pgvector-go
2. Define Model: Use the pgvector.Vector type in your struct.
3. Operations: Use standard NamedExec or Select methods. Pass pgvector.NewVector as a parameter in your SQL queries.

type Item struct {
    Embedding pgvector.Vector
}

// Insert
item := Item{
    Embedding: pgvector.NewVector([]float32{1, 2, 3}),
}
_, err := db.NamedExec(`INSERT INTO items (embedding) VALUES (:embedding)`, item)

// Query
var items []Item
db.Select(&items, "SELECT * FROM items ORDER BY embedding <-> $1 LIMIT 5", pgvector.NewVector([]float32{1, 1, 1}))

To use pgvector-go with GORM:

1. Install: go get github.com/pgvector/pgvector-go
2. Define Model: Use the pgvector.Vector type with the gorm:"type:vector(N)" tag.
3. Querying: Use db.Clauses with clause.OrderBy to pass the distance operator and the pgvector.NewVector value as a parameter.

type Item struct {
    Embedding pgvector.Vector `gorm:"type:vector(3)"`
}

// Query nearest neighbors
var items []Item
db.Clauses(clause.OrderBy{
    Expression: clause.Expr{SQL: "embedding <-> ?", Vars: []interface{}{pgvector.NewVector([]float32{1, 1, 1})}},
}).Limit(5).Find(&items)

To use pgvector-go with the Bun ORM:

1. Install: go get github.com/pgvector/pgvector-go
2. Define Model: Use the pgvector.Vector type with the bun:"type:vector(N)" tag.
3. Querying: Use OrderExpr with the distance operator (e.g., <->) to find nearest neighbors.
4. Indexing: You can implement the bun.AfterCreateTableHook to automatically create HNSW or IVFFlat indexes during table creation.

type Item struct {
    Embedding pgvector.Vector `bun:"type:vector(3)"`
}

// Insert
item := Item{
    Embedding: pgvector.NewVector([]float32{1, 2, 3}),
}
_, err := db.NewInsert().Model(&item).Exec(ctx)

// Query
var items []Item
err := db.NewSelect().
    Model(&items).
    OrderExpr("embedding <-> ?", pgvector.NewVector([]float32{1, 2, 3})).
    Limit(5).
    Scan(ctx)

To use pgvector-go with the pg driver:

1. Install: go get github.com/pgvector/pgvector-go
2. Enable extension: Execute CREATE EXTENSION IF NOT EXISTS vector.
3. Define Model: Use the pgvector.Vector type in your struct with the pg:"type:vector(N)" tag, where N is the dimension.
4. Operations: Use pgvector.NewVector([]float32{...}) for insertions and OrderExpr for distance-based queries.

type Item struct {
    Embedding pgvector.Vector `pg:"type:vector(3)"`
}

// Insert
item := Item{
    Embedding: pgvector.NewVector([]float32{1, 2, 3}),
}
_, err := db.Model(&item).Insert()

// Query nearest neighbors
var items []Item
err := db.Model(&items).
    OrderExpr("embedding <-> ?", pgvector.NewVector([]float32{1, 2, 3})).
    Limit(5).
    Select()

To use pgvector types (vector, halfvec, and sparsevec) with the pgx/v5 driver, you must call RegisterTypes. This function queries the database for the OIDs of the vector types and registers them in the connection's TypeMap. This enables seamless scanning and passing of vector types between Go and PostgreSQL.

Note that vector is treated as a required type; if it is not found in the database, RegisterTypes will return an error. halfvec and sparsevec are registered only if they exist in the database.

import (
	"context"
	"github.com/jackc/pgx/v5"
	"github.com/pgvector/pgvector-go/pgx"
)

// ... setup connection ...

err := pgx.RegisterTypes(ctx, conn)
if err != nil {
	// handle error
}

Use NewSparseVector or NewSparseVectorFromMap to initialize a SparseVector. SparseVector is a wrapper that implements sql.Scanner and driver.Valuer, making it compatible with standard Go database interfaces.

• NewSparseVector(vec []float32): Creates a sparse vector from a dense slice. It automatically filters out zero values and stores only non-zero elements.
• NewSparseVectorFromMap(elements map[int32]float32, dim int32): Creates a sparse vector from a map where keys are indices and values are the non-zero elements. The indices are automatically sorted.

import "github.com/pgvector/pgvector-go"

// From a dense slice
dense := []float32{1.0, 0.0, 2.0, 0.0}
vec := pgvector.NewSparseVector(dense)

// From a map of non-zero elements
elements := map[int32]float32{
    1: 0.5,
    3: 1.2,
}
vecFromMap := pgvector.NewSparseVectorFromMap(elements, 4)

The Vector type is a wrapper around []float32 designed to work seamlessly with SQL databases, JSON, and binary formats. It implements sql.Scanner and driver.Valuer, allowing you to use it directly in database queries using standard Go database/sql or compatible drivers like pgx.

To create a vector, use NewVector. To access the underlying data, use Slice().

import "github.com/pgvector/pgvector-go"

vec := pgvector.NewVector([]float32{1.0, 2.0, 3.0})
// Access underlying slice
raw := vec.Slice()