pyvespa Python API

To scale ColPali retrieval, use a two-phase ranking approach in your RankProfile:

1. First Phase: Use a fast metric like bm25 (for text) or an approximate MaxSim using inverted Hamming distance (for binary embeddings) to narrow down candidates.
2. Second Phase: Perform the full MaxSim operation using float representations of the embeddings to re-rank the top-k results.

Vespa's unpack_bits function is used to convert compressed binary vectors back to float for the second phase.

from vespa.package import RankProfile, Function, FirstPhaseRanking, SecondPhaseRanking

colpali_profile = RankProfile(
    name="default",
    inputs=[("query(qt)", "tensor<float>(querytoken{}, v[128])")],
    functions=[
        Function(
            name="max_sim",
            expression="""
                sum(
                    reduce(
                        sum(
                            query(qt) * unpack_bits(attribute(embedding)) , v
                        ),
                        max, patch
                    ),
                    querytoken
                )
            """,
        ),
        Function(name="bm25_score", expression="bm25(title) + bm25(text)"),
    ],
    first_phase=FirstPhaseRanking(expression="bm25_score"),
    second_phase=SecondPhaseRanking(expression="max_sim", rerank_count=100),
)
colpali_schema.add_rank_profile(colpali_profile)

When performing Approximate Nearest Neighbor (ANN) search in Vespa that combines vector similarity with metadata filtering, you can optimize performance using the VespaNNParameterOptimizer class from the vespa.evaluation module.

This optimizer analyzes a sample of queries to determine the best values for parameters that control how Vespa handles the interaction between ANN and filtering. It performs measurements and returns a report containing suggestions for the following parameters:

• ranking.matching.approximateThreshold
• ranking.matching.filterFirstThreshold
• ranking.matching.filterFirstExploration
• ranking.matching.postFilterThreshold

To use it, provide your Vespa application instance, a list of queries, and the number of target hits used in those queries.

from vespa.evaluation import VespaNNParameterOptimizer

optimizer = VespaNNParameterOptimizer(
    app=my_vespa_app,
    queries=my_list_of_queries,
    hits=number_of_target_hits_used_in_my_queries,
)
report = optimizer.run()

suggested_parameters = {
    "ranking.matching.approximateThreshold": report["approximateThreshold"]["suggestion"],
    "ranking.matching.filterFirstThreshold": report["filterFirstThreshold"]["suggestion"],
    "ranking.matching.filterFirstExploration": report["filterFirstExploration"]["suggestion"],
    "ranking.matching.postFilterThreshold": report["postFilterThreshold"]["suggestion"],
}

To improve the accuracy of AI coding assistants (like Claude Code, GitHub Copilot, or Cursor) when working with pyvespa, provide them with specific instructions to fetch Markdown versions of the documentation instead of scraping HTML. This ensures the LLM receives clean, structured text optimized for consumption.

Add the following block to your project's instruction files (e.g., CLAUDE.md, AGENTS.md, .cursorrules, or .github/copilot-instructions.md):

## pyvespa

When looking up pyvespa documentation, prefer fetching the Markdown versions
instead of scraping HTML pages:

- Documentation index: https://vespa-engine.github.io/pyvespa/llms.txt
- Full documentation: https://vespa-engine.github.io/pyvespa/llms-full.txt
- Per-page markdown: replace `.html` with `.md.txt` in any docs URL

Examples:
- https://vespa-engine.github.io/pyvespa/reads-writes.md.txt
- https://vespa-engine.github.io/pyvespa/api/vespa/application.md.txt