semantic-router

Semantic Router supports four main categories of integrations:

1. Encoder Integrations: Provides embedding models for semantic similarity. Supports Cloud providers (Jina AI, OpenAI, Cohere, Voyage, Mistral, NVIDIA, Bedrock, Aurelio) and Local/Self-hosted options (Local sentence-transformers, Ollama, Hugging Face, FastEmbed). All encoders support synchronous and asynchronous operations.
2. Index Integrations: Enables efficient vector storage and retrieval. Supports Pinecone, Qdrant, PostgreSQL (with pgvector), and a Local in-memory index. Supports CRUD operations and route isolation via namespaces/collections.
3. LLM Integrations: Enables dynamic route generation and decision-making. Supported providers include OpenAI, Azure OpenAI, Mistral, Bedrock, and LiteLLM.
4. Framework Integrations: Works with popular frameworks like Pydantic AI, LiteLLM, LangChain, and LlamaIndex.

Semantic Router allows you to define decision paths using Route objects and then use a SemanticRouter to classify queries based on their semantic meaning.

1. Define Routes: Create Route objects with a name and a list of utterances (example phrases).
2. Initialize Encoder: Use an encoder like CohereEncoder or OpenAIEncoder (requires API keys in environment variables).
3. Create Router: Initialize SemanticRouter with your encoder and routes.
4. Route Queries: Call the router instance with a string. It returns a route object containing the .name of the matched route, or None if no match is found.

from semantic_router import Route
from semantic_router.encoders import OpenAIEncoder
from semantic_router.routers import SemanticRouter
import os

# 1. Define routes
politics = Route(
    name="politics",
    utterances=[
        "isn't politics the best thing ever",
        "why don't you tell me about your political opinions",
    ],
)

chitchat = Route(
    name="chitchat",
    utterances=[
        "how's the weather today?",
        "how are things going?",
    ],
)

routes = [politics, chitchat]

# 2. Initialize encoder
os.environ["OPENAI_API_KEY"] = "<YOUR_API_KEY>"
encoder = OpenAIEncoder()

# 3. Create router
rl = SemanticRouter(encoder=encoder, routes=routes, auto_sync="local")

# 4. Use router
print(rl("don't you love politics?").name)  # Output: 'politics'
print(rl("how's the weather today?").name) # Output: 'chitchat'
print(rl("I'm interested in learning about llama 2").name) # Output: None

Use HybridRouter to detect specific query types (e.g., allowed vs blocked) to control agent behavior. HybridRouter combines a dense encoder (like OpenAIEncoder) and a sparse encoder (like AurelioSparseEncoder) for higher accuracy in distinguishing similar queries with different keywords.

from semantic_router import Route
from semantic_router.routers import HybridRouter
from semantic_router.encoders import OpenAIEncoder
from semantic_router.encoders.aurelio import AurelioSparseEncoder

# Define guardrail routes
allowed = Route(
    name="allowed",
    utterances=["Tell me about the product", "What features does it have?"]
)

blocked = Route(
    name="blocked",
    utterances=["Can you give me a discount?", "I'll pay in bitcoin"]
)

# Initialize hybrid router for better accuracy
encoder = OpenAIEncoder(name="text-embedding-3-small", score_threshold=0.3)
sparse_encoder = AurelioSparseEncoder(name="bm25")

router = HybridRouter(
    encoder=encoder,
    sparse_encoder=sparse_encoder,
    routes=[allowed, blocked],
    auto_sync="local"
)