DOCSEARCHbyAlgolia
Submit Docs

OpenLLMetry

repository·main·Indexed 11 days ago

https://github.com/traceloop/openllmetry

An open-source observability toolkit built on OpenTelemetry for LLM applications. It provides specialized instrumentations for LLM providers (such as Anthropic, Cohere, Groq, and Google Generative AI), vector databases (including Chroma and LanceDB), and AI frameworks like Langchain, CrewAI, Haystack, and Agno to enable deep tracing and monitoring.

Tokens
86.8K
Snippets
334
Records
400
Agent score
93%

What's inside OpenLLMetry

  1. Overview of OpenTelemetry Semantic Conventions for Gen-AI

    main
    The opentelemetry-semantic-conventions-ai package provides an extension to the standard OpenTelemetry Semantic Conventions specifically designed for Generative AI applications. It defines a standardized set of attributes for spans that allow developers to monitor and debug key Gen-AI metrics, such as prompts, completions, and token usage, in a consistent way across different LLM providers and frameworks.

  3. Format tool definitions as a single JSON array

    main

    In v0.5.x, tool definitions are no longer encoded as multiple flat, indexed attributes (e.g., gen_ai.tool.definitions.0.name). Instead, they are encoded as a single JSON-array attribute under GenAIAttributes.GEN_AI_TOOL_DEFINITIONS.

    Dashboard Impact: Dashboards that expand gen_ai.tool.definitions.{i}.name will no longer work. You must parse the JSON value of gen_ai.tool.definitions instead.

    import json
tool_defs = [
    {
        "name": "my_tool",
        "description": "Does something",
        "parameters": {...},
    }
]
span.set_attribute(GenAIAttributes.GEN_AI_TOOL_DEFINITIONS, json.dumps(tool_defs))

  4. Use OpenTelemetry instrumentations directly

    main
    If you already have an existing OpenTelemetry setup and do not want to use the traceloop-sdk, you can add any of the individual OpenLLMetry instrumentations directly to your project. The repository provides standard OpenTelemetry instrumentations for various LLM providers, Vector DBs, and frameworks.

  8. Initialize Traceloop for LLM monitoring

    main

    To start monitoring and debugging your LLM execution, use Traceloop.init(). This initializes the SDK using OpenTelemetry to perform non-intrusive tracing. You can configure the SDK to export traces to Traceloop or to your own existing observability stack. The app_name parameter identifies your service in your telemetry dashboard.

    Traceloop.init(app_name="joke_generation_service")

  12. Instrument Voyage AI with OpenTelemetry

    main

    To start capturing traces for Voyage AI, use the VoyageAIInstrumentor class. Call .instrument() before initializing your Voyage AI client to ensure all subsequent API calls are intercepted and recorded.

    from opentelemetry.instrumentation.voyageai import VoyageAIInstrumentor

# Initialize instrumentation
VoyageAIInstrumentor().instrument()

# Now use Voyage AI as usual
import voyageai
client = voyageai.Client()

# Embeddings
result = client.embed(texts=["Hello, world!"], model="voyage-3")

# Reranking
result = client.rerank(
    query="What is the capital of France?",
    documents=["Paris is the capital of France.", "London is in England."],
    model="rerank-2.5"
)