DOCSEARCHbyAlgolia
Submit Docs

Pezzo LLMOps Platform

repository·main·Indexed 25 days ago

https://github.com/pezzolabs/pezzo

Pezzo is a cloud-native, open-source LLMOps platform that enables developers to manage prompts, observe AI operations, and optimize LLM costs and latency. It provides centralized management and observability tools, supporting clients for Node.js (@pezzo/client), Python, and LangChain. The platform integrates with prompt service providers including OpenAI, Azure OpenAI, and Anthropic.

Tokens
11.8K
Snippets
39
Records
109
Agent score
85%

What's inside Pezzo

  1. Overview of Pezzo Prompt Management

    main

    Pezzo is a platform designed to streamline the lifecycle of Generative AI prompts. It provides tools for designing, testing, versioning, and deploying prompts across different providers and models without requiring code changes for every iteration.

    Key capabilities include:

    • Prompt Management: Centralized management of prompts regardless of the underlying AI provider or model.
    • Prompt Editor: A platform-based interface to ideate, design, and polish prompts before implementation.
    • Prompt Testing: Built-in testing tools to validate prompt performance within the Pezzo platform.
    • Environments: Support for managing multiple environments (e.g., development, production) for prompt isolation.
    • Version Control: Versioning capabilities that allow for tracking changes and rolling back to previous prompt versions.
    • Instant Deployment: The ability to deploy prompts to specific environments with a single click, bypassing traditional software release cycles.

  2. Overview of Pezzo features

    main

    Pezzo is an open-source toolkit for AI development that provides the following core capabilities:

    • Centralized Prompt Management: Manage all AI prompts in a single location.
    • Prompt Design & Versioning: Tools to create, edit, test, and version prompts.
    • Instant Deployments: Publish prompt updates instantly without a full application release cycle.
    • Observability: Access execution history, metrics (duration, prompt cost, completion cost), and statistics.
    • Troubleshooting: Use 'time travel' to inspect failed prompts, fine-tune them, and commit fixes immediately.
    • Cost Transparency: Monitor costs across different prompts and AI models.
    • Multi-language Support: Official clients available for Node.js and Python.

  3. Overview of Pezzo Proxy

    main

    The Pezzo Proxy is an open-source proxy server (Apache 2.0 license) designed to automatically route LLM requests through Pezzo. It is designed to be a transparent layer that works with any existing codebase without requiring SDK installations or code changes.

    Key Benefits:

    • Zero-code integration: Integrate Pezzo without changing how you write LLM consumption code.
    • Language agnostic: Works with any language (Node.js, Python, Golang, etc.) because it operates at the network/request level.
    • Always up-to-date: Supports new LLM provider features (like OpenAI) immediately upon release.
    • Enhanced capabilities: Enables features like a caching layer to reduce API costs.
    • Transparent maintenance: Fixes can be pushed to the proxy without requiring updates to your application code.

  4. Overview of Pezzo LLMOps Platform

    main
    Pezzo is a cloud-native, open-source LLMOps platform designed to help developers observe and monitor AI operations, troubleshoot issues, and manage prompts in a centralized location. It provides features for prompt management, observability, and caching to optimize costs and latency in AI applications.

  5. Monitor Generative AI operations with Pezzo Observability

    main

    Pezzo provides built-in observability for Generative AI operations, including:

    • Traces and Requests: Automatic tracing of operations and requests with detailed execution views.
    • Advanced Filtering: Ability to filter traces and requests by any field, including custom fields like user ID or correlation ID.
    • Metrics: Automatic collection of LLM call metrics such as execution time, cost, and error rates, presented in dashboards.
    • Alerts (Coming Soon): Future capability to define alerts on metrics for notifications.

  6. Filter requests in the Requests View

    main

    You can narrow down the list of requests in the Requests view using various criteria. Supported filtering options include:

    • Status: Filter by the success or failure of the request.
    • Date/Time: Filter requests within specific time ranges.
    • Cost: Filter by the cost of execution.
    • Custom Fields: Filter by any custom field you have included in your requests (e.g., user ID, correlation ID).

  7. Commit and Publish a Prompt

    main

    To make a prompt available for consumption in your application, you must follow a two-step process:

    1. Commit: Click the Commit button at the top right of the Prompt Editor. Provide a commit message (e.g., Initial version) and click Commit.
    2. Publish: Click the Publish button at the top right and select the target environment (e.g., Production).

    Note: Every Pezzo project includes a Production environment by default, but you can manage other environments via the Environments page.