DOCSEARCHbyAlgolia
Submit Docs

NVIDIA NeMo Guardrails

repository·develop·Indexed 27 days ago

https://github.com/nvidia-nemo/guardrails

An open-source toolkit for adding programmable guardrails to LLM-based conversational applications to control outputs, enforce topics, dialog paths, and response styles. Includes comprehensive benchmarking tools using AIPerf, Locust, and Mock LLMs to quantify latency tradeoffs and performance across different embedding backends.

Tokens
218.2K
Snippets
636
Records
1K
Agent score
91%

What's inside NeMo Guardrails

  1. Overview of NVIDIA NeMo Guardrails

    develop

    The NVIDIA NeMo Guardrails library is an open-source Python package designed to add programmable guardrails to LLM-based applications. It allows developers to block, alter, or validate user inputs and model responses to prevent unsafe, off-topic, malicious, or policy-violating content.

    Key features include:

    • Portability: Use the same YAML and Colang configurations for local development (via the Python library) and production deployment (via the NVIDIA NeMo Guardrails microservice).
    • Extensibility: Combine built-in guardrails, NVIDIA safety models, community models, third-party APIs, and custom Python actions.
    • Comprehensive Control: Inspect and control user inputs, retrieved content, tool calls, and model outputs within a single workflow.

  2. Introduction to NVIDIA NeMo Guardrails

    develop
    NVIDIA NeMo Guardrails is an open-source Python library designed to add programmable guardrails to LLM-based applications. It works by intercepting inputs and outputs to apply configurable safety checks, allowing you to block or modify content based on defined policies. It is used for tasks such as content moderation, jailbreak detection, and topic control.

  3. Overview of Colang

    develop

    Colang is a language used within NeMo Guardrails for defining conversational flows and logic. This package provides the Colang parser.

    Important Limitations:

    • Runtime Support: While the parser is capable of parsing the full Colang language, the current runtime only supports a stripped-down version of the language.
    • Internal Implementation: The parser may internally convert certain parts of Colang code into an intermediate Markdown format for secondary parsing; this is a legacy implementation detail and is not intended for end-user interaction.

  5. Compare LLMRails and IORails engines

    develop

    NeMo Guardrails provides two primary engines: LLMRails and IORails. Choosing between them depends on whether you need complex conversation flows, retrieval, or high-performance stateless validation.

    LLMRails

    Use LLMRails for full-featured guardrails. It runs the Colang runtime and supports:

    • All Rail Types: Input, output, dialog, retrieval (RAG), and execution (custom actions).
    • Colang Support: Both Colang 1.0 and Colang 2.x.
    • Advanced Features: Multi-turn conversation state, event-based APIs (generate_events), explain() for debugging, and multimodal (vision) rails.
    • Integrations: LangChain, custom LLM injection, and third-party rail catalogs (e.g., PII detection).
    • Deployment: Supports the bundled OpenAI-compatible REST API server.

    IORails

    Use IORails for high-performance, stateless input/output validation. It does not run the Colang runtime and supports:

    • Limited Rail Types: Input, output, and tool rails only.
    • Colang Support: Colang 1.0 only.
    • Performance Features: Speculative generation (racing input rails with model generation) and admission control via AsyncWorkQueue.
    • Observability: OpenTelemetry metrics for token usage and duration (exportable to Prometheus).
    • Constraints: Stateless (no conversation state), no retrieval/RAG, no custom actions, and no bundled REST server (use via Python API only).

  6. Understand the NeMo Guardrails Runtime Process

    develop

    The NeMo Guardrails runtime uses an event-driven architecture to process user input through three main stages. When a user sends an utterance, an UtteranceUserActionFinished event is triggered, initiating the following sequence:

    1. Generate canonical user message: The system converts the raw user utterance into a canonical form (intent) using a Colang flow. This involves a vector search of canonical examples and an LLM call to map the input to an intent (e.g., UserIntent).
    2. Decide next steps: The runtime determines the next action. This can be a pre-defined Colang flow or an LLM-driven decision. Next steps fall into two categories:
      • BotIntent: The bot should say something.
      • StartInternalSystemAction: The bot should execute an action (e.g., calling a tool or system). Actions return an InternalSystemActionFinished event.
    3. Generate bot utterance: If a BotIntent is identified, the system generates the actual text response. If a knowledge base (kb/ folder) is configured, it first performs retrieve_relevant_chunks to provide context to the LLM.

  7. Understand the NeMo Guardrails processing sequence

    develop

    When a user message is processed, NeMo Guardrails typically follows a three-step sequence to determine the bot's response:

    1. Compute Canonical Form: The system attempts to map the user's utterance to a canonical message (e.g., ask general question) using the generate_user_intent task.
    2. Determine Next Step: Using the canonical form, the system looks for matching Colang flows. If no flow matches, the LLM is prompted via the generate_next_steps task to predict the next logical step (e.g., bot response for general question).
    3. Generate Bot Message: Once the next step is determined, the system generates the final response message via the generate_bot_message task.

  8. Understand Colang and its versions

    develop

    Colang is an event-driven interaction modeling language interpreted by a Python runtime, used to define guardrails flows and bot behavior in .co files. The language version used depends on your NeMo Guardrails library version:

    • Colang 1.0: Used in NeMo Guardrails versions 0.1 through 0.7. It primarily supports text-based interactions and has limitations regarding parallel flows and explicit state management.
    • Colang 2.0-alpha: Introduced in version 0.8. It features a more powerful flows engine, support for parallel flows, asynchronous actions, and a core set of abstractions: flows, events, and actions.
    • Colang 2.0-beta: Supported in version 0.9 and later. It adds an import mechanism for the standard library, a generation operator (...), and standalone/flow parameter expression evaluation.

  9. Implement Content Safety, Jailbreak, and Topic Guardrails

    develop

    NeMo Guardrails supports several security and control use cases:

    • Content Safety: Protect against harmful inputs/outputs using LLM self-checking, NVIDIA safety models (e.g., Llama 3.1 NemoGuard 8B), community models (LlamaGuard), or third-party APIs (ActiveFence, Cisco AI Defense).
    • Jailbreak Protection: Prevent adversarial manipulation via self-check detection, heuristic pattern matching, or NVIDIA NemoGuard Jailbreak Detection NIM.
    • Topic Control: Ensure conversations stay within boundaries using Dialog rails (Colang flows), Topical rails, or NVIDIA NemoGuard Topic Control NIM.

  10. LLMRails Span Hierarchy and Structure

    develop

    When tracing is enabled in LLMRails, a single request produces a tree of spans reconstructed from the interaction log. The hierarchy is as follows:

    • guardrails.request (Root, Kind: SERVER): Represents the entire request.
    • guardrails.rail (Internal, Kind: INTERNAL): One span per activated rail.
    • guardrails.action (Internal, Kind: INTERNAL): One span per action executed by a rail.
    • {operation} {model} (Client, Kind: CLIENT): One span per LLM call made during an action.

  11. Core Building Blocks of NeMo Guardrails

    develop

    The library is structured around several key components that define how guardrails are implemented and executed:

    • Rails: Specific guardrail types that run at different stages of an LLM interaction, including Input, Retrieval, Dialog, Execution, and Output rails.
    • Configuration: YAML files used to define models, prompts, rails, tracing, and other runtime settings.
    • Colang flows: A specialized language (Colang) used to define conversational flows, guardrail logic, and event-driven behavior.
    • Custom actions: Python functions, tools, or external APIs that allow you to extend guardrails with application-specific logic.
    • Runtime interfaces: The Python SDK and guardrails server that provide the interface for applications to send messages through the guardrailed system.

  12. Observability Overview for NeMo Guardrails

    develop

    NVIDIA NeMo Guardrails provides three observability signals to debug development environments and monitor production behavior:

    1. Logging: Best for debugging single requests using verbose console output, the explain() method, or the log generation option for structured per-request data.
    2. Tracing: Best for following a request through activated rails and LLM calls. It provides full OpenTelemetry semantic-convention support.
    3. Metrics: Best for tracking aggregate behavior such as request volume, latency distributions, error rates, saturation, and per-LLM-call token usage for SLO dashboards and alerting.

    Note that Tracing and Metrics follow the OpenTelemetry library-instrumentation pattern: the library depends only on the OpenTelemetry API. The host application is responsible for configuring the SDK providers, exporters, and processors.