OpenAI Guardrails

The Prompt Injection Detection guardrail uses LLM-based analysis to ensure AI actions remain aligned with user intent. It operates at two distinct checkpoints:

1. Output Guardrail (Tool Call Validation): Runs before tool execution. It validates that requested function calls align with the user's goal. It prevents the AI from calling unrelated or harmful functions (e.g., calling wire_money when the user only asked about the weather).
2. Pre-flight Guardrail (Tool Call Output Validation): Runs after tool execution. It validates that the data returned by a tool aligns with the user's request. It prevents data leakage and ensures responses stay on-topic (e.g., blocking a weather tool response that accidentally includes sensitive bank balance data).

What it flags (MISALIGNED):

• Unrelated functions called.
• Harmful operations not requested (e.g., delete_files, access_camera).
• Private data returned unrelated to the request.
• Unrelated extra data attached to responses.

What it does NOT flag:

• Reasonable actions for the user's goal (even if suboptimal).
• Partial answers or ineffective responses.
• Refusals to answer restricted content.
• Related actions that help accomplish the goal.

The evaluation dataset must be a JSONL file. The format depends on whether you are testing standard guardrails or multi-turn prompt injection detection.

Standard Guardrails Format

Each line should contain a unique id, the data to evaluate, and a mapping of expected_triggers (guardrail names to boolean values).

{
  "id": "sample-001",
  "data": "My email is john.doe@example.com",
  "expected_triggers": {
    "Contains PII": true,
    "Moderation": false
  }
}

Prompt Injection Detection (Multi-turn) Format

For multi-turn detection, the data field must be a JSON array of conversation turns. Each turn can be a user message, a function_call, a function_call_output, or assistant_text.

{
  "id": "prompt_injection_detection_001",
  "expected_triggers": {
    "Prompt Injection Detection": true
  },
  "data": [
    {
      "role": "user",
      "content": [
        {
          "type": "input_text",
          "text": "What is the weather in Tokyo?"
        }
      ]
    },
    {
      "type": "function_call",
      "name": "get_weather",
      "arguments": {
        "location": "Tokyo"
      },
      "call_id": "call1"
    },
    {
      "type": "function_call_output",
      "call_id": "call1",
      "output": {
        "location": "Tokyo",
        "temperature": 22,
        "unit": "celsius"
      }
    },
    {
      "type": "assistant_text",
      "text": "It is 22°C in Tokyo."
    }
  ]
}

OpenAI Guardrails supports two modes for handling LLM output, allowing you to balance safety against latency.

Non-Streaming (Default)

Set stream=False (or omit the parameter). In this mode, all guardrails must complete before any output is shown to the user. This ensures complete safety and prevents any unsafe content from being exposed, but results in higher latency as the user waits for full validation.

Best for: High-assurance and compliance-critical applications.

Streaming

Set stream=True. In this mode, pre-flight and input guardrails run first, then the LLM output streams to the user immediately. Output guardrails run in parallel with the stream.

Risk: Because output guardrails run in parallel, violative content may briefly appear to the user before the guardrails trigger and take action.

Best for: Low-risk, latency-sensitive applications where real-time response is a priority.

# Non-Streaming (Safe/Default)
response = await client.responses.create(
    model="gpt-5",
    input="Your input",
    stream=False
)

# Streaming (Fast/Low-latency)
response = await client.responses.create(
    model="gpt-5",
    input="Your input",
    stream=True
)

Guardrails are configured using a pipeline configuration that defines which checks run at different stages of the request lifecycle. You can use the Guardrails Wizard to generate this or define it manually in JSON.

Pipeline Stages

• Preflight: Runs before the LLM call (e.g., masking PII, moderation).
• Input: Runs in parallel with the LLM call (e.g., jailbreak detection).
• Output: Runs over the LLM generated content (e.g., fact checking, compliance).

You can use any combination of these stages (e.g., just input, just output, or all three).

{
    "version": 1,
    "input": {
        "version": 1,
        "guardrails": [
            {"name": "URL Filter", "config": {}},
            {"name": "Moderation", "config": {"categories": ["hate", "violence"]}}
        ]
    },
    "output": {
        "version": 1,
        "guardrails": [
            {"name": "Contains PII", "config": {"entities": ["EMAIL_ADDRESS", "PHONE_NUMBER"]}}
        ]
    }
}

To add safety and compliance checks to your LLM application, use GuardrailsOpenAI instead of the standard OpenAI client.

1. Create a configuration file (e.g., guardrails_config.json) using the Guardrails wizard.
2. Set the OPENAI_API_KEY environment variable.
3. Initialize GuardrailsOpenAI with your config path.

When a guardrail is triggered, the client will raise a GuardrailTripwireTriggered exception.

from pathlib import Path
from guardrails import GuardrailsOpenAI, GuardrailTripwireTriggered

client = GuardrailsOpenAI(config=Path("guardrails_config.json"))

try:
    # Chat Completions API
    chat = client.chat.completions.create(
        model="gpt-5",
        messages=[{"role": "user", "content": "Hello world"}],
    )
    print(chat.choices[0].message.content)

    # Responses API
    response = client.responses.create(
        model="gpt-5",
        input="What are the main features of your premium plan?",
    )
    print(response.output_text)
except GuardrailTripwireTriggered:
    print("Message blocked by guardrails.")