Cupcake Governance and Augmentation Orchestrator

repository·main·Indexed 43 days ago

https://github.com/eqtylab/cupcake

A policy enforcement layer for agentic AI that ensures deterministic rule-following using OPA/Rego policies compiled to WebAssembly. It intercepts agent actions, enriches them with environmental signals, and supports integration with Claude Code and OpenCode via a dedicated CLI and plugin system.

Tokens
61.4K
Snippets
190
Records
271
Agent score
54%

What's inside Cupcake

  1. Core capabilities of Cupcake

    main

    Cupcake provides several key security and governance features:

    • Granular Tool Control: Prevent specific tools or arguments (e.g., blocking rm -rf /).
    • MCP Support: Native governance for Model Context Protocol tools (e.g., mcp__memory__*, mcp__github__*).
    • LLM-as-Judge: Use a secondary LLM or agent to evaluate actions for dynamic oversight.
    • Guardrail Libraries: Integrations with NeMo and Invariant for content and safety checks.
    • Observability: Structured logs and evaluation traces for all inputs, signals, and decisions.
  2. Supported AI coding agent harnesses

    main

    Cupcake supports several AI coding agent harnesses, each with varying levels of integration, context injection, and support for 'Ask' (interactive) prompts.

    • Claude Code: Integrated via external hooks (stdin/stdout). Supports full context injection and full 'Ask' support.
    • Cursor: Integrated via external hooks (stdin/stdout). Does not support context injection and has limited 'Ask' support.
    • OpenCode: Integrated via an in-process TypeScript plugin. Has limited context injection, and 'Ask' requests are converted to 'deny' decisions.
    • Factory AI: Integrated via external hooks (stdin/stdout). Supports full context injection and full 'Ask' support.
    | Harness                       | Integration                   | Context Injection | Ask Support       |
    | ----------------------------- | ----------------------------- | ----------------- | ----------------- |
    | [Claude Code](claude-code.md) | External hooks (stdin/stdout) | Yes               | Full              |
    | [Cursor](cursor.md)           | External hooks (stdin/stdout) | No                | Limited           |
    | [OpenCode](opencode.md)       | In-process TypeScript plugin  | Limited           | Converted to deny |
    | [Factory AI](factory-ai.md)   | External hooks (stdin/stdout) | Yes               | Full              |
  3. What is Watchdog and how does it work?

    main

    Watchdog is Cupcake's LLM-as-a-judge capability. It provides semantic security analysis by evaluating AI agent tool calls using an LLM before they execute. This complements deterministic policy rules by providing context awareness and dynamic reasoning to catch threats that don't match simple patterns.

    Workflow:

    1. An AI agent attempts a tool call.
    2. Cupcake intercepts the event.
    3. Watchdog sends the event to an LLM.
    4. The LLM returns a structured judgment (allow/deny, confidence, reasoning).
    5. This judgment is available to your policies as input.signals.watchdog.
    6. Your policies decide the final outcome based on this signal.
  4. What is Cupcake and how does it work?

    main

    Cupcake is a policy enforcement layer for AI coding agents. It intercepts agent events (such as shell command execution or file edits) and evaluates them against user-defined rules written in Open Policy Agent (OPA) Rego.

    Instead of relying on model context (which consumes tokens and can be ignored), Cupcake uses policy-as-code to provide deterministic guarantees. It sits in the agent hook path and returns decisions in milliseconds.

    Cupcake supports two evaluation models:

    1. Deterministic Policies: Written in OPA/Rego and compiled to WebAssembly (Wasm) for fast, sandboxed evaluation.
    2. LLM-as-Judge (Cupcake Watchdog): Uses a secondary LLM to evaluate actions for dynamic oversight when static rules are insufficient.
  5. Understand Cupcake's differentiated feedback for Cursor

    main

    When a policy denies an action in Cursor, Cupcake provides differentiated feedback. This allows the system to send a human-readable reason to the user while providing technical context to the AI agent, helping the agent learn from the violation without exposing sensitive details to the user.

    Decision Structure

    A deny decision in Rego can return a JSON object containing:

    • reason: The message shown to the user.
    • agent_context: The technical explanation provided to the AI agent.
    • rule_id: The identifier for the specific policy triggered.
    • severity: The criticality of the violation (e.g., CRITICAL).

    Example Rego Policy

    deny contains decision if {
        input.hook_event_name == "beforeShellExecution"
        contains(input.command, "rm -rf")
        decision := {
            "reason": "Dangerous command blocked: rm -rf",
            "agent_context": "This action violates system policies. Recursive deletion of directories is prohibited for security reasons.",
            "rule_id": "CURSOR-SECURITY-001",
            "severity": "CRITICAL"
        }
    }
  6. Compare Async vs Sync Evaluation Performance

    main

    Choosing between evaluate() and evaluateSync() depends on your application type:

    MethodEvent LoopRecommended Use Case
    evaluate() (async)✅ Non-blockingWeb servers, APIs, concurrent apps
    evaluateSync() (sync)❌ BlocksCLI scripts, startup validation, simple tools

    Performance Benchmarks:

    • Initialization: ~100-200ms (one-time, includes policy compilation)
    • Evaluation: <1ms for simple policies, 1-5ms for complex policies
    • Concurrency: Supports thousands of concurrent evaluations.
  7. Understand Claude Code event payload structure

    main

    All Claude Code hook events in Cupcake are unified by a common structure. Every payload implements the EventPayload trait, which provides access to CommonEventData.

    CommonEventData Fields

    • session_id: String
    • transcript_path: String
    • cwd: String
    • permission_mode: PermissionMode

    PermissionMode Enum

    Indicates the current permission level for the session:

    • Default: User is prompted for dangerous operations.
    • Plan: Claude creates plans without executing them.
    • AcceptEdits: File edits are auto-approved.
    • BypassPermissions: All tool calls are auto-approved (dangerous).

    Special Traits

    • InjectsContext: A marker trait implemented by payloads that support special stdout handling for context injection. These include SessionStart, UserPromptSubmit, and PreCompact.
  8. Differentiated feedback in Cursor policies

    main

    When a policy denies an action in Cursor, Cupcake supports providing differentiated feedback. This allows you to send a simplified message to the user while providing detailed technical context to the AI agent to help it learn from the violation.

    In a Rego policy, the decision object can include both reason (for the user) and agent_context (for the AI).

    Example Rego Decision Structure:

    deny contains decision if {
        input.hook_event_name == "beforeShellExecution"
        contains(input.command, "rm -rf")
        decision := {
            "reason": "Dangerous command blocked: rm -rf",
            "agent_context": "This action violates system policies. Recursive deletion of directories is prohibited for security reasons.",
            "rule_id": "CURSOR-SECURITY-001",
            "severity": "CRITICAL"
        }
    }
  9. Use Cupcake built-in security guardrails

    main

    Cupcake includes several Built-ins—special policies enabled by default in rulebook.yml that protect critical system functionality and cannot be easily bypassed by AI agents. These provide layered security at both the global and project levels.

    Common Built-ins:

    • rulebook_security_guardrails: Protects the .cupcake/ directory from being tampered with or deleted by an agent.
    • protected_paths: Blocks modifications to sensitive system paths like /etc/ or /System/.
    • git_pre_check: Validates git operations.
    • git_block_no_verify: Blocks git commands that attempt to skip verification (e.g., using --no-verify).
  10. Use Builtins to implement security policies

    main

    Builtins are high-level policy abstractions that allow you to implement common security patterns without writing Rego.

    Enabling/Disabling:

    • To enable: Simply configure the builtin. If a builtin has any configuration keys, it is enabled by default.
    • To disable: Set enabled: false or remove the configuration entirely.

    Scope:

    • Project-level: Configured in .cupcake/rulebook.yml.
    • Global-level: Configured in ~/.config/cupcake/rulebook.yml. Global configs take precedence and cannot be overridden by project configs.
  11. Use Signals to enrich policy decisions with external context

    main

    Signals are external scripts, strings, or commands that Cupcake uses to enrich an agent event with real-time, external data. This allows policies to make decisions based on the current state of a system (e.g., a database) rather than just the static event data.

    How Signals Work

    1. Event Trigger: An agent performs an action (e.g., an MCP tool call).
    2. Signal Execution: Cupcake pipes the event JSON via stdin to a signal script.
    3. External Query: The script queries an external system (like a PostgreSQL database) using data extracted from the event.
    4. Data Return: The script returns data (e.g., { "within_24_hours": true }) to the Cupcake engine.
    5. Policy Evaluation: The Rego policy evaluates the combined event data and signal output to permit or deny the action.

    When to use Signals

    • Context Enrichment: When you need information only available at runtime (e.g., checking if a database record exists or its current value).
    • Advanced Guardrailing: When integrating with LLM-based classifiers or 'AI as a judge' systems. Cupcake can accept the output of these advanced systems as a signal to make a final decision.
  12. Enrich policy decisions using Signals

    main

    Signals are external scripts, strings, or commands that provide real-time, runtime context to Cupcake policies. They allow policies to make decisions based on data from external systems (like databases or APIs) rather than just the agent's tool call.

    How Signals Work

    1. Event Trigger: An agent tool call occurs (e.g., a SQL UPDATE).
    2. Signal Execution: Cupcake pipes the event JSON via stdin to a signal script.
    3. External Query: The signal script queries an external system (e.g., a PostgreSQL database) using data extracted from the event.
    4. Context Return: The script returns data (e.g., { "within_24_hours": true }) to the Cupcake engine.
    5. Policy Decision: The Rego policy evaluates the returned signal data to allow or deny the action.

    When to use Signals

    • To enrich agent events with point-in-time information.
    • To implement advanced guardrails (e.g., querying an LLM-based classifier or an external security scanner).
    • To enforce business logic that requires real-world state (e.g., "cannot cancel appointments within 24 hours").