CopilotKit Generative UI

Open-ended Generative UI (MCP Apps) allows an agent to return a complete UI surface (like HTML or iframes) where the frontend acts as a container. In CopilotKit, you enable this by attaching MCPAppsMiddleware to your BuiltInAgent. This allows the runtime to connect to one or more MCP (Model Context Protocol) App servers.

Use this pattern when you want the agent to interact with external specialized UI services (e.g., Excalidraw) via a remote server.

import { BuiltInAgent } from "@copilotkit/runtime/v2";
import { MCPAppsMiddleware } from "@ag-ui/mcp-apps-middleware";

const agent = new BuiltInAgent({
  model: "openai/gpt-5.2",
  prompt: "You are a helpful assistant.",
}).use(
  new MCPAppsMiddleware({
    mcpServers: [
      {
        type: "http",
        url: "https://mcp.excalidraw.com/mcp", // or your local server: http://localhost:3001/mcp
        serverId: "my-server",
      },
    ],
  }),
);

Declarative Generative UI uses structured specifications like A2UI to allow agents to describe UI components (cards, forms, etc.) that the frontend then renders.

To implement A2UI:

1. Agent Side: Provide the agent with A2UI JSONL examples in its prompt so it learns the three required message envelopes: surfaceUpdate (components), dataModelUpdate (state), and beginRendering (render signal).
2. Frontend Side: Use createA2UIMessageRenderer from @copilotkit/a2ui-renderer and pass the resulting renderer into the renderActivityMessages prop of the CopilotKitProvider.

# Agent side: Injecting A2UI examples into instructions
UI_EXAMPLES = """
---BEGIN FORM_EXAMPLE---
{"surfaceUpdate":{"surfaceId":"form-surface","components":[ ... ]}}
{"dataModelUpdate":{"surfaceId":"form-surface","path":"/","contents":[ ... ]}}
{"beginRendering":{"surfaceId":"form-surface","root":"form-column","styles":{ ... }}}
---END FORM_EXAMPLE---
"""

instruction = AGENT_INSTRUCTION + get_ui_prompt(self.base_url, UI_EXAMPLES)

return LlmAgent(
    model=LiteLlm(model=LITELLM_MODEL),
    name="ui_generator_agent",
    description="Generates dynamic UI via A2UI declarative JSON.",
    instruction=instruction,
    tools=[],
)

// Frontend side: Registering the A2UI renderer
import { CopilotKitProvider, CopilotSidebar } from "@copilotkitnext/react";
import { createA2UIMessageRenderer } from "@copilotkit/a2ui-renderer";
import { a2uiTheme } from "../theme";

const A2UIRenderer = createA2UIMessageRenderer({ theme: a2uiTheme });

export function A2UIPage({ children }: { children: React.ReactNode }) {
  return (
    <CopilotKitProvider
      runtimeUrl="/api/copilotkit-a2ui"
      renderActivityMessages={[A2UIRenderer]}   // ← hook in the A2UI renderer
    >
      {children}
      <CopilotSidebar defaultOpen labels={{ modalHeaderTitle: "A2UI Assistant" }} />
    </CopilotKitProvider>
  );
}

To integrate MCP Apps into a Next.js application, configure a BuiltInAgent with MCPAppsMiddleware and export a POST handler using copilotRuntimeNextJSAppRouterEndpoint. This setup allows the agent to call tools on an MCP server (like create_view) and renders the resulting content (e.g., an iframe) directly in the chat interface.

import {
  CopilotRuntime,
  ExperimentalEmptyAdapter,
  copilotRuntimeNextJSAppRouterEndpoint,
} from "@copilotkit/runtime";
import { BuiltInAgent } from "@copilotkit/runtime/v2";
import { NextRequest } from "next/server";
import { MCPAppsMiddleware } from "@ag-ui/mcp-apps-middleware";

const agent = new BuiltInAgent({
  model: "openai/gpt-5",
  prompt: `You are an AI diagramming assistant powered by Excalidraw...`,
}).use(
  new MCPAppsMiddleware({
    mcpServers: [
      {
        type: "http",
        url: process.env.MCP_SERVER_URL ?? "http://localhost:3001/mcp",
        serverId: "excalidraw",
      },
    ],
  }),
);

const serviceAdapter = new ExperimentalEmptyAdapter();

const runtime = new CopilotRuntime({
  agents: {
    default: agent,
  },
});

export const POST = async (req: NextRequest) => {
  const { handleRequest } = copilotRuntimeNextJSAppRouterEndpoint({
    runtime,
    serviceAdapter,
    endpoint: "/api/copilotkit",
  });

  return handleRequest(req);
};

Controlled Generative UI allows you to pre-build UI components that an agent can trigger. You own the layout and styling, while the agent controls the execution and data passing.

In CopilotKit, use the useFrontendTool hook to register a tool and define a render function. The render function receives a status (e.g., inProgress, executing, complete), args (the parameters passed by the agent), and result (the output of the tool handler), allowing you to show different UI states like loading spinners or data cards based on the tool's lifecycle.

// Weather tool - callable tool that displays weather data in a styled card
useFrontendTool({
  name: "get_weather",
  description: "Get current weather information for a location",
  parameters: z.object({ location: z.string().describe("The city or location to get weather for") }),
  handler: async ({ location }) => {
    await new Promise((r) => setTimeout(r, 500));
    return getMockWeather(location);
  },
  render: ({ status, args, result }) => {
    if (status === "inProgress" || status === "executing") {
      return <WeatherLoadingState location={args?.location} />;
    }
    if (status === "complete" && result) {
      const data = JSON.parse(result) as WeatherData;
      return (
        <WeatherCard
          location={data.location}
          temperature={data.temperature}
          conditions={data.conditions}
          humidity={data.humidity}
          windSpeed={data.windSpeed}
        />
      );
    }
    return <></>;
  },
});

The useComponent hook is used for 'Open Generative UI'. Unlike MCP Apps which rely on remote servers returning iframe URLs, useComponent allows a LangGraph (or similar) agent to generate raw content (like HTML/SVG strings) directly in a tool call. The frontend then receives this structured output and renders it using a specified React component.

This is ideal for algorithm visualizations, 3D animations (Three.js), or interactive charts where the agent produces the content directly rather than calling a remote service.

import { useComponent } from "@copilotkit/react-core/v2";
import {
  WidgetRenderer,
  WidgetRendererProps,
} from "@/components/generative-ui/widget-renderer";

// Register a named component: the agent calls "widgetRenderer" with
// { title, description, html } and the frontend renders it in a sandboxed iframe.
useComponent({
  name: "widgetRenderer",
  description: 
    "Renders interactive HTML/SVG visualizations in a sandboxed iframe. " + 
    "Use for algorithm visualizations, diagrams, widgets, and simulations.",
  parameters: WidgetRendererProps,
  render: WidgetRenderer,
});