Convex Agent Component

The Agent Playground is a tool designed for experimenting with @convex-dev/agent. It provides a visual interface to interact with agent threads and test various agent behaviors.

Key capabilities include:

• Thread Management: List and browse threads associated with a specific user.
• Message Inspection: View messages within a thread, including tool call details and message metadata.
• Context Experimentation: Test contextual message lookup by adjusting different context options.
• Message Interaction: Send messages to threads with configurable saving options.
• Secure Communication: Uses API keys to communicate securely with the backend.

The RAG (Retrieval-Augmented Generation) component is a Convex component that enables LLMs to search through custom knowledge bases. It automates the process of breaking data into chunks and generating embeddings for vector-based semantic search.

Key features include:

• Namespaces: Isolate search domains for specific users or teams.
• Semantic Search: Vector-based search using configurable embedding models.
• Custom Filtering: Efficiently filter vector search results.
• Chunk Context: Retrieve surrounding chunks for improved context.
• Importance Weighting: Assign a value from 0 to 1 to documents to influence search results.
• Chunking Flexibility: Use default chunking or provide your own logic.

The Agent component automatically manages the tool call lifecycle. When a tool is called, the call and its result are stored as messages in the thread associated with the source message.

To ensure the Agent automatically processes the tool result and continues generating (re-generating) if necessary, pass stopWhen: stepCountIs(num) to generateText or streamText, where num > 1.

When a model calls multiple tools in a single step, every pending approval must be resolved (either approved or denied) before generation can continue.

If a new generation attempt is started while approvals are still unresolved, the system will automatically deny the unresolved approvals with the reason "auto-denied: new generation started". This ensures message history integrity by preventing tool calls from existing without results.

Messages in a thread use order and stepOrder (incrementing integers) to maintain sequence.

• User Messages: Added at the next available order with stepOrder: 0.
• Assistant/Tool Responses: Appended to the current order with incrementing stepOrder values.
• Manual Placement: To start a standalone message at the next available order, pass order: "next" to saveMessage. To place it at a specific position, pass a non-negative integer.
• Context Linking: Use promptMessageId to associate a response with a specific previous message. Note that if the promptMessageId is not the latest message, the LLM context will only include messages up to that ID.

The Agent component extends the standard AI SDK UIMessage type with additional metadata optimized for rendering agentic workflows.

Standard AI SDK fields:

• parts: Array of parts (e.g., text, file, image, toolCall, toolResult).
• content: String of the message content.
• role: The role (e.g., user, assistant, system).

Agent-specific helper fields:

• key: Unique identifier for the message.
• order: The message's order in the thread.
• stepOrder: The step order within a specific message order.
• status: The message status (e.g., streaming).
• agentName: The name of the agent that generated the message.
• text: The text content of the message.
• _creationTime: Timestamp of the message.

To use this type, import UIMessage from @convex-dev/agent.

When building agentic workflows, you need a mechanism to decide if a thread should be handled by an LLM or escalated to a human. Common patterns include:

1. Explicit State: Storing a flag in your database indicating whether the user or LLM is currently assigned to the thread.
2. LLM Routing: Using a fast, cheap LLM call to classify if a user question requires human intervention.
3. Vector Search: Using embeddings to compare the user's question against a corpus of questions known to require human handling.
4. Structured LLM Output: Having the primary LLM generate an object that includes a field indicating if human assistance is needed.
5. Tool-based Escalation: Providing the LLM with a tool that, when called, signals the need for human input.

Tools allow an LLM to interact with external services, databases, or APIs. You can provide tools at multiple layers of the Agent lifecycle. Specifying tools at a more specific layer will overwrite defaults following this precedence: args.tools ?? thread.tools ?? agent.options.tools.

Available injection points:

• Agent constructor: new Agent(components.agent, { tools: {...} })
• Creating a thread: createThread(ctx, { tools: {...} })
• Continuing a thread: continueThread(ctx, { tools: {...} })
• On thread functions: thread.generateText({ tools: {...} })
• Outside of a thread: supportAgent.generateText(ctx, {}, { tools: {...} })

Tool-based RAG allows the LLM to intelligently decide when it needs to search for context or add new information by providing it with a search tool. This is ideal for dynamic knowledge management where the agent needs to be adaptive.

In this pattern, the message history includes the original prompt, the tool call, and the tool response, allowing the LLM to reference the retrieved information during the conversation.

searchContext: createTool({
  description: "Search for context related to this user prompt",
  args: z.object({ query: z.string().describe("Describe the context you're looking for") }),
  handler: async (ctx, { query }) => {
    const context = await rag.search(ctx, { namespace: userId, query });
    return context.text;
  },
}),