ai-sdk-provider-claude-code

repository·main·Indexed 18 days ago

https://github.com/ben-vargas/ai-sdk-provider-claude-code

An unofficial community provider for the Vercel AI SDK that enables the use of Claude via the official Anthropic Claude Agent SDK and Claude Code CLI. It supports AI SDK v7 (version 4.0.1) and provides features such as tool streaming, structured output via Zod, MCP server integration, warm start latency optimization, and context window monitoring. Requires a Claude Pro/Max subscription and local installation of the Claude CLI.

Tokens
38.5K
Snippets
107
Records
147
Agent score
62%

What's inside ai-sdk-provider-claude-code

  1. Understand the project structure

    main

    The project is organized into several key directories:

    • src/: Contains the core implementation, including the provider factory (claude-code-provider.ts), the language model implementation (claude-code-language-model.ts), and utilities for message conversion, JSON extraction, and error handling.
    • examples/: Provides a wide range of usage scripts, from basic text generation to complex scenarios like tool management, streaming, and session management.
    • docs/: Contains comprehensive guides and troubleshooting information.
    • run-all-examples.sh: A shell script available to execute all provided examples at once.
  2. Core features of the Claude Code AI SDK provider

    main

    The ai-sdk-provider-claude-code provider offers the following capabilities:

    • Vercel AI SDK compatibility: Works seamlessly with the standard AI SDK ecosystem.
    • Streaming support: Supports streaming responses.
    • Multi-turn conversations: Handles conversational context.
    • Native structured outputs: Provides schema compliance for supported features via constrained decoding.
    • AbortSignal support: Allows cancelling requests.
    • Tool management: Manages MCP (Model Context Protocol) servers and permissions.
    • Callbacks: Provides hooks for onSdkMessage, task/hook/MCP status events, canUseTool, and onElicitation.
    • Query controller access: Enables safe live-session controls.
    • Telemetry: Exposes standard AI SDK v7 metadata and stream parts for DevTools and OpenTelemetry (OTel) registration.
  3. Access AI SDK v5 (Historical) documentation

    main

    The documentation in this directory covers legacy provider versions 1.x–2.x, which correspond to AI SDK v5. While much of this guidance remains applicable to version 3.x, new features and current implementation details should be consulted in the main README.

    Available resources for v5 include:

    • Usage Guide: GUIDE.md
    • Troubleshooting: TROUBLESHOOTING.md
    • Tool Streaming Support: TOOL_STREAMING_SUPPORT.md (details tool streaming event semantics)
    • Migration Guide: V5_BREAKING_CHANGES.md (covers migration from v0.x to v1.x)
  4. Understand Claude Code model capabilities

    main

    The Claude Code provider supports text and object generation, but has specific limitations regarding tool calling and image input.

    ModelText GenerationObject GenerationImage InputAI SDK Tool CallingMCP Tools
    opus
    sonnet

    Note on Tool Calling: While the underlying models support tool use, this provider does not implement the standard AI SDK tool calling interface. Instead, you can use MCP (Model Context Protocol) servers for tool functionality. Claude can also use built-in tools (such as Bash, Read, and Write) directly through the Claude Code SDK.

  5. Optimize latency with Warm Start

    main

    To reduce time-to-first-token (TTFT), you can pre-spawn the CLI using startup() or WarmQuery. This creates a pre-spawned CLI subprocess that can be reused.

    Timing Metadata: You can inspect performance via finalStep.providerMetadata['claude-code']. Available keys include:

    • ttftMs: Time to first token in milliseconds.
    • ttftStreamMs: Time to first stream token in milliseconds.
    • timeToRequestMs: Time to request in milliseconds.
    • warmSpareClaimed: Boolean indicating if a warm spare was used.

    Note: WarmQuery cannot accelerate generateText or streamText calls directly; it is used to drive the raw SDK message stream.

    npx tsx examples/warm-start.ts
  6. Maintain Conversation Context with Message History

    main

    To maintain context in a conversation, pass the full array of ModelMessage objects (including previous user and assistant turns) to the messages property. While the provider returns session IDs in metadata, explicit message history is the recommended approach for continuity.

    import { ModelMessage } from 'ai';
    
    const messages: ModelMessage[] = [
      { role: 'user', content: [{ type: 'text', text: 'My name is Alice' }] },
      { role: 'assistant', content: [{ type: 'text', text: 'Nice to meet you, Alice!' }] },
      { role: 'user', content: [{ type: 'text', text: 'What is my name?' }] },
    ];
    
    const result = await generateText({
      model: claudeCode('sonnet'),
      messages,
    });
    import { ModelMessage } from 'ai';
    
    const messages: ModelMessage[] = [
      { role: 'user', content: [{ type: 'text', text: 'My name is Alice' }] },
      { role: 'assistant', content: [{ type: 'text', text: 'Nice to meet you, Alice!' }] },
      { role: 'user', content: [{ type: 'text', text: 'What is my name?' }] },
    ];
    
    const result = await generateText({
      model: claudeCode('sonnet'),
      messages,
    });
  7. Implement a custom `SessionStore` (Alpha)

    main

    For backends like Postgres, S3, or Redis, you can use the SessionStore adapter.

    • Mirroring: Use the sessionStore setting in claudeCode() to mirror transcripts to your store in addition to local files while queries run.
    • Helper Redirection: Passing a sessionStore in the options object of a helper (e.g., getSessionInfo(id, { sessionStore: myStore })) redirects that helper to use your store instead of the local filesystem.
    • Constraints: You cannot combine sessionStore with persistSession: false or enableFileCheckpointing: true.
    • Implementation: Use foldSessionSummary(), SessionKey, SessionStoreEntry, and SessionSummaryEntry as building blocks for your store implementation.
  8. Understand provider limitations

    main

    The Claude Code provider has several functional limitations:

    • No Image Support: supportsImageUrls is set to false.
    • No Embedding Support: Text embeddings are unavailable.
    • No Tool Calling: The AI SDK's tool calling interface is not implemented; only object-json mode works.
    • Text-Only: No support for file generation or other modalities.
    • Unsupported Settings: The following AI SDK settings are ignored and will trigger warnings:
      • temperature
      • maxTokens
      • topP, topK
      • presencePenalty, frequencyPenalty
      • stopSequences
      • seed
  9. Tool Streaming Limitations and Performance

    main

    When implementing tool streaming, be aware of the following constraints:

    Limitations

    • Delta Emission: The provider currently emits a single tool-input-delta payload per tool call unless the SDK sends partial updates. If Claude sends non-prefix updates (corrections/replacements), deltas are skipped and only the final input is sent in the tool-call event.
    • Images: Remote image URLs are not supported. Convert images to base64 data URLs and ensure streamingInput is configured correctly.

    Performance and Size Limits

    • Delta Calculation: Only performed for tool inputs $\le$ 10KB and for prefix-only updates. Larger or non-prefix updates skip deltas.
    • Input Size Thresholds:
      • > 100KB: Logs a warning due to performance impact.
      • > 1MB: Throws an error.
    • Memory: Tool state is retained until the stream completes to prevent duplicate tool-call emissions during multiple result/error chunks.
  10. Manage sessions using sessionId and resume

    main

    Experimental session management allows you to continue a previous CLI session or track sessions using custom IDs.

    • Resuming a session: Extract the sessionId from result.providerMetadata?.['claude-code']?.sessionId and pass it to the resume option in the model configuration.
    • Custom tracking: Pass a deterministic sessionId in the model configuration to correlate requests.
    // Resume using the session ID
    const sessionId = result.providerMetadata?.['claude-code']?.sessionId;
    
    const response = await generateText({
      model: claudeCode('sonnet', { resume: sessionId }),
      messages: [{ role: 'user', content: [{ type: 'text', text: 'What is my name?' }] }],
    });
    
    // Use a custom session ID for tracking
    const result = await generateText({
      model: claudeCode('sonnet', { sessionId: 'my-custom-session-id' }),
      messages: [{ role: 'user', content: [{ type: 'text', text: 'Hello' }] }],
    });
  11. Resume an existing session (Experimental)

    main

    You can resume a previous CLI session by extracting the sessionId from the providerMetadata of a previous response and passing it to the resume option in the claudeCode model configuration.

    import { generateText } from 'ai';
    import { claudeCode } from 'ai-sdk-provider-claude-code';
    
    // First message
    const { text, providerMetadata } = await generateText({
      model: claudeCode('sonnet'),
      messages: [{ role: 'user', content: 'My name is Bob.' }],
    });
    
    // Resume using the session ID
    const sessionId = providerMetadata?.['claude-code']?.sessionId;
    
    const { text: response } = await generateText({
      model: claudeCode('sonnet', { resume: sessionId }),
      messages: [{ role: 'user', content: 'What is my name?' }],
    });