DOCSEARCHbyAlgolia
Submit Docs

AI SDK Python Streaming Preview

repository·main·Indexed 18 days ago

https://github.com/vercel-labs/ai-sdk-preview-python-streaming

A demonstration template for streaming AI chat completions from a Python FastAPI backend to a Next.js frontend using the Vercel AI SDK Data Stream Protocol. It features a /api/chat endpoint that accepts ClientMessage objects and returns a text/event-stream response.

Tokens
1.5K
Snippets
6
Records
7
Agent score
62%

What's inside ai-sdk-preview-python-streaming

  2. Run the example locally

    main

    To run the full stack (Next.js frontend and FastAPI backend) locally, follow these steps:

    1. API Keys: Obtain API keys from your chosen AI providers (e.g., OpenAI, Anthropic).
    2. Environment Variables: Create a .env file based on the provided .env.example and populate it with your API keys.
    3. Node Dependencies: Install frontend dependencies using pnpm install.
    4. Python Environment:
      • Create a virtual environment: virtualenv venv
      • Activate it: source venv/bin/activate
      • Install Python dependencies: pip install -r requirements.txt
    5. Start Development: Run pnpm dev to launch the development server.
    # 1. Install Node dependencies
pnpm install

# 2. Setup Python virtual environment
virtualenv venv
source venv/bin/activate
pip install -r requirements.txt

# 3. Launch development server
pnpm dev

  3. Bootstrap the AI SDK Python Streaming Preview example

    main

    You can bootstrap a local instance of this example using create-next-app with any of the supported package managers (npm, Yarn, or pnpm) by pointing to the repository URL.

    # Using npx
npx create-next-app --example https://github.com/vercel-labs/ai-sdk-preview-python-streaming ai-sdk-preview-python-streaming-example

# Using yarn
yarn create next-app --example https://github.com/vercel-labs/ai-sdk-preview-python-streaming ai-sdk-preview-python-streaming-example

# Using pnpm
pnpm create next-app --example https://github.com/vercel-labs/ai-sdk-preview-python-streaming ai-sdk-preview-python-streaming-example

  4. Merge CSS classes with cn

    main

    The cn utility function is a helper for conditionally joining CSS class names. It combines clsx for conditional logic and tailwind-merge to ensure that Tailwind CSS classes do not conflict (e.g., ensuring the last class provided takes precedence in the case of conflicts).

    import { cn } from '@/lib/utils';

// Example usage:
const className = cn('px-2 py-1', isError && 'text-red-500', 'bg-blue-500');

  5. Use the /api/chat endpoint for streaming chat completions

    main

    The /api/chat endpoint accepts a POST request containing a list of ClientMessage objects and returns a streaming response using the Data Stream Protocol.

    Request Format

    • Method: POST
    • Path: /api/chat
    • Query Parameter: protocol (defaults to 'data'). This determines the streaming protocol used.
    • Body: A JSON object matching the Request schema:
      {
  "messages": [
    { "role": "user", "content": "Hello!" }
  ]
}

    Response Format

    • Media Type: text/event-stream
    • Behavior: The response is a streamed sequence of events (text, tool calls, etc.) patched with necessary headers for the specified protocol via patch_response_with_headers.
    # Example request structure
import requests

url = "http://localhost:8000/api/chat?protocol=data"
payload = {
    "messages": [
        {"role": "user", "content": "What is the capital of France?"}
    ]
}

response = requests.post(url, json=payload, stream=True)
for line in response.iter_lines():
    if line:
        print(line.decode('utf-8'))

  6. Sanitize UI messages with sanitizeUIMessages

    main

    Use sanitizeUIMessages to clean up an array of UIMessage objects, typically before rendering them in a chat interface. This function filters out incomplete or empty assistant messages and ensures that tool-related parts are only included if they have available output.

    Specifically, it:

    1. Filters assistant message parts to only include text parts or tool- parts where state === 'output-available'.
    2. Removes any messages that end up with no valid parts.
    3. Ensures that a message is only kept if it contains at least one non-empty text part or a valid tool- part with available output.
    import { sanitizeUIMessages } from '@/lib/utils';
import { UIMessage } from '@ai-sdk/react';

const sanitized = sanitizeUIMessages(messages as UIMessage[]);