airweave

repository·main·Indexed 27 days ago

https://github.com/airweave-ai/airweave

An open-source context retrieval layer for AI agents and RAG systems. Airweave connects to apps, databases, and documents to provide a unified, LLM-friendly search interface via SDKs, REST APIs, or MCP. It includes a Connect widget for managing integrations and a React SDK (@airweave/connect-react) for embedding the connection modal into applications.

Tokens
135.7K
Snippets
217
Records
575
Agent score
92%

What's inside airweave

  1. Overview of Airweave Webhooks

    main

    Webhooks provide real-time notifications for sync, source connection, and collection lifecycle events. Instead of polling the API, you register a webhook endpoint, and Airweave pushes updates to you via HTTP POST.

    Note: This is a Beta Feature. The API is stable but may receive enhancements.

    Workflow:

    1. An event occurs in Airweave (e.g., a sync completes).
    2. Airweave creates an event with a full payload.
    3. Svix (the delivery infrastructure) sends an HTTP POST to your registered endpoint.
    4. Your server processes the payload and responds with 200 OK.
  2. Overview of Airweave

    main
    Airweave is an open-source context retrieval layer designed for AI agents. It provides a unified search interface across various applications, databases, and documents. Instead of building individual integrations for every data source, Airweave connects to existing tools and continuously syncs information to create a searchable retrieval layer. This allows AI agents to access current, source-grounded context at query time.
  3. Understand Slack Federated Search

    main

    The Slack connector uses federated search instead of data synchronization.

    • Real-Time: Queries are sent to Slack's search.all API endpoint at query time.
    • No Syncing: Messages and files are not stored in Airweave, avoiding Slack's restrictive rate limits and keeping data within Slack.
    • Freshness: Results are always up-to-date as they are fetched live.
    • Merging: Slack results are automatically merged and ranked with results from your other connected Airweave sources.
  4. Understand Airweave MCP Server Architecture

    main

    The Airweave MCP server acts as a bridge between AI Assistants and the Airweave API. It supports two operational modes:

    1. Local mode: A single process per user communicating via stdio. Configured via environment variables (AIRWEAVE_API_KEY, AIRWEAVE_COLLECTION, AIRWEAVE_BASE_URL).
    2. Hosted mode: A stateless HTTP server that creates a fresh instance per request. It supports API key authentication via headers and OAuth 2.0 (via Auth0 + Redis) on the managed platform.

    The server translates MCP tool calls into Airweave API requests to the following endpoint format: POST /collections/{id}/search/{tier} where {tier} can be instant, classic, or agentic.

  5. Understand Google Slides connector behavior

    main

    The Google Slides connector integrates with the Google Drive API to retrieve presentations. It treats presentations as files that are processed through Airweave's standard file processing pipeline:

    • Export Format: Presentations are exported as PDF to enable full-text semantic search.
    • Capabilities: Handles presentation listing, filtering, content export, metadata preservation (ownership, sharing, timestamps), and incremental sync via the Drive Changes API.
  6. Understand the Retrieval Agent Goal and Constraints

    main

    The retrieval agent's goal is to act as a search engine, building a comprehensive result set for a user query by searching a vector database.

    Key Principles:

    • Breadth over Selectivity: Aim to find results that basic vector similarity searches might miss. It is better to return many results (even if some are borderline) than to be overly picky and miss relevant data.
    • Iteration Budget: The agent operates under a strict budget of {max_iterations} iterations. Each response containing tool calls counts as one iteration.
    • Strategy: Plan to search broadly early, read and collect in the middle, and wrap up at the end. Results collected before the budget is exhausted will still be returned.
  7. Architecture of Airweave MCP Search Server

    main

    The Airweave MCP Search Server operates in two distinct modes depending on your deployment needs:

    1. Local Mode (Stdio Transport): Used with local clients like Claude Desktop or Cursor. The client communicates via Stdio to the MCP Server, which then calls the Airweave API using environment variables (API Key, Collection) for authentication and context.
    2. Hosted Mode (HTTP Transport): A distributed architecture where a Cloud AI Platform communicates via HTTPS to a Load Balancer and multiple MCP HTTP Server pods. It uses a Redis Session Store for distributed sessions and connects to the Airweave API.

    Key Components:

    • Session Manager: Redis-backed distributed sessions with local caching.
    • OAuth Validator: Token validation with 1-hour caching.
    • Rate Limiter: Per-API-key rate limiting (100 sessions/hour).
    • Transport Layer: Supports Stdio (local) or Streamable HTTP (hosted).
  8. Understand SharePoint Online Access Control (ACL) mapping

    main

    The connector extracts permissions from Graph API drive items and maps them to canonical principals. This allows Airweave to enforce per-user search filtering by expanding Entra ID group memberships recursively.

    | Principal Type | Format | Example |
    |---|---|---|
    | User | `user:{email}` | `user:alice@contoso.com` |
    | Entra ID group | `group:entra:{group_id}` | `group:entra:abc-123-def` |
    | SP site group | `group:sp:{group_name}` | `group:sp:site_members` |
  9. Understand Airweave Core Concepts

    main

    Airweave organizes data using the following hierarchy:

    • Source: An external application, database, or document store (e.g., Notion, Slack, GitHub, Google Drive, Salesforce).
    • Source Connection: A configured, authenticated instance linking to a specific account or workspace.
    • Entity: A single indexed item extracted from a source (e.g., a Slack message, Notion page, GitHub issue).
    • Collection: A searchable knowledge base composed of entities from one or more source connections. Note: Search is always performed against exactly one collection.