Spacedrive

The Archive system is Spacedrive's data archival engine designed to index external data sources that exist outside the traditional filesystem. While the VDFS (Virtual Data File System) manages files, the Archive system manages structured data like emails, notes, messages, bookmarks, calendar events, and contacts.

Core Capabilities:

• Universal Indexing: Uses adapters to ingest data via a script-based protocol (stdin/stdout JSONL) from sources like Gmail, Obsidian, Slack, and GitHub.
• Hybrid Search: Combines full-text search (SQLite FTS5) with semantic vector search (LanceDB + FastEmbed) using Reciprocal Rank Fusion.
• Safety Screening: Uses Prompt Guard 2 to classify indexed text for injection attacks, categorizing content into trust tiers (authored, collaborative, external).
• Schema-driven Sources: Each source is self-contained with its own SQLite database, vector index, and TOML schema that auto-generates tables and indexes.
• AI-Ready: Provides structured search APIs for AI agents (like Spacebot) with built-in safety metadata to prevent prompt injection.

The KeyManager is Spacedrive's unified cryptographic secret storage system. It provides encrypted storage for sensitive data such as device keys, library encryption keys, paired device session keys, and cloud credentials.

Architecture

• Storage Backend: Uses redb, an embedded key-value database located at <data_dir>/secrets.redb.
• Encryption: Uses XChaCha20-Poly1305 AEAD cipher. All secrets are encrypted at rest using a Device Key.
• Root Key: The Device Key (256-bit) is stored in the OS keychain (macOS Keychain, Linux Secret Service API, or Windows Credential Manager). A plaintext file fallback exists for development/testing only.

Device Key Hierarchy

1. Device Key (from OS keychain)
   • Library Keys: Per-library encryption used for cloud credentials and library-specific secrets.
   • Paired Device Data: Session keys and device info used for P2P networking.
   • Arbitrary Secrets: Application-level secrets used by extensions or custom storage.

A Location is any directory on your device that Spacedrive tracks and monitors. When you add a location, Spacedrive immediately indexes its contents, detects changes in real-time, and syncs metadata across devices (if enabled).

Key Characteristics

• Root Entry: Every location creates a 'root entry' in the database with parent_id = NULL. This root serves as the ancestor for all files and folders within that location.
• Nested Locations: Spacedrive supports adding a sub-directory of an existing location as a new location. Because of the root entry design, this does not duplicate the file tree in the database; it simply creates a new pointer to an existing entry.
• Ownership: Ownership is inherited from the location. A Location has a device_id, and all entries within its tree are considered owned by that device. This allows for efficient ownership transfers (e.g., moving an external drive between devices) by updating a single location record.

// Location points to root entry
Location {
    entry_id: Some(123),  // References entries table
    // ... other fields
}

// Root entry for "/Users/alice/Documents"
Entry {
    id: 123,
    name: "Documents",
    kind: Directory,
    parent_id: None,  // Root has no parent
    // ... other fields
}

Spacedrive uses a peer-to-peer networking model powered by Iroh (built on QUIC) to enable secure, direct device-to-device communication without cloud servers. The architecture is centered around the NetworkingService, which coordinates connections, protocols, and device states.

Core Components

• NetworkingService: The central coordinator managing the Iroh endpoint, device registry, protocol routing, and cryptographic identity.
• NetworkIdentity: Manages the device's persistent cryptographic identity using Ed25519 keys (node_id, signing_key, and verifying_key).
• DeviceRegistry: The source of truth for all known devices and their current states (e.g., Discovered, Pairing, Paired, Connected, Disconnected).
• Protocol System: Uses ALPN (Application-Layer Protocol Negotiation) to route connections to specific handlers like pairing/1.0, sync/1.0, or transfer/1.0.

pub struct NetworkingService {
    endpoint: Endpoint,              // Iroh's QUIC endpoint
    device_registry: DeviceRegistry, // Tracks all known devices
    protocol_registry: ProtocolRegistry, // Routes messages
    identity: NetworkIdentity,       // Cryptographic identity
}

Spacebot integration in the Spacedrive interface follows a specific data flow pattern to ensure consistency and live updates. Developers should follow this architecture:

1. Context Provider: SpacebotContext provides the necessary state.
2. Data Fetching: Use TanStack Query for managing conversations, history, and workers.
3. Live Updates: Use SSE (Server-Sent Events) via EventSource for real-time updates.
4. State Changes: Use Mutations for sending messages and creating conversations.
5. Consumption: UI Routes consume the context via the useSpacebot() hook.

New Data Sources: When adding new features (like Agent lists, Status, Tasks, Memory, or Cron jobs), follow the pattern of using TanStack Query for fetching, mutations for writes, and SSE for live updates.

API Client: All interactions with the Spacebot HTTP API must go through the @spacebot/api-client package. Do not implement direct HTTP calls in the UI components; add new methods to the client instead.

// Pattern for new data sources:
// 1. TanStack Query for fetching
// 2. Mutations for writes
// 3. SSE for live updates

The Spacedrive Inspector uses a composition-based architecture to display information about different resource types. Instead of a single monolithic component with complex conditionals, the Inspector acts as a generic container that renders specific variant components (like FileInspector or LocationInspector) based on the provided InspectorVariant state.

Architecture Overview

• Inspector.tsx: The generic container that manages the variant state and handles the high-level layout.
• Variant Components: Specialized components (e.g., FileInspector.tsx, LocationInspector.tsx) that implement the specific UI for a resource type.
• Shared Components: Reusable UI primitives located in shared/ used across all variants to maintain visual consistency.

Inspector.tsx                  # Generic container
├── FileInspector.tsx         # File variant
├── LocationInspector.tsx         # Location variant
├── DeviceInspector.tsx       # Device variant (future)
├── VolumeInspector.tsx       # Volume variant (future)
└── shared/                   # Shared tab components