Cairn Background Agent System

The repository is organized into several key functional areas:

• agent_worker/: The agent execution engine. Contains worker.py and the CLI entry point __main__.py.
• cairn_utils/: The core logic library. Includes agents/ (prompts and implementations), github_utils.py (GitHub integration), toolbox.py (agent tools), task_storage.py (SQLite operations), and agent_classes.py (base classes).
• fastapi_app/: The web interface powered by FastAPI (app.py).
• static/: Assets for the Web UI.
• examples/: Provided usage examples.

Cairn uses a hierarchical three-tier agent system to execute complex software engineering tasks. This architecture allows for parallelizing work and maintaining context across different parts of a codebase (e.g., frontend and backend).

1. FULLSTACK PLANNER: The top-level orchestrator. It breaks down large tasks into subtasks that can run in parallel and manages communication between those parallel tasks.
2. PM (Project Manager): Specialized in delegating specific instructions to SWE agents, checking their changes, tracking dependencies, and creating Pull Requests.
3. SWE (Software Engineer): The bottom-level agent specialized in making actual code changes.

Agents can communicate with each other (e.g., a2a comms) to ensure consistency in API routes and data formats when working on parallel subtasks.

Cairn uses a local .cairn/ directory in your project root to manage persistent settings and agent memory.

Customizing Agent Behavior with settings.json

You can define global or repository-specific rules in .cairn/settings.json to guide agent behavior (e.g., coding styles or preferred libraries).

{
    "general_rules": [
        "Always use TypeScript instead of JavaScript",
        "Follow the existing code style and patterns"
    ],
    "repo_specific_rules": {
        "my-frontend": [
            "Use React hooks instead of class components"
        ]
    }
}

Repository Memory

Agents generate memory files in .cairn/memory/<repo-name>.json. This persistent memory helps agents understand repository structure more efficiently, saving time and LLM costs by reducing the need for repetitive tool calls.

The agent worker follows a specific lifecycle to ensure task state is managed correctly in the database:

1. Task Loading: The worker retrieves the task details from the database using TaskStorage.
2. Status Update: The task status is updated to Running.
3. Execution: The task logic is executed via a wrapper function.
4. Completion: Once finished, the worker updates the task record with the resulting data.

This module is intended to be run as a separate process to achieve true parallelism when processing multiple tasks.

To execute tasks, use the "New Task" button or Ctrl+K in the UI. You must select one of three agent types based on the complexity of the task:

• SWE: Recommended for simple, self-contained subtasks. The output is a new branch containing the changes.
• PM: Recommended for slightly more complex subtasks. It delegates software changes to an SWE agent. The output is a Pull Request (PR) detailing the changes.
• Fullstack Planner: Recommended for fullstack or multi-step tasks. It generates a list of subtasks that can run in parallel and manages communication between agents working on cross-subtask code.

After selecting the type, choose the target repositories from your connected list and monitor progress via real-time logs.

You can run a specific agent task from the command line using the agent_worker module. Pass the <task_id> as a positional argument to the module.

python -m agent_worker <task_id>

You can run Cairn using a single startup script or by manually starting the backend and frontend services in separate terminals.

Option 1: Startup Script (Recommended) This starts both servers and opens your browser automatically.

chmod +x run.sh
./run.sh

Option 2: Manual Startup

• Terminal 1 (Backend): Runs the FastAPI app on port 8000.

  python fastapi_app/app.py

• Terminal 2 (Frontend): Runs the development server on port 5173.

  cd frontend
  npm run dev

Access the UI at http://localhost:5173/.

Cairn provides insights like Contributor Statistics, Language Distribution, Code Ownership, and Commit Patterns. To enable these, you must provide a GitHub Personal Access Token.

1. Generate a classic token at GitHub Settings > Tokens.
2. Select the following scopes:
   • For public repositories: public_repo
   • For private repositories: repo
3. Add the token to your .env file:

   GITHUB_TOKEN=your_github_token_here

If no GITHUB_TOKEN is provided, core agent functionality remains available, but analytics will be disabled.

The backend server is an Express application that integrates API routes, error handling, and environment-specific static file serving (Vite for development, static serving for production).

To run the server, ensure the PORT environment variable is set; otherwise, it defaults to 3000. The server uses registerRoutes(app) to attach API endpoints and includes a middleware that logs API requests (method, path, status code, duration, and response body) to the console.

# The server listens on the PORT env var or 3000
PORT=3000 npm run start

To execute an agent task within a Python application, import run_agent_task from the agent_worker package. Since it is an asynchronous function, you must run it within an event loop (e.g., using asyncio.run()).

import asyncio
from agent_worker import run_agent_task

# Run a specific task
asyncio.run(run_agent_task("task_12345"))

The frontend uses Drizzle ORM for database schema management and migrations. Configuration is handled via drizzle.config.ts and requires a DATABASE_URL environment variable to be set. The configuration specifies the migration output directory, the schema definition file, and the database dialect.

import { defineConfig } from "drizzle-kit";

export default defineConfig({
  out: "./migrations",
  schema: "./shared/schema.ts",
  dialect: "postgresql",
  dbCredentials: {
    url: process.env.DATABASE_URL,
  },
});