LLM Council

The application follows a three-stage process to generate a high-quality response through multi-model collaboration:

1. Stage 1: First opinions: The user query is sent to all models in the COUNCIL_MODELS list individually. Responses are displayed in a tabbed view for inspection.
2. Stage 2: Review: The models review each other's work. To prevent bias, model identities are anonymized. Each model ranks the other responses based on accuracy and insight.
3. Stage 3: Final response: The CHAIRMAN_MODEL reviews all initial responses and the subsequent reviews to compile a single, definitive final answer.

You can start the application using a single script or by running the backend and frontend manually in separate terminals.

Option 1: Start script

Run the provided shell script from the project root:

./start.sh

Option 2: Manual execution

Terminal 1 (Backend):

uv run python -m backend.main

Terminal 2 (Frontend):

cd frontend
npm run dev

Once running, access the application at http://localhost:5173.

# Option 1
./start.sh

# Option 2
# Terminal 1
uv run python -m backend.main

# Terminal 2
cd frontend
npm run dev

The project uses uv for Python package management and npm for the frontend. Follow these steps to install all necessary dependencies:

1. Backend: Run uv sync in the project root.
2. Frontend: Navigate to the frontend directory, run npm install, and return to the root.

# Backend
uv sync

# Frontend
cd frontend
npm install
cd ..

LLM Council uses OpenRouter to communicate with multiple LLMs. You must provide an API key by creating a .env file in the project root directory.

1. Obtain an API key from openrouter.ai.
2. Create a .env file in the root folder.
3. Add your key using the OPENROUTER_API_KEY environment variable.

OPENROUTER_API_KEY=sk-or-v1-...

You can define which models participate in the council and which model acts as the final aggregator by editing backend/config.py.

• COUNCIL_MODELS: A list of OpenRouter model strings that will provide initial opinions and review each other.
• CHAIRMAN_MODEL: The specific model designated to compile all responses into the final answer.

COUNCIL_MODELS = [
    "openai/gpt-5.1",
    "google/gemini-3-pro-preview",
    "anthropic/claude-sonnet-4.5",
    "x-ai/grok-4",
]

CHAIRMAN_MODEL = "google/gemini-3-pro-preview"

To launch the full LLM Council application (both the backend and the frontend), execute the start.sh script from the repository root. This script automates the following:

1. Starts the backend server using uv on http://localhost:8001.
2. Starts the frontend development server using npm on http://localhost:5173.
3. Manages the lifecycle of both processes, allowing you to stop both servers simultaneously by pressing Ctrl+C.

./start.sh

The api object provides asynchronous methods to interact with the LLM Council backend. It targets http://localhost:8001 by default. Use these methods to manage conversation lifecycles and exchange messages.

import { api } from './api';

// List all conversations
const conversations = await api.listConversations();

// Create a new conversation
const newConv = await api.createConversation();

// Get a specific conversation by ID
const conversation = await api.getConversation(newConv.id);

// Send a standard message
const messageResponse = await api.sendMessage(newConv.id, 'Hello Council!');

Send a user message and receive real-time updates via Server-Sent Events (SSE) as each stage of the council process completes. This is ideal for UI implementations that want to show progress.

POST /api/conversations/{conversation_id}/message/stream

Request Body:
{
  "content": "string"
}

Returns: StreamingResponse (text/event-stream)

Event Types:
- `stage1_start` / `stage1_complete` (data: stage1_results)
- `stage2_start` / `stage2_complete` (data: {stage2_results, metadata: {label_to_model, aggregate_rankings}})
- `stage3_start` / `stage3_complete` (data: stage3_result)
- `title_complete` (data: {title: string})
- `complete` (no data)
- `error` (data: {message: string})

Initialize a new conversation session. This generates a unique conversation_id and returns the initial conversation object.

POST /api/conversations

Request Body: {}

Response Model: Conversation

Conversation Schema:
- id: str
- created_at: str
- title: str
- messages: List[Dict[str, Any]]

To receive real-time, streaming updates (such as token-by-token generation), use sendMessageStream. This method uses Server-Sent Events (SSE) under the hood.

It accepts a callback function onEvent(eventType, data) which is invoked for every data: line received from the stream. The data parameter is the parsed JSON object from the event.

import { api } from './api';

await api.sendMessageStream(
  'conversation-id',
  'Tell me a story',
  (eventType, event) => {
    console.log(`Received event: ${eventType}`, event);
    // Handle different event types (e.g., token updates, finished status)
  }
);

Fetch the full details of a specific conversation, including its entire message history, using its unique ID.

GET /api/conversations/{conversation_id}

Response Model: Conversation

Returns 404 if the conversation does not exist.

Send a user message to a conversation and trigger the complete 3-stage LLM Council process. This is a blocking request that waits for all stages to complete before returning the full result set.

POST /api/conversations/{conversation_id}/message

Request Body:
{
  "content": "string"
}

Response Body:
{
  "stage1": <stage1_results>,
  "stage2": <stage2_results>,
  "stage3": <stage3_result>,
  "metadata": <metadata>
}