Sourcebot Documentation

Sourcebot is a self-hosted tool for codebase understanding with the following core capabilities:

• Ask Sourcebot: Natural language interface to ask complex questions about your code. It uses code search and navigation to provide answers with inline citations and navigable snippets.
• Code Search: Fast search across repositories and branches supporting regular expressions, filters (repo/language), and boolean logic.
• Code Navigation: IDE-level navigation including 'goto definition' and 'find references' across all indexed repositories.
• Built-in File Explorer: A modern UI for exploring files with syntax highlighting, a file tree, and integrated code navigation.

Sourcebot supports connections to a wide variety of Git hosting providers and repository types. You can configure connections for the following platforms:

• GitHub
• GitLab
• Bitbucket Cloud
• Bitbucket Data Center
• Azure Dev Ops Cloud
• Azure Dev Ops Server
• Gitea
• Gerrit
• Other Git hosts (Generic Git hosts)
• Local Git repos

Sourcebot chat uses two distinct concepts to ground LLM responses in the codebase:

1. source: An artifact from the codebase (such as a file or a specific commit) used to provide context to the LLM.
2. references: Citations or pointers to a source included in the LLM's response, allowing developers to verify the LLM's reasoning.

Sources are attached to a chat thread via:

• Manual @ mentions: A developer explicitly mentions a source in a prompt (e.g., "what does @auth.ts do?").
• LLM Tool Calls: The LLM invokes a tool (e.g., readFile) during its response generation.

References are embedded in the LLM response using a specific pattern (e.g., @file:{auth.ts:12-24}) which is then parsed and rendered via a remark plugin. Note that reference resolution is performed on a best-effort basis due to the non-deterministic nature of LLMs.

API metrics track usage from non-web sources, including MCP integrations and direct API access.

API Active Users

Shows unique users interacting via non-web sources:

• Any: Users who used either MCP or the API.
• MCP: Users from IDE extensions and other MCP clients.
• API: Users from direct HTTP API access (e.g., via API keys), excluding web app and MCP traffic.

API Requests

Total request counts for:

• MCP: Code searches, file reads, tree listings, repo listings, and Ask chats from MCP clients.
• API: Direct HTTP API requests, excluding web app and MCP traffic.

Sourcebot is distributed as a single Docker container that manages multiple services via supervisord. The architecture consists of the following core components:

• Web Server: A Next.js application providing the Sourcebot UI.
• Backend Worker: A Node.js process responsible for incremental synchronization with code hosts (e.g., GitHub, GitLab) and asynchronous repository indexing.
• Zoekt: A trigram indexing code search engine used for high-performance code search.
• Postgres: A transactional database for business-logic data.
• Redis Job Queue: An in-memory store using BullMQ to manage asynchronous work queues.
• .sourcebot/ cache: A file-system cache used for persistent data storage.

Global analytics track usage across all Sourcebot interfaces (Web App, MCP, and API).

• Active Users: The number of unique users who performed any tracked action, such as code searches, navigations, Ask chats, file views, and tree browsing.

Note: Web repo listings are excluded from these metrics to reduce noise from passive page loads.

Web App metrics track interactions specifically within the Sourcebot web interface.

Web Active Users

Shows unique users broken down by activity type:

• All: Users who performed any web action (code searches, navigations, Ask chats, or file views), excluding repo listings.
• Search: Users who performed code searches.
• Ask: Users who created Ask chat sessions.

Web Activity

Total event counts for:

• Code Searches: Searches performed in the web search bar.
• Ask Chats: Conversations created through the web interface.
• Navigations: "Go to Definition" and "Find All References" actions in the code viewer.

Sourcebot uses Auth.js as its underlying authentication framework.

Key details:

• Providers: Supports various providers including credentials, Google, GitHub, etc.
• Mechanism: Uses JSON Web Tokens (JWT) to provide secure and reliable user authentication information.
• Data Privacy: Authentication data (emails, passwords, etc.) and your code remain within your deployment. If using credential login, passwords are encrypted at rest and in transit.

Sourcebot provides a high-performance code search engine that supports searching across multiple repositories and branches. Key capabilities include:

• Regex support: Use regular expressions for precise code matching.
• Query language: Use a rich query language to scope searches to specific files, repositories, languages, or symbol definitions.
• Branch search: Search across specific lists of branches.
• Filtering: Use the filter panel to narrow results by repository or programming language.
• Scalability: Powered by trigram indexing to handle massive codebases efficiently.
• Syntax Highlighting: Supports over 100+ languages.