CTX Development Toolkit

The RAG (Retrieval-Augmented Generation) module is organized within the ctx/generator project. When implementing or extending RAG features, follow this directory structure:

• rag/Config/: Contains configuration classes for RAG, Stores, Vectorizers, and Transformers.
• rag/Console/: Contains CLI commands related to RAG.
• rag/MCP/Tools/: Contains MCP (Model Context Protocol) tool actions and handlers.
• rag/Service/: Contains core business logic like IndexerService and RetrieverService.
• rag/Store/: Contains the StoreFactory and store implementations.
• rag/RagParserPlugin.php: The plugin responsible for parsing RAG configurations.

The ProjectService in ctx-mcp-server manages project state, including paths, aliases, and the current active project. When implementing or modifying these methods, follow these core behaviors:

1. Path Normalization: Always normalize paths using FSPath::create($path)->toString() to ensure consistency across the state.
2. Alias Cleanup: When a project is removed via removeProject(), all aliases pointing to that project path must be automatically removed.
3. Current Project Handling: If the project being removed is the currentProject, set currentProject to null. Do not automatically switch to another project; that logic belongs to the UI/command layer.
4. State Persistence: Always call $this->saveState($state) after modifying the state to ensure changes are persisted.

The tools system uses a registry pattern to manage external command execution. It consists of four main components:

• ToolRegistry: Stores tool definitions and provides access via unique IDs.
• ToolDefinition: A configuration object containing metadata (ID, description) and a list of commands.
• ToolCommand: Represents a specific command, including its arguments, working directory, and environment variables.
• CommandExecutor: The engine that executes commands using the Symfony Process component.
• ToolHandlerInterface: An interface (implemented by RunToolHandler) that manages the execution logic based on tool type.

The Document Compilation System transforms configuration-defined documents into output files. It follows a structured lifecycle:

1. Validation: Ensures the Document has a description and outputPath.
2. Path Resolution: Resolves variables within the outputPath.
3. Overwrite Check: If overwrite is false and the file exists, compilation is skipped.
4. Content Building:
   • Adds a title (from the description).
   • Adds tags if present.
   • Processes each Source: adds source descriptions, parses content, and applies both source-specific and document-level modifiers.
5. Error Collection: Gathers any issues into an ErrorCollection.
6. File Writing: Creates necessary directories and writes the final content to the outputPath.

The prompt filtering system is an architectural layer integrated into the import process. It works as follows:

1. Configuration: You define filter rules in your YAML configuration.
2. Parsing: LocalSourceConfig or UrlSourceConfig parses these rules into a FilterConfig object.
3. Instantiation: The PromptFilterFactory consumes the FilterConfig to create concrete implementations of PromptFilterInterface (such as IdPromptFilter, TagPromptFilter, or CompositePromptFilter).
4. Execution: During the import process, the PromptConfigMerger calls shouldInclude(array $promptConfig) on the generated filters. If it returns false, that prompt is excluded from the final merged configuration.

Filtering is performed in-memory during the merge phase, ensuring minimal performance overhead.

The RAG system exposes three Model Context Protocol (MCP) tools that allow an AI to interact directly with the knowledge store:

1. rag-store: For managing the storage of knowledge.
2. rag-search: For retrieving relevant information from the store.
3. rag-manage: For administrative tasks related to the RAG system.

These tools are automatically registered when the RagBootloader is enabled and integrated into the ActionsBootloader.

CTX operates in two distinct modes:

1. MCP Server Mode: The AI interacts with your codebase in real-time. The AI Assistant communicates with the CTX MCP Server, which in turn interacts with your project files, custom tools (tests, deploy, lint), multiple projects, and context documents.

2. Context Generation Mode: This mode is used to build structured, static markdown documents for any LLM. The workflow follows: context.yaml $\rightarrow$ Sources $\rightarrow$ Filters $\rightarrow$ Modifiers $\rightarrow$ Markdown Documents.

To keep ToolAction classes thin, move all business logic into a separate Handler class. The ToolAction class should inject the Handler via its constructor and call a processing method (e.g., process()) within the __invoke method.

final readonly class ToolHandler
{
    public function __construct(
        private SomeDependency $dependency,
    ) {}

    public function process(RequestDto $request): ResultType
    {
        // Business logic here
    }
}

The CTX configuration system uses a layered architecture to separate concerns:

1. Entry Point: ConfigurationProvider initiates the process.
2. Loading: ConfigLoaderFactory creates either a ConfigLoader (for single files) or a CompositeConfigLoader (for scanning multiple files/formats).
3. Reading: ConfigReaderRegistry selects a reader based on file extension:
   • .json $\rightarrow$ JsonReader
   • .yaml or .yml $\rightarrow$ YamlReader
   • .php $\rightarrow$ PhpReader
4. Parsing: ConfigParser processes the raw data using a ParserPluginRegistry. This registry manages plugins like ImportParserPlugin, VariablesParserPlugin, and ExcludeParserPlugin.
5. Storage: The final processed configuration is stored in a ConfigRegistry.

The StoreRegistry implementation manages the lifecycle of vector stores. When getStore(string $collectionName) is called:

1. It verifies the collection exists in the configuration.
2. If the store for that collection has not been created yet, it uses the StoreFactory to instantiate it.
3. The newly created store is stored in an internal array ($stores) and returned.
4. Subsequent calls for the same collection name return the cached instance.

This pattern ensures that resources (like connections to Qdrant) are only consumed when a specific collection is actually needed.

All MCP filesystem tools in CTX now support exclude patterns. When a tool attempts to access a file or directory that matches an exclude pattern defined in the project configuration, the operation will be blocked. This ensures that sensitive files, build artifacts, or large dependency directories (like node_modules) are not accidentally read, modified, or deleted by the AI.

Supported tools with exclude support:

• directory-list
• file-search
• file-read
• file-write
• file-replace-content
• file-insert-content
• file-delete-content

The exclusion system prevents AI tools from interacting with sensitive or unwanted files by using a centralized registry.

Core Mechanism:

1. Centralized Registry: The ExcludeRegistryInterface acts as the single source of truth for all exclusion patterns.
2. Dependency Injection: Filesystem handlers (like FileReadHandler or FileWriteHandler) receive this registry via dependency injection.
3. Pre-operation Check: Before any file operation (read, write, delete, etc.) is performed, the handler builds the relative path and calls shouldExclude(string $path) on the registry.
4. Enforcement: If shouldExclude returns true, the handler aborts the operation and returns an error (e.g., Result::error("Path excluded")), preventing the underlying filesystem operation from ever occurring.