DOCSEARCHbyAlgolia
Submit Docs

gortex

repository·main·Indexed 21 days ago

https://github.com/zzet/gortex

A high-performance code-intelligence engine for AI agents and IDEs that indexes code into a persistent, provenance-tiered knowledge graph. It provides Model Context Protocol (MCP) tools for analyzing pull requests, including PR impact scoring and triage, and features a specialized evaluation framework for measuring agent performance on software engineering tasks.

Tokens
101.5K
Snippets
287
Records
445
Agent score
74%

What's inside gortex

  1. Overview of the Gortex evaluation methodology

    main

    Gortex uses an agent-graded self-eval methodology to measure real-world quality. Instead of relying on synthetic benchmarks like NDCG@10, Gortex evaluates how actual AI agents (such as Claude Sonnet 4.6, GPT 5.4, and Copilot CLI) perform when using Gortex tools compared to when they do not.

    The methodology is built on three core principles to ensure honesty and reliability:

    1. Multi-agent testing: Tasks are scored against at least three distinct agent/model combinations to ensure results are not specific to a single provider.
    2. Bias-of-prompt check: Every task is run with both the default agent prompt and a deliberately worse "ablation prompt" to detect if performance is merely a result of prompt tuning.
    3. Negative-delta requirement: Scoring uses a classifier to distinguish between "gortex helped", "no measurable difference", and "gortex hurt". Results must explicitly report cases where Gortex caused a negative impact.

  2. Use the Gortex Eval Framework for SWE-bench evaluation

    main
    The Gortex Eval Framework is a specialized harness designed to measure the impact of Gortex's code intelligence on AI agent performance. It utilizes the SWE-bench evaluation methodology to quantify how much Gortex improves an agent's ability to solve software engineering tasks.

  3. How contract matching and workspace boundaries work

    main

    Gortex uses a matching mechanism to pair API contracts (HTTP routes, gRPC, GraphQL, etc.) between providers and consumers. Matching is strictly scoped by a tuple of (EffectiveWorkspace, EffectiveProject, ID, Role).

    To prevent spurious pairings (e.g., pairing a production service with a personal project that happen to share the same endpoint), Gortex enforces boundaries using WorkspaceID and ProjectID slugs.

    • Scoping Rule: Providers and consumers in different workspaces or different projects will never pair, even if their IDs and routes are identical.
    • Fallback Behavior: If workspace or project slugs are empty, Gortex falls back to using the RepoPrefix. This ensures single-repo setups still function correctly.

  4. Configure the Gortex Multi-server Roster

    main

    The Gortex daemon can act as a router, directing MCP traffic across multiple servers (e.g., a local Unix socket for local repos and remote HTTPS servers for cloud indexes).

    • Configuration File: The roster is stored at ~/.gortex/servers.toml.
    • Management: Use the CLI to manage the roster: gortex daemon server list, add, or remove.
    • Authentication: Tokens for remote servers can be embedded directly in the config (--auth-token) or pulled from an environment variable at request time using --auth-token-env (recommended).
    • Note: You must restart the daemon to apply changes to the roster.

  5. Understand the Gortex Evaluation Judge Prompt

    main

    The Gortex evaluation framework uses a judge model to compare two coding agent outputs: one produced with access to the Gortex MCP tool surface (WITH) and one produced without it (WITHOUT).

    To ensure reproducibility, any changes to the prompt must be accompanied by a bump to the judge_prompt_revision field in published results. The current revision is 1 (2026-05-18).

    The judge evaluates based on three possible labels:

    • (a) WITH was measurably better than WITHOUT: Higher accuracy, better completeness, fewer hallucinations, or substantially fewer tokens for the same quality.
    • (b) WITH and WITHOUT were roughly equivalent: No meaningful difference in accuracy, completeness, or cost.
    • (c) WITH was measurably worse than WITHOUT: Less accuracy, more confusion (e.g., tool noise), or noticeably more tokens for the same quality.

    If the judge is uncertain between (a)/(b) or (c)/(b), it must default to (b).

  6. Understand Temporal workflow contract detection

    main

    Gortex provides specialized detection for Temporal workflows and activities. It uses a via Meta value on EdgeCalls edges to categorize the relationship. When a call is resolvable (like a stub or a start call), Gortex rewrites it to point directly to the registered handler or workflow node, allowing standard tools like find_usages and explain_change_impact to work seamlessly.

    Temporal Edge Taxonomy

    viaDirectionEmitted fromtemporal_kindResolved?
    temporal.registerprovider tagworker.RegisterActivity(WithOptions) / RegisterWorkflow(WithOptions) / RegisterActivitiesactivity / workflowindexed, not rewritten
    temporal.stubworkflow → activity / child-workflowworkflow.ExecuteActivity / ExecuteLocalActivity / ExecuteChildWorkflowactivity / workflowyes → registered handler
    temporal.startservice → workflowclient.ExecuteWorkflow / SignalWithStartWorkflowworkflowyes → registered workflow
    temporal.handlerworkflow exposesworkflow.SetQueryHandler / GetSignalChannel / SetUpdateHandlerquery / signal / updateprovider edge
    temporal.signal-sendsender → running workflowworkflow.SignalExternalWorkflow / client.SignalWorkflowsignalconsumer edge
    temporal.query-callcaller → running workflowclient.QueryWorkflowqueryconsumer edge

    Metadata and Roles

    • Node Roles: Nodes are stamped with temporal_role (e.g., activity, workflow, activity_interface, workflow_interface, signal, query, update).
    • temporal_registered_name: The actual dispatch key used (from RegisterOptions{Name}).
    • temporal_register_plural: Indicates registration via RegisterActivities(&Struct{}) where methods are promoted.
    • temporal_name_origin=env_default: Indicates a dispatch name resolved from an environment variable with a literal default.

  7. Understand Cross-Repo API Contracts

    main

    Gortex automatically detects and matches API contracts across different repositories. It identifies providers (where an API is defined/implemented) and consumers (where it is called) to help manage dependencies and impact analysis.

    Supported contract types include:

    • HTTP routes: Detected via framework annotations (e.g., gin, Express, FastAPI, Spring). Matches route handlers to fetch or http.Get calls.
    • gRPC: Matches Proto service definitions to client stub calls.
    • GraphQL: Matches schema type/field definitions to query/mutation strings.
    • Message topics: Matches Kafka, RabbitMQ, NATS, or Redis pub/sub calls (Publish vs. Subscribe).
    • WebSocket: Matches emit() patterns to on() listeners.
    • Env vars: Matches os.Getenv, process.env, or .env files to Setenv or usage calls.
    • OpenAPI: Links Swagger/OpenAPI spec paths to HTTP routes.
    • Temporal workflows: Matches Go/Java SDK annotations to ExecuteActivity or ExecuteChildWorkflow calls.

  8. Manage File Exclusions and Ignore Layers

    main

    Gortex uses layered exclusion logic. Exclusions are applied in this order: builtin $\rightarrow$ repo's .gitignore $\rightarrow$ global config $\rightarrow$ per-repo entry $\rightarrow$ workspace config.

    Supported Ignore Files:

    • .gitignore: Respected by default.
    • .gortexignore: Gortex-specific ignore file (sibling to .gitignore).
    • .ignore / .rgignore: Ripgrep ignore files.

    All ignore files are scoped to the directory containing them. You can use !pattern in a later layer to re-include a file excluded by an earlier layer.

  9. Use Editor Overlays for unsaved buffer simulation

    main

    The OverlayManager allows MCP clients to simulate tool outputs (like find_usages) using unsaved file buffers. This is managed via the HTTP endpoint /v1/overlay/sessions/....

    To prevent incorrect data from being returned due to file changes, the system uses BaseSHA drift detection. If the underlying file has changed such that the overlay is stale, the merge is refused to prevent wrong-line-number errors.

  10. Use Projects to scope queries

    main

    A project is a sub-bucket within a workspace. This is useful for organizing many repositories into logical groups for specific tasks.

    When using the MCP interface, you can load only specific repositories by specifying a project. Inside an AI agent session, calling set_active_project will switch the default scope for all subsequent queries, removing the need to repeat the project parameter.

    # Only load repositories belonging to a specific project via MCP
gortex mcp --project my-saas

  11. Evaluation Methodology Best Practices (Anti-gaming)

    main

    To maintain the integrity of Gortex evaluations, adhere to these methodological constraints:

    • Anonymity: The judge must never see the identity of the agent (e.g., do not include "WITH agent was Claude Sonnet 4.6"). The only signal should be the WITH / WITHOUT split.
    • Cost Integrity: Token counts must be computed internally (using cl100k_base) before being passed to the judge to prevent agents from falsifying cost data.
    • Human-Authored Ground Truth: Canonical answers must be written by human experts before any agent runs. Writing them after seeing agent outputs is considered methodology fraud.
    • Budget Enforcement: Per-task token and wall-clock budgets must be identical for both WITH and WITHOUT runs. If a run exceeds its budget, it should be scored as "no answer" rather than being penalized under label (c).

  12. Manage response budgets with MCP and per-call caps

    main

    Gortex implements budget-by-default responses for list-shaped tools to prevent token overflow.

    • Default Behavior: Responses cap each page at the project default budget and return a next_cursor for subsequent pages.
    • Per-call Caps: You can compose limits using max_bytes and max_tokens. The tighter of the two limits will be applied. Truncation markers are included in the response to indicate where data was cut.