DOCSEARCHbyAlgolia
Submit Docs

TrueCourse Documentation

repository·main·Indexed 19 days ago

https://github.com/truecourse-ai/truecourse

An AI-powered architecture and code intelligence platform designed to detect code defects through deterministic rules and LLM analysis, and identify business-logic drift by verifying code against documentation specifications. The platform includes a CLI for repository analysis and a False Positive (FP) Detection Automation system that uses a continuous loop of discovery, fixing, and regression testing to improve rule accuracy.

Tokens
181K
Snippets
422
Records
750
Agent score
67%

What's inside TrueCourse

  1. Overview of TrueCourse Enterprise Edition (EE)

    main

    TrueCourse Enterprise Edition (ee/) provides commercial-licensed features that layer onto the open-source core. It is designed to be modular: the OSS core remains free of commercial code and can run without ee/ present. Enterprise features are integrated via sanctioned runtime seams (server plugin loader and client route/slot registry) rather than static imports.

    Key Enterprise Packages:

    • @truecourse/ee-server: Enterprise server code (e.g., WorkOS SSO/auth) that registers into the dashboard server's plugin seam.
    • @truecourse/ee-client: Enterprise UI components (Workspace, GitHub integration, and Models pages) contributed into the dashboard client's registries.
    • @truecourse/ee-github-app: The hosted GitHub App PR gate, composed by ee-server.
    • @truecourse/ee-llm: API-backed LLM transport built on the Vercel AI SDK.

  2. What TrueCourse catches

    main

    TrueCourse validates documented behavior through CLI or HTTP API drivers. It detects:

    • Wrong responses and exit codes.
    • Missing or mistyped output fields.
    • Illegal state transitions.
    • Bypassed validation and authentication rules.
    • Silently-dropped side effects.
    • Incorrect formula results.
    • Stale scenarios caused by specification changes.

  3. Understand the Spec Guard architecture and rollout phases

    main

    Spec Guard is a system designed to replace traditional contract verification with section-bound scenario tests. It ensures that documented behaviors (sections in your documentation) are actively tested against the actual implementation.

    Core Workflow

    1. Scenario Generation: Scenarios are generated based on documentation sections (the 'spec').
    2. Execution: A guard-runner executes these scenarios in a sandbox environment.
    3. Verification: If a documented behavior changes or breaks, the scenario turns 'red' and provides an evidence transcript.

    Development Roadmap

    The project is organized into distinct phases:

    • OSS v1 (Phases 0-5): Focuses on the core engine, including the guard-runner, guard-generator, and a local guard store for managing runs and evidence.
    • Drivers (Phases 6-7): Extends testing capabilities to API-driven scenarios (using request/capture/expect verbs), TUI, Playwright, and programmatic APIs.
    • Enterprise Edition (Phase 8): Moves Guard to a hosted service that acts as a GitHub PR gate, running scenarios server-side on every Pull Request.

  4. Manage credentials and server allowlists in Spec Guard

    main

    Spec Guard implements a credential allowlist to ensure scenarios only access credentials valid for their bound server. This prevents 401 errors from being misattributed to the application when a scenario attempts to use a credential intended for a different service.

    Credential Scoping

    • Bound Servers: A scenario only sees the credentials its bound server accepts.
    • Foreign Credentials: If a scenario attempts to use a credential via {{cred:...}} that is not in its server's allowlist, it is treated as a foreignCredential. This results in an error explicitly naming the servers the credential does authenticate against, rather than a generic 401 error from the application.

    Implementation Details

    • The fromRequest method accepts a server parameter to specify which server the login/request should be minted against.
    • All server names used in credentials must be validated against the declared set in api.servers or api.serve.

  5. Understand how Birth Findings are preserved during regeneration

    main

    In TrueCourse, 'Birth Findings' (failures identified during the initial discovery/generation phase) are designed to persist even if a generate command is run without re-running those specific tests.

    Persistence Logic: A finding from a previous guard/result.json is carried forward into a new report only if:

    1. The scenario is still listed as failing in the manifest.
    2. The current generation produced no fresh finding for that scenario.
    3. The scenario was not re-authored (re-written) during the current run.

    Important Notes:

    • Fidelity Rejections: These are treated as per-generate advisories and are never carried forward.
    • Missing Evidence: If guard/evidence/ is gitignored, evidence paths in the report may 404 after a clone. The system treats the existence of a path as proof that a run occurred, regardless of whether the files are currently present on disk.
    • Recovery: If findings were already wiped before this persistence logic was implemented, they cannot be resurrected. Running guard run will rebuild the details for every committed test.

  6. Runner Semantics and Outcomes

    main

    The runner executes self-contained YAML scenarios. Key behaviors include:

    • Staleness: If any bound section is stale, the scenario is marked stale. If a flow is orphaned, it is marked orphaned.
    • Journey Drift: If the code surface has changed (mismatched journey fingerprints), the runner annotates the result with journeyDrifted: true. The test still executes using its frozen steps.
    • Known-Red Tests: Tests that fail during their initial 'birth' execution are committed as status: 'failing'. They are treated like any other test in run totals.
    • Rollups: Results are rolled up per flow. LATEST.json per-scenario results include a flowId to support flow-first rollups.

  7. Understand the transition from Contract Verify to Spec Guard

    main

    The legacy verify, drifts, contracts, and infer command families and the BL Drift dashboard have been fully removed from the CLI, OSS dashboard, and Enterprise Edition (EE).

    These have been replaced by Spec Guard, which uses section-bound scenario tests. While the high-level user commands are gone, the underlying Matching Engine remains available for programmatic use via the contract-extractor package and the contract-verifier engine (which handles code-fact extraction, .tc parsing, and inference).

  8. Handle OpenAPI as a dual-purpose source

    main

    In TrueCourse, a committed OpenAPI document serves two roles to ensure consistency between specification and implementation:

    1. As Corpus Sections (Spec Side): It defines the expected surface area.
    2. As Journey Entries (Code-Truth Proxy): It acts as a proxy for discovering entry points.

    Drift Detection:

    • If an OpenAPI operation has no matching route registration in the code, the journey is marked specOnly: true.
    • If a scenario is realized through an OpenAPI operation but fails during 'birth' (initial validation), it signals a documented-but-unimplemented drift.

  9. Interpret Spec Guard coverage statuses

    main

    When viewing the Coverage surface, each section of your documentation is assigned a status band indicating its current state:

    StatusMeaning
    GreenGuarded and passing.
    RedFailing (drift detected in this section).
    AmberStale (the section has been edited since the last generation).
    GreyUntestable (includes a reason for why it cannot be guarded).
    UnmarkedNo coverage yet (has not been scanned/generated).

    In the dashboard, you can filter these using the totals-strip chips with either Blur (dims non-matching sections to preserve context) or Hide (collapses non-matching sections).

  10. Identify stale PRs for monitoring

    main

    A PR is considered stale and added to the cur_set if it meets all the following criteria:

    1. It is currently open.
    2. Its head branch starts with claude/<SCOPE>fp-fix/.
    3. It does not carry the <SCOPE>fp-reviewed label.
    4. It does not carry the <SCOPE>fp-reviewing label (this prevents false alarms on PRs currently being actively reviewed).
    5. It has been open for more than STALE_HOURS (where STALE_HOURS = 1).

  11. Manage relevance filtering for documentation

    main

    The relevance filter determines which documents in a repository are relevant to the spec corpus. To avoid including irrelevant documentation (like third-party API references or test fixtures), you can use the following methods:

    Manual Exclusion

    • Force-exclude: Use the skip field on a doc row to manually exclude it.
    • .truecourseignore: Add patterns like tests/fixtures/** to your .truecourseignore file to prevent them from being processed.

    Automatic Filtering Logic

    • Repo Identity: The system uses a repo-identity.ts utility to resolve the repository's name and aliases (from package.json, README, etc.). This identity is injected into the prompt to ensure the model distinguishes between the repository's own product and third-party systems.
    • Path-aware relevance: The system passes the repo-relative PATH to the model. Documents located under tests/fixtures/, tests/sample/, or tests/example/ trees that describe external products are typically treated as irrelevant.

  12. Manage seeded state and external datastores

    main

    Guard boots one fresh server per scenario. Because of this, any state kept in-memory is lost between scenarios. To ensure seeded data survives, your application must use an external datastore (e.g., Postgres, Redis) managed by api.services.up.

    If your application is currently in-memory, you have two options:

    1. Provide a real store for Guard runs using the api.env configuration.
    2. Have each scenario create its own required data through the API itself.