SpecStory Documentation

repository·dev·Indexed 23 days ago

https://github.com/specstoryai/getspecstory

SpecStory captures, indexes, and makes searchable AI coding interactions from agents like Claude Code, Cursor, and Gemini CLI. It includes a CLI for session saving and cloud synchronization, and 'Lore' (@specstory/lore), a tool that mines coding histories to forge reusable agent skills based on recurring successful patterns (Beats) and theme mining.

Tokens
91.7K
Snippets
128
Records
400
Agent score
79%

What's inside SpecStory

  1. What is SpecStory Lore and how to use it

    dev

    SpecStory Lore is a skill used to mine your coding history (transcripts from agents like Claude Code, Cursor, Gemini, etc.) into a persistent corpus of 'beats'. It identifies reproducible workflows and surfaces 'skill candidates' that can be forged into reusable SKILL.md packages.

    Key Concepts:

    • Mining: The process of parsing SpecStory transcripts to find patterns.
    • Skill Candidates: Corroborated workflows identified by the engine that are candidates for being turned into formal skills.
    • Forging: The final act of creating a SKILL.md package from a selected candidate.
    • Harness-Portable: This skill is designed to work across different AI agent environments (agentskills.io format).
  2. Overview of Cloud Resume and Search functionality

    dev

    The specstory resume and specstory search CLI commands are being extended from local-only functionality to include sessions stored in SpecStory Cloud. This allows users to surface sessions that have been pushed to the cloud via sync from other machines.

    Important Deployment Note: The server must be deployed before shipping the CLI. If the CLI makes uncap cloud reads against an old server, the request will degrade to the default limit of 200.

  3. Overview of SpecStory CLI

    dev

    SpecStory CLI is a cross-platform command-line tool designed to save AI coding conversations from various coding agents (terminal agents and IDEs) as local markdown files. It can optionally sync these files to SpecStory Cloud to create a searchable, centralized knowledge system.

    Key features include:

    • Integration with terminal agents like Claude Code, Cursor CLI, Gemini CLI, etc.
    • Automatic redaction of secrets (API keys, tokens, credentials).
    • Syncing capabilities to SpecStory Cloud.
    • Support for Linux and macOS.
  4. Understand the Antigravity CLI directory layout

    dev

    The Antigravity CLI (agy) stores all session and configuration data in ~/.gemini/antigravity-cli/. Understanding this layout is critical for locating logs, configuration, and conversation data.

    Key directories and files:

    • ~/.gemini/oauth_creds.json: Shared OAuth credentials used for authentication.
    • ~/.gemini/config/projects/<projectId>.json: Maps a projectId (UUID) to a workspace path.
    • ~/.gemini/antigravity-cli/settings.json: Contains {enableTelemetry, model, trustedWorkspaces[]}.
    • ~/.gemini/antigravity-cli/conversations/<conversationId>.db: The canonical SQLite database used to resume sessions.
    • ~/.gemini/antigravity-cli/brain/<conversationId>/: The primary directory for conversation data, containing logs and system messages.
    • ~/.gemini/antigravity-cli/brain/<conversationId>/.system_generated/logs/transcript_full.jsonl: The primary source for parsing clean, native-JSON tool arguments.
    ~/.gemini/                                          # shared with Gemini CLI
    ├── oauth_creds.json                                # OAuth creds
    ├── config/
    │   └── projects/<projectId>.json                   # projectId(UUID) -> workspace path map
    └── antigravity-cli/                                # all Antigravity data lives here
        ├── history.jsonl                               # interactive-TUI prompt log
        ├── installation_id                             # UUID string
        ├── settings.json                               # {enableTelemetry, model, trustedWorkspaces[]}
        ├── keybindings.json
        ├── last_check.timestamp                        
        ├── cache/onboarding.json
        ├── bin/{agentapi,webm_encoder}                 
        ├── updater/update.lock
        ├── knowledge/knowledge.lock
        ├── cli.log -> log/cli-YYYYMMDD_HHMMSS.log       
        ├── log/cli-YYYYMMDD_HHMMSS.log                 
        ├── conversations/<conversationId>.db           # SQLite conversation store
        ├── conversation_summaries.db                   # SQLite index of conversations
        ├── scratch/                                    # default workspace
        └── brain/<conversationId>/                      # one dir per conversation
            └── .system_generated/
                ├── logs/
                │   ├── transcript_full.jsonl           # PRIMARY SOURCE
                │   └── transcript.jsonl                
                ├── messages/                            
                │   ├── <messageId>.json                
                │   ├── read.json                       
                │   └── cursor.json                     
                └── tasks/                               
                    └── task-<N>.log                    
  5. What is AIttributor and how does it work?

    dev

    AIttributor is a lightweight git hook designed to detect active AI agents and append attribution metadata to git commit messages. It acts as a "license plate camera at commit time," providing a low-friction way to mark commits that were assisted by AI.

    Core Mechanism:

    1. It operates as a prepare-commit-msg git hook.
    2. It detects active agents using four signals: agent-specific environment variables, process ancestry, sibling process tree matching the repository's current working directory, and agent breadcrumb files in known paths.
    3. If an agent is detected, it appends specific git trailers to the commit message.

    Key Limitations:

    • It only provides commit-level attribution. It does not capture prompts, tool calls, line-level provenance, or micro-versioning.
    • Correlation is coarse; it correlates by repository working directory and active agent presence at the time of commit. It may be inaccurate if multiple agents are active simultaneously.
  6. How Cloud Session Resuming works

    dev

    Resuming a session from the cloud follows a specific reconstruction path:

    1. Fetch: The CLI fetches the SessionData blob from the cloud via the API.
    2. Reconstruct: The CLI uses ReconstructSession(target) to turn the blob into a native session.
    3. Localize: The reconstructed session is written as a native file on the local machine.
    4. Launch: The agent is launched using the newly created local file.

    Version Safety: SessionData includes a schemaVersion. If the cloud blob's version is newer than the local CLI's supported version, the CLI will refuse to resume and prompt for an update.

  7. Understand the Lore Idempotency Contract

    dev

    Lore is designed to be idempotent, meaning re-running the engine on an unchanged repository will result in no changes. The engine uses a fingerprint based on size + mtime + PARSER_VERSION to skip redundant processing.

    Key behaviors include:

    • Accumulation: New sessions are appended to the corpus.
    • Updates: If a session file grows or changes, it is updated rather than duplicated.
    • Upgrades: A PARSER_VERSION bump triggers an automatic one-time full re-parse.
    • Force Re-indexing: Use the --force flag to re-index everything regardless of fingerprints.
    • Pruning: Running prune removes orphaned rows if a transcript is deleted or moved, and flags duplicate-identity groups if a project gains a new git_id (remote).
    • Forging: The forged registry is authoritative. Running /lore after forging allows forged check to compare skill evidence against the current corpus and recommend actions like up-to-date, update, update-carefully (for hand-edited files), or orphaned.
  8. Understand the `sessions.db` indexing logic

    dev

    The sessions.db file acts as a derived cache for all SpecStory agent sessions. It is designed to be a high-performance, searchable index of user and agent interactions.

    Core Principles

    • Derived Cache: sessions.db is not the source of truth; the native agent files are. The index is built by observing these files.
    • Project Identity: Identity is determined by the filesystem structure (walking up to the .git root) rather than stored configuration files. This ensures that monorepo subdirectories are correctly grouped under the same git_id in the cloud.
    • Soft Deletes: The index supports soft deletion via a d key.
    • FTS (Full Text Search): The index includes a reconstructed, flattened version of SessionData (user/agent turns) to allow for efficient searching across all providers.
  9. Understand Cloud Sync Debouncing Behavior

    dev

    When running specstory run, the CLI implements a debouncing mechanism to optimize cloud synchronization and prevent excessive API requests. This mechanism is designed to handle rapid sequences of events by coalescing updates and limiting the sync rate.

    Debounce Rules

    • Immediate Sync: A sync occurs immediately if no sync is currently in progress and more than 10 seconds have passed since the last sync for that specific session.
    • Queue and Coalesce: If an event occurs within the 10-second debounce window, it is queued. If multiple events occur while an event is queued, only the latest event is kept (coalescing).
    • Auto-flush: If an event is queued, the system will automatically trigger a sync once the 10-second debounce timer expires.
    • Forced Flush: All pending syncs are flushed immediately when the specstory run process exits to prevent data loss.
    • Per-session Isolation: Debouncing is applied per session; different sessions sync independently of one another.
  10. How the SpecStory CLI version check works

    dev

    The SpecStory CLI automatically checks for updates during startup by performing an HTTP HEAD request to https://github.com/specstoryai/getspecstory/releases/latest. It follows the 302 redirect to extract the latest version tag from the final URL using the pattern /releases/tag/v?(.+)$.

    Key behaviors:

    • Notification: If a newer version is detected, a non-intrusive message is displayed:
      A new version of SpecStory CLI is available!
      Visit https://get.specstory.com/claude-code for update instructions.
    • Exclusions: Version checks are automatically skipped if the current CLI version is a "dev" version.
    • Silent Mode: The version check respects the existing --silent flag.
  11. Filter and sort sessions by machine

    dev

    SpecStory provides machine-based filtering to manage sessions across different devices. This is handled via the machine filter (m) in the picker or search views.

    Machine Filter (m) Behavior:

    • Cycling: The filter cycles through options: all $\rightarrow$ "local only" $\rightarrow$ <each remote machine> $\rightarrow$ (wraps back to all).
    • "local only": This option shows sessions belonging to the current machine, including those that were synced and then pruned from the local disk.
    • Remote Machines: Remote machines are identified by their hostnames captured at sync time. If duplicate names exist, they are disambiguated as hostname (1), hostname (2), etc.
    • Sorting: All views remain sorted by recency using the updated_at field.
  12. How workthread clustering and lifecycle status work

    dev

    The workthreads engine processes .specstory/history transcripts using specific clustering rules and lifecycle logic to prevent fragmentation (too many small threads) and mega-threads (one giant thread for the whole codebase).

    Clustering Rules

    1. Session as Unit: A single session's beats are unioned into one thread. A multi-prompt session is treated as one thread.
    2. Rare Key Requirement: To merge two sessions, they must share at least two RARE keys. A key is a file or a distinctive symbol (snake_case, camelCase, or ALL_CAPS_WITH_UNDERSCORE). Ubiquitous keys (found in >4 sessions), config files (.env, package.json), and short abbreviations (AI, API, EOF) are ignored.
    3. Bounded Union: A thread is capped at a maximum of 5 sessions. Any merge that would exceed this cap is refused.

    Lifecycle Statuses

    Threads are classified based on activity relative to the current date:

    • closed: Latest outcome was a success, the thread has been quiet for $\ge 3$ days, and the last activity was $\le 30$ days ago. (Note: Threads are flagged as reverted if a beat ran a rollback command like git revert or git reset --hard).
    • new: First activity occurred within the last 7 days.
    • open: Activity occurred within the last 14 days.