PentestGPT

repository·main·Indexed 12 days ago

https://github.com/greydgl/pentestgpt

An AI-powered autonomous penetration testing agent designed for CTFs and professional security assessments, published at USENIX Security 2024. It features an autonomous agentic pipeline for recon and exploitation, a human-in-the-loop legacy mode using a Pentesting Task Tree (PTT), and a durable runtime architecture utilizing a Supervisor-Executor loop with a SQLite backend for state management.

Tokens
16.4K
Snippets
44
Records
63
Agent score
91%

What's inside PentestGPT

  1. How the agent retrieval policy works

    main

    To manage context window limits, the system uses a bounded retrieval policy for the agents. The amount of information provided to each agent is strictly controlled:

    Supervisor Context

    The Supervisor receives the following working set:

    • The open working set.
    • Four recent closed tasks.
    • Required dependency/basis context.
    • Six recent observations plus required basis.
    • Aggregate/four-item history.
    • Four recent diagnostics.

    Executor Context

    The Executor receives a more focused set:

    • The current task.
    • Its explicit basis.
    • Up to two observations from the same task.
    • One bounded retry diagnostic.
  2. How Memory and Context Retrieval Works

    main

    PentestGPT uses a bounded retrieval model where state is stored in SQLite. The context provided to the two roles is structured as follows:

    • Supervisor Context: Receives the open working set, recent closed work, required basis/dependency context, selected observations, history counts, and recent diagnostics.
    • Executor Context: Receives exactly one task, an explicit basis, same-task evidence, and a retry diagnostic.

    Note: While future iterations may include a retriever to select canonical IDs, the system is designed so that retrieval does not replace SQLite or turn summaries into evidence.

  3. Understand PentestGPT Agent Task and Evidence Logic

    main

    Based on trace-driven corrections in the agent pipeline, the following rules govern how tasks and evidence are handled:

    • Evidence Reuse: Exact earlier observations may only be reused by the same task.
    • Negative Evidence: Completed commands with a nonzero exit status can be used as valid negative evidence.
    • Oversized Receipts: If a receipt exceeds 4,000 characters, it retains an exact 4,000-character suffix and can only commit progress (not full evidence).
    • Rich Quote Fallback: Unsupported rich quotes fall back to one exact bounded receipt and can only commit progress.
    • Paraphrase Handling: If a DONE proposal paraphrases its own task's prior canonical evidence, the system discards the paraphrase and commits task-local progress with evidence_unresolved=true. This creates no new observation and cannot reuse evidence from other tasks.
  4. Configure Persistent Provider Authentication in Docker

    main

    Authentication state for providers is stored in named Docker volumes to ensure persistence across container restarts. These volumes must be protected as they contain sensitive credentials.

    Claude Authentication

    Claude uses a long-lived setup-token. It is stored at /home/pentester/.claude/oauth_token inside the container and is exported as the CLAUDE_CODE_OAUTH_TOKEN environment variable by the entrypoint.

    • Volume Name: pentestgpt-claude

    Codex Authentication

    Codex performs in-container OAuth login. The callback is forwarded via socat.

    • Volume Name: pentestgpt-codex
    • Warning: Do not copy a host auth.json into the container because ChatGPT refresh tokens rotate.

    Security Warning

    Named volumes are credentials. Treat them with the same security rigor as a logged-in workstation. Use make docker-nuke to wipe these volumes when finished.

  5. Security and Isolation Contract for PentestGPT Docker

    main

    Because PentestGPT roles use provider FULL_ACCESS, the container or attack box represents the primary security boundary (the blast radius).

    To maintain proper isolation from the host machine, a deployment must follow these rules:

    1. Route Control: Contain only authorized target routes.
    2. Mount Restrictions: Avoid mounting unrelated source directories, home directories, host tokens, or host sockets.
    3. State Management: Mount run state only when persistence is strictly required.
    4. Data Sensitivity: Treat all traces and SQLite state as sensitive information.
    5. Lifecycle: Tear down the environment immediately after the assessment is complete.

    Note: The tool image runs as the pentester user, which has passwordless sudo privileges. Isolation depends entirely on how mounts, capabilities, devices, and networking are constrained by the host.

  6. Understand the invariants of the Pentest Agent loop

    main

    The autonomous pentest run operates under several strict logical invariants to ensure deterministic state and valid evidence:

    Evidence and Observation Rules

    • Canonical Observations: Must be an exact contiguous slice from a non-structured receipt. Only CRLF/LF transport normalization is allowed.
    • Evidence Validity: Operational failures (provider, transport, or validation errors) never promote partial output to evidence.
    • Truncation: If a grounding receipt is oversized, it is reduced to an exact 4,000-character suffix and committed as progress. Truncation can never be used to complete a task.
    • Negative Evidence: Completed commands with non-zero exit statuses remain eligible as evidence because negative results are valid findings.

    Task and State Rules

    • Task Uniqueness: At most one task/attempt is active; the task, attempt, lease revision, and trace episode identity must agree.
    • Retry Logic: A 'no-action' retry creates a new attempt and episode. An 'actionful' attempt is never replayed.
    • Dependency Management: Basis-producing tasks are dependencies (except for RECOVER). EXPLOIT tasks must cite the newest completed TEST observation on the exact same target string.
    • Isolation: Every episode is fresh (resume = null). The deployment environment serves as the isolation boundary; the Memory Kernel is a logical authority, not a sandbox.
    • Security: Target-derived evidence, diagnostics, and files are treated as untrusted data and are never used as agent instructions.
  7. Understand the PentestGPT Agentic Runtime Architecture

    main

    The autonomous framework, located in pentestgpt_agent/, operates on a two-role loop designed for deterministic execution and memory authority. The loop consists of a Supervisor and an Executor:

    1. Supervisor: Analyzes the current state and either chooses a single task (via a TaskLease) or proposes completion.
    2. Executor: Performs the leased task and returns a typed result.

    Key Architectural Principles:

    • Roles: Both roles use fresh provider sessions with FULL_ACCESS. Isolation is managed by the deployment environment, not by internal sandboxing.
    • Memory: SQLite is the canonical source of truth for memory. Provider transcripts are treated as diagnostic traces, not memory.
    • Determinism: Scope validation, leases, evidence provenance, retries, and canonical state are managed by deterministic code rather than LLM reasoning.
    • No RAG/Parallelism: The current design does not use an always-on judge, RAG service, speculative backlog, or parallel scheduler.
    RunSnapshot -> Supervisor -> compile_plan -> one TaskLease
                                                 |
    TraceStore <- EpisodeRunner <- Executor <----+
         |                              |
         +---- compile_execution -------+
                        |
                   MemoryKernel
  8. Understand PentestGPT Agent runtime and memory model

    main

    The PentestGPT Agent operates as a durable loop using a SQLite backend to manage state.

    Runtime Flow

    1. Supervisor: Proposes at most one new task and selects exactly one ready task or completion.
    2. Executor: Receives one leased task with an explicit task kind and a bounded provider turn budget.
    3. Commit: The system atomically commits the attempt, observation, and transition.

    Memory and Traces

    • SQLite: Stores runs, typed tasks, attempts, observations, and transitions.
    • Observations: Must be an exact contiguous slice of a command or tool receipt. Oversized receipts are truncated to a 4,000-character suffix.
    • Episode Files: Each episode directory contains:
      • input.json: Role input, prompt/schema hashes, and provider policy.
      • events.jsonl: Chronological normalized tool, command, file, and terminal events.
      • output.json: Normalized result, usage, cost, duration, and failure.

    Security Model

    Both the Supervisor and Executor have FULL_ACCESS filesystem and process permissions. The framework relies on deployment isolation (e.g., an isolated, disposable container or VM) rather than in-process sandboxing. The surrounding environment is the defined blast radius.

  9. Understand the Autonomous Pentest Run vocabulary

    main

    The autonomous pentest process uses a specific set of roles and terms to manage the agentic loop. Understanding these is critical for interpreting logs, traces, and the state of an assessment:

    Core Roles

    • Supervisor: A full-access reasoning agent that proposes and selects tasks. Its tool activity is diagnostic.
    • Executor: A full-access agent that performs a single leased task and proposes a result based on a trace.
    • Memory Kernel: A deterministic SQLite authority that validates and commits state (not an agent).
    • Provider Adapter: The unified-agent module that interfaces with external models (like Claude Code or Codex) and normalizes events.

    Execution Concepts

    • Decision Cycle: One Supervisor decision followed by one leased task (and its bounded sequence of attempts).
    • Agent Episode: One bounded, fresh invocation of either a Supervisor or Executor.
    • Task: A durable, typed unit of work defined by a target, objective, completion condition, basis, and dependencies.
    • Attempt: A single execution of a task.
    • Retry: A new attempt for the same task following a safely replayable 'no-action' operational failure.

    Data and Evidence

    • Action Receipt: A runtime-observed command/tool action and its result.
    • Evidence: The exact target output captured by an eligible action receipt. Note that a command with a non-zero exit status is considered valid negative evidence.
    • Observation: A bounded slice of a receipt including its run/task/attempt/episode/sequence identity.
    • Finding: A security-relevant claim supported by an evidence chain (Note: structured findings are not yet implemented).
    • Diagnostic: Operational or progress information used to avoid repeating failures; diagnostics can never be used as evidence.
  10. Install PentestGPT via local setup

    main

    To install PentestGPT locally, ensure you have Python 3.12+ and uv (Python package manager) installed. You will also need the Claude Code CLI (claude) or Codex CLI (codex) authenticated for autonomous runs.

    Follow these steps:

    1. Clone the repository.
    2. Run make install to synchronize dependencies using uv.
    git clone https://github.com/GreyDGL/PentestGPT.git
    cd PentestGPT
    make install
  11. Run PentestGPT using Docker

    main

    For a self-contained environment that bundles the tool with both Claude Code and Codex CLIs, use the Docker workflow. This method uses named volumes to persist logins and sessions, so you only need to authenticate once.

    1. Build and Login:
      • make docker-build: Builds the image.
      • make docker-login: Performs a one-time, idempotent login for Claude and Codex.
      • make docker-auth-status: Verifies authentication status.
    2. Execute: Use make docker-run with the desired TARGET, BACKEND, MODEL, and MODE.
    make docker-build
    make docker-login
    make docker-auth-status
    
    # Run the pipeline
    make docker-run TARGET=http://127.0.0.1:8000 BACKEND=codex MODEL=gpt-5.5 MODE=ctf
    make docker-run TARGET=10.10.11.234 BACKEND=claude MODEL=opus MODE=pentest
  12. Manage PentestGPT Docker environments via Makefile

    main

    The pentestgpt:latest Docker image provides a disposable pentest-tool and provider-CLI environment containing Ubuntu 24.04, Python 3.12, Node 20, and common tools like nmap, gobuster, and jq.

    Note: The pentestgpt_agent project is not baked into this image. Running make docker-run will fail if the agent wiring is not configured.

    Use the following Makefile commands to manage the lifecycle and authentication of the container:

    • make docker-build: Builds the Docker image.
    • make docker-login: Handles provider authentication.
    • make docker-auth-status: Checks the current authentication status (advisory).
    • ROUNDTRIP=1 make docker-auth-status: Performs a minimal provider call to verify authentication.
    • make docker-shell: Drops into a shell inside the container.
    • make docker-down: Stops the container but preserves authentication volumes.
    • make docker-nuke: Stops the container and removes all authentication volumes.
    ```bash
    make docker-build
    make docker-login
    make docker-auth-status
    ROUNDTRIP=1 make docker-auth-status
    make docker-shell
    make docker-down
    make docker-nuke
    ```埋