DOCSEARCHbyAlgolia
Submit Docs

deepsec

repository·main·Indexed 26 days ago

https://github.com/vercel-labs/deepsec

An agent-powered vulnerability scanner for large-scale codebases. deepsec uses high-reasoning AI models to perform on-demand security reviews, utilizing a pipeline that moves from fast regex-based matching to deep AI investigation. It supports distributed scans via Vercel Sandbox microVMs and integrates with AI providers like Anthropic, OpenAI, and the Vercel AI Gateway.

Tokens
46.5K
Snippets
63
Records
249
Agent score
87%

What's inside deepsec

  1. Understand the deepsec pipeline stages

    main

    deepsec operates via an append-only, idempotent pipeline where each stage is a separate CLI subcommand. Re-running a stage merges new information into existing records rather than overwriting them.

    1. scan: Globs the project and runs regex matchers to identify candidates. Writes these to FileRecords with status: "pending".
    2. process: Sends batches of pending files to an AI agent backend to generate findings. Updates status to "analyzed" and appends to analysisHistory.
    3. revalidate: Re-checks existing findings using an AI agent to assign a verdict (true-positive, false-positive, fixed, or uncertain).
    4. enrich: Attaches git committer info and ownership data to FileRecords containing findings.
    5. export / report / metrics: Read-only stages that shape data into JSON, markdown, or cross-project metrics without modifying the source FileRecords.

  2. Understand the deepsec workspace structure

    main

    A mature deepsec workspace evolves from the minimal .deepsec/ scaffold into a more robust configuration. A complete workspace typically includes:

    • package.json: Declares deepsec as a dependency.
    • deepsec.config.ts: Loads INFO.md inline and registers custom matchers via plugins.
    • matchers/*.ts: Custom matcher files tuned for specific codebase patterns.
    • INFO.md: Provides AI prompt context, such as authentication shapes, threat models, and sources of false positives.
    • config.json: Optional per-project configuration containing keys like priorityPaths, promptAppend, and ignorePaths.

  3. Understand the deepsec data layout

    main

    Deepsec uses a data/ directory to store the on-disk state for projects, files, runs, and findings. Each project has its own subdirectory. Files within the files/ directory are append-only; re-scanning merges new candidates, re-processing appends to analysisHistory, and revalidation annotates existing findings.

    Directory Structure:

    data/<projectId>/
├── project.json              # rootPath, githubUrl (auto-managed)
├── INFO.md                   # repo context injected into AI prompts
├── config.json               # priorityPaths, promptAppend, ignorePaths (optional)
├── files/                    # one JSON per scanned source file (FileRecord)
│   └── path/to/source.ts.json
├── runs/                     # one JSON per run (RunMeta)
│   └── 20260429215021-19ac.json
└── reports/                  # generated markdown + JSON reports

    Note: data/ is gitignored by default. To version it for CI or sharing, commit it explicitly.

  4. Understand deepsec technology detection and scanning

    main

    deepsec automatically detects frameworks and ecosystems using sentinel files or lockfile shapes. Detection results are persisted to data/<projectId>/tech.json.

    When a technology is detected, deepsec performs two primary actions:

    1. Activates Gated Matchers: Specific security matchers (e.g., js-express-route) are triggered only when their corresponding technology tag is present.
    2. Injects Prompt Highlights: Per-tech "threat highlights" are injected into the AI prompt to provide context-specific security guidance (e.g., highlighting common vulnerabilities in Next.js Server Actions).

    Matchers without a technology gate run on every repository regardless of detected tech.

  5. Identify tech-stack specific threats

    main

    Next.js

    • middleware.ts at the edge is NOT sufficient auth; it can be bypassed via routes escaping the matcher.
    • Server Actions are public POST endpoints and require explicit auth + authorization checks.
    • JSON.stringify() inside dangerouslySetInnerHTML or inline <script> tags is an XSS risk unless output escapes </ (look for safeJsonStringify or \u003c).
    • searchParams and dynamic route segments ([id], [...slug]) are untrusted user input.
    • unstable_cache / revalidateTag on user-supplied keys can leak data across tenants.

    React

    • dangerouslySetInnerHTML with user-influenced strings (including DB values/usernames) is an XSS risk.
    • Refs/effects touching document.location or window.opener can lead to open-redirect or tabnabbing.
    • Server-rendered JSON in <script> tags must escape </ to be safe.

    Django

    • @csrf_exempt views handling state-changing POSTs without alternative auth (signature/token) are vulnerable to CSRF.
    • Model.objects.raw(...) or cursor.execute() using f-string interpolation is SQL injection (flag any %-formatted SQL).
    • mark_safe(), format_html(), or {% autoescape off %} on user input is an XSS risk.
    • ModelForm without explicit fields = [...] (or using __all__) allows mass-assignment.
    • DEBUG=True + ALLOWED_HOSTS=['*'] leaks tracebacks and SECRET_KEY material.

    Gin (Go)

    • Auth middleware applied via r.Use(...) must precede route registration in the same group to be effective.
    • c.Query, c.Param, and c.PostForm are user input surfaces for SQL, exec, fs, and URL injection.
    • c.HTML with untrusted strings is XSS unless using auto-escaped templates ({{.X}}) and avoiding {{.X | safehtml}}.

  7. Understand plugin execution order

    main

    Plugins are evaluated in the order they appear in the plugins array:

    • Additive behavior: For matchers, notifiers, and agents, contributions from all plugins are registered.
    • Last-write-wins behavior: For ownership, people, and executor, a later plugin's provider will replace an earlier one.

  8. Configure AI providers and credentials

    main

    Deepsec uses AI models for investigation. You can configure how it accesses these models using environment variables.

    Vercel AI Gateway (Recommended)

    For production-scale scans, use the Vercel AI Gateway. This provides the necessary quota for highly concurrent research. Set the following environment variable:

    • AI_GATEWAY_API_KEY

    Direct Provider Access

    To bypass the gateway and use providers directly, set the following pairs:

    • Anthropic: ANTHROPIC_AUTH_TOKEN and ANTHROPIC_BASE_URL
    • OpenAI: Set the corresponding OpenAI credentials.

    Note: Explicit provider values take precedence over AI_GATEWAY_API_KEY.

    AI_GATEWAY_API_KEY=vck_...

  9. Configure Pi backend with custom AI providers

    main

    The pi backend can be pointed at OpenAI/Anthropic-compatible gateways (like Martian) using generic provider override flags. This allows you to use the Pi harness with different model providers.

    Required/Supported flags:

    • --agent pi
    • --model <model-id>
    • --ai-provider <provider-name>
    • --ai-base-url <url>
    • --ai-api-key-env <env-var-name>
    • --ai-header name=value (can be repeated for provider-specific headers)
    MARTIAN_API_KEY=... 
pnpm deepsec process --project-id my-app \
  --agent pi \
  --model openai/gpt-5.5 \
  --ai-provider openai \
  --ai-base-url https://api.withmartian.com/v1 \
  --ai-api-key-env MARTIAN_API_KEY

  10. Identify Django-specific security threats

    main

    When reviewing Django applications, look for these specific threat patterns:

    • CSRF: @csrf_exempt views handling state-changing POSTs without an alternate authentication mechanism (like a signature or token).
    • SQL Injection: Use of Model.objects.raw(...) or cursor.execute() with f-string interpolation or %-formatted SQL.
    • XSS: Use of mark_safe() or format_html() on user-controlled input, or {% autoescape off %} blocks in templates.
    • Mass Assignment: ModelForm without a defined fields = [...] attribute (or using __all__) which exposes all model columns.
    • Information Disclosure: DEBUG=True combined with ALLOWED_HOSTS=['*'] in reachable settings files, which can leak tracebacks and SECRET_KEY material.

  11. Run a deepsec scan on a project

    main

    To run a scan using a sample configuration, follow these steps:

    1. Copy the sample directory next to your real project.
    2. Point the root configuration to your codebase.
    3. Execute the scan command from inside the directory containing the configuration.
    pnpm deepsec scan

  12. Optimize matcher file patterns

    main

    To prevent the scanner from becoming bogged_down (especially with noisy matchers), always set filePatterns as tightly as possible. Avoid broad globs like **/*.{ts,tsx} on large repositories.

    Recommended patterns:

    • Language-specific: **/*.go, **/*.lua, **/*.tf
    • Directory-anchored: **/api/**/*.ts, **/services/**/handlers/*.ts
    • Combined: **/services/**/*.{ts,go}