Titus Secrets Scanner

Because Titus uses Hyperscan internally, each titus.Scanner instance requires exclusive access to its scratch memory. Do not share a single scanner instance across multiple goroutines simultaneously.

Choose one of the following patterns for concurrent scanning:

Pattern 1: Multiple Scanner Instances

Create a new scanner per goroutine. This is the simplest approach but has higher setup overhead.

Pattern 2: Worker Pool with Scanner Pool

For high-throughput, maintain a channel of pre-initialized scanners. A worker pulls a scanner from the channel, performs the scan, and returns it when finished.

Pattern 3: Sequential Scanning

Use a single scanner instance to scan files one after another in a loop. This is safe and efficient for single-threaded workflows.

// Pattern 2: Worker Pool with Scanner Pool
type ScannerPool struct {
    scanners chan *titus.Scanner
}

func NewScannerPool(size int) (*ScannerPool, error) {
    pool := &ScannerPool{
        scanners: make(chan *titus.Scanner, size),
    }
    for i := 0; i < size; i++ {
        s, err := titus.NewScanner()
        if err != nil {
            return nil, err
        }
        pool.scanners <- s
    }
    return pool, nil
}

func (p *ScannerPool) Scan(content string) ([]*titus.Match, error) {
    scanner := <-p.scanners
    defer func() { p.scanners <- scanner }()
    return scanner.ScanString(content)
}

Every finding is assigned a numeric score (0–100) and a severity tier based on the rule's base_score and various modifiers.

Scoring Modes

• Static Scoring (Default): Uses only static rule metadata.
• Enhanced Scoring: Use --score-scope to enable live API calls (e.g., AWS STS/IAM, GitHub API) to verify the actual blast radius of a credential. This adjusts the score based on real-world permissions.

Accessibility Context

Scores are adjusted based on whether the code is public or private:

• --accessibility public: No penalty for public repositories.
• --accessibility private: Applies a -25 penalty (default for local scans).

# Score findings using static rule metadata only (no network calls)
titus scan path/to/code

# Score findings AND make live API calls to verify credential blast radius
titus scan path/to/code --score-scope

# Override the accessibility context
titus scan path/to/code --accessibility public
titus scan path/to/code --accessibility private

Titus uses a scoring pipeline to assign a numeric value (0–100) to every finding, which is then mapped to a severity tier. This helps prioritize remediation.

The Scoring Pipeline:

1. Base Score: Every detection rule starts with a base_score (0–100).
2. Modifiers: Registered scorers check the rule ID. If they match, they contribute modifiers.
3. Priority: Modifiers are applied in descending priority order (highest number first). If priorities are equal, they apply in the order they are defined in the YAML.
4. Application: Modifiers either add/subtract a value (delta) or replace the score entirely (set_score).
5. Clamping: The final score is clamped to the range [0, 100].
6. Accessibility: An accessibility modifier may apply a penalty based on whether the repository is public or private.

Severity Tiers:

• info: 0 – 20
• low: 21 – 40
• medium: 41 – 60
• high: 61 – 80
• critical: 81 – 100

Titus persists full CommitMetadata (timestamps, author/committer info, and commit messages) from git enumeration into its SQLite datastore. This metadata is used to provide temporal context for discovered secrets.

Key Behaviors:

• Storage Format: Timestamps are stored as RFC 3339 strings (e.g., 2025-06-15T14:32:07Z).
• Display in Explore TUI: In the details pane, a Date: line is displayed beneath the Author: line, showing the committer timestamp formatted as YYYY-MM-DD HH:MM:SS. This line only appears if the CommitterTimestamp is non-zero.
• Display in Reports: The report command includes a Date: line in human-readable output for each match's provenance, provided the provenance is GitProvenance and contains a populated timestamp.
• Backward Compatibility: New code can open old datastores (it attempts to add columns via ALTER TABLE and ignores errors if they exist). Old code can open new datastores because it only selects the original 4 columns, ignoring the additional metadata columns.

    File: path/to/file.yml
    Date: 2025-06-15 14:32:07
    Blob: abc123...

To perform basic secrets detection, initialize a scanner using titus.NewScanner() and use ScanString to find matches in text content. Remember to defer scanner.Close() to release resources.

package main

import (
    "fmt"
    "log"

    "github.com/praetorian-inc/titus"
)

func main() {
    // Create a scanner with builtin rules (444+ detection patterns)
    scanner, err := titus.NewScanner()
    if err != nil {
        log.Fatal(err)
    }
    defer scanner.Close()

    // Scan a string for secrets
    content := `
        # Config file
        aws_access_key_id = AKIAIOSFODNN7EXAMPLE
        aws_secret_access_key = wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
    `

    matches, err := scanner.ScanString(content)
    if err != nil {
        log.Fatal(err)
    }

    // Process results
    fmt.Printf("Found %d potential secrets:\n", len(matches))
    for _, match := range matches {
        fmt.Printf("  - %s (rule: %s) at line %d\n",
            match.RuleName,
            match.RuleID,
            match.Location.SourceSpan.Start.Line,
        )
    }
}

When YAML-based scoring conditions are insufficient for complex logic (such as multi-step SDK flows or stateful checks), you can implement the scoring.Condition interface in Go.

Key Methods

• Evaluate(ctx context.Context, m *types.Match) (bool, error): The core logic. It receives the Match object, allowing you to extract data from m.NamedGroups. Return true to trigger the modifier, or false to skip it.
• markDynamic(): If you implement this method, the condition is gated behind the --score-scope flag (or titus.ScopeEnabled(true) in the Go API). Omit this method for static conditions that should always run.

Registration

To use your custom scorer, register it in buildScoringEngine() within cmd/titus/scan.go. Note that Go scorers take precedence: if a Go scorer and a YAML scorer target the same Rule ID, the Go scorer wins.

package scoring

import (
    "context"
    "github.com/praetorian-inc/titus/pkg/types"
)

type myCondition struct{}

// markDynamic gates this condition behind --score-scope.
// Omit this method entirely for static conditions that always run.
func (c *myCondition) markDynamic() {}

func (c *myCondition) Evaluate(ctx context.Context, m *types.Match) (bool, error) {
    // Extract named capture groups from the match
    token, ok := m.NamedGroups["token"]
    if !ok {
        return false, nil
    }

    // Perform your API or SDK call here.
    // Return true to fire the modifier, false to skip it.
    return checkTokenActive(ctx, string(token))
}

You can enable active validation by passing titus.WithValidation() to NewScanner. This allows you to check if a detected secret is still valid (active), expired/revoked, or if verification was undetermined.

When validation is enabled, check the match.ValidationResult.Status field.

scanner, err := titus.NewScanner(titus.WithValidation())
if err != nil {
    log.Fatal(err)
}
defer scanner.Close()

matches, err := scanner.ScanString(content)
if err != nil {
    log.Fatal(err)
}

for _, match := range matches {
    if match.ValidationResult != nil {
        switch match.ValidationResult.Status {
        case titus.StatusValid:
            fmt.Printf("ACTIVE SECRET: %s\n", match.RuleName)
        case titus.StatusInvalid:
            fmt.Printf("Expired/revoked: %s\n", match.RuleName)
        case titus.StatusUndetermined:
            fmt.Printf("Could not verify: %s\n", match.RuleName)
        }
    }
}

You can extend or override scoring for any rule ID (including custom rules) by writing a YAML scorer. Scorers define which rule_ids they target and a list of modifiers.

Modifier Types

• delta: Adds or subtracts a fixed value from the running score.
• set_score: Replaces the running score with an absolute value.

Static Modifier Triggers

• match_group: Matches a regex against a named capture group from the rule's regex pattern.
• surrounding_context_contains: Matches a pattern against the text surrounding the finding.
• match_length: Matches based on the length of the secret (e.g., min: 40).

Dynamic HTTP Modifiers

When --score-scope is enabled, you can use the http key to make live requests.

• Authentication: Use auth.bearer: "{{ .Groups.<name> }}" to inject a named capture group as a Bearer token.
• Firing Conditions (fires_when):
  • status_code_is: Match HTTP status code.
  • header_contains: Match a substring in a response header.
  • json_path_equals: Match a JSONPath expression against a value.
  • json_path_matches: Match a JSONPath expression against a regex.
  • json_array_length_gte: Match if a JSONPath array has at least N elements.

Note: Identical HTTP requests within a single scan are cached to optimize performance.

scorers:
  - name: my-scorer-name
    rule_ids:
      - np.aws.6
      - my.custom.rule.1
    modifiers:
      # Static: match a capture group
      - name: my-static-modifier
        priority: 100
        match_group:
          name: key_id
          matches: '^PREFIX'
        delta: 15

      # Static: match surrounding context
      - name: config-file-context
        priority: 80
        surrounding_context_contains:
          pattern: 'api_key\s*='
        delta: 5

      # Dynamic: HTTP call with Bearer token injection
      - name: check-active
        priority: 90
        http:
          method: GET
          url: "https://api.example.com/me"
          auth:
            bearer: "{{ .Groups.token }}"
        fires_when:
          status_code_is: 200
        delta: 20

Titus provides several ways to scan for secrets depending on your target. Results are printed to the console and written to a datastore (defaulting to titus.ds).

Basic Scanning

• Scan a file: titus scan path/to/file.txt
• Scan a directory: titus scan path/to/directory
• Scan git history: titus scan --git path/to/repo
• Scan a Docker/OCI image: titus scan --docker alpine:latest
• Validate secrets: Add the --validate flag to check detected secrets against their source APIs.

GitHub & GitLab Scanning

You can scan public repositories directly via URL without a token.

# Scan a GitHub repository
titus scan github.com/kubernetes/kubernetes

# Scan a GitLab project
titus scan gitlab.com/gitlab-org/cli

For organization-wide or user-wide scanning (including private repos), use the dedicated subcommands and provide a token via --token or the GITHUB_TOKEN/GITLAB_TOKEN environment variables.

# Scan all public repos in a GitHub org
titus github --org kubernetes

# Scan all repos in a GitHub org with a token
titus github --org kubernetes --token $GITHUB_TOKEN

# Scan all repos for a GitHub user
titus github --user octocat

# Scan all projects in a GitLab group
titus gitlab scan --group mygroup --token $GITLAB_TOKEN

# Scan a single repo with git history (finds deleted secrets)
titus github owner/repo --git

# Scan a file for secrets
titus scan path/to/file.txt

# Scan a directory for leaked credentials
titus scan path/to/directory

# Scan a public GitHub repository (no token needed)
titus scan github.com/org/repo

# Scan a public GitLab project (no token needed)
titus scan gitlab.com/namespace/project

# Scan git history for secrets in past commits
titus scan --git path/to/repo

# Scan a Docker / OCI image (pulled from a registry — no docker daemon required)
titus scan --docker alpine:latest

# Validate detected secrets against source APIs
titus scan path/to/code --validate

The extension adds a Titus tab to Burp Suite with three sub-tabs:

Secrets Tab

Displays all detected secrets. You can filter by Type, Host, or Status (Active/Inactive/Unknown). Use the search box for text and regex matching.

• Bulk actions: Select multiple rows to validate or mark as false positive in batch.
• Secret Details panel: Provides rule info, category, full secret value, first seen timestamp, all associated URLs, validation results (e.g., AWS account ID), and the full HTTP Request/Response with the secret highlighted.

Statistics Tab

Provides an aggregate view of secrets:

• Summary: Total unique secrets, hosts scanned, validation breakdown, and false positive count.
• Secrets by Type: Count of each secret type with category classification.
• Secrets by Host: Number of secrets found per host.

Settings Tab

Configure scanning behavior:

• Scan Settings: Enable/disable Passive scanning, Request body scanning, and Validation (outbound API checks).
• Scan Parameters: Adjust worker threads, max file size, and context snippet length.
• Severity Configuration: Customize severity levels per secret category.
• Actions: Clear cache, reset settings, or save/export findings to JSON.

Note: A Titus tab also appears in the response inspector when viewing any request if secrets are detected.

Titus can be built with SIMD-accelerated regex matching (using Vectorscan or Hyperscan) or as a pure-Go binary.

Standard Build (Vectorscan-accelerated)

Requires CGO, a C library (Vectorscan/Hyperscan), and pkg-config. The make build command attempts to install missing dependencies via Homebrew, apt, or dnf.

# Build the CLI binary (outputs to dist/titus)
make build

# Build the Burp Suite extension JAR
make build-burp

# Build the Chrome browser extension
make build-extension

# Run tests
make test
make integration-test

If embedding Titus via go build, you must enable CGO and provide the vectorscan tag:

# macOS (Homebrew)
CGO_ENABLED=1 PKG_CONFIG_PATH="$(brew --prefix vectorscan)/lib/pkgconfig" \
  go build -tags vectorscan -o dist/titus ./cmd/titus

# Linux
CGO_ENABLED=1 go build -tags vectorscan -o dist/titus ./cmd/titus

Pure-Go Build (No C dependencies)

Use these commands if you cannot install C libraries or want a portable, static binary:

# Portable pure-Go binary (no CGO, no vectorscan)
make build-pure

# Fully static binary
make build-static