detect-secrets

Scanning is primarily managed through the SecretsCollection class. The engine uses a combination of:

• Plugins: Codified rules that scan strings/lines to find potential secrets.
• Filters: Pure functions that exclude false positives based on specific criteria.
• Transformers: Convert files into line-proxies to handle multi-line formats.

To scan files or diffs programmatically, use the SecretsCollection interface. While the tool provides high-level CLI commands, developers can call SecretsCollection.scan_diff directly for diff-based scanning.

The detect-secrets project provides three distinct tools depending on your workflow stage:

1. detect-secrets scan: Use this to scan a repository and generate or update a baseline of identified secrets.
2. detect-secrets-hook: Use this to alert on or block new secrets that are not present in your existing baseline (typically used in pre-commit hooks).
3. detect-secrets audit: Use this to analyze an existing baseline file to distinguish between true and false positives and optimize plugin settings.

Filters are functions that return a boolean value to indicate whether a specific condition should be skipped during a scan. They allow you to systematically exclude results, such as specific files, lines, or secret patterns, from being reported as vulnerabilities.

Filters are executed at different stages of the scanning process:

1. Filename stage: Filters operate on the filename alone (e.g., to skip non-existent files).
2. Line stage: Filters operate on the content of a line.
3. Secret stage: Filters operate on the filename, line, the raw secret value, and the plugin that identified it.

If you want to exclude a specific false positive from blocking a commit without updating the global baseline, use inline comments to allowlist the secret or the next line.

secret = "hunter2"      # pragma: allowlist secret

//  pragma: allowlist nextline secret
const secret = "hunter2";

You can use detect-secrets audit --diff to perform differential analysis between two baseline files. This is useful for determining how changes in configuration (such as plugin options) affect the number of secrets detected. The output will show the status of secrets, such as >> REMOVED << if a secret found in the first baseline is no longer detected in the second.

$ detect-secrets scan test_data --base64-limit 4 > limit4
$ detect-secrets scan test_data --base64-limit 5 > limit5
$ detect-secrets audit --diff limit4 limit5

Via CLI

Use the --disable-filter flag followed by the full import path of the filter function.

$ detect-secrets scan test_data --disable-filter detect_secrets.filters.heuristic.is_prefixed_with_dollar_sign

Via Python API

If using detect-secrets as a library, customize your settings object by providing a filters_used list containing dictionaries with the filter's path.

from detect_secrets.core import baseline
from detect_secrets.settings import transient_settings

config = {
    'filters_used': [
        {
            'path': 'detect_secrets.filters.heuristic.is_potential_uuid',
        },
    ],
}

with transient_settings(config):
    secrets = baseline.create('.')

The Gibberish Detector uses an ML model to determine if a value is likely gibberish rather than a real secret. Note that this may ignore common secrets like password because they are word-like.

To use this feature, install the gibberish-detector package:

pip install detect-secrets[gibberish]

You can use the default pre-trained model or provide your own via the --gibberish-model flag.

detect-secrets scan --gibberish-model custom.model

The detect-secrets audit command provides features for manual analysis of existing baselines. Key capabilities include:

• Manual Labeling: Distinguish between true and false positives.
• Baseline Comparison: Determine how changes in configuration affect scan results.

If you have a large list of strings to exclude (rather than a few regex patterns), use the --word-list flag. This requires the pyahocorasick package.

To install the necessary dependency:

pip install detect-secrets[word_list]

detect-secrets scan --word-list wordlist.txt

Use the audit command on a pre-generated baseline to manually label secrets. This allows you to distinguish between true positives and false positives by providing feedback for each detected secret. This metadata is added to the secrets but does not prevent new, unlabelled secrets from being detected in future scans.

$ detect-secrets scan test_data > .secrets.baseline
$ detect-secrets audit .secrets.baseline