ripsecrets

repository·main·Indexed 21 days ago

https://github.com/sirwart/ripsecrets

A high-performance, local-only command-line tool designed to prevent secret keys from being committed to source control. Version 0.1.11 uses probability-based randomness detection and predefined regex patterns to identify sensitive information like API keys and private keys while maintaining low false-positive rates. It supports integration as a git pre-commit hook, custom regex patterns via --additional-pattern, and secret exclusions through .secretsignore files or inline pragma comments.

Tokens
3.4K
Snippets
13
Records
17
Agent score
75%

What's inside ripsecrets

  1. How ripsecrets detects secrets

    main

    The tool uses two primary methods to identify secrets:

    1. Known Patterns: Uses regular expressions to match API keys with predefined prefixes (e.g., Stripe, Slack).
    2. Randomness Detection: For secrets without a known pattern (like AWS secret access keys), ripsecrets looks for variables or properties assigned words like token, secret, or password. It then calculates the probability that the assigned string occurred by random chance. If the probability is less than 1 in 10,000, it is flagged as a secret.

    ripsecrets operates entirely locally and never sends data to third-party services for verification.

  2. Configure ripsecrets with pre-commit framework

    main

    If you use the pre-commit framework, add ripsecrets to your .pre-commit-config.yaml. There are two hook types available:

    1. ripsecrets (Recommended): Automatically sets up a Rust environment to compile and run the tool.
    2. ripsecrets-system: Looks for a ripsecrets binary already present on your PATH. This requires manual installation via Homebrew, Cargo, or pre-built binaries.

    Example Configuration

    repos:
      - repo: https://github.com/sirwart/ripsecrets
        rev: v0.1.8 # Use latest tag on GitHub
        hooks:
          - id: ripsecrets
            # args:
            # - --additional-pattern 'mytoken*'
            # - --additional-pattern 'mykey*'
    repos:
      - repo: https://github.com/sirwart/ripsecrets
        rev: v0.1.8
        hooks:
          - id: ripsecrets
  3. Install ripsecrets as a pre-commit hook

    main

    To prevent secrets from being committed to your Git repository, you can integrate ripsecrets into your workflow.

    Automatic Installation

    Run this command to automatically configure a pre-commit hook in your current repository:

    ripsecrets --install-pre-commit

    Manual Installation

    If you are managing your own pre-commit scripts, add the following command. Using --strict-ignore ensures that your .secretsignore file is respected:

    ripsecrets --strict-ignore `git diff --cached --name-only --diff-filter=ACM`
    ripsecrets --install-pre-commit
  4. Install ripsecrets

    main

    You can install ripsecrets using several different methods depending on your environment:

    Homebrew (macOS and Linux)

    brew install ripsecrets

    Cargo (Rust/Cargo users)

    cargo install --git https://github.com/sirwart/ripsecrets --branch main

    Nix Flake

    nix profile install github:sirwart/ripsecrets

    Pre-built Binaries

    Download the latest binary directly from the GitHub releases page.

    brew install ripsecrets
  5. Use ripsecrets via CLI

    main

    By default, running ripsecrets recursively searches source files in your current directory for secrets. For every secret found, it prints the file, line number, and the secret itself. If secrets are found, the process exits with a non-zero status code.

    Basic Usage

    ripsecrets

    Scanning specific files or directories

    You can pass specific paths as arguments:

    ripsecrets file1 file2 dir1

    Finding custom secrets with regex

    Use the --additional-pattern flag to detect custom secret formats. Any capturing groups in your regex will be tested for randomness. To avoid randomness testing on a pattern, use non-capturing groups (e.g., (?:foo|bar) instead of (foo|bar)).

    ripsecrets --additional-pattern 'my-secret-\*' 
    ripsecrets
  6. Predefined secret regex patterns in ripsecrets

    main

    The library includes a set of built-in regex patterns to detect common secrets. These include:

    • URLs/Credentials: Generic URL patterns with embedded credentials.
    • Tokens: JWT/JWE, GitHub (personal access tokens), GitLab, Stripe, Square, Azure Storage, GCP API Keys, npm (modern and legacy), Slack, and Slack Webhooks.
    • Private Keys: RSA, EC, DSA, OpenSSH, PGP, and AGE secret keys.
    • Other: Twilio, Mailchimp, and Intra42 patterns.
    • Random Strings: Uses matcher::RANDOM_STRING_REGEX for high-entropy string detection.
  7. Ignore secrets using .secretsignore or pragma comments

    main

    ripsecrets provides two ways to prevent specific strings from being flagged as secrets:

    1. .secretsignore file: You can provide a set of specific byte sequences (secrets) to the IgnoringMatcher that should be skipped during scanning.
    2. Inline Pragma: You can suppress a secret detection on a specific line by adding a pragma comment. The matcher looks for the pattern pragma: allowlist secret on the same line to skip the match.
  8. How ripsecrets identifies random strings

    main

    ripsecrets uses a specific regex pattern to identify potential secrets. It looks for lines containing keywords like key, token, secret, or password (case-insensitive) followed by assignment operators (e.g., :, =, :=, =>, <-, >) and a high-entropy string between 15 and 90 characters long.

    To reduce false positives, the tool applies a randomness check (is_random) on the captured string. A string is only flagged if it passes a statistical randomness threshold (based on the p_random algorithm). If the string contains no numbers, the threshold for randomness is stricter.

    // The regex used for identifying secret patterns:
    // (?i:key|token|secret|password)\w*["' ]?]?\s*(?:[:=]|:=|=>|<-|>)\s*[\t "'`]?([\w+./=~\-\\\^]{15,90})(?:[\t\n "'`]|</|$)
  9. Ignore secrets using .secretsignore or comments

    main

    You can exclude specific files, directories, or individual secrets from being scanned.

    Using .secretsignore

    Create a .secretsignore file in your repository. It supports syntax similar to .gitignore. You can also use a [secrets] section to ignore specific string values.

    # Ignore files/directories
    test/*
    dummy
    
    # Ignore specific secret strings
    [secrets]
    pAznMW3DsrnVJ5TDWwBVCA

    Using allowlist comments

    ripsecrets is compatible with detect-secrets style allowlist comments. Add the comment # pragma: allowlist secret to the same line as the detected secret:

    test_secret = "pAznMW3DsrnVJ5TDWwBVCA" # pragma: allowlist secret
    test/*
    dummy
    
    [secrets]
    pAznMW3DsrnVJ5TDWwBVCA
  10. Configure file and secret exclusions with .secretsignore

    main

    You can control which files are ignored during a scan and which specific secret patterns are excluded by using a .secretsignore file in your current directory.

    File Exclusions

    If the .secretsignore file does not contain the [secrets] header, all entries are treated as standard file/directory ignore patterns (similar to .gitignore).

    Secret Exclusions

    To exclude specific secret patterns while still ignoring certain files, use the [secrets] section header. Everything above the header is treated as a file ignore pattern, and everything below the header is treated as a secret exclusion. Lines starting with # in the [secrets] section are treated as comments.

    Example .secretsignore structure

    # File ignores (standard gitignore style)
    node_modules/
    *.log
    
    [secrets]
    # Secret exclusions
    MY_PRIVATE_KEY_PATTERN
    INTERNAL_TOKEN_VALUE
    # Example .secretsignore content
    node_modules/
    *.log
    
    [secrets]
    MY_PRIVATE_KEY_PATTERN
    INTERNAL_TOKEN_VALUE
  11. Calculate randomness probability with `p_random`

    main

    When a potential secret does not match any known patterns, p_random provides a heuristic to determine if the string is likely random or non-random text. It calculates the probability that the string's characteristics occurred by chance based on three metrics:

    1. Distinct Values: Non-random text typically has fewer distinct characters than random text.
    2. Character Classes: It evaluates the probability of the observed distribution of numbers, uppercase, and lowercase letters based on the detected base (Hex, Alphanumeric, or Base64).
    3. Bigrams: For Base64 strings, it checks the frequency of common source-code bigrams. A random string is expected to contain a much lower frequency of these common bigrams.

    The function returns an f64 representing the calculated probability. A very low probability suggests the string is likely random (and thus a potential secret), while a higher probability suggests it is likely structured text.

    pub fn p_random(s: &[u8]) -> f64
  12. Install ripsecrets as a git pre-commit hook

    main

    The install_pre_commit function automates the setup of a git pre-commit hook to ensure ripsecrets runs automatically before every commit.

    It follows these installation logic rules:

    1. Detection: It looks for a .git directory at the provided repo_root.
    2. Pre-commit.d Support: If a .git/pre-commit.d directory exists, it installs the hook as .git/pre-commit.d/ripsecrets.
    3. Standard Hook Support: If no pre-commit.d exists, it installs/appends to the standard .git/hooks/pre-commit file.
    4. Idempotency: If the hook command is already present in the file, it returns an error to prevent duplicate entries.

    The hook command installed is: ripsecrets --strict-ignore git diff --cached --name-only --diff-filter=ACM``

    It automatically sets the executable bit (0o100) on the created hook file.

    use std::path::Path;
    
    // Assuming the library is imported as ripsecrets
    install_pre_commit(Path::new("./")).expect("Failed to install pre-commit hook");