ClusterFuzzLite Documentation

repository·main·Indexed 19 days ago

https://github.com/google/clusterfuzzlite

A continuous fuzzing solution that integrates into CI workflows (GitHub Actions, GitLab, Google Cloud Build, Prow) to detect vulnerabilities in pull requests and perform long-running batch fuzzing. Supports C, C++, Java (JVM), Go, Python, Rust, and Swift. Provides capabilities for crashing testcase downloads, coverage reports, and modular vulnerability discovery.

Tokens
19.3K
Snippets
50
Records
75
Agent score
68%

What's inside ClusterFuzzLite

  1. Overview of ClusterFuzzLite

    main

    ClusterFuzzLite is a continuous fuzzing solution designed to run within Continuous Integration (CI) workflows. It enables developers to find vulnerabilities by fuzzing pull requests before code is committed.

    Key capabilities include:

    • Pull Request Fuzzing: Quick fuzzing on code changes to catch bugs early.
    • Batch Fuzzing: Continuous, longer-running fuzzing to find deeper bugs and build a corpus for code-change fuzzing.
    • Crash Reporting: Automatic downloads of crashing testcases.
    • Coverage Reports: Visibility into which parts of the codebase are being exercised by the fuzzer.
    • Modular Design: Users can select specific features based on their needs.
  2. Use FuzzedDataProvider for complex fuzz targets

    main

    The FuzzedDataProvider class (part of the Jazzer API) simplifies fuzzing by translating raw input bytes into useful Java types (e.g., int, long, String).

    • Availability: The library is located at $JAZZER_API_PATH in ClusterFuzzLite base images.
    • Local Development: You can obtain the library from Maven Central under the group com.code-intelligence and artifact jazzer-api.
    • Usage: Pass FuzzedDataProvider as the argument to fuzzerTestOneInput instead of byte[].
  3. Write Rust fuzzers using the fuzzing feature

    main

    ClusterFuzzLite's use of cargo-fuzz automatically enables the fuzzing feature. You can use this to organize code and manage dependencies specifically for fuzzing, similar to how #[cfg(test)] works for unit tests.

    Conditional Compilation

    You can wrap fuzzing-specific logic in modules using the fuzzing attribute:

    #[cfg(fuzzing)]
    pub mod fuzz_logic {
        use super::*;
        // ... fuzzing implementation
    }

    To combine testing and fuzzing logic into a single block, use:

    #[cfg(any(test, fuzzing))]

    Fuzzing-specific Dependencies

    In your Cargo.toml files, you can define dependencies that are only compiled when the fuzzing feature is active:

    [target.'cfg(fuzzing)'.dependencies]
    # fuzzing-only dependencies here
    #[cfg(fuzzing)]
    pub mod fuzz_logic {
        use super::*;
    
        ...
    }
    
    [target.'cfg(fuzzing)'.dependencies]
  4. Choose or reuse an existing Filestore

    main

    When setting up a new platform, you can choose between implementing a custom filestore or reusing existing ones:

    • Reuse existing filestores: You can use gsutil to support Google Cloud Storage Buckets or Amazon S3 Buckets.
    • Use no_filestore: If your platform does not support file storage, use the no_filestore implementation. This makes all filestore operations no-ops. This allows ClusterFuzzLite to run without exceptions, but most important features (like saving crashes or coverage) will not function.
  5. Understand the supported sanitizers in ClusterFuzzLite

    main

    ClusterFuzzLite uses sanitizers to detect bugs by instrumenting code at compile-time. When configuring ClusterFuzzLite, use the following shorthand names to specify the sanitizer:

    • address: Refers to AddressSanitizer (ASan). Used for detecting memory safety issues and memory leaks. This is the most critical sanitizer for fuzzing.
    • ubsan: Refers to UndefinedBehaviorSanitizer (UBSan). Used for detecting undefined behavior, such as integer overflows.
    • memory: Refers to MemorySanitizer (MSan). Used for detecting the use of uninitialized memory.

    Note on MSan: To avoid false positives, an MSan instrumented binary must be entirely instrumented with MSan; if any part of the binary lacks MSan instrumentation, it will report errors.

  6. How ClusterFuzzLite architecture works

    main

    ClusterFuzzLite's architecture is centered around two main functions performed by distinct Docker images, controlled primarily via environment variables.

    Configuration

    All configuration is handled through environment variables. Most logic is managed by config_utils.py, while platform-specific configurations (like CI systems) are handled in platform_config.

    Core Functions

    1. Building Fuzzers: Performed by the gcr.io/oss-fuzz-base/clusterfuzzlite-build-fuzzers image. This involves:

      • Building the builder image and the fuzzers (using SANITIZER and LANGUAGE variables).
      • Deleting unaffected fuzzers: During code change fuzzing, ClusterFuzzLite diffs the repo against a base_ref or base_commit. It uses coverage data to identify which fuzzers cover modified files. Unaffected fuzzers are deleted to save time, unless they have no coverage data.
      • Checking for common mistakes: Uses OSS-Fuzz's bad build check to ensure fuzzers don't crash trivially and that sanitizer instrumentation matches the expected SANITIZER.
      • Uploading builds: If UPLOAD_BUILD is set, builds are uploaded to the filestore.
    2. Running Fuzzers: Performed by the gcr.io/oss-fuzz-base/clusterfuzzlite-run-fuzzers image. The behavior is determined by the MODE environment variable, which can be set to:

      • code change fuzzing
      • batch fuzzing
      • corpus pruning
      • coverage report generation
  7. ClusterFuzzLite features and capabilities

    main

    ClusterFuzzLite provides several key capabilities for vulnerability discovery:

    • Quick code change (pull request) fuzzing: Finds bugs during the PR process before code is merged.
    • Crashing testcase downloads: Allows users to download the specific inputs that caused a crash.
    • Continuous batch fuzzing: Runs longer-term fuzzing asynchronously to find deeper bugs and build a corpus to improve code-change fuzzing efficiency.
    • Coverage reports: Provides visibility into which parts of the codebase are being exercised by the fuzzer.
    • Modular functionality: Users can opt-in to specific features as needed.
  8. Understand ClusterFuzzLite fuzzing modes

    main

    ClusterFuzzLite operates in several modes to suit different stages of the development lifecycle. You select a mode using the mode option.

    Primary Fuzzing Modes

    • Code Change Fuzzing (code-change): The default mode. Designed for pull requests or commits. It is optimized for speed: it defaults to 10 minutes of fuzzing and exits immediately after finding a single crash. It aims to find bugs introduced by specific changes before they are merged.
    • Batch Fuzzing (batch): Designed for scheduled runs (e.g., daily). It runs all fuzzers for a longer, preset duration and does not exit upon finding a bug. It is used to find existing bugs in the codebase and to build a corpus (a collection of testcases) that improves coverage for other modes.

    Helper Modes

    • Corpus Pruning (prune): A maintenance mode used to minimize corpuses by removing redundant testcases that do not increase code coverage. This is highly recommended if you use batch mode to keep fuzzing efficient.
    • Code Coverage Report Generation (coverage): Uses the corpus developed during batch fuzzing to generate an HTML report showing which parts of the code are covered. This data helps code-change mode determine which fuzzers are affected by a specific code change.

    Continuous Builds Task

    While not a mode, the Continuous Builds task can be enabled. It saves builds for later use by code-change mode. This allows ClusterFuzzLite to distinguish between pre-existing crashes and those introduced by a new code change; if a crash is pre-existing, it will not be reported in code-change mode.

  9. Getting Started with ClusterFuzzLite

    main

    To begin using ClusterFuzzLite, choose your starting point based on your familiarity with fuzzing:

    1. New to fuzzing? If you are unfamiliar with libFuzzer and sanitizers, review the project's [Overview] documentation to understand the fuzzing process and terminology.
    2. Experienced with fuzzing? If you already understand libFuzzer and sanitizers, proceed directly to [Step 1: Build Integration] to integrate fuzzing into your build system.
  10. Accessing and Viewing Crashes and Artifacts

    main

    Since Google Cloud Build does not provide an easy way to download build files directly, you must access them via your Google Cloud Storage bucket.

    Downloading Crashes:

    1. Inspect the build logs to find the name of the crashing input file.
    2. Download the crash file from: <your-cloud-bucket>/crashes/<fuzzer>/<sanitizer>/<crash-file>

    Viewing via Browser: You can navigate to the following URL in a web browser: https://console.cloud.google.com/storage/browser/<your-cloud-bucket-without-gs>/crashes/<fuzzer>/<sanitizer>

    Note: <your-cloud-bucket-without-gs> is your bucket name without the gs:// prefix (e.g., if your bucket is gs://my-bucket, use my-bucket).

  11. Set up Continuous Builds with GitHub Actions

    main

    Continuous builds trigger a build and upload the result as a GitHub Actions artifact whenever a push is made to your main/default branch.

    This is used to determine if a crash found during PR fuzzing is novel. If a build for the current main branch already contains the crash, PR fuzzing will not report it as a new failure.

    Warning: Large builds may consume significant GitHub Actions storage and minute quotas. Use with caution.

    Create .github/workflows/cflite_build.yml and ensure upload-build: true is set in the build_fuzzers action.

    name: ClusterFuzzLite continuous builds
    on:
      push:
        branches:
          - main
    permissions: read-all
    jobs:
      Build:
       runs-on: ubuntu-latest
       concurrency:
         group: ${{ github.workflow }}-${{ matrix.sanitizer }}-${{ github.ref }}
         cancel-in-progress: true
       strategy:
         fail-fast: false
         matrix:
            sanitizer:
            - address
       steps:
       - name: Build Fuzzers (${{ matrix.sanitizer }})
         id: build
         uses: google/clusterfuzzlite/actions/build_fuzzers@v1
         with:
            language: c++
            sanitizer: ${{ matrix.sanitizer }}
            upload-build: true