DOCSEARCHbyAlgolia
Submit Docs

RepoAudit Documentation

repository·main·Indexed 19 days ago

https://github.com/purcl/repoaudit

A multi-agent, repository-level bug detection framework that uses LLMs and tree-sitter parsing to mimic manual code auditing. It supports C/C++, Java, Python, and Go, detecting bugs such as Null Pointer Dereference (NPD), Memory Leaks (MLK), and Use After Free (UAF) without requiring compilation. The framework features a multi-agent pipeline consisting of MetaScanAgent and DFBScanAgent, a three-layer memory system (Syntactic, Semantic, and Report), and a Streamlit-based web UI for reviewing bug reports.

Tokens
9.2K
Snippets
24
Records
50
Agent score
64%

What's inside RepoAudit

  1. Use TSAnalyzer for parsing-based analysis

    main

    The TSAnalyzer uses tree-sitter to derive the Abstract Syntax Tree (AST) of repository code. It extracts constructs such as function parameters, arguments, return values, branches (if-statements), and loops (for/while).

    It is used to construct:

    • Call graphs (based on function names and parameter counts).
    • Control-flow order analysis.
    • CFL-reachability analysis.

    Different sub-classes of TSAnalyzer exist for different programming languages. These are located in src/tstool/analyzer/.

  2. How RepoAudit Agents work

    main

    RepoAudit is a multi-agent framework that uses different agents for different stages of the audit:

    • MetaScanAgent (metascan.py): Uses tree-sitter–powered parsing-based analyzers to obtain the basic syntactic properties of the program.
    • DFBScanAgent (dfbscan.py): Performs inter-procedural data-flow analysis to detect data-flow bugs, such as source-must-not-reach-sink (e.g., Null Pointer Dereference) and source-must-reach-sink (e.g., Memory Leak) bugs.

  3. How RepoAudit's multi-agent pipeline works

    main

    RepoAudit operates as a multi-agent framework where specialized agents perform code auditing tasks (e.g., bug detection, program slicing). The workflow follows a specific pipeline:

    1. Code Indexing: A TSAnalyzer (parsing-based) is initialized to perform AST parsing and build syntactic information.
    2. Semantic Analysis: An Agent is invoked, which can utilize TSTool (parsing-based tools) and LLMTool (LLM-driven tools) to perform semantic analysis.
    3. Reporting: The agent produces final results (e.g., bug reports).

    Throughout this process, information is persisted in three types of memory:

    • Syntactic Memory: Stores AST-derived info like Functions, APIs, and Values.
    • Semantic Memory: Stores agent-specific intermediate states.
    • Report Memory: Stores final scan results like bug or debug reports.
    Code → TSAnalyzer (AST Parsing) → Agents (Semantic Analysis) → Reports (Final Results)

Memory Layers:
- Syntactic Memory (Syntactic Info: Value/Function/API)
- Semantic Memory (Semantic Properties: Agent State)
- Report Memory (Scan Report: Bug/Debug results)

  4. Understand RepoAudit's Memory system

    main

    RepoAudit uses three distinct memory layers to manage data during an audit:

    • Syntactic Memory (src/memory/syntactic/): Maintains critical code constructs extracted via TSAnalyzer. It stores Function, API, and Value information. This is retrieved by agents when invoking LLM-driven tools.
    • Semantic Memory (src/memory/semantic/): Maintains the intermediate states of agents. Each agent defines its own state as a sub-class of State. For example, DFBScanState stores data-flow facts and relevant parameters/return values.
    • Report Memory (src/memory/report/): Maintains the final outputs of agents, such as bug_report.py or debug reports.

  5. Extend RepoAudit with new data-flow bug types

    main

    If you want to add a new bug type that can be reduced to reachability analysis on a data-dependency graph (a data-flow bug), you should follow the pattern used by DFBScanAgent.

    To implement a new data-flow bug detector, follow these two steps:

    1. Implement a DFBScanExtractor subclass: Create a new class that inherits from DFBScanExtractor to define the source and sink extractors for your target programming language. Place this in the appropriate dfbscan_extractor directory.
    2. Provide Prompt Templates: Add JSON files containing prompt templates for intra-procedural data-flow analysis and path feasibility validation. These should be placed in the corresponding sub-directories under prompt/ (e.g., prompt/<Language>/dfbscan/).

    Note on Prompt Templates: If your bug's data-flow facts propagate similarly to Null Pointer Dereference, Memory Leak, or Use-After-Free, you can reuse the existing templates found in intra_dataflow_analyzer.json and path_validator.json.

    Running the Scan: When executing the scan, you must specify whether the bug is a "source-must-reach-sink" or "source-must-not-reach-sink" type by using the --is-reachable option in your run command.

  6. Quick Start with Benchmarks

    main

    To run a quick scan using the provided benchmark programs:

    1. Initialize Submodules: Ensure all benchmark submodules are initialized.
    2. Run Scan: Use the src/run_repoaudit.sh script to scan the benchmark/Java/toy/NPD directory.
    3. View Results: Check the generated JSON and log files after the scan completes.
    # Initialize submodules
cd RepoAudit
git submodule update --init --recursive

# Run the scan script
cd src
sh run_repoaudit.sh

  7. Initialize Benchmark Programs

    main

    RepoAudit includes prepared benchmark programs in the benchmark directory. Because some are Git submodules, you must initialize them before running scans on benchmarks.

    cd RepoAudit
git submodule update --init --recursive
    git submodule update --init --recursive

  8. Install RepoAudit

    main

    Follow these steps to set up the RepoAudit environment using Conda and Pip.

    1. Create and activate a Conda environment with Python 3.13:

      conda create -n repoaudit python=3.13
conda activate repoaudit

    2. Install dependencies:

      cd RepoAudit
pip install -r requirements.txt

    3. Build Tree-sitter bindings:

      cd lib
python build.py

    4. Configure API Keys: Set your OpenAI and Anthropic API keys in your environment (e.g., in ~/.bashrc):

      export OPENAI_API_KEY=xxxxxx
export ANTHROPIC_API_KEY=xxxxxx
    conda create -n repoaudit python=3.13
conda activate repoaudit
cd RepoAudit
pip install -r requirements.txt
cd lib
python build.py

  9. Support a new programming language in RepoAudit

    main

    To add support for a new programming language to the RepoAudit framework, complete the following steps:

    1. Update Build Configuration: Add the repository link for the new language in lib/build.py, then run python lib/build.py from the root directory to install the necessary components.
    2. Implement a TSAnalyzer: Create a parsing-based analyzer by implementing all abstract methods defined in the TSAnalyzer class. You can use existing implementations like Cpp_TSAnalyzer or Java_TSAnalzyer in src/tstool as references.
    3. Implement Bug Detectors: Once the language parsing is supported, implement the specific bug detectors for that language following the data-flow bug extension process.
    4. Update RepoAudit Entry Point: Modify src/repoaudit.py to append the new language and bug type choices to the available options, enabling them for analysis.
    # Step 1: Update build config and install
python lib/build.py

  10. Run a Code Scan with run_repoaudit.sh

    main

    The src/run_repoaudit.sh script is the primary way to scan a target project folder for bugs. It supports three usage modes:

    1. Basic Usage

    Scans the default toy project (../benchmark/Python/toy) for NPD (Null Pointer Dereference) bugs.

    cd src
sh run_repoaudit.sh

    2. Specify Project Path

    Scans a specific project using a relative or absolute path. Defaults to NPD bugs.

    sh run_repoaudit.sh /path/to/your/project

    3. Specify Project and Bug Type

    Scans a specific project for a chosen bug type. The bug type argument is case-insensitive.

    sh run_repoaudit.sh /path/to/your/project UAF
    sh run_repoaudit.sh /path/to/your/project UAF

  11. Enable Parallel Auditing

    main

    To accelerate analysis for large repositories, you can use parallel auditing.

    • Neural Workers: Use the --max-neural-workers flag to increase the number of neural workers. The default value is 6.
    • Parsing-based Analysis: This is enabled in parallel mode by default, with a maximum of 10 workers.
    # Example of increasing neural workers (conceptually)
# Use the --max-neural-workers flag with your execution command

  12. Launch the Web UI

    main

    RepoAudit provides a Streamlit-based web interface to review bug reports. The UI allows you to:

    • Examine functions within a bug trace.
    • View LLM-generated explanations.
    • Classify results as TP (True Positive), FP (False Positive), or Unknown. Labeled results are stored locally on your machine.

    To start the UI, run the streamlit command pointing to src/ui/web_ui.py.

    cd RepoAudit
streamlit run src/ui/web_ui.py