Bashkit

repository·main·Indexed 41 days ago

https://github.com/everruns/bashkit

A high-performance, secure, in-process virtual sandbox for executing bash scripts. It provides a POSIX-compliant shell environment with a virtual filesystem, resource limits, and built-in support for LLM tool integration. Includes a C API for cross-platform deployment and a benchmarking tool, bashkit-bench, for comparing interpreter performance.

Tokens
272.5K
Snippets
639
Records
994
Agent score
54%

What's inside Bashkit

  1. Overview of Bashkit

    main

    Bashkit is a virtual Bash interpreter designed for sandboxed, in-process shell execution. It is intended to be used as a runtime for scripts, an LLM tool, or a library integrated into Rust, Python, JavaScript, or TypeScript applications.

    Important Security Note: Bashkit operates as a sandboxed shell runtime. It is not a normal host shell. Access to the host filesystem or network typically requires explicit opt-in and should be treated as a privileged operation.

  2. Overview of Bashkit security testing coverage

    main

    Bashkit maintains extensive security testing suites to mitigate various attack vectors. Key testing areas include:

    • Threat Model Tests: Comprehensive coverage of identified threat IDs.
    • Unicode Security: Tests for TM-UNI-* vulnerabilities.
    • Nesting Depth: Tests for parser depth attacks (positive, negative, and misconfiguration scenarios).
    • Fail-Point Tests: Security-specific fail-point testing.
    • Network Security: Tests for network-related vulnerabilities.
    • Builtin & Logging Security: Validation of error handling and logging safety.
    • Git Security: Tests for Git and Git remote operations.
    • Fuzz Testing: Continuous parser and lexer fuzzing.
  3. Review Bashkit Evaluation Task Results

    main

    The bashkit-eval package produces detailed reports evaluating the performance of LLMs (like openai/gpt-5.2) on various automation tasks. Each task result includes:

    • Status: [PASS] or [FAIL].
    • Metrics: Number of turns, tool calls (successful vs error), duration, and token usage (input/output).
    • Score: A numerical score representing task completion.
    • Checks: A granular list of validation steps (e.g., dir_exists, file_contains, stdout_contains, exit_code) that must pass for the task to succeed.

    Common task categories include file_operations, text_processing, pipelines, scripting, data_transformation, json_processing, complex_tasks, error_recovery, system_info, archive_operations, code_search, environment, database_operations, config_management, and build_simulation.

  4. Understand the Bashkit crate structure

    main

    Bashkit is organized as a Cargo workspace. Depending on your integration needs, you should target the specific crate:

    • crates/bashkit/: The core library containing the parser, interpreter, VFS, builtins, and the tool contract. Use this for native Rust integrations.
    • crates/bashkit-cli/: The CLI binary for direct terminal usage.
    • crates/bashkit-python/: Python bindings via PyO3.
    • crates/bashkit-js/: JavaScript bindings via NAPI-RS.
    • crates/bashkit-capi/: A versioned native C ABI for C/C++ integrations.
    • crates/bashkit-eval/: An LLM evaluation study using the mira framework.
  5. Bashkit Design Principles

    main

    The architecture of Bashkit is guided by four core principles:

    1. Async-first: All filesystem and execution operations are asynchronous (utilizing tokio).
    2. Virtual: The interpreter does not access the real filesystem by default; it operates against a Virtual Filesystem (VFS).
    3. Multi-tenant safe: Each Bash instance maintains isolated state.
    4. Trait-based: Extensibility is achieved through the FileSystem and Builtin traits.
  6. Access Bashkit official resources and documentation

    main

    Use the following official links for general information, issue tracking, and ecosystem documentation:

  7. Use the embedded jq JSON processor in Bashkit

    main

    Bashkit includes an embedded jq JSON processor powered by jaq with a compatibility shim. It is designed to be compatible with jq versions 1.7 and 1.8. This is particularly useful for LLM agents that need to generate and execute JSON filters.

    Key constraints to note:

    • Input Stream: Multiple files are concatenated into a single stream via shell redirection. Consequently, per-file metadata like input_filename and input_line_number are not available (they return null and 0 respectively).
    • Error Handling: Filter parse failures return exit code 3. Runtime errors return exit code 5. Error messages are summarized to prevent exceeding LLM context windows.
    # Check the version of the embedded jq
    jq --version
  8. Use the @everruns/bashkit-wasm package in JS runtimes

    main

    The @everruns/bashkit-wasm package is a slim, single-threaded WebAssembly module designed for any JavaScript host, including browsers, edge/serverless workers (Cloudflare Workers, Vercel Edge, Deno Deploy), Node, Deno, and Bun.

    Unlike the native @everruns/bashkit (napi) package, this version does not require SharedArrayBuffer or cross-origin isolation (COOP/COEP) headers, making it suitable for embedded contexts and constrained edge runtimes.

    Key Features:

    • Full bash syntax and text-tool builtins (grep, sed, awk, find, jq, etc.).
    • Binary-safe virtual filesystem (VFS).
    • Support for JS custom builtins (both sync and async).
    • Streaming output and cancellation.
    • Static script analysis.

    Limitations:

    • No Network/Sockets: http_client (curl/wget) and ssh are absent. To access the network, implement a custom JS builtin that calls the host's fetch.
    • No Threads: It is strictly single-threaded.
    • No tzdata: To keep the binary size small, the IANA database is omitted. Using a named TZ= zone (e.g., TZ=America/New_York) will cause the date command to fail with an error. Use TZ=UTC or leave it unset.
    import { initBashkit, Bash } from '@everruns/bashkit-wasm';
    
    // Initialize the WASM module once
    await initBashkit();
    
    // Construct the Bash instance
    const bash = new Bash();
  9. Compare Bashkit performance against standard Bash

    main

    Bashkit benchmarks demonstrate significant performance advantages over standard bash across various categories. Key areas of improvement include:

    • Arithmetic & Control Flow: Sub-millisecond execution for basic operations and loops.
    • Arrays & Strings: Highly efficient memory and manipulation operations.
    • Subshells & Pipes: Drastic reduction in overhead for subshell creation and pipeline execution.
    • Tooling: Faster execution of common patterns for grep, sed, awk, and jq.
    • Startup: Near-instantaneous startup times compared to the millisecond-scale overhead of standard bash.

    Note: While most benchmarks show a Match (✓), some specific cases like large_multiline_script, large_assoc_array, str_uppercase, and str_lowercase may show mismatches (✗) or errors, indicating differences in behavior or implementation between Bashkit and standard Bash.

  10. Review Bashkit Evaluation Task Categories

    main

    The bashkit-eval suite tests agent capabilities across several functional categories. Developers can use these categories to benchmark how well an LLM or agent performs specific types of shell-based tasks:

    • file_operations: Creating directory structures, backing up/renaming files, and finding/deleting files.
    • text_processing: Extracting data from logs, replacing text in config files, and CSV manipulation.
    • pipelines: Using pipes, process substitution, and xargs for batch processing.
    • scripting: Writing logic (FizzBuzz, health checks), using trap for cleanup, and managing shell functions.
    • data_transformation: Converting between formats (CSV, JSON, TSV, NDJSON) and using regex for extraction.
    • json_processing: Deep-merging, querying, and schema migrations for JSON data.
    • complex_tasks: Building CLI apps, generating release notes, and performing diff reports.
    • error_recovery: Handling missing files or broken JSON formats.
    • system_info: Reporting environment variables and date calculations.
    • archive_operations: Creating and selectively extracting archives (e.g., tar.gz).
    • code_search: Recursive searching and find-and-replace across codebases.
    • environment: Managing environment variables and sourcing config files.
    • database_operations: Performing GROUP BY and joins on CSV data.
    • config_management: Merging INI/JSON configs and generating .env files.
    • build_simulation: Multi-stage build pipelines and build script generation.
  11. Use Bashkit structured-data builtins

    main

    Bashkit provides specialized builtins for common data formats to simplify script automation. These tools read from files or stdin and can be piped together.

    BuiltinFormatBest Use Case
    jqJSONComplex transformations, filters, and construction.
    yqYAML / JSONApplying jq-style expressions to YAML or safe in-place updates.
    jsonJSONQuick access (get/set/keys) without learning jq syntax.
    csvCSVColumn selection, row filtering, and tabular data manipulation.
    tomlqTOMLExtracting values from configuration files like Cargo.toml or pyproject.toml.
  12. Security Tooling and Analysis in Bashkit

    main

    Bashkit employs several layers of static and dynamic analysis to ensure security and memory safety.

    Static Analysis

    • cargo-audit: Scans Cargo.lock files against the RustSec Advisory Database for known CVEs.
    • cargo-deny: Checks for license compliance and security advisories.
    • cargo-clippy: Performs linting with security-focused warnings.
    • cargo-geiger: Tracks the usage of unsafe code blocks.

    Dynamic and Memory Safety Analysis

    • cargo-fuzz: Uses LibFuzzer to find crashes, hangs, or memory issues in parsers and interpreters.
    • Miri: Detects Undefined Behavior (UB) in unsafe code.
    • proptest: Performs property-based testing to check for panics and boundary conditions.
    • AddressSanitizer (ASAN): Detects memory errors and buffer overflows during local testing or CI.