depyler

repository·main·Indexed 18 days ago

https://github.com/paiml/depyler

A Python-to-Rust transpiler that uses Python type annotations to generate idiomatic, memory-safe Rust code. It features semantic verification to ensure behavioral equivalence, a multi-stage pipeline (Parser, HIR, Type System, Verification, and Codegen), and support for various Python constructs including async/await, context managers, and classes. Users can transpile source files, compile directly to standalone binaries, or integrate it as a Rust library via the depyler crate.

Tokens
467.5K
Snippets
1.4K
Records
1.9K
Agent score
62%

What's inside depyler

  1. Overview of Python Standard Library Testing Roadmap

    main

    The Python Standard Library Testing Roadmap tracks the validation and testing of Python 3.12 standard library modules for Depyler transpilation. The goal is to achieve 100% coverage of the ~200+ modules in the Python 3.12 standard library.

    Modules are prioritized based on:

    1. Usage Frequency: How common they are in real-world code.
    2. Transpilation Complexity: Rated as LOW, MEDIUM, HIGH, or EPIC.
    3. Dependencies: Prerequisites required for other modules.
    4. Value: The capability unlocked for users upon successful transpilation.
  2. What is Depyler?

    main

    Depyler is a Python-to-Rust transpiler designed for energy-efficient, safe code generation. It converts a practical subset of Python into idiomatic, high-performance Rust code while providing formal correctness guarantees through progressive verification.

    Key benefits include:

    • Energy Efficiency: Optimized Rust output reduces runtime energy consumption.
    • Type Safety: Leverages Rust's type system to enforce safety on Python code.
    • Zero-Runtime Overhead: Safety is verified at compile-time.
    • Formal Verification: Uses property-based testing and contract systems to ensure semantic equivalence between Python and Rust.
  3. Overview of Depyler Performance Benchmarks

    main

    Depyler provides significant performance, memory, and energy efficiency improvements by transpiling Python code to Rust. Benchmarks demonstrate that compared to a Python baseline, Depyler-generated Rust code is typically:

    • 15.3x faster in execution time.
    • 6.2x more memory efficient (84% reduction).
    • 11.8x more energy efficient (91% reduction).
    • 12.7x higher in throughput.

    These improvements are observed across mathematical workloads, data processing, web servers, and scientific computing.

  4. Python Standard Library Testing Roadmap: Tier 3 Modules

    main
    The Depyler Python Standard Library testing roadmap outlines the planned coverage for various Python modules to ensure high-fidelity transpilation and validation. Tier 3 modules are categorized by functional groups such as Data Types & Structures, Algorithms & Utilities, File Processing, Execution & Processes, and Runtime Services. Each module is assigned a priority (P1-P3), complexity level, and an estimated development effort.
  5. Assess Depyler transpiler capabilities and limitations

    main

    Based on validation of Python standard library modules, the Depyler transpiler is suitable for basic operations but has known limitations regarding advanced type systems and binary data.

    Supported Operations (Production-Ready)

    • String manipulation: e.g., textwrap module functions.
    • System access: e.g., sys.platform, sys.argv.
    • Data serialization: e.g., json.loads, json.dumps.
    • Typed arrays: e.g., array module (integers, floats, indexing).
    • Basic types: Functions returning str, int, list, or dict are generally stable.

    Unsupported or Broken Operations

    • Binary Data: struct module (packing/unpacking), memoryview, bytearray, and bytes literals (b"...").
    • Advanced Types: Regex Match objects (from the re module) and Option<Captures>.
    • Buffer Protocol: Operations involving memory views or byte buffers.
    • Specific Object Operations: copy.copy() for list types may fail due to incorrect codegen.
  6. Understand Depyler's Python Standard Library Coverage

    main

    Depyler's ability to transpile Python to Rust is measured by its coverage of Python's standard library and core language features. As of October 2025, the current pass rate is approximately 71.3% (72/101 tests).

    Key Coverage Areas:

    • Complete (100%): Built-in types (literals, operators), control flow (if/else, loops), basic functions, collections (lists, dicts, sets, strings), basic classes (init, methods, properties), and asyncio (async/await).
    • Partial: Context managers (60%), Type annotations (60%), and Iterators (60%).
    • Not Implemented (0%): Exception handling, Generators, Pattern Matching (match/case), and Advanced features like Lambdas/Closures.

    Use this analysis to understand the limitations of the current transpiler when processing real-world Python codebases.

  7. View the Depyler development roadmap priorities

    main

    The development of Depyler is organized into the following sequential priorities:

    • Priority 1: Core Language Completeness (v1.1.0) - COMPLETED
    • Priority 2: Method Resolution & Attributes (v1.2.0) - Focuses on OOP support (instance/static/class methods, attribute resolution, and inheritance).
    • Priority 3: Advanced Type Features (v1.3.0) - Focuses on decorators, context managers (with statements), and iterators/generators.
    • Priority 4: Async/Await Support (v1.4.0) - Full async Python support including async functions, iteration, and context managers.
    • Priority 5: Module System (v1.5.0) - Full module and package support (imports, __init__.py, circular imports).
    • Priority 6: Standard Library Mapping (v1.6.0) - Mapping Python stdlib (os, sys, json, re, datetime, etc.) to Rust equivalents.
    • Priority 7: Optimization & Polish (v2.0.0) - Production-ready performance, diagnostics, and tooling (IDE integration, profiling).
  8. Access Depyler documentation modules

    main

    The documentation for Depyler is organized into modules. For the test_docs_minimal module, you can access the following resources:

    • Module Documentation: High-level overview of the module.
    • API Reference: Detailed technical specifications of the public API.
    • Usage Guide: Practical instructions and examples for using the module.
  9. Performance summary of Depyler vs Python/NumPy

    main

    Depyler (Rust-based) provides significant performance improvements over Python and NumPy across various domains. Key performance characteristics include:

    • Scientific Computing: Up to 8.5x faster simulation time and 2.6x less peak memory usage compared to Python + NumPy in molecular dynamics workloads.
    • Fluid Dynamics: Up to 8.5x faster execution and significantly lower energy consumption compared to NumPy/SciPy.
    • Edge/Mobile: Up to 7.5x less energy consumption and significantly lower operating temperatures on devices like Raspberry Pi 4.
    • IoT/Battery: Up to 6.8x longer battery life on ESP32 and 7.4x longer on Arduino-class devices.
    • General Gains: Significant improvements in execution speed (up to 35.3x), memory efficiency (up to 13.0x less), and energy reduction (up to 28.8x less).
  10. Use Renacer features to debug Depyler transpilation

    main

    Renacer is a pure Rust system call tracer used to discover hidden transpilation patterns and debug issues in Depyler (specifically targeting bugs like DEPYLER-0456 and DEPYLER-0457). It provides several key capabilities for analyzing the transpilation process:

    • Transpiler Source Mapping: Maps Rust compilation errors back to the original Python source code.
    • Decision Trace: Captures specific compile-time decisions made by the transpiler.
    • Function Profiling: Identifies hot paths and I/O bottlenecks during transpilation.
    • Syscall Tracing: Reveals file access patterns occurring during the transpilation process.
    • OTLP Export: Exports transpilation telemetry to Jaeger for visual analysis.
  11. Understand the Depyler development roadmap and versioning

    main

    Depyler follows a strictly sequential development plan where each priority level must be 100% complete before moving to the next. Each completed priority triggers a new minor version release (e.g., v1.1.0, v1.2.0).

    Versioning Scheme:

    • v1.0.x: Bug fixes only.
    • v1.x.0: Released upon completion of each priority level.
    • v2.0.0: Major release for production readiness, including optimizations and tooling.

    Development Principles:

    • Sequential Execution: No skipping priorities.
    • Quality Gates: Every priority must pass all quality checks (tests, documentation, benchmarks, etc.) before release.
    • Incremental Releases: Each priority completion results in a new minor version.
  12. Python Standard Library Coverage Report

    main
    This report details the validation status of the Python Standard Library within Depyler (v3.19.20). As of the report date (2025-10-27), 27 modules have been validated with a 100% pass rate across 151 tests. This coverage includes critical categories such as Data Serialization, Cryptography, File System I/O, and Data Structures, indicating that Depyler is production-ready for the validated subset of the Python standard library.