JSPyBridge Documentation

repository·master·Indexed 21 days ago

https://github.com/extremeheat/jspybridge

A bridge enabling bidirectional interoperability between Python and Node.js. It allows calling JavaScript from Python and vice versa using a Proxy-based system, supporting asynchronous property access, function calls, and memory management via Foreign Object Reference IDs (FFID). Key features include the @AsyncTask decorator for non-blocking JS tasks, @On/@Once for EventEmitter integration, and the PyClass system for extending Python classes in JavaScript.

Tokens
7.5K
Snippets
25
Records
36
Agent score
74%

What's inside JSPyBridge

  1. Create Python classes in JavaScript using `PyClass`

    master

    You can extend a Python class in JavaScript by creating a class that extends PyClass. This allows you to add JavaScript methods to a Python object or override existing ones.

    Lifecycle and Methods

    • constructor(superclass: PythonRef = null, superArguments = [], superKwargs = {}): Used to initialize JS properties and specify the Python superclass. The constructor is called before the Python __init__ method.
    • init(): This method is called after the Python superclass has been initialized. Variables defined here exist on the Python side but remain accessible from JS.
    • this.parent: Acts like super. in standard JavaScript, allowing you to call methods on the Python superclass to avoid recursion.

    Performance Note

    While variables can exist on both sides, accessing a variable frequently across the bridge incurs overhead. For high-frequency access, keep the variable on the same side as the logic using it.

    import { python, PyClass } from 'pythonia'
    const calc = await python('./calc.py')
    
    class MyCalculator extends PyClass {
      constructor() {
        // super(PythonClass, positionalArgs, keywordArgs)
        super(calc.Calc, [true], { integers: false })
      }
    
      async mul (a, b) {
        let res = a
        for (let i = 1; i < b; i++) {
          res = await this.add(res, b)
        }
        return res
      }
    
      async div(a, b) {
        // Call the superclass's div() using this.parent
        return await this.parent.div(a, b)
      }
    }
    
    // Instantiate the class
    const calculator = await MyCalculator.init()
  2. How the jspybridge communication works

    master

    The bridge facilitates communication between Python and JavaScript using standard input/output pipes or network sockets, without requiring native modules.

    Core Mechanism:

    • Property Access & Function Calls: A communication protocol allows one side to access properties or execute functions on the other side.
    • Foreign Object Reference IDs (FFID): Non-primitive values are passed as FFIDs. Both sides maintain a map that associates these numeric IDs with actual object references.
    • Proxy Objects:
      • In JavaScript, an ES6 Proxy is used to intercept operations.
      • In Python, a class with custom magic methods (like __getattr__) acts as a proxy.
      • Every property access on a proxy is mirrored across the bridge to the host environment.
    • Memory Management: Proxy objects are automatically garbage collected (GC tracked). JavaScript uses FinalizationRegistry for Python proxies, and Python uses __del__ for JavaScript proxies. When a proxy is destroyed on one side, its reference is automatically removed from the other side.
  3. How JavaScript proxy chains work

    master
    On the JavaScript side, the bridge utilizes Proxy chains to handle complex operations. This mechanism allows the bridge to build up a call stack for property accesses or function calls. The full call stack is only sent and executed in Python once a terminal operation, such as a .then() call (for property access) or an actual function execution, is completed.
  4. Handle asynchronous tasks with AsyncTask and TaskState

    master

    To avoid blocking the dedicated callback thread (where all JS callbacks run), use the @AsyncTask decorator to spawn new threads for long-running or async JS tasks.

    Key Concepts:

    • @AsyncTask(start=True/False): Wraps a function to run in a thread. If start=False, you must call start(fn) manually.
    • TaskState: The first argument passed to an AsyncTask. It provides a stopping boolean to signal the thread to exit and a wait(seconds) (or sleep(seconds)) method.
    • task.sleep(seconds): Crucial for non-blocking loops. It sleeps the thread but also automatically exits the process once the stopping flag is set to True.
    • Thread Control:
      • start(fn): Starts a decorated function.
      • stop(fn): Sends a stopping signal to the task. It is a polite request and does not force an exit.
      • abort(fn, killAfterSeconds): Forces the thread to kill if it hasn't stopped within the specified seconds. Use this sparingly as killing Python threads is generally discouraged.
    import time
    from javascript import AsyncTask, start, stop, abort
    
    @AsyncTask(start=False)
    def routine(task: TaskState):
      while not task.stopping:
        # ... do some repeated task ...
        task.sleep(1) # Use task.sleep instead of time.sleep to respect the stopping flag
    
    start(routine)
    time.sleep(1)
    stop(routine)
  5. Manage the NodeJS process lifecycle

    master

    By default, JSPyBridge manages a single NodeJS process. You can manually control this process to clear memory or recover from crashes using terminate() and init().

    Important: Re-initialization Pattern When you call javascript.init() to start a new process, global objects like globalThis or console are re-assigned. To avoid holding references to the old, terminated process, always access these objects via the full module import rather than importing them directly from the module.

    Correct Pattern:

    • import javascript
    • javascript.globalThis.Date() (Correct)
    • from javascript import globalThis -> globalThis.Date() (Incorrect after re-init)
    import javascript
    
    # Use the first process
    javascript.eval_js('console.log("Hello from 1st NodeJS process!")')
    
    # Terminate and start a fresh process
    javascript.terminate()
    javascript.init()
    
    # Access globals via the module to ensure you get the new process
    javascript.eval_js('console.log("Hello from 2nd NodeJS process!")')
  6. Iterate over Python objects in JavaScript

    master

    When iterating over Python objects in JavaScript, you must use for await loops instead of standard for-of loops to handle the asynchronous nature of the bridge. Additionally, you can use py.enumerate to mimic Python's enumerate() behavior.

    for await (const item of pythonObject) {
      // ...
    }
    
    // Using enumerate
    for await (const [index, value] of py.enumerate(pythonObject)) {
      // ...
    }
  7. Use the PyBridge for Python-to-JS interop

    master

    The PyBridge class manages the communication between JavaScript and Python. It uses a Proxy-based system to allow JavaScript to interact with Python objects as if they were native JS objects, though most operations are asynchronous and require await.

    Key Concepts

    • Chaining: You can chain property access (e.g., obj.attr.subattr), but you must await the final result to resolve the chain.
    • Asynchronous Nature: Because calls cross the bridge, almost all interactions return Promises. Failing to await will return an Intermediate object instead of the actual value.
    • Iteration: To iterate over Python objects (like lists), use for await...of instead of a standard for...of loop.
    // Example of property access and function calling
    const value = await pyObject.some_method(arg1, arg2);
    
    // Example of property chaining
    const subValue = await pyObject.nested.property;
    
    // Example of iteration
    for await (const item of pyList) {
      console.log(item);
    }
  8. How to call Python methods and access properties

    master

    The bridge uses Proxies to allow you to interact with Python objects as if they were native JavaScript objects. However, because all communication with Python is asynchronous, you must use await for almost every interaction.

    Property Access

    To access a property, use await object.property.

    Method Calls

    To call a method, use await object.method(args).

    Keyword Arguments and Timeouts

    You can pass keyword arguments to a Python method by appending a special object to the arguments list using the $ suffix pattern. You can also specify a custom timeout for a specific call using $timeout within that object.

    Iteration

    To iterate over a Python object (like a list), you must use for await...of.

    Common Pitfalls

    • Forgetting await: If you forget to await a call, you will receive an Intermediate object instead of the result. Logging this object will trigger a warning: [You must use await when calling a Python API].
    • Synchronous access: You cannot access Python properties synchronously. Always use await.
    // Property access
    const val = await pyObject.some_property;
    
    // Method call
    const result = await pyObject.some_method('arg1', 42);
    
    // Method call with keyword arguments and custom timeout
    // The '$' suffix indicates the next argument is a kwargs object
    const result = await pyObject.some_method('arg1', { timeout: 5000 });
    
    // Iteration
    for await (const item of pyList) {
      console.log(item);
    }
  9. How JSBridge manages object references between Python and JavaScript

    master

    The JSBridge class acts as the intermediary for interop between Python and JavaScript. It maintains a reference map (this.m) that associates unique identifiers (ffid) with actual JavaScript objects.

    When a JavaScript object is passed to Python, it is wrapped in a proxy that carries an ffid. When Python calls a method or accesses a property, the JSBridge uses that ffid to look up the corresponding object in its internal map.

    Key behaviors:

    • FFID Generation: The bridge increments this.ffid whenever a new complex object (class, function, or object) is returned to Python to ensure unique identification.
    • Type Mapping: The bridge automatically detects the type of the JavaScript value (e.g., string, num, big, py, class, fn, obj, void) and communicates this type back to Python so the correct Python proxy can be used.
    • Memory Management: The bridge provides a free method to manually delete references from the internal map, preventing memory leaks when Python-side proxies are no longer needed.
  10. Iterate over Python objects using `for await...of`

    master

    When dealing with Python collections (like lists or generators) in JavaScript, you cannot use a standard for...of loop because the elements are fetched asynchronously across the bridge.

    Instead, use the for await...of syntax. Using a standard for...of loop will result in a SyntaxError.

    // Correct way to iterate over a Python list/sequence
    for await (const item of pythonList) {
      console.log(item);
    }
  11. Import JavaScript packages with require()

    master

    Use the require function to manage and import npm dependencies. If a package is not found in the local or global registry, the library will attempt to install it automatically. For relative imports (starting with . or /), the file is loaded relative to your calling script.

    Package Versioning: Providing a package_version creates a unique ID (e.g., chalk--1.0) to prevent collisions between different versions of the same package.

    def require ( package_name: str, package_version: Optional[str] = None ) -> Void
  12. Bind event listeners with @On and @Once

    master

    The library provides specialized wrappers for Node.js EventEmitter objects. You should use these instead of the standard .on or .once methods to ensure proper integration with the bridge.

    Available Wrappers:

    • @On(emitter, eventName): A decorator to bind a function to an event.
    • @Once(emitter, eventName): A decorator to bind a function that triggers only once.
    • off(emitter, eventName, handlerFn): A top-level function to remove an event listener.

    Note on this: When using the @On decorator, the first argument of the decorated function is the JavaScript this context.

    from javascript import require, On, Once, off, once
    MyEmitter = require('./emitter.js')
    myEmitter = MyEmitter()
    
    @On(myEmitter, 'increment')
    def handleIncrement(this, counter):
        print("Incremented", counter)
        # To stop listening:
        off(myEmitter, 'increment', handleIncrement)
    
    myEmitter.inc()