lupa

repository·master·Indexed 22 days ago

https://github.com/scoder/lupa

A high-performance Python wrapper and bridge that integrates Lua and LuaJIT runtimes into CPython. It allows developers to switch between Python and Lua at runtime, providing tools for mapping Lua tables to Python mappings, using Lua coroutines as Python generators, and managing multiple Lua versions (including Lua 5.1 through 5.4 and LuaJIT).

Tokens
8K
Snippets
35
Records
41
Agent score
78%

What's inside lupa

  1. Handle None vs. nil mapping

    master

    Lupa maps Python's None to Lua's nil by default. However, because Lua uses nil as a termination marker for iterators, Lupa uses a special constant python.none during iteration to prevent loops from terminating prematurely when encountering a None value.

    To check for None inside Lua code, compare the value against python.none.

    # In Lua, use python.none to represent Python's None during iteration
    func = lua.eval('''
        function(items) 
            for value in python.iter(items) do
                if value == python.none then
                    -- This was a Python None
                end
            end
        end
    ''')
    
    items = [1, None, 2]
    func(items)
  2. Use Lua coroutines as Python generators

    master

    Lupa wraps Lua coroutines so they behave like Python generators. You can create a wrapped coroutine from a Lua function using the .coroutine() method or by creating it directly in Lua code.

    Once created, you can:

    • Iterate over the coroutine using standard Python iteration.
    • Send values into the coroutine using the .send(value) method.

    Note: The .throw() method is not supported.

    >>> lua_code = '''\n...     function(N)\n...         for i=0,N do\n...             coroutine.yield( i%2 )\n...         end\n...     end\n... '''
    >>> lua = LuaRuntime()
    >>> f = lua.eval(lua_code)
    
    >>> gen = f.coroutine(4)
    >>> list(enumerate(gen))
    [(0, 0), (1, 1), (2, 0), (3, 1), (4, 0)]
  3. Use Lua tables as Python mappings

    master

    Lupa maps Lua tables to Python-like mapping objects.

    Key behaviors:

    • Indexing: For array-like tables, indexing starts at 1 (Lua standard), not 0. Negative indexing is not supported.
    • Lookups: Accessing a non-existent key or index returns None (similar to Python's dict.get()).
    • Attributes: You can use attribute access (table.key) to access table members or Lua library functions (e.g., string.lower('A')).
    • Length: len() works for array tables but returns 0 for mapping tables (because it relies on Lua's # operator).
    # Array-like table (1-based indexing)
    table = lua.eval('{10, 20, 30, 40}')
    print(table[1])  # 10
    
    # Mapping table
    mapping = lua.eval('{ [20] = -20, [3] = -3 }')
    print(mapping[20])  # -20
    print(mapping['missing'])  # None
    
    # Attribute access
    string_lib = lua.eval('string')
    print(string_lib.lower('A'))  # 'a'
  4. Control Python object access protocols in Lua

    master

    Lua does not distinguish between attribute access (obj.x) and item access (obj[x]); both map to indexing. Lupa uses a heuristic to decide which Python protocol to use, preferring __getitem__ if available.

    To explicitly force a specific protocol, use these helper functions:

    • lupa.as_itemgetter(obj): Restricts the view to the item access protocol (__getitem__).
    • lupa.as_attrgetter(obj): Restricts the view to the attribute access protocol (getattr).

    These helpers can be used in Python or accessed via the python global inside Lua.

    import lupa
    
    d = {'get': 'value'}
    
    # Force item access (d['get'])
    item_getter = lupa.as_itemgetter(d)
    
    # Force attribute access (d.get)
    attr_getter = lupa.as_attrgetter(d)
  5. Build Lupa with standard Lua 5.x

    master

    Lupa can be used with standard (non-JIT) Lua runtimes.

    Option 1: Using the bundled Lua submodule (Easiest)

    1. Clone the desired Lua submodule (e.g., Lua 5.4):
      git submodule update --init third-party/lua54
    2. Build Lupa using the --use-bundle flag:
      python3 setup.py bdist_wheel --use-bundle --with-cython

    Option 2: Using a system-installed Lua package Lupa's setup.py uses pkg-config to find Lua or LuaJIT2 automatically, preferring LuaJIT2.

    • To force Lupa to ignore LuaJIT2 and use standard Lua, pass the --no-luajit option to setup.py.
    • If neither is found automatically, use --no-luajit to prevent build failure and provide parameters externally via environment variables.
    git submodule update --init third-party/lua54
    python3 setup.py bdist_wheel --use-bundle --with-cython
  6. Restrict Lua access to Python objects

    master

    When executing untrusted Lua code, you should restrict access to Python's builtins and attributes. Lupa provides two primary ways to implement access control:

    1. Attribute Filtering

    Use the attribute_filter argument in LuaRuntime to provide a function that intercepts every attribute access. The filter receives (obj, attr_name, is_setting).

    2. Dedicated Attribute Handlers

    Since Lupa 1.0, you can provide attribute_handlers consisting of a getter and a setter function. This is often cleaner for implementing whitelists.

    Security Recommendation: To safely restrict access, use a whitelist of safe attribute names or provide a dedicated set of API objects to the Lua environment.

    >>> def getter(obj, attr_name):
    ...     if attr_name == 'yes':
    ...         return getattr(obj, attr_name)
    ...     raise AttributeError('not allowed to read attribute "%s"' % attr_name)
    ...
    >>> def setter(obj, attr_name, value):
    ...     if attr_name == 'put':
    ...         setattr(obj, attr_name, value)
    ...         return
    ...     raise AttributeError('not allowed to write attribute "%s"' % attr_name)
    ...
    >>> lua = lupa.LuaRuntime(
    ...     register_eval=False,      # disallow python.eval('...')
    ...     register_builtins=False,  # disallow python.builtins.*
    ...     attribute_handlers=(getter, setter))
    >>> # ... usage follows
  7. Configure Lua version during build

    master

    By default, the Lupa build process searches for an installed version of LuaJIT, then Lua, and finally falls back to bundled versions. You can override this behavior using specific setup options during installation.

    # Example usage of build options
    # (Note: These are passed to the build system/setup tool)
    --lua-lib <libfile>
    --lua-includes <incdir>
    --use-bundle
    --no-bundle
    --no-luajit
  8. Build Lupa with LuaJIT2

    master

    To build Lupa using LuaJIT2, you must first download and build LuaJIT2 manually, placing it within the lupa base directory.

    Steps:

    1. Download and unpack lupa.
    2. Download LuaJIT2 and unpack it into a subdirectory of the lupa base directory (e.g., .../lupa-0.1/LuaJIT-2.0.2).
    3. Build LuaJIT:
      cd LuaJIT-2.0.2
      make
      cd ..
      Note: Use make CFLAGS="..." if specific compiler flags are required.
    4. Build Lupa:
      python setup.py build_ext -i

    Platform Specifics:

    • Windows: Ensure lua51.lib is generated in addition to lua51.dll. MSVC produces this, but MinGW does not.
    • macOS (64-bit):
      • You may need to set export ARCHFLAGS="-arch x86_64" for both LuaJIT and Lupa to avoid fat binary issues.
      • Additional compiler flags may be required: -pagezero_size 10000 -image_base 100000000.
    cd LuaJIT-2.0.2
    make
    cd ..
    python setup.py build_ext -i
  9. Install Lupa via package managers

    master

    You can install Lupa using pip if the necessary Lua development headers are already present on your system.

    Debian/Ubuntu (Lua 5.2):

    apt-get install liblua5.2-dev
    pip install lupa

    Debian/Ubuntu (LuaJIT2):

    apt-get install libluajit-5.1-dev
    pip install lupa

    OS X (Lua 5.2 via Homebrew):

    brew install lua
    brew install pkg-config
    pip install lupa
    apt-get install liblua5.2-dev
    pip install lupa
  10. Choose a specific Lua version

    master

    By default, import lupa uses the latest available Lua version. If you need a specific version (e.g., LuaJIT or a specific Lua 5.x release), you should import from the specific submodule. This allows you to control which Lua runtime is integrated into your CPython environment.

    Available submodules include lupa.luajit21, lupa.lua54, lupa.lua53, etc.

    try:
        import lupa.luajit21 as lupa
    except ImportError:
        try:
            import lupa.lua54 as lupa
        except ImportError:
            try:
                import lupa.lua53 as lupa
            except ImportError:
                import lupa
    
    print(f"Using {lupa.LuaRuntime().lua_implementation} (compiled with {lupa.LUA_VERSION})")
  11. Build Lupa with the Limited API (abi3 wheels)

    master

    Starting with Lupa 2.7, you can build abi3 wheels to support multiple Python versions. This can be done by passing the --limited-api option to setup.py or by setting the LUPA_LIMITED_API environment variable.

    Examples:

    • Build for the ABI version of the currently running Python:
      python3.11 setup.py build_wheel --limited-api=true
    • Build for a specific Python ABI version (e.g., 3.9):
      python3 setup.py build_wheel --limited-api=3.9
    python3.11 setup.py build_wheel --limited-api=true
  12. Importing Lua binary modules

    master

    To use Lua binary modules (C modules) within Lupa, they must be compiled against the same LuaJIT header files used to build Lupa, but they should not be linked against the LuaJIT library.

    CPython requires global symbol visibility enabled for shared libraries to load these modules. While lupa attempts to set these flags automatically using the DLFCN module, you may need to set them manually if the automatic setup fails. To do this, set sys.setdlopenflags() to the sum of your system's RTLD_NEW and RTLD_GLOBAL values.

    import sys
    orig_dlflags = sys.getdlopenflags()
    # Example: if RTLD_NEW=2 and RTLD_GLOBAL=256, use 258
    sys.setdlopenflags(258)
    import lupa
    sys.setdlopenflags(orig_dlflags)
    
    lua = lupa.LuaRuntime()
    posix_module = lua.require('posix')  # Example of loading a binary module