wrapt

repository·develop·Indexed 25 days ago

https://github.com/grahamdumpleton/wrapt

A Python module providing transparent object proxies, universal decorators, and monkey patching utilities. It is designed to preserve introspection (such as signatures and annotations) more effectively than standard mechanisms like functools.wraps(), allowing a single decorator implementation to be applied to functions, classes, staticmethods, classmethods, or instance methods.

Tokens
50.9K
Snippets
122
Records
194
Agent score
79%

What's inside wrapt

  1. Overview of wrapt capabilities

    develop

    The wrapt module is designed to provide transparent object proxies for Python, serving as a foundation for:

    • Function wrappers and decorators: Creating robust decorators that preserve signatures and metadata.
    • Monkey patching: Providing utilities like post-import hooks to safely patch code.
    • Object proxies: Creating transparent proxies for Python objects.

    To maintain high performance, wrapt uses a C extension for critical components, with an automatic fallback to a pure Python implementation if a compiler is unavailable.

  2. Type Hinting in wrapt

    develop

    As of version 2.0.0, wrapt includes type hints for its public APIs to improve interoperability with static type checkers like pyright, mypy, and ty.

    Compatibility Notes:

    • Python Version: Type metadata is available when running on Python 3.10 or later.
    • Best Experience: pyright (used within the VS Code Pylance extension) provides the best experience.
    • Limitations: mypy and ty may have limitations when dealing with static methods of classes decorated with wrapt decorators. pyrefly currently does not work correctly with decorators applied to class methods.
    • Dynamic Nature: Because decorators are dynamic, some patterns cannot be expressed precisely. You may occasionally need to add explicit annotations or helper functions to guide the type checker.
  3. Understand the limitations of basic Python decorators

    develop

    A basic Python decorator implemented via a function closure or a class instance often breaks introspection and the Python object model. Common issues include:

    • Loss of Metadata: The wrapped function's __name__ and __doc__ attributes are replaced by the wrapper's attributes.
    • Broken Introspection: Tools like inspect.getargspec() or inspect.getsource() will return information about the wrapper instead of the original function, or fail entirely.
    • Descriptor Protocol Failure: Simple decorators can break @classmethod and @staticmethod because they do not correctly handle descriptors that need to be bound to a class or instance before being called.
    • Attribute Errors: Using functools.wraps or functools.update_wrapper on certain objects (like classmethod objects in older Python versions) can raise AttributeError if attributes like __module__ are missing.
  4. Understand C Extension vs. Pure Python implementations

    develop

    For performance, several core classes in wrapt have two implementations: a C extension and a pure Python fallback in wrapt.wrappers.

    When you import from the top-level wrapt module, the C extension is used by default if available. The pure Python version is used only if the C extension cannot be imported.

    Classes with C implementations:

    • BaseObjectProxy (exposed as ObjectProxy in the C extension)
    • CallableObjectProxy
    • PartialCallableObjectProxy
    • FunctionWrapper
    • BoundFunctionWrapper

    All other features (like wrapt.decorator, wrapt.synchronized, and wrapt.ObjectProxy) are pure Python code that transparently use whichever implementation of these core classes is active.

  5. Use version-specific test files

    develop

    To ensure certain tests only run on appropriate Python versions, use a specific naming convention. The tests/conftest.py configuration automatically detects these suffixes and skips the tests if the current Python version is too old.

    Naming Convention

    • test_name_py310.py: Runs only on Python 3.10 and later.
    • test_name_py312.py: Runs only on Python 3.12 and later.

    Standard test files named test_*.py will run on all supported Python versions.

  6. Understand the performance trade-offs of wrapt.decorator

    develop

    The wrapt module prioritizes correctness and universal compatibility over raw speed. Unlike standard decorators that use function closures, wrapt implements wrappers as classes that act as descriptors. This ensures decorators work correctly across all scenarios (functions, methods, classmethods, and staticmethods) but introduces additional runtime overhead compared to simple function closures.

    Key architectural differences:

    • Function Closures: Faster, but cannot be easily wrapped around staticmethod or classmethod decorators (they must be placed inside them).
    • wrapt.decorator: Highly compatible and handles descriptor protocols correctly, but has higher overhead. Performance varies depending on whether the C extension is installed.
  7. Understand __qualname__ divergence between Python and C implementations

    develop

    The pure-Python and C implementations of ObjectProxy handle the __qualname__ attribute differently:

    • Pure-Python implementation: Takes a snapshot of the wrapped object's __qualname__ at construction time. If the wrapped object's __qualname__ is mutated directly later, the proxy still returns the original value.
    • C extension implementation: Performs a live-read of the wrapped object's __qualname__ on every access.

    Consistency: If you set __qualname__ through the proxy (e.g., proxy.__qualname__ = "new"), both implementations will agree because the change is applied to the wrapped object and the local snapshot (in Python) simultaneously.

    import wrapt
    
    def foo(): pass
    
    proxy = wrapt.ObjectProxy(foo)
    foo.__qualname__ = "Changed"
    
    # Pure-Python: proxy.__qualname__ returns the original value (snapshot)
    # C extension: proxy.__qualname__ returns "Changed"
  8. Understand performance overhead when decorating methods

    develop

    When using wrapt decorators on class methods (instance methods, class methods, or static methods), there is additional overhead compared to decorating simple functions. This is because wrapt implements decorators as descriptors to correctly honor the Python execution model (ensuring proper binding of methods to instances).

    Key Performance Insights:

    • Pure Python Implementation: A pure Python implementation of the wrapt decorator factory incurs significantly higher overhead (e.g., ~6.67 $\mu$s per call) because it must handle descriptor logic via __get__ to manage method binding.
    • C Extension Implementation: Using the wrapt C extension significantly reduces this overhead (e.g., ~0.836 $\mu$s per call), making it much closer to the performance of a simple function closure decorator.
    • Why the overhead exists: To be 'correct' and avoid breaking code during monkey patching, wrapt must perform extra steps during the __get__ call to ensure the method is properly bound to the class or instance.

    If you are performing high-frequency calls (e.g., thousands of calls per web request), the difference between a pure Python implementation and the C extension implementation can be the difference between milliseconds and microseconds of added latency.

  9. Understand the interaction between decorators and descriptors

    develop

    In Python, decorators can fail when applied to objects that implement the descriptor protocol, such as @classmethod or @staticmethod.

    The Problem

    A standard decorator (often implemented as a function closure) typically wraps a function and calls it directly. However, descriptors like @classmethod do not have a __call__ method themselves; instead, they implement __get__ to return a bound method object that is callable.

    If a decorator is applied on top of a descriptor:

    @decorator
    @classmethod
    def my_method(cls): ...

    The decorator attempts to call the @classmethod object directly, resulting in a TypeError: 'classmethod' object is not callable because it bypassed the descriptor's __get__ method.

    The Solution

    To correctly support descriptors, a decorator must itself be a descriptor. It must implement the __get__ method to ensure that when the decorated object is accessed as a class attribute, the descriptor protocol is propagated correctly.

  10. Perform type comparisons with `isinstance()` on proxies

    develop

    The type of an ObjectProxy instance is ObjectProxy (or a derived class). However, ObjectProxy implements __class__ to return the class of the wrapped object.

    To correctly check the type of the underlying object, use isinstance() instead of direct type comparison. isinstance() will return True for the wrapped type and also for wrapt.ObjectProxy itself.

    >>> value = 1
    >>> proxy = wrapt.ObjectProxy(value)
    >>> isinstance(proxy, int)
    True
    >>> proxy.__class__
    <class 'int'>
    >>> isinstance(proxy, wrapt.ObjectProxy)
    True
  11. How to use the wrapt public API

    develop

    The wrapt package follows a strict public API contract. All intended public features are re-exported from the top-level wrapt module and listed in wrapt.__all__.

    To ensure your code is compatible with future updates, you should always import names directly from the wrapt module rather than from its submodules. Submodules like wrapt.wrappers, wrapt.decorators, wrapt.synchronization, etc., are considered private implementation details and their internal structure may change without notice.

    For example, if a decorator is moved from wrapt.decorators to wrapt.synchronization in a new release, code using import wrapt and @wrapt.decorator_name will continue to work, while code that imported from the submodule directly will break.

    import wrapt
    
    @wrapt.synchronized
    def function():
        ...
  12. Handle explicit instance passing in instance methods

    develop

    When an instance method is called directly via the class (e.g., Class.method(instance, *args)), the decorator factory must ensure the instance is correctly identified rather than being treated as the first element of args.

    To achieve this, the bound_function_wrapper should check if self.instance is None. If it is, it should pop the first element from args to use as the instance and then use functools.partial to bind that instance to the wrapped function before passing it to the user's wrapper.

    class bound_function_wrapper(object_proxy):
        def __call__(self, *args, **kwargs):
            if self.instance is None:
                # Handle Class.method(instance, ...)
                instance, args = args[0], args[1:]
                wrapped = functools.partial(self.wrapped, instance)
                return self.wrapper(wrapped, instance, args, kwargs)
            return self.wrapper(self.wrapped, self.instance, args, kwargs)