asyncstdlib

asyncstdlib is a toolbox that provides asynchronous re-implementations of Python standard library functions and classes. It is designed to make standard library helpers compatible with async callables, iterables, and context managers.

Key features include:

• Async versions of standard helpers: Provides asynchronous equivalents for functions like zip, map, enumerate, functools.reduce, itertools.tee, and itertools.groupby.
• Safe async iterator handling: Includes helpers to ensure prompt cleanup and simplify the use of custom asynchronous iterators.
• Sync-to-Async integration: Provides tools to integrate existing synchronous code into asynchronous programs and libraries.
• Event loop agnostic: Works seamlessly with asyncio, trio, or any custom asynchronous event loop.

The asyncstdlib.itertools module provides asynchronous versions of Python's standard itertools functions. These functions are designed to work with both synchronous iterables and asynchronous iterables.

Important Resource Management Note: To prevent resource leaks, all utilities in this module explicitly close their iterable arguments when they are finished. This behavior applies even to non-exhausting utilities like dropwhile. You may need to use explicit scoping to manage this behavior correctly.

Converting synchronous iterators: If a function from the standard itertools is not explicitly provided in asyncstdlib, you can convert a synchronous iterator into an asynchronous one using asyncstdlib.iter().

import itertools
import asyncstdlib

# Convert a standard itertools.count to an async iterator
async_count = asyncstdlib.iter(itertools.count(5))

In asyncstdlib, an async neutral type is a type that supports both a regular (synchronous) and an asynchronous implementation.

For example, an async neutral iterable can be used with a standard for _ in iterable loop or an asynchronous async for _ in iterable loop. This pattern is frequently used for callables, where parameters are made async neutral to allow the function to be used easily with a mixture of synchronous and asynchronous arguments.

The asyncstdlib.asynctools module provides utilities to manage the lifecycle of asynchronous iterators.

• borrow(iterator): Returns an asynchronous iterator that ensures the original iterator is properly handled during its lifetime.
• scoped_iter(iterable): An asynchronous context manager that yields an asynchronous iterator. This is useful for ensuring that resources associated with an iterator are cleaned up when the context is exited.

The library is organized into submodules that mirror the Python standard library. You can import from these specific submodules or use the top-level asyncstdlib namespace, which exposes all individual functions and classes directly.

Submodules:

• asyncstdlib.builtins: Async versions of built-in functions (e.g., zip, sum, list).
• asyncstdlib.functools: Async versions of functools tools (e.g., reduce, cached_property, lru_cache).
• asyncstdlib.contextlib: Async versions of contextlib tools (e.g., contextmanager, closing).
• asyncstdlib.itertools: Async versions of itertools tools (e.g., cycle, chain, accumulate).
• asyncstdlib.heapq: Async versions of heapq tools (e.g., merge, nlargest, nsmallest).
• asyncstdlib.asynctools: Core tools for building well-behaved async helpers and programs.

The asynctools library provides several functions for transforming synchronous/asynchronous functions and iterators into unified asynchronous interfaces:

• sync(function): Converts a function (which may be a regular function or an async function) into a consistent asynchronous function.
• any_iter(iter): Transforms an iterator of awaitables into an asynchronous iterator of the yielded values.
• await_each(awaitables): An asynchronous iterator that yields the results of awaiting each item in the provided collection of awaitables.
• apply(func, *args, **kwargs): Asynchronously applies a function to arguments that are themselves awaitables.

In asyncstdlib, async iterators are designed to be resource-safe by default. Most asyncstdlib iterators assume sole ownership of the iterators passed to them. This means that as soon as the asyncstdlib iterator is cleaned up (e.g., after being exhausted), it will automatically call .aclose() on the underlying iterator it was wrapping.

While this is safe for simple exhaustion patterns, it can be problematic if you need to use the same underlying iterator across multiple different utility calls (like multiple calls to islice), as the first utility call might trigger the cleanup of the underlying source. To handle these cases, you should use explicit scoping.

Many objects in asyncstdlib are async neutral, meaning they accept both regular (synchronous) and asynchronous arguments.

How it works:

• Detection: The library determines whether a callable is regular or async by inspecting its return type at runtime. This allows it to support async-producing factories (like an async def function wrapped in functools.partial).
• Constraint: The result must consistently be either regular or async.
• Consistency: While arguments can be async neutral, all asyncstdlib callables themselves consistently return awaitables, asynchronous iterators, or asynchronous context managers.

Cleanup for async iterables is unique because the aclose() method may require an active event loop.

Cleanup Behavior:

• Eager Cleanup: By default, all asyncstdlib utilities that work on async iterators will eagerly call aclose() on them.
• Preventing Automatic Cleanup: If you want to prevent automatic cleanup, use asyncstdlib.asynctools.borrow.
• Guaranteed Cleanup: To guarantee cleanup in your own custom code, use asyncstdlib.asynctools.scoped_iter.

Because asyncstdlib.heapq mimics Python's heapq but for async contexts, it does not implement its own internal key parameter for all operations. Instead, when working with heap structures manually, you should use the (key, item) tuple pattern. If your key function is asynchronous, you must await the key before pushing the tuple onto the heap.

Pattern: heappush(heap, (await key_func(item), item))