DOCSEARCHbyAlgolia
Submit Docs

Python Free-threaded Compatibility

repository·main·Indexed 43 days ago

https://github.com/quansight-labs/free-threaded-compatibility

A coordination hub for ecosystem-wide work on Python free-threading compatibility. Provides guides for implementing support in Python packages, debugging concurrency issues with pytest, GDB, LLDB, and sanitizers, and configuring dependencies like CFFI for Python 3.14+.

Tokens
35.3K
Snippets
67
Records
107
Agent score
55%

What's inside Free-threaded Compatibility

  1. Overview of the Python Free-Threading Guide

    main

    The Python Free-Threading Guide is a centralized resource for Python package maintainers and end users interested in supporting or experimenting with free-threaded CPython (PEP 703).

    Key resources provided by this guide include:

    • Compatibility Status Tracker: A tracker for the compatibility status of packages that include native code (tracking.md).
    • Installation & Execution Guides: Instructions for installing free-threaded Python and running it with the GIL disabled.
    • Porting Guides: Documentation for updating pure Python packages and native extensions to support free-threading.
    • Testing & Debugging: Guidance on improving multithreaded test coverage, setting up CI, and debugging thread safety issues.

  2. Examples of performance gains with free-threaded Python

    main

    This repository provides several examples demonstrating how to use free-threaded Python to improve performance by utilizing the Python threading module. In these scenarios, the free-threaded build outperforms the standard GIL-enabled build because it avoids the lock contention that prevents the GIL-enabled build from scaling well with multiple threads.

    Internal Examples

    • Mandelbrot Set Visualization: Demonstrates computational scaling.
    • Monte Carlo Simulation: Demonstrates statistical simulation scaling.
    • Web Scraping with asyncio: Demonstrates I/O bound tasks in a free-threaded environment.

    External Use Cases

    • Web Services: Benchmarking web services on free-threaded builds.
    • GPU Data Loading: Improving data loading on GPUs using threads.
    • Multithreaded Aggregation: Using AtomicDict.reduce() for efficient aggregation.

  3. Understand Limited API constraints in free-threaded builds

    main

    The free-threaded build does not support the CPython Limited API.

    If your extension uses the limited API to build version-independent wheels, you cannot use it when shipping binaries for the free-threaded build. You must build against the full C API instead of setting Py_LIMITED_API at build time.

    However, code inside #ifdef Py_GIL_DISABLED blocks can use full C API constructs that are outside the limited API, provided those uses are removed once the free-threaded build gains official support for the limited API.

  4. How native extensions interact with the free-threaded CPython runtime

    main

    In the free-threaded build, the GIL is disabled, allowing multiple threads to simultaneously call into the CPython C API. However, threads must still explicitly attach and detach from the interpreter runtime.

    Key Concepts:

    • Attached Threads: Threads that have an attached thread state and can use interpreter state. In the free-threaded build, many threads can be attached and executing Python code simultaneously.
    • Detached Threads: Threads that have detached from the runtime (e.g., via Py_BEGIN_ALLOW_THREADS or PyGILState_Release).
    • Why Detach?: Even without a GIL, you must detach during long-running I/O or computations. The free-threaded interpreter uses a 'stop-the-world' garbage collection scheme that requires all threads to be detached before it can proceed. If threads do not detach, the interpreter may be blocked.
    • API Compatibility: The code used to attach/detach (like PyGILState_Ensure and PyGILState_Release) is identical to the code used in GIL-enabled builds. Despite the 'GIL' naming, these functions manage thread state attachment/detachment in the free-threaded build.

  5. Implement lock-free concurrency with atomics

    main

    For high-performance scenarios where mutex contention is a concern, use atomic operations instead of locks.

    • C++ and Rust: It is highly recommended to use the built-in atomics provided by the respective standard libraries.
    • C: Use C11/C17 standard library atomics.
    • MSVC Note: To enable experimental support for C atomics in MSVC, use the /experimental:c11atomics compiler flag.

    Warning: Writing lock-free code is technically difficult. Always test atomic-based code using Thread Sanitizer on multiple architectures, especially ARM, due to differences in memory ordering semantics.

  6. Configure TSan suppressions

    main

    To prevent TSan from reporting known, non-critical, or external data races, use a suppressions file.

    1. Create a file containing race:<function_name> entries for each race to ignore.
    2. Set the suppressions option in the TSAN_OPTIONS environment variable pointing to your file.

    Example suppression file content:

    race:llvm::RuntimeDyldELF::registerEHFrames
race:partial_vectorcall_fallback
race:dnnl_sgemm
    TSAN_OPTIONS="suppressions=$PWD/tsan-suppressions" python my_test.py

  7. Avoid anti-patterns in free-threading documentation

    main

    To maintain high-quality documentation, avoid these common mistakes:

    • Vague Statements: Avoid using the term "thread-safe" in isolation. It is too ambiguous. Instead, specify exactly what is atomic and what requires synchronization.
    • Implementation Details in User Docs: Do not explain the internal mechanics (e.g., how the GIL is replaced) in user-facing documentation. Instead, state the functional impact (e.g., "Free-threaded Python removes the GIL, enabling parallel execution") and refer to PEP 703 for technical details.
    • Scattered Thread Safety Notes: Do not mention thread safety inconsistently across many different function documentations. Use a centralized thread safety page and link to it.

  8. Implement lock-free sharing with Copy-on-Write (CoW)

    main

    Copy-on-Write (CoW) is a thread-safe pattern for lock-free sharing of data structures, ideal when reads are much more frequent than writes (e.g., caching).

    Implementation Pattern

    1. Atomic Read: Copy the shared reference to a local variable at the start of the operation.
    2. Copy on Write: When a change is needed, create a new copy of the data structure, mutate the copy, and then atomically update the global shared reference to point to the new version.
    3. Avoid In-place Mutation: Never modify the shared object directly.

    Critical Requirement

    Readers must not assume the shared reference remains unchanged between accesses. You must capture the reference in a local variable first to ensure consistency.

    Incorrect (Not Thread-Safe):

    if nth < len(cache):
    return cache[nth] # 'cache' might be replaced by another thread between the check and the access

    Correct (Thread-Safe):

    local_cache = cache
if nth < len(local_cache):
    return local_cache[nth]
    cache = [0, 1]

def fib(nth: int) -> int:
    global cache
    if nth < 1:
        raise ValueError("nth must be a positive integer")

    # Atomically read shared reference to global cache
    local_cache = cache

    if nth > len(local_cache) + 1:
        # Make a new un-shared list
        local_cache = local_cache.copy()

        while nth >= len(local_cache):
            local_cache.append(local_cache[-1] + local_cache[-2])

        # Atomically update global shared reference
        cache = local_cache

    return local_cache[nth]

  9. Handle thread-safety for the warnings module

    main

    The warnings module's functionality for tracking warnings is inherently thread-unsafe on Python versions prior to 3.14.

    • Python 3.13 and older: warnings.catch_warnings is not thread-safe.
    • Python 3.14 (GIL-enabled): Thread-safe warnings are not the default.
    • Python 3.14 (Free-threaded): Thread-safe warnings are enabled by default.

    If running tests via pytest-run-parallel, it will automatically mark tests using warnings as thread-unsafe on Python 3.13 or older, or on 3.14 if thread-safe warnings are not configured. If using a custom multithreaded execution mechanism, you must skip warning checks unless the interpreter is configured for thread-safe warnings using the following configuration options:

    • PYTHON_CONTEXT_AWARE_WARNINGS
    • PYTHON_THREAD_INHERIT_CONTEXT

  10. Convert global state to thread-local state

    main

    In free-threaded Python, C static global variables are susceptible to data races. To prevent memory corruption and segfaults, convert global state to thread-local state using one of the following methods:

    • Python/Cython: Use threading.local.
    • C/C++ (Python API): Use the Py_tss API (Thread-Specific Storage).
    • C/C++ (Standard): Use the thread_local storage specifier. Note that on MSVC, you should only use thread_local for statically defined integers and pointers.
    • NumPy Users: Use the NPY_TLS macro provided in numpy/npy_common.h (available in NumPy 2.1 or newer).

  12. Coordinate documentation for free-threaded Python

    main

    The dedicated free-threading community documentation meeting is currently on hold. To discuss improving CPython documentation for the free-threaded build, participate in the regular meetings coordinated by the CPython Documentation Workgroup.

    If you are working on documentation for a community project and need to coordinate or chat, use the Free-threaded Python Community Discord.