threadpoolctl Documentation

threadpoolctl provides Python helpers to limit the number of threads used by the thread-pool-backed implementations of common native libraries used in scientific computing and data science, such as BLAS and OpenMP.

Fine control over the underlying thread-pool size is particularly useful for workloads involving nested parallelism to mitigate oversubscription issues (where too many threads compete for CPU resources, degrading performance).

Threading Constraint

threadpoolctl is currently designed for situations where BLAS and OpenMP are only called from the main Python thread.

To ensure consistent behavior, threadpoolctl and the BLAS/OpenMP APIs should only ever be called from the same, single Python thread.

Safe use cases include:

• Configuring a worker in a process pool, where the worker then calls BLAS or OpenMP APIs directly in its main thread.
• Running code in a Jupyter notebook cell's main thread.

Unsafe use cases:

• Calling BLAS/OpenMP APIs and threadpoolctl from multiple different Python threads. This will lead to inconsistent results.

To control threadpools for native libraries not currently supported (beyond OpenMP and main BLAS), you can implement a custom controller.

1. Subclass LibController.
2. Implement the required attributes and methods defined in the LibController docstring.
3. Register the new class using threadpoolctl.register(YourCustomController).

To use threadpoolctl in your projects, install the latest version from PyPI using pip.

pip install threadpoolctl

On Linux and macOS, loading a mix of compiled extensions linked with libomp (LLVM/Clang) and libiomp (ICC) can cause crashes or deadlocks. This is an unrecoverable incompatibility.

Note: Using threadpoolctl may trigger these crashes in this specific environment. This issue typically arises when using packages from certain distributions (like conda's default channel) that use LLVM/Clang alongside packages linked against Intel MKL.

For FlexiBLAS (a BLAS wrapper), you can switch the backend at runtime. This feature is currently experimental. You can switch to a backend predefined at build time (found in available_backends) or provide a direct path to a shared library.

Warning: This API is experimental and subject to change.

from threadpoolctl import ThreadpoolController
import numpy as np

controller = ThreadpoolController()
# Retrieve the flexiblas controller from the loaded libraries
flexiblas_ct = controller.select(internal_api="flexiblas").lib_controllers[0]

# Switch to a predefined backend
flexiblas_ct.switch_backend("OPENBLASPTHREAD")

# Or switch by providing a direct path to a shared library
flexiblas_ct.switch_backend("/path/to/libmkl_rt.so")

Use threadpool_info() to get a list of dictionaries describing the current state of threadpool-enabled runtime libraries loaded in the process. Alternatively, use the object-oriented ThreadpoolController class and its .info() method for similar results.

from threadpoolctl import threadpool_info
from pprint import pprint

# Get info as a list of dicts
pprint(threadpool_info())

# Using the object-oriented API
from threadpoolctl import ThreadpoolController
controller = ThreadpoolController()
pprint(controller.info())

Both threadpool_limits and ThreadpoolController can be used as decorators to set thread limits for a specific function. Use the .wrap() method to create the decorator.

from threadpoolctl import ThreadpoolController, threadpool_limits
import numpy as np

controller = ThreadpoolController()

@controller.wrap(limits=1, user_api='blas')
def my_func():
    # BLAS calls inside this function are limited to 1 thread
    a = np.random.randn(1000, 1000)
    return a @ a

# Alternatively using threadpool_limits directly:
@threadpool_limits.wrap(limits=1, user_api='blas')
def my_func_alt():
    pass

You can limit the number of threads used by underlying runtime libraries in specific code blocks using threadpool_limits or ThreadpoolController.limit(). This is useful for preventing oversubscription when using thread-parallelism alongside BLAS/OpenMP calls.

Note: ThreadpoolController will not act on libraries loaded after the controller is instantiated.

from threadpoolctl import threadpool_limits
import numpy as np

# Using the context manager
with threadpool_limits(limits=1, user_api='blas'):
    a = np.random.randn(1000, 1000)
    a_squared = a @ a

# Using the object-oriented API
from threadpoolctl import ThreadpoolController
controller = ThreadpoolController()
with controller.limit(limits=1, user_api='blas'):
    a = np.random.randn(1000, 1000)
    a_squared = a @ a