Bottleneck Documentation

repository·master·Indexed 22 days ago

https://github.com/pydata/bottleneck

Bottleneck is a collection of fast NumPy array functions written in C, optimized for performance when dealing with NaN values and moving window operations. It provides specialized subpackages for reduction operations (e.g., nansum, nanmean), moving window operations (e.g., move_sum, move_mean), and non-reduction operations. The library includes a benchmark suite via the bottleneck.benchmark package to compare execution speeds against NumPy.

Tokens
1.9K
Snippets
5
Records
17
Agent score
79%

What's inside Bottleneck

  1. Overview of Bottleneck subpackages

    master

    Bottleneck is organized into several subpackages that provide specialized functions for fast NumPy array operations. Key functional areas include:

    • bottleneck.reduce: Contains fast implementations of reduction operations (e.g., sum, mean, min, max).
    • bottleneck.move: Provides fast moving window operations (e.g., moving sum, moving mean).
    • bottleneck.nonreduce: Contains non-reduction operations that operate on arrays.
    • bottleneck.nonreduce_axis: Specialized non-reduction operations that operate along a specific axis.
    • bottleneck.benchmark: Tools for benchmarking Bottleneck performance.
    • bottleneck.slow: Fallback implementations for environments where the optimized C extensions are unavailable.
  2. Understand Bottleneck licensing

    master

    Bottleneck is distributed under a Simplified BSD license.

    Note that Bottleneck includes components from other projects which carry their own licenses:

    • NumPy: BSD license
    • SciPy: BSD license
    • Setuptools: MIT license (used for configuration and installation)
  3. Use the bottleneck.benchmark package for performance testing

    master

    The bottleneck.benchmark package provides tools for benchmarking Bottleneck's performance. It is organized into several submodules designed for different levels of benchmarking detail:

    • bottleneck.benchmark.autotimeit: Automated timing utilities.
    • bottleneck.benchmark.bench: Standard benchmarking tools.
    • bottleneck.benchmark.bench_detailed: Detailed benchmarking for granular performance analysis.

    Users can access these modules to run performance tests and compare execution speeds of various Bottleneck operations.

  4. Use Bottleneck for fast NumPy operations

    master

    Bottleneck provides fast NumPy array functions written in C. It is particularly useful for handling NaN values and performing moving window operations.

    Note on Acceleration: Only arrays with dtype int32, int64, float32, and float64 are accelerated. Other dtypes will use slower, unaccelerated functions. Additionally, byte-swapped input arrays (e.g., big-endian on a little-endian system) will not be accelerated.

    import numpy as np
    import bottleneck as bn
    
    a = np.array([1, 2, np.nan, 4, 5])
    
    # Find the nanmean
    print(bn.nanmean(a))
    # Output: 3.0
    
    # Moving window mean
    print(bn.move_mean(a, window=2, min_count=1))
    # Output: array([ 1. ,  1.5,  2. ,  4. ,  4.5])
  5. Install Bottleneck from source

    master

    To install from source, ensure you meet the requirements:

    • Python: >3.9
    • NumPy: 1.16.0+
    • Compiler: gcc, clang, MinGW, or MSVC

    Linux and macOS

    Use pip to install the current directory:

    pip install .

    Windows

    1. Install MinGW and add it to your system path.
    2. Run the installation using the following command:
    python setup.py install --compiler=mingw32
  6. Install Bottleneck via Anaconda

    master

    The easiest way to install Bottleneck is using Anaconda or Miniconda, which provides a pre-compiled version by default.

    Note: For users seeking optimal performance, it may be beneficial to uninstall the pre-compiled version and install from source using the pip method described in other guides to allow local compilers to perform optimizations.

  7. Install Build Dependencies for Bottleneck

    master

    Because Bottleneck is distributed as a source package to allow for local compiler optimizations, you must install build dependencies before installing the package via pip.

    Debian & Ubuntu

    Install gcc and python3-dev:

    sudo apt install gcc python3-dev

    (Note: python3-dev can be skipped if using Anaconda.)

    RHEL, Fedora & CentOS

    Install gcc and python3-devel:

    sudo yum install gcc python3-devel

    Windows

    Refer to the Python Wiki for instructions on which Visual Studio version to install: https://wiki.python.org/moin/WindowsCompilers

  8. Update pip and setuptools before installing Bottleneck

    master

    Bottleneck leverages PEP 517. It is recommended to update pip and setuptools before installation to ensure compatibility and leverage recent improvements.

    Using Anaconda:

    conda update setuptools pip

    Using pip:

    pip install --upgrade setuptools pip
  9. Run Bottleneck benchmarks

    master

    Bottleneck includes a benchmark suite to compare its performance against NumPy. Speed is reported as NumPy time / Bottleneck time.

    To run the standard benchmark suite:

    import bottleneck as bn
    bn.bench()

    To run a detailed benchmark for a specific function, use bn.bench_detailed(). For example, to benchmark move_median with a specific fraction of NaNs:

    import bottleneck as bn
    bn.bench_detailed("move_median", fraction_nan=0.3)
    import bottleneck as bn
    
    # Run standard benchmark
    bn.bench()
    
    # Run detailed benchmark for a single function
    bn.bench_detailed("move_median", fraction_nan=0.3)