xtensor

repository·master·Indexed 26 days ago

https://github.com/xtensor-stack/xtensor

A C++ library for numerical analysis using multi-dimensional array expressions. It features an extensible expression system with lazy broadcasting, an API inspired by the C++ standard library, and compatibility with NumPy, Julia, and R data structures. The library provides xarray for dynamic dimensionality and xtensor for compile-time dimensionality, along with specialized bindings for BLAS, I/O, and language integration.

Tokens
50.5K
Snippets
124
Records
325
Agent score
86%

What's inside xtensor

  1. Introduction to xtensor

    master
    xtensor is a C++ library designed for numerical analysis using multi-dimensional array expressions. It features an extensible expression system that enables lazy broadcasting and provides an API that follows C++ standard library idioms. The library's containers are inspired by NumPy, and it supports processing NumPy data structures in-place via Python's buffer protocol (through the xtensor-python project).
  2. Understand xtensor containers and views

    master

    In xtensor, containers are in-memory expressions that implement the xexpression API. While the primary user-facing classes are xt::xarray and xt::xtensor (which handle constructors and value semantics), the core functionality of the xexpression API is implemented in the underlying xstrided_container and xcontainer classes.

    Views (such as xview, xstrided_view, etc.) allow for non-owning manipulations of data, whereas containers like xarray own the underlying data.

  3. Use z5 for Zarr and N5 storage in C++

    master
    The z5 project implements the zarr and n5 storage specifications in C++. It uses xtensor to represent arrays in memory. This is useful for chunked nd-array storage that leverages the filesystem for parallel write access and efficient cloud-based storage. It also provides a Python wrapper via xtensor-python.
  4. Use xtensor language bindings for Python, Julia, and R

    master

    xtensor provides specialized bindings to wrap native arrays from other languages into xtensor containers, allowing for in-place modification and reshapes:

    • Python: Use xtensor-python to get pyarray and pytensor containers which wrap NumPy arrays. It also includes utilities to generate NumPy-style universal functions from scalar functions.
    • Julia: Use xtensor-julia to get jlarray and jltensor containers which wrap Julia arrays. It includes utilities to generate NumPy-style universal functions.
    • R: Use xtensor-r to get rarray and rtensor containers which wrap R arrays. It includes utilities to generate NumPy-style universal functions.
  5. Understand xtensor closure semantics

    master

    xtensor uses lazy evaluation. Operations like x + y do not return a container, but an expression that holds references, const references, or copies of the operands. These operands are called closure types.

    To avoid dangling references:

    • When an argument is an rvalue, the closure type is a value (the rvalue is moved).
    • When an argument is an lvalue reference, the closure type is a reference to that type.

    If you want to avoid the complexities of lazy evaluation and closure semantics, you can use xt::eval() to return an evaluated container instead of an expression.

  6. Use xtensor library bindings for BLAS and I/O

    master

    Extend xtensor functionality using these specialized library bindings:

    • Linear Algebra: Use xtensor-blas to provide bindings to BLAS libraries, enabling linear-algebra operations directly on xtensor expressions.
    • File I/O: Use xtensor-io to load various file formats into xtensor expressions, including image files, sound files, HDF5 files, and NumPy .npy and .npz files.
  7. Understand xtensor lazy evaluation and expression trees

    master

    Most expressions in xtensor are lazy-evaluated, meaning they do not hold values immediately. Instead, they represent a node in an expression tree that computes values only upon access or when assigned to a container.

    Nodes in the tree are often represented by the xfunction template class, which stores:

    1. A functor describing the mathematical operation.
    2. The closures of the child expressions (the most optimal way to store each child, such as constant references or moved rvalues).
  8. Iterate over slices along a specified axis using xtensor iterators

    master

    Beyond the standard iterators provided by expression types, xtensor provides specialized iterator classes designed to iterate over slices of an expression along a specific axis. This allows for more granular control when traversing multidimensional data structures. The available iterator types include:

    • xaxis_iterator: For iterating over elements along a specific axis.
    • xaxis_slice_iterator: For iterating over slices along a specific axis.
  9. Understand xtensor tensor types

    master

    xtensor provides several container types depending on whether you need dynamic or static shapes:

    • xarray<T>: A tensor that can be reshaped to any number of dimensions (dynamic shape).
    • xtensor<T, N>: A tensor where the number of dimensions N is fixed at compile time.
    • xtensor_fixed<T, xshape<I, J, K>>: A tensor where both the number of dimensions and the specific shape are fixed at compile time.
    • xchunked_array<CS>: A chunked array using the specified CS chunk storage.

    Most methods described in the documentation apply to xarray, xtensor, and xtensor_fixed unless otherwise specified.

  10. Optimize xtensor build on Windows

    master

    Windows users must activate the /bigobj flag to prevent compilation failures. For optimization, it is recommended to link against xtensor::optimize and disable the manifest.

    If XTENSOR_USE_XSIMD is enabled, you must also specify a target instruction set (e.g., /arch:AVX2, /arch:AVX, or /arch:ARMv7VE).

    target_link_libraries(... xtensor xtensor::optimize)
    set(CMAKE_EXE_LINKER_FLAGS /MANIFEST:NO)
    
    # OR
    
    target_compile_options(target_name PRIVATE /EHsc /MP /bigobj)
    set(CMAKE_EXE_LINKER_FLAGS /MANIFEST:NO)
    
    # If using XSIMD, specify instruction set:
    target_compile_options(target_name PRIVATE /arch:AVX2)
  11. Load and save NPY (NumPy) data with xtensor

    master

    Use xt::load_npy to load data from a .npy file and xt::dump_npy to save xtensor data to a .npy file. When calling xt::load_npy, you must provide the template argument for the data type being loaded.

    #include <istream>
    #include <iostream>
    #include <fstream>
    
    #include <xtensor/containers/xarray.hpp>
    #include <xtensor/io/xnpy.hpp>
    
    int main()
    {
        // Note: you need to supply the data type you are loading
        //       in this case "double".
        auto data = xt::load_npy<double>("in.npy");
    
        xt::xarray<double> a = {{1,2,3,4}, {5,6,7,8}};
        xt::dump_npy("out.npy", a);
    
        return 0;
    }