Apache Mahout Documentation

repository·main·Indexed 25 days ago

https://github.com/apache/mahout

An environment for creating scalable and performant machine learning applications. It includes Qumat, a high-level Python library for quantum machine learning (v0.7.0.dev0), and QDP (Quantum Data Plane) for high-performance quantum data encoding. QDP features a trait-based architecture for reading quantum data from formats such as Parquet, Arrow IPC, NumPy, and PyTorch, supporting both batch and streaming reading via DataReader and StreamingDataReader traits.

Tokens
64.2K
Snippets
137
Records
351
Agent score
80%

What's inside Apache Mahout

  1. Overview of QuMat

    main
    QuMat is a high-level Python library designed to interface with multiple quantum computing backends. It provides an abstraction layer that allows developers to write quantum code once and run it across different backends without worrying about backend-specific implementation details.
  2. Overview of the QDP Python API (qumat_qdp)

    main

    The qumat_qdp package is the primary Python interface for Quantum Data Processing (QDP). It provides a unified encoding facade, benchmark builders, and data loaders for quantum machine learning workflows.

    Key components include:

    • QdpEngine: The main entry point for encoding data using CUDA or AMD backends.
    • QdpBenchmark: A builder for measuring throughput and latency.
    • QuantumDataLoader: A builder for iterating over encoded batches (synthetic or file-backed).
    • QdpTensor / QuantumTensor: A DLPack-based facade for accessing backend-native tensors in PyTorch.
    • Backend Selection: Supports native Rust/CUDA (backend="cuda"), Triton AMD (backend="amd"), and a PyTorch reference implementation (backend="pytorch").
  3. Overview of the QuMat Codebase

    main

    The qumat codebase provides a unified interface for quantum circuit simulation across multiple backends, including Qiskit, Cirq, and Amazon Braket.

    Core components include:

    • Backend Modules: Specific modules (e.g., qiskit_backend.py, cirq_backend.py, amazon_braket_backend.py) that handle backend-specific circuit manipulation.
    • QuMat Class (qumat.py): An abstraction layer that provides methods to create circuits and apply standard quantum gates.

    Currently, users can create circuits, apply fixed gates (NOT, Hadamard, CNOT, Toffoli, SWAP, Pauli X/Y/Z), execute circuits to get measurement results, and draw circuits (backend-dependent).

  4. Overview of Qumat for Quantum Computing

    main

    Qumat is a high-level Python library designed for quantum computing workflows. It provides two primary capabilities:

    1. Quantum Circuit Abstraction: Allows you to build quantum circuits using standard gates (such as Hadamard, CNOT, and Pauli) through a unified API. This abstraction enables you to write code once and execute it across different backends, including Qiskit, Cirq, or Amazon Braket.

    2. QDP (Quantum Data Plane): Facilitates encoding classical data into quantum states using GPU-accelerated kernels. It supports zero-copy tensor transfers via DLPack, allowing efficient data movement between PyTorch, NumPy, and TensorFlow without overhead.

  5. Access Mahout communication channels (Slack, IRC, and Archives)

    main

    Mahout maintains several real-time and asynchronous communication channels:

    • Slack: Used for discussion, but decisions made here should be moved to the dev@ mailing list or GitHub to ensure visibility for non-Slack users.
    • IRC: The channel name is #mahout (though real-time discussion has largely moved to Slack).
    • Mailing List Archives: You can view historical discussions via the Official Apache Archive:
      • https://mail-archives.apache.org/mod_mbox/mahout-dev/
      • https://mail-archives.apache.org/mod_mbox/mahout-user/
  6. Explore Apache Mahout Python APIs

    main

    Apache Mahout provides experimental Python APIs for quantum computing and machine learning tasks. The current API documentation is organized into two primary packages:

    1. Qumat: Focused on quantum circuit execution and management.
    2. QDP: Focused on encoding data for Quantum Machine Learning (QML).

    Note that these APIs are currently experimental and the documentation is evolving. Developers should focus on the public interfaces provided in these packages for their integration needs.

  7. Overview of Qumat and QDP

    main

    Apache Mahout includes two primary components for quantum computing workflows:

    • Qumat: A quantum circuit abstraction layer used for defining and managing quantum circuits.
    • QDP (Quantum Data Plane): A GPU-accelerated data encoding layer used for preparing data for Quantum Machine Learning (QML).
  8. What is Qumat and how does it prevent vendor lock-in?

    main
    Qumat is an Apache Mahout project designed to allow users to write quantum circuits and algorithms once using a vendor-neutral Python library. These circuits can then be transpiled to run on various quantum computing backends, including IBM's qiskit, Google's cirq, and Amazon Braket. This approach follows the philosophy of the Apache Mahout Samsara project, which provides a vendor-neutral language for machine learning to avoid dependency on specific hardware or software providers.
  9. Understand engine maintenance changes in Mahout 14.1

    main

    In version 14.1, several execution engines have been moved to the community module. This means the code remains available for use, but it is no longer actively maintained by the core Mahout team.

    Engines moved to community include:

    • H2o
    • Flink-batch
    • Map Reduce

    Users can still contribute to these engines via Pull Requests.

  10. What QDP produces: GPU-resident state vectors

    main

    QDP encodes classical data into a quantum state vector $\vert\psi\rangle$ for $n$ qubits. The output is always a 2D tensor on the GPU with the following characteristics:

    • State length: $2^{n}$
    • Shape:
      • Single sample: [1, 2^n]
      • Batch: [batch_size, 2^n]
    • Precision:
      • complex64 (2×float32) when output precision is float32
      • complex128 (2×float64) when output precision is float64
  11. How backend selection works in qumat_qdp

    main

    QDP uses two distinct backend-selection surfaces that serve different purposes:

    1. Unified Engine Routing: Used with QdpEngine(..., backend=...).

      • "cuda" routes to the native _qdp CUDA engine.
      • "amd" routes to the Triton AMD implementation.
    2. Builder Fallback/Reference Routing: Used via .backend(...) on QdpBenchmark or QuantumDataLoader.

      • "rust" (default) uses the high-performance Rust pipeline.
      • "pytorch" uses the pure-PyTorch reference implementation for validation or fallback.

    Note on force_backend: The force_backend(backend: Backend | None) function only affects the value returned by get_backend() and the cached BACKEND constant at import time. It does not change which implementation QdpBenchmark or QuantumDataLoader use; for those, you must use the .backend() method on the builders.