Apache Mahout Documentation
repository·main·Indexed 25 days ago
https://github.com/apache/mahoutAn environment for creating scalable and performant machine learning applications. It includes Qumat, a high-level Python library for quantum machine learning (v0.7.0.dev0), and QDP (Quantum Data Plane) for high-performance quantum data encoding. QDP features a trait-based architecture for reading quantum data from formats such as Parquet, Arrow IPC, NumPy, and PyTorch, supporting both batch and streaming reading via DataReader and StreamingDataReader traits.
What's inside Apache Mahout
- QuMat is a high-level Python library designed to interface with multiple quantum computing backends. It provides an abstraction layer that allows developers to write quantum code once and run it across different backends without worrying about backend-specific implementation details.
Overview of the QDP Python API (qumat_qdp)
mainThe
qumat_qdppackage is the primary Python interface for Quantum Data Processing (QDP). It provides a unified encoding facade, benchmark builders, and data loaders for quantum machine learning workflows.Key components include:
QdpEngine: The main entry point for encoding data using CUDA or AMD backends.QdpBenchmark: A builder for measuring throughput and latency.QuantumDataLoader: A builder for iterating over encoded batches (synthetic or file-backed).QdpTensor/QuantumTensor: A DLPack-based facade for accessing backend-native tensors in PyTorch.- Backend Selection: Supports native Rust/CUDA (
backend="cuda"), Triton AMD (backend="amd"), and a PyTorch reference implementation (backend="pytorch").
Overview of the QuMat Codebase
mainThe
qumatcodebase provides a unified interface for quantum circuit simulation across multiple backends, including Qiskit, Cirq, and Amazon Braket.Core components include:
- Backend Modules: Specific modules (e.g.,
qiskit_backend.py,cirq_backend.py,amazon_braket_backend.py) that handle backend-specific circuit manipulation. - QuMat Class (
qumat.py): An abstraction layer that provides methods to create circuits and apply standard quantum gates.
Currently, users can create circuits, apply fixed gates (NOT, Hadamard, CNOT, Toffoli, SWAP, Pauli X/Y/Z), execute circuits to get measurement results, and draw circuits (backend-dependent).
- Backend Modules: Specific modules (e.g.,
Introduction to QDP (Quantum Data Plane)
mainQDP (Quantum Data Plane) is a high-performance, GPU-accelerated library designed for encoding classical data into quantum states. It serves as a data plane for quantum machine learning workflows.Overview of Qumat for Quantum Computing
mainQumat is a high-level Python library designed for quantum computing workflows. It provides two primary capabilities:
Quantum Circuit Abstraction: Allows you to build quantum circuits using standard gates (such as Hadamard, CNOT, and Pauli) through a unified API. This abstraction enables you to write code once and execute it across different backends, including Qiskit, Cirq, or Amazon Braket.
QDP (Quantum Data Plane): Facilitates encoding classical data into quantum states using GPU-accelerated kernels. It supports zero-copy tensor transfers via DLPack, allowing efficient data movement between PyTorch, NumPy, and TensorFlow without overhead.
Access Mahout communication channels (Slack, IRC, and Archives)
mainMahout maintains several real-time and asynchronous communication channels:
- Slack: Used for discussion, but decisions made here should be moved to the
dev@mailing list or GitHub to ensure visibility for non-Slack users. - IRC: The channel name is
#mahout(though real-time discussion has largely moved to Slack). - Mailing List Archives: You can view historical discussions via the Official Apache Archive:
https://mail-archives.apache.org/mod_mbox/mahout-dev/https://mail-archives.apache.org/mod_mbox/mahout-user/
- Slack: Used for discussion, but decisions made here should be moved to the
Explore Apache Mahout Python APIs
mainApache Mahout provides experimental Python APIs for quantum computing and machine learning tasks. The current API documentation is organized into two primary packages:
- Qumat: Focused on quantum circuit execution and management.
- QDP: Focused on encoding data for Quantum Machine Learning (QML).
Note that these APIs are currently experimental and the documentation is evolving. Developers should focus on the public interfaces provided in these packages for their integration needs.
Overview of Qumat and QDP
mainApache Mahout includes two primary components for quantum computing workflows:
- Qumat: A quantum circuit abstraction layer used for defining and managing quantum circuits.
- QDP (Quantum Data Plane): A GPU-accelerated data encoding layer used for preparing data for Quantum Machine Learning (QML).
What is Qumat and how does it prevent vendor lock-in?
mainQumat is an Apache Mahout project designed to allow users to write quantum circuits and algorithms once using a vendor-neutral Python library. These circuits can then be transpiled to run on various quantum computing backends, including IBM'sqiskit, Google'scirq, and AmazonBraket. This approach follows the philosophy of the Apache Mahout Samsara project, which provides a vendor-neutral language for machine learning to avoid dependency on specific hardware or software providers.Understand engine maintenance changes in Mahout 14.1
mainIn version 14.1, several execution engines have been moved to the
communitymodule. This means the code remains available for use, but it is no longer actively maintained by the core Mahout team.Engines moved to
communityinclude:- H2o
- Flink-batch
- Map Reduce
Users can still contribute to these engines via Pull Requests.
What QDP produces: GPU-resident state vectors
mainQDP encodes classical data into a quantum state vector $\vert\psi\rangle$ for $n$ qubits. The output is always a 2D tensor on the GPU with the following characteristics:
- State length: $2^{n}$
- Shape:
- Single sample:
[1, 2^n] - Batch:
[batch_size, 2^n]
- Single sample:
- Precision:
complex64(2×float32) when output precision isfloat32complex128(2×float64) when output precision isfloat64
How backend selection works in qumat_qdp
mainQDP uses two distinct backend-selection surfaces that serve different purposes:
Unified Engine Routing: Used with
QdpEngine(..., backend=...)."cuda"routes to the native_qdpCUDA engine."amd"routes to the Triton AMD implementation.
Builder Fallback/Reference Routing: Used via
.backend(...)onQdpBenchmarkorQuantumDataLoader."rust"(default) uses the high-performance Rust pipeline."pytorch"uses the pure-PyTorch reference implementation for validation or fallback.
Note on
force_backend: Theforce_backend(backend: Backend | None)function only affects the value returned byget_backend()and the cachedBACKENDconstant at import time. It does not change which implementationQdpBenchmarkorQuantumDataLoaderuse; for those, you must use the.backend()method on the builders.