DOCSEARCHbyAlgolia
Submit Docs

Velox Execution Engine

repository·main·Indexed 26 days ago

https://github.com/facebookincubator/velox

A high-performance, composable C++ execution engine library for data management systems. Velox provides optimized components for vectorized execution, memory management, and I/O, including a generic typing system, Arrow-compatible columnar memory layout, and relational operators. It is designed for integration into compute engines and includes PyVelox Python bindings. Note that Velox is not a standalone database and does not include a SQL parser or optimizer.

Tokens
188.8K
Snippets
355
Records
1K
Agent score
87%

What's inside Velox

  1. Overview of Velox GPU Acceleration components

    main

    Velox provides several experimental components for GPU execution located under velox/experimental/. These components fall into three categories:

    1. Execution Backends: Run Velox operators on the GPU.
      • Wave: A whole-pipeline backend that JIT-compiles contiguous operator runs into fused CUDA kernels.
      • cuDF: An operator-level backend that replaces operators one-to-one using NVIDIA libcudf.
    2. Portable Primitives Library (Breeze): Provides data-parallel building blocks (e.g., reduce, scan, sort) used to build GPU kernels. It is multi-platform (CUDA, HIP, OpenCL, etc.).
    3. GPU-to-GPU Data Exchange (UCX exchange): A transport mechanism that shuffles data between workers directly between GPUs, avoiding host memory staging.

    Note: All GPU components are experimental. Operators not supported by a backend will automatically fall back to CPU execution.

  2. Overview of SetDigest data sketches

    main

    A SetDigest is a data sketch used for estimating set cardinality and performing set operations such as intersection cardinality and Jaccard index. It combines HyperLogLog for cardinality estimation with MinHash for exact counting and intersection operations.

    Key characteristics:

    • Internal Types: Supports bigint (all numeric types are converted to this) and varchar (for strings).
    • Accuracy: When cardinality is below a maximum hash limit, the digest is 'exact' and returns precise results. As cardinality increases, it transitions to approximate estimation.
    • Compatibility: The serialization format is compatible with Presto.
    • Storage: SetDigests can be cast to and from VARBINARY for storage and retrieval.

  4. Overview of Velox Memory Management

    main

    Velox uses a custom memory management system built on std::mmap to avoid std::malloc fragmentation. The system provides:

    • Optimized Allocation: Supports large contiguous buffers (e.g., for HashTable::allocateTables) and small non-contiguous buffers using arena techniques (e.g., StreamArena, HashStringAllocator).
    • Fair Memory Sharing: Uses a MemoryArbitrator to adjust query capacities at runtime, ensuring total memory stays within system limits and preventing individual queries from exceeding per-query limits via techniques like disk spilling.
    • Transparent File Cache: An integrated cache for accelerating table scans that dynamically shares memory with query execution by shrinking when queries need more space.
    • OOM Prevention: Manages physical memory via std::mmap to control the Resident Set Size (RSS) and prevent server-level Out-Of-Memory (OOM) events.

  5. Overview of Velox execution engine

    main

    Velox is a composable, high-performance C++ execution engine library designed for building data management systems (batch, interactive, stream processing, and AI/ML).

    Note for Users: Velox is not a standalone database; it does not provide a SQL parser, a dataframe layer, or a query optimizer. It is intended for developers integrating and optimizing compute engines by providing a fully optimized query plan as input.

    Core Components:

    • Type: Generic typing system (scalar, complex, nested types like structs, maps, arrays).
    • Vector: Arrow-compatible columnar memory layout (Flat, Dictionary, Constant, Sequence/RLE).
    • Expression Eval: Vectorized expression evaluation engine.
    • Functions: Vectorized scalar, aggregate, and window functions (Presto/Spark semantics).
    • Operators: Relational operators (scans, writes, projections, filtering, joins, etc.).
    • I/O: Connector interface for file formats (ORC, Parquet, Nimble) and storage (S3, HDFS, GCS, ABFS, local).
    • Network Serializers: Wire protocol interfaces (PrestoPage, Spark UnsafeRow).
    • Resource Management: Memory arenas, buffer management, tasks, drivers, and thread pools.

  6. Overview of Axiom Composable Query Engines

    main
    Axiom is a C++ library built on top of Velox designed for building fully composable, high-performance query engines. It decomposes the query engine into independent, reusable components: frontends, optimizer, runtime, connectors, and execution. These components are connected via stable, engine-agnostic APIs, specifically the logical plan (between frontends and the optimizer) and the physical plan (between the optimizer and the runtime). This architecture allows developers to swap or extend components, such as adding a new SQL dialect or a new deployment mode (local, streaming, or batch), without rebuilding the entire stack.

  7. Overview of Velox-cuDF

    main
    Velox-cuDF is an extension module for Velox that provides a GPU-accelerated backend using the cuDF library. It integrates with libcudf (the CUDA C++ core of cuDF) to implement the Velox DriverAdapter interface via CudfDriverAdapter. This allows Velox to rewrite query plans for GPU execution, replacing CPU operators with GPU-accelerated ones. It uses Arrow-compatible data layouts and relies on Velox's pipeline-based execution model to manage concurrent work on the GPU.

  8. Understand the Velox Tracing Framework components

    main

    The Velox tracing framework is designed to record execution data in production or shadow environments and replay it locally for debugging. It consists of three main functional layers:

    1. Trace Writers: Components that record metadata, input data, and scan splits during real execution.
    2. Trace Readers: Components that load the recorded metadata, input data, and scan splits for analysis or replay.
    3. Trace Replayers: Tools that use the readers to display query summaries or replay the execution of a specific target operator.

    Tracing memory usage is managed via a dedicated system pool named tracePool, accessible via memory::MemoryManager::getInstance()->tracePool().

  10. Understand Velox Task, Split, and PlanNode relationships

    main

    In Velox, a query fragment within a worker node is represented as a velox::exec::Task. A Task is defined by two primary components:

    • Plan: velox::core::PlanNode specifies the logic and operations the Task must perform.
    • Splits: velox::exec::Split specifies the specific data the Task operates on.

    Common split types include:

    • velox::connector::ConnectorSplit: Used in initial stage tasks (like table scans) to identify specific pieces of files to read.
    • velox::exec::RemoteConnectorSplit: Used in subsequent stages to identify a running Task from which to read input.

    The distributed engine provides the PlanNode and Split definitions, which Velox then uses to instantiate and execute the Task.

  11. Understand Task, Driver, and Operator lifecycles

    main

    In Velox, the lifetime hierarchy is strictly ordered: Task > Driver > Operator.

    • Task: The top-level entity that outlives both Drivers and Operators. It holds shared ownership of Drivers via drivers_ and SplitGroupState.barriers.
    • Driver: Owned by the Task. A Driver holds exclusive ownership of its DriverCtx (which contains a std::shared_ptr<Task>) and its Operators. A Driver's lifetime cannot exceed the Task's lifetime due to this circular reference, which is cleared when the Task drops its references to the Drivers.
    • Operator: Does not hold a direct reference to a Task; instead, it accesses the Task via a raw pointer to the DriverCtx provided by the Driver.

    Key Lifecycle Management:

    • Circular references between Tasks and Drivers are cleared when Task::terminate is called or when Task::removeDriver is invoked (e.g., via Driver::close).
    • Task::terminate acts as a catch-all to clear references in the event of abnormal or early termination.

  12. Understand Velox Query Plan Nodes and Operators

    main

    Velox represents a query plan as a tree of PlanNode objects. To execute this plan, Velox converts the tree into a set of linear pipelines. A pipeline is a sequence of operators derived from a linear sub-tree of the plan.

    When converting nodes to operators, Velox follows these rules:

    • One-to-one mapping: Most nodes map to a single operator.
    • Node fusion: A FilterNode followed by a ProjectNode is fused into a single FilterProject operator.
    • Node splitting: Nodes with multiple children are split into multiple operators. For example, a HashJoinNode is converted into a pair of operators: HashProbe and HashBuild.
    • Source Operators: Operators corresponding to leaf nodes in the plan tree are called source operators. Only specific plan nodes can serve as leaves.