DOCSEARCHbyAlgolia
Submit Docs

hls4ml Documentation

repository·main·Indexed 24 days ago

https://github.com/fastmachinelearning/hls4ml

A package for machine learning inference on FPGAs that translates models (such as Keras) into High-Level Synthesis (HLS) code for ultra-low-latency inference. It supports multiple backends including Xilinx Vivado HLS, Vitis HLS, Intel HLS, and Catapult HLS. Key features include automatic precision inference, Distributed Arithmetic (DA) optimization via da4ml, external BRAM weight loading using BramFactor, and an Extension API for implementing custom layers.

Tokens
36.8K
Snippets
62
Records
154
Agent score
83%

What's inside hls4ml

  1. What is PQuantML?

    main
    PQuantML is a hardware-aware model compression framework designed for deploying compact neural networks on resource-constrained hardware like FPGAs and embedded accelerators. It supports joint pruning and quantization, layer-wise precision configuration, and flexible training pipelines for both PyTorch and Keras V3. It is designed to integrate with hardware-friendly toolchains such as hls4ml.

  2. What is hls4ml and how does it work?

    main

    hls4ml is a tool designed to translate machine learning models from open-source frameworks (such as Keras and PyTorch) into High Level Synthesis (HLS) code. This HLS code can then be transpiled to run on FPGAs.

    Key capabilities include:

    • Rapid Prototyping: Enables fast iteration of ML algorithms for hardware by automating the conversion from software models to hardware design languages.
    • Hardware Optimization: Leverages FPGA architectures to implement optimizations like pipelining and parallel evaluation.
    • Deployment Options: The resulting HLS project can be used to produce an IP core for complex designs or a kernel for CPU co-processing.
    • Customization: Users can define various algorithm parameters to balance the trade-offs between performance, resource utilization, and latency.

  3. Supported ML Frontends and Architectures

    main

    hls4ml supports several machine learning frameworks (Frontends) and neural network architectures.

    Supported Frontends

    • Keras:
      • Keras v2 (including QKeras and HGQ)
      • Keras v3 (including QKeras-v3 and HGQ2)
    • PyTorch (Torch)
    • ONNX (including QONNX)

    Supported Architectures

    • Fully connected NN (MLP)
    • Convolutional NN (1D and 2D)
    • Recurrent NN (RNN, LSTM, GRU)
    • GarNet
    • Einsum and EinsumDense
    • Multi-head attention (MHA) (experimental)

    Supported HLS Backends

    • Vivado HLS
    • Intel HLS
    • Vitis HLS
    • Catapult HLS
    • oneAPI (experimental)

  4. What is High Granularity Quantization (HGQ2)?

    main

    HGQ2 (High Granularity Quantization 2) is a quantization-aware training (QAT) framework built on Keras v3. It is designed for real-time deep learning applications on edge devices like FPGAs and integrates directly with hls4ml for hardware synthesis.

    Key capabilities include:

    • Multi-backend support: Uses Keras v3 to work with TensorFlow, JAX, and PyTorch.
    • Flexible quantization: Supports schemes like fixed-point and minifloat.
    • Gradient-based optimization: Allows for bitwidth optimization at arbitrary granularity (per-weight and per-activation levels).
    • Effective Bit-Operations (EBOP): Provides accurate resource estimation for deployed firmware during the training phase.
    • Advanced layer support: Supports layers such as einsum, einsum dense, and multi-head attention with quantization and hardware synthesis support.

  5. How flows and optimizer registration work

    main

    A hls4ml.model.flow.flow.Flow is an ordered sequence of optimizers representing a single stage in the conversion process.

    Execution Logic:

    1. Optimizers within a flow are applied in sequence.
    2. If an optimizer's transform method returns True, the flow is re-run until no more changes are detected.
    3. Once a flow stabilizes, the next flow in the sequence begins.

    Key API Methods for Customization:

    • hls4ml.model.optimizer.optimizer.register_pass: Use this to register a new optimizer pass.
    • hls4ml.model.flow.flow.register_flow: Use this to register a new flow.
    • hls4ml.model.graph.ModelGraph.apply_flow: Use this to apply a registered flow to a model graph.

    Flow Types:

    • Backend-agnostic flows: Common model-level flows that run regardless of the target hardware.
    • Backend-specific flows: Flows tailored for specific hardware (e.g., the Vivado backend's IP flow which requires several sub-flows to run first).

  6. Choose between Vivado and Vitis backends for AMD/Xilinx FPGAs

    main

    When targeting AMD/Xilinx FPGAs, hls4ml provides two backends:

    • Vitis backend: Targets the Vitis HLS compiler. This is the recommended backend for new designs. While the full Vitis accelerator flow is not directly supported, the HLS produced can be easily incorporated into a Vitis kernel.
    • Vivado backend: Targets the discontinued Vivado HLS compiler. This backend is primarily for legacy support; new hls4ml developments may not be backported to it.

    Both backends are designed to produce IP that can be incorporated into Vivado designs.

  7. How optimizer passes work in hls4ml

    main

    To prepare a model for code generation, hls4ml performs a series of transformations called optimization passes. These passes modify the internal model graph or layer attributes.

    All optimizer passes must derive from the hls4ml.model.optimizer.optimizer.OptimizerPass class. They operate using two main functions:

    • match: A function that returns a boolean indicating if the pass should be applied to a specific node/layer.
    • transform: Performs the actual transformation. It returns a boolean indicating if the model graph was modified. If transform returns True, the current flow of optimizers may be re-run to ensure all dependencies are satisfied.

    Types of Optimizer Passes:

    • General/Backend-agnostic: Located in hls4ml.model.optimizer.passes.
    • Backend-specific: Located in backend-specific modules, e.g., hls4ml.backends.vivado.passes or hls4ml.backends.quartus.passes.
    • GlobalOptimizerPass: Matches every node in the graph (e.g., for type transformations).
    • LayerOptimizerPass: Matches nodes of a specific layer type (e.g., for generating HLS code for a specific node).
    • ModelOptimizerPass: Applied to the entire model rather than individual nodes (e.g., hls4ml.model.optimizer.passes.stamp.MakeStamp).
    • ConfigurableOptimizerPass: An optimizer that accepts configurable parameters.
    • Template: An optimizer that populates a code template and assigns it to a layer attribute, used for generating code blocks in later stages.

  8. Ensure correct input precision for BitExact

    main

    When using the BitExact pass, quantizers should be placed immediately after the inputs to ensure input precision is properly inferred.

    • HGQ/HGQ2: This is typically handled automatically.
    • QKeras: You must manually ensure a QActivation with quantized_bits is placed right after the input layer so that the input precision can be derived.

  9. Configure Convolution Layer implementations

    main

    Convolutional layers (Conv1D/2D) are implemented based on the chosen io_type:

    io_parallel

    Best for small models requiring low latency.

    • Vivado/Vitis: Uses the im2col transformation to turn convolution into matrix-multiplication. You can control parallelism via ParallelizationFactor. Use the Resource strategy for larger models.
    • Catapult: Uses a direct implementation via nested loops.
    • Winograd: Available in Quartus, oneAPI, and Catapult backends by setting implementation to Winograd or combination. This can be faster but is less concerned with bit accuracy and overflow.

    io_stream

    Processes data serially (one pixel at a time).

    • LineBuffer: The default implementation; generally produces better results.
    • Encoded: Available in Catapult and Vivado backends, configurable via the ConvImplementation option.

    Special Convolution Types

    • Depthwise convolution: Replaces matrix-vector multiplication with elementwise multiplication. Only the Latency strategy is available.
    • Pointwise convolution: For 1x1 (2D) or 1 (1D) filters. The Vivado/Vitis backend has a dedicated io_parallel/Latency strategy where the reuse_factor (RF) splits execution and limits multipliers. Note: in_width must be divisible by the RF.

  10. Supported Keras versions and quantization frameworks

    main

    hls4ml supports both Keras v2 (tf.keras) and Keras v3. The mechanism for conversion differs by version:

    • Keras v2: Conversion is based on parsing the serialized JSON representation of the model.
    • Keras v3: Conversion uses direct model inspection.

    Supported quantization frameworks include:

    • QKeras: Supported via its equivalent API and quantizers. Note that QKeras is not compatible with Keras v3.
    • HGQ: Supported, but deprecated in favor of HGQ2.
    • HGQ2: Supported, including additional advanced operators like QEinsum, QMultiHeadAttention, and QUnaryFunctionLUT (arbitrary unary function as a 1-d lookup table).

  11. How MultiModelGraph works for modular hardware design

    main

    The MultiModelGraph class allows you to divide a large neural network into multiple smaller subgraphs (each a ModelGraph) by specifying split points at certain layers. This is useful for managing very large models, enabling step-wise optimization, or implementing modular design flows.

    Key benefits include:

    • Parallelism: Subgraphs can be built in parallel.
    • Stitched Designs: Subgraphs can be synthesized independently and then merged (stitched) in hardware.
    • Simulation: You can perform simulation and performance estimation on the complete stitched design.
    hls_multigraph_model = hls4ml.model.to_multi_model_graph(hls_model, ['layer3', 'layer7'])

  12. SNN Backend and Execution Model

    main

    The SNN flow in hls4ml has the following characteristics:

    • Backend Support: Currently supports only the Vitis backend.
    • Execution Model: Implementations are synchronous (clock-driven). Neuron state updates and layer computations run in standard HLS pipelines/streams each cycle according to interface handshakes. It is not a native asynchronous/event-routed neuromorphic architecture.