QKeras Documentation

repository·master·Indexed 20 days ago

A quantization extension for Keras that provides drop-in replacement layers (such as QDense, QConv2D, and QLSTM) and activation functions to enable the training and deployment of quantized deep learning models. It includes QTools for hardware mapping and energy consumption estimation, AutoQKeras for automated quantization hyperparameter search via Keras-Tuner, and codebook-based quantization for compressing activations and model weights.

Tokens: 7.2K
Snippets: 11
Records: 22
Agent score: 67%

What's inside QKeras

What is QKeras and how to use it
master
QKeras is a quantization extension for Keras designed to provide drop-in replacements for standard Keras layers. It allows you to quickly create deep quantized versions of Keras networks by replacing variable-creating layers (like Dense or Conv2D) with their quantized counterparts (like QDense or QConv2D) and applying quantization to arithmetic operations and activations.
To successfully quantize a model, you must:
1. Replace variable-creating layers with QKeras equivalents.
2. Quantize any layers that perform mathematical operations.
```
from keras.layers import *
from qkeras import *

# Example of replacing a standard layer with a quantized one
x = QConv2D(18, (3, 3),
        kernel_quantizer="stochastic_ternary",
        bias_quantizer="ternary", name="first_conv2d")(x)
```
https://github.com/google/qkeras/blob/master/README.md
Automate quantization with AutoQKeras
master
AutoQKeras treats quantization and rebalancing as a hyperparameter search problem. It uses Keras-Tuner (supporting random search, hyperband, or Gaussian processes) to automatically find optimal quantization parameters for an existing deep neural network.
To manage the large hyperparameter space, users can group tasks by patterns and perform distributed training. Detailed documentation is available in the notebook/AutoQKeras.ipynb notebook.
https://github.com/google/qkeras/blob/master/README.md
Use QTools for hardware mapping and energy estimation
master
QTools assists in hardware implementation and energy consumption estimation for quantized models.
Data Type Map Generation
Generates a map for weights, bias, multipliers, adders, etc., including operation type, variable size, and quantizer details.
- Inputs: A quantized model and a list of input quantizers.
- Outputs: A JSON file (via qtools_stats_to_json) or a printed map (via qtools_stats_print).
- Example location: qkeras/qtools/examples/example_generate_json.py
Energy Consumption Estimation
Estimates energy consumption in Pico Joules (pJ) for memory access and MAC operations. This is useful for comparing power consumption between two models on the same device.
- Energy Model: Based on 45nm process data from Horowitz M.
- Usage: Can be integrated into a total loss function to balance accuracy and energy cost.
- Example location: qkeras/qtools/examples/example_get_energy.py
https://github.com/google/qkeras/blob/master/README.md
What is QKeras?
master
QKeras is a quantization extension for Keras that provides drop-in replacements for Keras layers. It allows developers to quickly create deep quantized versions of Keras networks by replacing standard layers (like Dense or Conv2D) with quantized equivalents (like QDense or QConv2D) and adding quantization to activation layers and arithmetic operations. It is designed to be minimally intrusive to native Keras functionality.
https://github.com/google/qkeras/blob/master/notebook/QKerasTutorial.ipynb
Quantized Layers in QKeras
master
QKeras implements several quantized versions of standard Keras layers. If quantizers are not specified, most layers behave like their unquantized counterparts (except QBatchNormalization).
Implemented layers include:
- QDense
- QConv1D
- QConv2D
- QDepthwiseConv2D
- QSeparableConv2D (implemented as depthwise + pointwise expansions)
- QActivation (merged quantization and activation function)
- QAveragePooling2D (implemented as AveragePooling2D stacked with QActivation)
- QBatchNormalization
- QOctaveConv2D
Note: Some high-level operations like Bidirectional wrappers may not be fully compatible with all quantized layers. In such cases, it is recommended to use quantization functions invoked as strings.
https://github.com/google/qkeras/blob/master/notebook/QKerasTutorial.ipynb
Reduce search space by grouping layers in AutoQKeras
master
If the quantization search space is too large, you can group layers using regular expressions in the limit dictionary. This forces multiple layers to share the same quantization choice.
Note that layer class names (like "Conv2D") select different quantizers by default, so you must use layer name patterns (e.g., "^conv2d_") to group them.
Example of grouping convolution layers and activations:
```
limit = {
    "Conv2D": [4, 8, 4], # Default for all Conv2D
    "^conv2d_0$": [["binary", "ternary"], 8, 4], # Specific group for first conv
    "^conv2d_[1234]$": [4, 8, 4], # Group for subsequent convs
    "^act_[0123]$": [4], # Group for activations
}
```
https://github.com/google/qkeras/blob/master/notebook/AutoQKeras.ipynb
How QKeras quantization works
master
QKeras achieves quantization by tagging variables (weights/biases) and the outputs of arithmetic layers with quantized functions.
To quantize a model, you must follow two main patterns:
1. Replace Variable-Creating Layers: Replace standard Keras layers that create trainable or non-trainable variables (e.g., LSTM, Conv2D, Dense) with their QKeras equivalents (e.g., QLSTM, QConv2D, QDense).
2. Quantize Math Operations: Any layer performing mathematical operations should be followed by a quantization step. This is typically done using QActivation, which acts as a merged quantization and activation function.
Note on Clipping: Quantized values are clipped between their maximum and minimum quantized representations. For po2 (power-of-two) type quantizers, it is recommended to explicitly specify the max_value parameter.
https://github.com/google/qkeras/blob/master/notebook/QRNNTutorial.ipynb

Example: Quantizing a Keras network

master

This example demonstrates how to convert a standard Keras CNN into a quantized QKeras model using QConv2D, QSeparableConv2D, QDense, and QActivation layers with specific quantizers.

from keras.layers import *
from qkeras import *

x = x_in = Input(shape)
x = QConv2D(18, (3, 3),
        kernel_quantizer="stochastic_ternary",
        bias_quantizer="ternary", name="first_conv2d")(x)
x = QActivation("quantized_relu(3)")(x)
x = QSeparableConv2D(32, (3, 3),
        depthwise_quantizer=quantized_bits(4, 0, 1),
        pointwise_quantizer=quantized_bits(3, 0, 1),
        bias_quantizer=quantized_bits(3),
        depthwise_activation=quantized_tanh(6, 2, 1))(x)
x = QActivation("quantized_relu(3)")(x)
x = Flatten()(x)
x = QDense(NB_CLASSES,
        kernel_quantizer=quantized_bits(3),
        bias_quantizer=quantized_bits(3))(x)
x = QActivation("quantized_bits(20, 5)")(x)
x = Activation("softmax")(x)

Quantized layers implemented in QKeras
master
QKeras provides several quantized layer implementations that serve as replacements for standard Keras layers:
- Convolutional Layers: QConv1D, QConv2D, QDepthwiseConv2D, QSeparableConv1D, QSeparableConv2D, QMobileNetSeparableConv2D, QConv2DTranspose, QOctaveConv2D.
- Dense Layers: QDense.
- Recurrent Layers: QSimpleRNN, QSimpleRNNCell, QLSTM, QLSTMCell, QGRU, QGRUCell, QBidirectional.
- Other Layers: QActivation, QAdaptiveActivation, QAveragePooling2D (implemented as AveragePooling2D stacked with a QActivation layer), and QBatchNormalization (experimental).
Note on Layer Wrappers: Some functionality may not be safe with high-level wrappers (e.g., Bidirectional wrappers for RNNs). If you encounter issues, use quantization functions invoked as strings instead of the actual functions.
https://github.com/google/qkeras/blob/master/README.md
Quantized activation functions in QKeras
master
QKeras implements various activation functions for quantization. Many stochastic functions (like stochastic_binary or quantized_relu) draw a random number from a uniform distribution based on the _hard_sigmoid of the input to achieve the expected value of the activation function.
Key Parameters:
- bits: The number of bits for quantization.
- integer: The number of bits to the left of the decimal point.
- symmetric: Used to generate symmetric ranges (e.g., [-1, 1)) to help convergence and eliminate bias.
Available Activations:
- Sigmoid/Tanh variants: smooth_sigmoid(x), hard_sigmoid(x), binary_sigmoid(x), binary_tanh(x), smooth_tanh(x), hard_tanh(x).
- Quantized/Stochastic variants: quantized_bits(bits=8, integer=0, symmetric=0, keep_negative=1)(x), bernoulli(alpha=1.0)(x), stochastic_ternary(alpha=1.0, threshold=0.33)(x), ternary(alpha=1.0, threshold=0.33)(x), stochastic_binary(alpha=1.0)(x), binary(alpha=1.0)(x), quantized_relu(bits=8, integer=0, use_sigmoid=0, negative_slope=0.0)(x), quantized_ulaw(bits=8, integer=0, symmetric=0, u=255.0)(x), quantized_tanh(bits=8, integer=0, symmetric=0)(x), quantized_po2(bits=8, max_value=-1)(x), quantized_relu_po2(bits=8, max_value=-1)(x).
Range Adjustment Rule: If a quantization for weights or bias generates numbers outside the range [-1.0, 1.0], you must adjust the *_range to match. For example, quantized_bits(bits=6, integer=2) requires a weight range of $2^2$.
https://github.com/google/qkeras/blob/master/README.md

Manually Create a Quantized Model

master

To create a quantized model manually, replace standard Keras layers with Q-prefixed layers and specify kernel_quantizer, bias_quantizer, or use QActivation for activations.

from qkeras import *

def CreateQModel(shape, nb_classes):
    x = x_in = Input(shape)
    x = QConv2D(18, (3, 3),
        kernel_quantizer="stochastic_ternary", 
        bias_quantizer="quantized_po2(4)",
        name="conv2d_1")(x)
    x = QActivation("quantized_relu(2)", name="act_1")(x)
    x = QConv2D(32, (3, 3), 
        kernel_quantizer="stochastic_ternary", 
        bias_quantizer="quantized_po2(4)",
        name="conv2d_2")(x)
    x = QActivation("quantized_relu(2)", name="act_2")(x)
    x = Flatten(name="flatten")(x)
    x = QDense(nb_classes,
        kernel_quantizer="quantized_bits(3,0,1)",
        bias_quantizer="quantized_bits(3)",
        name="dense")(x)
    x = Activation("softmax", name="softmax")(x)

    model = Model(inputs=x_in, outputs=x)
    return model

Replace Keras layers with QKeras layers

master

To manually build a quantized model, replace standard Keras layers with QKeras layers and specify quantizers for kernels, recurrent activations, and biases.

Common QKeras layers include:

QLSTM
QGRU
QSimpleRNN
QBidirectional
QDense
QConv2D
QSeparableConv2D
QActivation (for merging quantization and activation)

Example of a quantized recurrent model:

def create_qmodel(batch_size=None):
  x = x_in = Input(shape=(maxlen,), batch_size=batch_size, dtype=tf.int32)
  x = Embedding(input_dim=max_features, output_dim=embedding_dim)(x)
  x = QActivation('binary', name='embedding_act')(x)
  x = QLSTM(
    units,
    activation='quantized_tanh(4)',
    recurrent_activation='quantized_relu(4,0,1)',
    kernel_quantizer='stochastic_ternary("auto")',
    recurrent_quantizer='quantized_bits(2,1,1,alpha=1.0)',
    bias_quantizer='quantized_bits(4,0,1)')(x)
  x = QDense(
    1, 
    kernel_quantizer="quantized_bits(4,0,1)",
    bias_quantizer='quantized_bits(4,0,1)')(x)
  x = QActivation('sigmoid')(x)
  model = tf.keras.Model(inputs=[x_in], outputs=[x])
  return model

QKeras Documentation

What's inside QKeras

Data Type Map Generation

Energy Consumption Estimation