calflops: PyTorch FLOPs and MACs Calculator

calflops is a tool designed to compute theoretical FLOPs (floating-point operations), MACs (multiply-add operations), and Parameters for various neural network architectures.

Supported Architectures:

• Linear layers
• CNN (Convolutional Neural Networks)
• RNN (Recurrent Neural Networks)
• GCN (Graph Convolutional Networks)
• Transformers (including BERT, LLaMA, and other Large Language Models)
• Custom models: Any model based on a PyTorch implementation using torch.nn.function.* is supported.

Key Features:

• Provides submodule-level granularity: It can print the FLOPs, Parameter count, and the proportion of total consumption for each submodule, helping users identify performance bottlenecks.

calflops is a tool designed to theoretically calculate FLOPs (Floating Point Operations), MACs (Multiply-Accumulate operations), and the number of model parameters for various neural networks.

Key features include:

• Broad Model Support: Supports Linear, CNN, RNN, GCN, and Transformer models (including large language models like BERT and LLaMA).
• Custom Model Support: Can calculate metrics for any custom model as long as it is implemented using torch.nn.function.* operations in PyTorch.
• Granular Analysis: Can print the FLOPs, MACs, and parameter counts for each submodule, including their relative proportions, to help identify performance bottlenecks.
• LLM Convenience: For large Transformer models, it can automatically construct the required input_shape if you provide the corresponding transformers_tokenizer.

If a model cannot be inferred on a meta device, use the standard calflops.calculate_flops() function.

To handle model inputs automatically, you can pass a transformers_tokenizer to the function. calflops will then automatically construct the input data based on the provided input_shape.

Alternatively, you can manually pass in pre-constructed input data for models that require multiple inputs.

When analyzing Large Language Models (LLMs) with calflops, the following metrics are provided:

• fwd FLOPs: The Floating Point Operations required for a single forward propagation.
• bwd + fwd FLOPs: The combined FLOPs for both forward and backward propagation.

Important Note on Calculation Accuracy: According to arXiv:2205.05198, the fwd + bwd metric provided by calflops does not include the computational overhead of model parameter activation. To include this overhead and align with the paper's methodology, you should multiply the fwd result by 4.

Typical input data format for LLM benchmarks in this project is batch_size=1, seq_len=128.

When reviewing results from calflops, note the following definitions:

• fwd FLOPs: The Floating Point Operations for the model's forward propagation.
• bwd + fwd FLOPs: The combined Floating Point Operations for both forward and backward propagation.
• MACs: Multiply-Accumulate operations.
• Total (numerical representation): In benchmark tables, 'Total' refers to the absolute numerical value without unit abbreviations (like G or M).

You can customize the scope of the calculation using these parameters:

• include_backPropagation: Set to True to include backward pass computations. Defaults to False (forward pass only).
• compute_bp_factor: Determines the ratio of backward to forward computation. Defaults to 2.0.
• forward_mode: Set to 'generate' if you want to calculate the FLOPs for the model.generate() method instead of the standard forward pass.

By default, calflops prints a summary. You can enable more granular reporting using these parameters:

• print_results=True: Prints the model profile (default is True).
• print_detailed=True: Prints the calculation results and the proportion of FLOPs, MACs, and Parameters for every submodule in the model (default is True).

You can install or upgrade to the latest version of calflops using PyPI.

To install the latest version:

pip install --upgrade calflops

Alternatively, you can download the .whl files from PyPI and install them locally:

pip install calflops-*-py3-none-any.whl

For models with a single input (like standard CNNs), you can provide an input_shape tuple. calflops will automatically generate a random tensor of that shape to perform the calculation. Use output_as_string=True and output_precision to format the returned values.

from calflops import calculate_flops
from torchvision import models

model = models.alexnet()
batch_size = 1
input_shape = (batch_size, 3, 224, 224)
flops, macs, params = calculate_flops(model=model, 
                                      input_shape=input_shape,
                                      output_as_string=True,
                                      output_precision=4)
print("Alexnet FLOPs:%s   MACs:%s   Params:%s \n" %(flops, macs, params))

If you want to use specific, manually constructed data (e.g., from a tokenizer) instead of auto-generated inputs, pass the data via the kwargs parameter. When using kwargs, do not provide an input_shape.

# Assuming 'inputs' is a dictionary of tensors from a tokenizer
flops, macs, params = calculate_flops(model=model,
                                      kwargs = inputs,
                                      print_results=False)

For models with a single input (like standard CNNs), you can provide the input_shape parameter. calflops will automatically generate a random input tensor of that shape to perform the calculation.

If your model has multiple inputs, you must use the args (for positional arguments) or kwargs (for keyword arguments) parameters instead of input_shape.

from calflops import calculate_flops
from torchvision import models

model = models.alexnet()
batch_size = 1
input_shape = (batch_size, 3, 224, 224)
flops, macs, params = calculate_flops(model=model, 
                                      input_shape=input_shape,
                                      output_as_string=True,
                                      output_precision=4)
print("Alexnet FLOPs:%s   MACs:%s   Params:%s \n" %(flops, macs, params))