Hyperledger Caliper Documentation

Hyperledger Caliper is a blockchain performance benchmark framework. It enables users to test various blockchain solutions using predefined use cases to generate performance test results.

Supported Blockchain Solutions

• Hyperledger Fabric v2.X (via Deprecated Network SDK and Gateway SDK)

Performance Indicators Measured

Caliper tracks several key metrics to evaluate blockchain performance:

• Success rate
• Throughput: Transactions per second (TPS) and Read throughput.
• Latency: Minimum, maximum, and average latency for Transactions and Reads.
• Resource consumption: CPU, Memory, Network IO, and other system resources.

Installing and running Hyperledger Caliper follows a three-step lifecycle:

1. Acquire the Caliper CLI: Obtain the CLI binary via the @hyperledger/caliper-cli NPM package or the hyperledger/caliper Docker image.
2. Bind SDK packages: Execute the bind command through the CLI to pull the specific version of SDK packages required for your target platform.
3. Run the benchmark: Start the benchmark using the CLI or by starting the Docker container.

When using sample artifacts, it is recommended to use the caliper-benchmarks repository as your workspace.

The packages/ directory contains both public/published packages and internal utility packages:

Public/Published Packages:

• caliper-cli: The command line interface (CLI) for Caliper.
• caliper-core: The core and common codebase used by other packages.
• caliper-fabric: The Hyperledger Fabric connector implementation.

Internal Packages:

• caliper-publish: Utility CLI for publishing Caliper to NPM and DockerHub.
• caliper-tests-integration: Collection of CI integration tests.

Caliper commands can pass runtime configuration settings to control the lifecycle of a benchmark. You can use specific flags to skip or isolate certain phases of the execution flow.

Available Phases:

• start
• init
• install
• test
• end

Example Usage: If you have an existing network and want to skip the setup phases and only run the actual benchmark, use the --caliper-flow-only-test flag.

Caliper uses two types of monitoring modules to collect data during test execution for inclusion in generated reports:

1. Resource monitors: Collect statistics on resource utilization (e.g., CPU, memory, I/O) during benchmarking. Monitoring is reset between test rounds.
2. Transaction monitors: Collect worker transaction statistics and can provide conditional dispatch actions.

Resource monitors are configured in the benchmark configuration file using the monitors.resource array.

Caliper uses connector modules to provide a unified interface for different System Under Test (SUT) types. Connectors hide the specific complexities of a blockchain platform from Caliper's internal modules and workload modules.

Instead of writing SUT-specific logic in your workload, you call a simplified connector API (e.g., "initialize the connector/SUT"), and the connector implementation handles the underlying platform-specific tasks (like using a specific SDK or managing channel creation in Hyperledger Fabric).

Workload modules are the core of a Caliper benchmark. They act as the 'brain' of an emulated System Under Test (SUT) client, responsible for constructing and submitting transactions (TXs). They implement the specific business logic, benchmark scenarios, or user behaviors you wish to test.

Workload modules are Node.js modules that must export a factory function named createWorkloadModule. This function is used by Caliper to instantiate the module for each worker process.

/**
 * Create a new instance of the workload module.
 * @return {WorkloadModuleInterface}
 */
function createWorkloadModule() {
    return new MyWorkload();
}

module.exports.createWorkloadModule = createWorkloadModule;

The zero-rate controller stops workload generation for the duration of a round. It is not useful as a standalone controller for a round, but it is a powerful building block for composite-rate controllers to create 'cooldown' periods or pauses in a workload profile.

Important: The zero-rate controller can only be used in duration-based rounds.

{
  "type": "composite-rate",
  "opts": {
    "weights": [30, 10, 10, 30],
    "rateControllers": [
      { "type": "fixed-rate", "opts": {"tps" : 100} },
      { "type": "fixed-rate", "opts": {"tps" : 500} },
      { "type": "zero-rate", "opts": { } },
      { "type": "fixed-rate", "opts": {"tps" : 100} }
    ],
    "logChange": true
  }
}

Because Caliper is in a pre-release lifecycle (< v1.0.0), minor version bumps can introduce breaking changes. Understanding the versioning types is critical for stability:

• Stable Releases (0.7.1): Deemed stable by maintainers. These have corresponding GitHub tags and matching documentation versions.
• Unstable Releases (0.6.1-unstable-20240422122901): Published on every merged pull request. These allow testing of new features from the main branch before they become stable.
• Unstable Tag (unstable): A mutable pointer to the very latest unstable release (the vNext version).

Recommendation: Always use explicit version numbers (e.g., @0.7.1) in your npm install commands to avoid unexpected breaking changes.

Workload modules interact with the Fabric adapter during three distinct phases of a test run:

1. Initialization: Occurs during the initializeWorkloadModule callback. This is where you receive the sutAdapter (the connector) and the sutContext (a FabricConnectorContext instance).
2. Execution: Occurs during the submitTransaction callback, where you use the adapter to send requests to the blockchain.
3. Cleanup: Occurs during the cleanupWorkloadModule callback at the end of the round, used for releasing resources.

The sutAdapter is an instance of ConnectorInterface. For Fabric, calling sutAdapter.getType() returns the string 'fabric'.