Apache bRPC

bRPC (better RPC) is an industrial-grade C++ RPC framework designed for high-performance systems such as search, storage, machine learning, advertising, and recommendation engines.

Key capabilities include:

Multi-protocol Support on a Single Port

You can host or access various protocols on a single port, including:

• HTTP/HTTPS/h2/gRPC: Provides a convenient HTTP implementation (easier than libcurl) and allows accessing Protobuf-based protocols via HTTP/h2+json from other languages.
• Redis & Memcached: Thread-safe clients that are more convenient than official clients.
• Streaming Media: Supports RTMP, FLV, and HLS (can be used to build media servers).
• Other Protocols: Hadoop_RPC, Thrift (thread-safe), RDMA, and various Baidu-specific protocols (e.g., baidu_std, streaming_rpc, sofa_pbrpc, nova_pbrpc, public_pbrpc, ubrpc, and nshead-based protocols).
• Distributed Systems: Supports building high-availability systems using an industrial-grade RAFT implementation (available via braft).

Flexible Request Handling

• Server: Supports both synchronous and asynchronous request processing.
• Client: Supports synchronous, asynchronous, and semi-synchronous access. It also offers combo_channel to simplify complex sharding or concurrent access patterns.

Observability and Debugging

• Built-in HTTP Interface: Debug services via a web interface.
• Profilers: Includes built-in CPU, heap, and contention profilers.
• Customization: Easy to add new protocols, custom components, naming services (DNS, ZK, etcd), and load balancing algorithms (RR, random, consistent hashing).

bvar (better variable) is a library for high-performance, thread-safe monitoring in C++. It provides various classes for different statistical needs.

Common bvar Types

Usage Patterns

• Pushing values: Use the << operator to update variables (e.g., g_read_error << 1;).
• Derived variables: Window and PerSecond are derived from existing bvars and update automatically. Do not push values to them manually.

#include <bvar/bvar.h>

namespace foo {
namespace bar {

// Counter for read errors
bvar::Adder<int> g_read_error;

// Windowed version (60s window) of the error counter
bvar::Window<bvar::Adder<int>> g_read_error_minute("foo_bar", "read_error", &g_read_error, 60);

// Composite variable for latency and QPS
bvar::LatencyRecorder g_write_latency("foo_bar", "write");

// Counter for tasks pushed
bvar::Adder<int> g_task_pushed("foo_bar", "task_pushed");

// Per-second average of tasks pushed (default 10s window)
bvar::PerSecond<bvar::Adder<int>> g_task_pushed_second("foo_bar", "task_pushed_second", &g_task_pushed);

} // namespace bar
} // namespace foo

// In application code:
foo::bar::g_read_error << 1;
foo::bar::g_write_latency << 23; // records 23ms
foo::bar::g_task_pushed << 1;

Based on historical benchmarks, brpc demonstrates high performance across several key metrics compared to other RPC frameworks like UB, Thrift, gRPC, and sofa-pbrpc:

• Throughput (QPS): brpc shows excellent scalability. For single-connection scenarios, it outperforms others for request sizes under 16KB. In multi-connection scenarios, it achieves high throughput (up to 2.3GB/s in tests). It also scales well with increasing thread counts and client counts.
• Latency: brpc maintains low average latency and is highly resistant to 'long-tail' latency (tail latency) issues. It is rarely affected by the performance spikes that impact other frameworks.
• Scalability: brpc exhibits strong multi-threading and multi-client scalability, meaning adding more threads or clients results in a significant increase in total QPS.

Note on Client Implementation: While running brpc clients within bthread can provide a 10%-20% QPS boost and lower latency, the benchmarks provided used standard pthread for a fair comparison with other frameworks.

A ParallelChannel (or "pchan") is a combo channel that sends requests to all its internal sub-channels in parallel and merges their responses into a single result. It provides a unified interface for synchronous and asynchronous access, supports cancellation, and supports timeouts.

Key features:

• Composability: Any subclass of brpc::ChannelBase (including other combo channels) can be added as a sub-channel.
• Failure/Success Limits: You can control when the overall RPC ends early using ParallelChannelOptions:
  • fail_limit: The maximum number of failed responses allowed before the RPC is terminated immediately.
  • success_limit: The maximum number of successful responses required before the RPC is terminated. Note that fail_limit has higher priority; success_limit only takes effect if fail_limit is not set.
• Duplicate Sub-channels: You can add the same sub-channel multiple times to initiate multiple parallel RPCs to the same service.

// Example of setting limits in ParallelChannelOptions
brpc::ParallelChannelOptions options;
options.fail_limit = 4;
// options.success_limit = 10; // Only used if fail_limit is not set

bvar is a high-performance counter library designed for multi-threaded environments. It supports single-dimensional bvar and multi-dimensional mbvar to record various numerical values in user programs.

Key Characteristics

• Performance: It uses thread-local storage to minimize cache bouncing and contention. The write overhead is extremely low (~20ns) and remains constant regardless of the number of threads.
• Trade-off: It shifts the cost from writing to reading. While writing is nearly free, reading requires aggregating data from all threads, making reads slower.

When to use bvar

• Use it when: You need to monitor system metrics or user-defined counters in a high-concurrency environment where write performance is critical and reads are infrequent (e.g., for monitoring/display).
• Do NOT use it when: You need to perform logic based on the most recent value immediately, or when both reading and writing are extremely frequent and require low latency. In these cases, standard atomic operations are preferred.

mbvar provides multi-dimensional statistical counters. It consists of two main classes: MVariable (the base class providing registration, enumeration, and dumping) and MultiDimension (a template derived class for specific metric types).

Supported metric types for MultiDimension include:

• bvar::Adder<T>: A counter. varname << N performs varname += N.
• bvar::Maxer<T>: Tracks the maximum value. varname << N performs varname = max(varname, N).
• bvar::Miner<T>: Tracks the minimum value. varname << N performs varname = min(varname, N).
• bvar::IntRecorder: Tracks the average value since start (not a sliding window; use WindowEx for time-windowed averages).
• bvar::LatencyRecorder: Specialized for latency and QPS. Provides average latency, max latency, QPS, and total count.
• bvar::Status<T>: Records and displays a value; includes a set_value function.
• bvar::WindowEx<R, T>: Provides statistics over a previous time window. It is independent and requires data to be sent to it.
• bvar::PerSecondEx<T>: Provides average per-second statistics over a previous time window. It is independent and requires data to be sent to it.

#include <bvar/bvar.h>
#include <bvar/multi_dimension.h>

namespace foo {
namespace bar {
// Define a global multi-dimensional mbvar variable with labels: idc, method, status
bvar::MultiDimension<bvar::Adder<int>> g_request_count("request_count", {"idc", "method", "status"});

int process_request(const std::list<std::string>& request_label) {
    // Get the single-dimension bvar pointer for the specific labels
    // e.g., request_label = {"tc", "get", "200"}
    bvar::Adder<int>* adder = g_request_count.get_stats(request_label);
    
    if (!adder) {
        return -1;
    }
    // Note: adder must only be accessed within the lifetime of g_request_count
    *adder << 1 << 2 << 3; // adder adds up to 6
    return 0;
}

} // namespace bar
} // namespace foo

A SelectiveChannel (or "schan") is a combo channel that uses a load balancing algorithm to select and access exactly one of its internal sub-channels. Unlike ordinary channels that connect to servers directly, SelectiveChannel sends requests to groups of machines (sub-channels).

Key Characteristics

• Load Balancing: It provides a high-level abstraction for balancing traffic between groups of machines.
• Retries: Retries performed by SelectiveChannel are independent of retries in its sub-channels. If a call to a sub-channel fails (even after its own internal retries), SelectiveChannel may retry by selecting a different sub-channel.
• Async Requirement: Unlike other combo channels, SelectiveChannel requires that the request remains valid until the RPC completes. If you are using SelectiveChannel asynchronously, you must ensure the request is deleted inside the done callback.

An ExecutionQueue provides asynchronous, ordered execution of tasks in a separate thread (or bthread). It is designed to eliminate resource contention by using message passing instead of mutexes.

Key Features

• Asynchronous Ordered Execution: Tasks are executed in the exact order they are submitted.
• Multi-Producer: Multiple threads can submit tasks to the same queue simultaneously.
• Wait-free Submission: Task submission is wait-free, meaning it won't be blocked by system scheduling or other threads.
• High-Priority Support: Allows high-priority tasks to jump ahead of pending normal-priority tasks (while maintaining order within the high-priority group).
• Task Cancellation: Supports canceling tasks that have been submitted but not yet executed.
• Batch Processing: The execution thread can process tasks in batches, improving CPU cache locality.

ExecutionQueue vs. Mutex

Recommendation:

• Use a Mutex if the critical section is very small and contention is low.
• Use an ExecutionQueue if you need strict ordering, or if you want to improve throughput via batching in high-contention scenarios.