DOCSEARCHbyAlgolia
Submit Docs

eCAL (enhanced Communication Abstraction Layer)

repository·master·Indexed 21 days ago

https://github.com/eclipse-ecal/ecal

A high-performance, brokerless middleware for scalable interprocess communication supporting publish-subscribe and server-client patterns across local shared memory and networked UDP/TCP environments. Includes the Machine Monitoring Agent (MMA) for hardware usage tracking and a comprehensive suite of Google Benchmark microbenchmarks for evaluating Pub/Sub and Service layer performance.

Tokens
79.4K
Snippets
185
Records
385
Agent score
77%

What's inside eCAL

  1. Overview of eCAL applications

    master

    eCAL ships with several specialized applications to help you manage, debug, and record communication within your eCAL network:

    • eCAL Monitor: A debugging tool used to visualize and inspect real-time activity across your eCAL network.
    • eCAL Recorder & eCAL Player: A pair of tools designed to capture snapshots of eCAL traffic and replay them later.
    • eCAL Sys: A management application used to handle distributed startup configurations and monitor active eCAL Tasks.

  2. Overview of Machine Monitoring Agent (MMA) capabilities

    master

    The Machine Monitoring Application (MMA) collects hardware usage information. It monitors and displays:

    • CPU usage
    • RAM memory: Both available and total capacity.
    • Logical partitions: Displays name, capacity, available space, and I/O information (read/write bytes per second).
    • Network interface cards: Displays name, IP address, and I/O information (send/receive bytes per second).

    On Linux, the agent retrieves this data using the /proc virtual file system, the df command, and the /etc/os-release file.

  3. Overview of eCAL ecosystem tools

    master

    eCAL provides several tools for developing, testing, and debugging data flows:

    • eCAL Monitor: Visualizes data flow and inspects messages between publishers and subscribers.
    • eCAL Recorder: Records data sent between eCAL nodes using the HDF5 format.
    • eCAL Player: Replays eCAL recordings.
    • eCAL Sys: Defines system configuration and monitors applications.

  4. Overview of available eCAL benchmarks

    master

    The eCAL benchmark suite uses the Google Benchmark framework and is organized into several functional categories. Each category tests specific aspects of the communication abstraction layer:

    • pubsub: Tests core Publish/Subscribe performance, including Send, Send and Receive, and Receive Latency.
    • pubsub_config: Tests Send performance using various configuration combinations such as Zero Copy, Double Buffer, and Handshake.
    • pubsub_multi: Tests Send performance when multiple publishers are active simultaneously.
    • service: Tests the Ping mechanism within the service layer.
    • setup: Measures overhead and initialization costs, including Initialize, Initialize and Finalize, Publisher Creation, Subscriber Creation, and Registration Delay.
    • util: Contains Python scripts for auxiliary tasks, such as running 7-Zip compression benchmarks or calculating frequency and datarate from PubSub benchmark results.

  5. Overview of eCAL Sys

    master

    eCAL Sys is an application designed for starting, stopping, and monitoring applications across one or multiple machines. It is useful for automating system startup and monitoring whether applications report errors.

    Key capabilities include:

    • Configuring, starting, stopping, and restarting applications.
    • Monitoring started applications and local/remote PCs.
    • Providing a cross-platform remote-control technique.

    Note: eCAL Sys has been available since eCAL version 5.8.

  6. Use eCAL Rec CLI for recordings

    master

    ecal_rec is the command-line version of the eCAL Rec Host application (a replacement for the eCAL Rec GUI). It allows you to load, create, and save .ecalrec configuration files, and manage both local and distributed recordings.

    Important Distinction:

    • ecal_rec: The recorder client used for distributed recordings (this is the primary tool for automation).
    • ecal_rec_client: The recorder host application in command-line flavor.

    Operational Modes:

    1. Direct mode (default): Loads a configuration and executes commands immediately, then exits.
    2. Remote-control mode (--remote-control): Does not execute commands itself but relays them to another eCAL Rec Host (like a GUI or another CLI instance). Useful for triggering recordings automatically while using a GUI.
    3. Interactive mode (--interactive): Opens an interactive console that accepts commands from STDIN and prevents the application from exiting automatically. This mode can be used simultaneously with remote-control mode.

  7. Understand eCAL Setup & Lifecycle Benchmarks

    master

    The eCAL Setup & Lifecycle Benchmarks suite provides microbenchmarks to measure the performance costs associated with the eCAL middleware lifecycle. These benchmarks focus on initialization, teardown, and the creation of communication entities (publishers and subscribers).

    All benchmarks use the topic name benchmark_topic and are implemented using the Google Benchmark framework, building into a standalone binary via BENCHMARK_MAIN().

  8. Understand the eCAL Pub/Sub microbenchmarks

    master

    The benchmark_pubsub.cpp microbenchmarks measure eCAL publish/subscribe performance within a single process using Google Benchmark. These tests evaluate how eCAL handles byte payloads on a specific topic (benchmark_topic) across different payload sizes ranging from 1 B to 16 MiB.

    There are three primary benchmark types:

    1. Send/Bandwidth: Measures the cost of publishing messages. A receiver thread is present to maintain the publisher's flow, but receiver latency is not part of the measurement. Uses wall-clock time (UseRealTime).
    2. Send + Receive/Round-trip (1→1): Measures the cost of publishing a message and blocking until a receive callback is triggered. Uses wall-clock time (UseRealTime).
    3. Receive Latency (manual timing): Measures the delivery latency by timing the interval from immediately before blocking to the moment the callback timestamp is recorded. Uses manual timing (UseManualTime and SetIterationTime).

    Note: Because the publisher and subscriber reside in the same process (different threads), these benchmarks target intra-host and intra-process behavior.

  9. Understand the eCAL Service Call Benchmark

    master

    The eCAL Service Call Benchmark is a microbenchmark designed to measure the request-response performance overhead of the eCAL RPC (Remote Procedure Call) path.

    Key Characteristics:

    • Scenario: A service server exposes a ping method. A client repeatedly calls CallWithResponse("ping", "request", ...) and waits for a small constant payload response.
    • Execution Model: To isolate the RPC path overhead, the server and client run in separate threads within the same process (intra-process/intra-host).
    • Synchronization: The benchmark includes a startup synchronization step that waits for discovery/registration to ensure at least one client instance is matched to the server before timing begins.

    Note: This benchmark does not isolate network or cross-process effects as it targets intra-process behavior.

  10. Overview of the eCAL Recorder

    master

    The eCAL Recorder allows you to capture any data transmitted through your eCAL network and save it to disk. The resulting output is known as an eCAL Measurement. These measurements can be used for later analysis or to stimulate other applications by replaying the recorded data as input.

    There are two primary recording modes:

    1. Centralized recording: The recorder captures all topics on your main machine over the network. This is the simplest mode.
    2. Distributed recording: Each machine in the network records its own local topics. This mode is used to prevent network congestion in large systems.

  11. Structure of an ecaldhf5 measurement file

    master

    An ecalhdf5 measurement is structured using HDF5 datasets and attributes.

    Top-level Attributes

    The top-level HDF5 file contains two primary attributes:

    • Version (String): The version of the ecaldhf5 format (e.g., 5.0).
    • Channels (String): A comma-separated list of all channels present in the measurement.

    Data Organization

    • Payload Datasets: Each individual data entry is stored in a unique dataset named with a unique ID assigned by the library. The payload is stored as a char array.
    • Channel Datasets: Each channel has a dedicated dataset that acts as a table. This table contains metadata for every message frame (row) in that channel and attributes describing the channel itself.

  12. Compare eCAL and ROS for communication requirements

    master

    When deciding between eCAL and ROS, consider the following architectural differences:

    Message Protocol Agnosticism

    • eCAL: Transports raw binary data. It is not coupled to any specific serialization format, allowing you to use Protobuf, Flatbuffers, Capnproto, or MessagePack. This provides better support for message evolution and backwards compatibility.
    • ROS: Uses a built-in IDL system that is tightly coupled to the ROS ecosystem. While ROS 2 RMWs allow for transport flexibility, tools like rosbag are still tied to specific message definitions.

    Integration Model

    • eCAL: A modular library (+ tools). You only need to link to the eCAL core library to build communication components, making it easy to integrate into existing software ecosystems.
    • ROS: A complete ecosystem. It includes its own build system and bundled C++ libraries, which provides a high-level experience but can be harder to integrate into pre-existing, non-ROS projects.

    Key Feature Differences

    • Distributed Recordings: eCAL supports distributed recording where data is recorded on the host where it is created. These recordings can later be merged and played back in synchronization.
    • Brokerless Communication: eCAL is brokerless. It uses a dedicated UDP-based "monitoring layer" to exchange necessary information for communication, rather than requiring a central broker.
    • High Bandwidth: eCAL is recommended for scenarios where high bandwidth and throughput are critical, as it typically outperforms standard ROS RMWs.