DOCSEARCHbyAlgolia
Submit Docs

scaphandre

repository·main·Indexed 23 days ago

https://github.com/hubblo-org/scaphandre

A metrology agent used to measure and expose the electric power and energy consumption of technical services, ranging from bare metal hosts to virtual machines. It supports GNU/Linux and Windows, providing capabilities to quantify energy footprints and export metrics to Prometheus, Riemann, and Warp10. Scaphandre can measure power on bare metal, QEMU/KVM virtual machines, and Kubernetes environments, utilizing sensors like PowercapRAPL and MsrRAPL to estimate per-process energy consumption in watt-hours.

Tokens
25.3K
Snippets
48
Records
129
Agent score
84%

What's inside scaphandre

  1. What is Scaphandre?

    main

    Scaphandre is a metrology agent designed to measure electric power and energy consumption metrics. It allows companies and individuals to quantify the energy footprint of their tech services and export this data to various monitoring and data analysis toolchains.

    Key capabilities include:

    • Measuring power/energy on bare metal hosts.
    • Measuring power/energy of QEMU/KVM virtual machines from the host.
    • Exposing VM power metrics to the guest OS (via hypervisor features).
    • Exporting metrics to Prometheus (HTTP exporter or Push Gateway), Riemann, and Warp10.
    • Supporting Kubernetes environments.
    • Storing metrics in JSON files or displaying them in the terminal.

    Supported Operating Systems:

    • Gnu/Linux
    • Windows 10, 11, and Server 2016/2019/2022

  2. Overview of Scaphandre

    main
    Scaphandre is a monitoring agent dedicated to measuring energy consumption metrics. Its primary purpose is to help users measure and understand the energy consumption patterns of technical services, facilitating a shift towards more sustainable technology practices.

  4. Use the Qemu exporter to expose VM energy metrics

    main

    The Qemu exporter computes energy consumption metrics for each Qemu/KVM virtual machine found on the host. It exposes these metrics as filetrees compatible with the powercap_rapl sensor.

    Note: This feature is currently experimental. While metrics are considered trustworthy, the methods for sharing data with guests are still under discussion.

  5. Understanding RAPL domains and power measurements

    main

    RAPL (Running Average Power Limit) is a feature on Intel and AMD x86 CPUs (manufactured after 2012) used to set power limits and provide power usage measurements. Note that RAPL measurements are often based on estimations and modeling rather than direct physical measurement.

    Common RAPL domains include:

    • Core/PP0: Energy consumed by the CPU Cores.
    • Uncore/PP1: Energy consumed by components close to the CPU, typically the embedded GPU chipset.
    • Dram: Energy consumed by the memory/RAM sticks.
    • Package/PKG: Includes both 'Core' and 'Uncore'. Depending on the platform documentation, it may or may not include 'Dram'.
    • PSys: Monitors and controls the thermal and power specifications of the entire SoC. It is used when power consumption originates from sources other than the CPU or GPU (e.g., motherboard components, WiFi/Bluetooth cards). In multi-socket server systems, each socket reports its own PSys values.

  6. Understand the PowercapRAPL sensor mechanism

    main

    The PowercapRAPL sensor is used when Scaphandre can measure the host's power consumption directly.

    • RAPL (Running Average Power Limit): A technology in most Intel and AMD x86 CPUs (post-2012) that allows tracking energy consumption of the CPU, CPU sockets, and sometimes the DRAM controller.
    • Data Flow: The powercap kernel module writes energy consumption data to system files. Scaphandre reads these files, buffers the data, and passes it to exporters.
    • Process-level Calculation: When an exporter requests a measurement, the sensor:
      1. Reads energy counters from powercap.
      2. Reads CPU usage statistics from /proc/stats.
      3. Reads per-process statistics from /proc/PID/stats.
      4. Computes the ratio of CPU time spent by a specific PID relative to the total active CPU time to derive the process's share of the power consumption.

  7. How Scaphandre computes per-process power consumption

    main

    Scaphandre estimates the power consumption of individual processes by combining CPU resource sharing data with absolute power readings from hardware sensors.

    The Core Logic

    1. Resource Sharing (Jiffies): Computers use timesharing, allocating small intervals of time called jiffies to different processes. Scaphandre tracks the total jiffies allocated to a specific process.
    2. Absolute Power (Sensors): Scaphandre uses hardware sensors (like Intel's RAPL) to measure the total power (in Watts) being drawn by the machine or specific components (CPUs, DRAM controllers).
    3. The Calculation: Scaphandre correlates the power readings with the process's CPU usage. It identifies the power drawn during the specific jiffies allocated to a process. By aggregating these readings over a period, it calculates the energy used in watt-hours.

    Virtualization and Multi-processor Support

    • Multi-processor: Scaphandre can read sensors that provide power usage per processor and combine that with the time allocation for processes on those specific processors.
    • Virtualization: While VMs often lack direct access to host power sensors, Scaphandre can expose these readings to guest VMs or containers if the host infrastructure is running Scaphandre and providing the data.

  8. Understand Scaphandre stdout output format

    main

    When using the stdout exporter, the output provides a breakdown of power consumption:

    1. Host Line: Total power consumption of the machine (calculated between the last two measurements).
    2. Socket Lines: Power consumption per CPU socket, including details by RAPL Domain (Core, Uncore, DRAM, etc.). Multiple lines appear if there are multiple sockets.
    3. Top 5 consumers: A list of the 5 processes consuming the most power during the measurement period, showing:
      • Power: Consumption in Watts (W).
      • PID: Process ID.
      • Exe: The executable path.

    Example output:

    Host:    9.391334 W    Core        Uncore        DRAM
Socket0  9.392    W    1.497082 W
Top 5 consumers:
Power    PID    Exe
4.808363 W    642    "/usr/sbin/dockerd"
4.808363 W    703    "/usr/bin/docker-containerd"
4.808363 W    1028   "/usr/local/bin/redis-server"
0 W        1    "/usr/lib/systemd/systemd"
0 W        2    ""
------------------------------------------------------------

  9. What are Scaphandre sensors?

    main

    Sensors are the data collection component of Scaphandre. Their primary responsibilities are:

    1. Obtaining power consumption metrics from the host.
    2. Making those metrics available to the exporter.

    Key Sensor Details:

    • PowercapRAPL: Uses the powercap Linux kernel module to access the RAPL (Running Average Power Limit) feature of x86 CPUs. This requires bare-metal access and typically will not work in a virtual machine unless metrics are provided by a hypervisor.
    • Estimation-based sensors (Planned): Designed for cloud environments where bare-metal access is unavailable, estimating power based on resource consumption (CPU/GPU/RAM/IO) and hardware characteristics.
    • Interface: Sensors communicate with exporters through a Topology object via the get_topology method.

  10. Measure power consumption of containers and orchestrators

    main

    Scaphandre supports several container-related use cases:

    1. Container Power Consumption: You can measure the power consumption of specific containers running on a host. This typically involves querying a Time Series Database (TSDB) using process-level metrics.
    2. Kubernetes Orchestration: Scaphandre can be installed on Kubernetes clusters via a Helm chart to monitor the power consumption of nodes, containers, and applications. This is often used in conjunction with Prometheus and Grafana for visualization.

  11. How Scaphandre's sensors and exporters work together

    main

    Scaphandre follows an extensible architecture composed of two main components: sensors and exporters.

    1. Sensors are responsible for collecting or pre-computing power consumption metrics from the host (e.g., via the PowercapRAPL sensor which interfaces with the Linux kernel's RAPL feature). Sensors provide these metrics to exporters via a Topology object using the get_topology method.
    2. Exporters are responsible for requesting new metrics from a sensor and publishing them to a destination (e.g., Stdout for terminal output or prometheus for an HTTP endpoint).

    When running Scaphandre from the command line, you select a sensor using the -s flag, followed by the exporter subcommand.

  12. What are Scaphandre exporters?

    main

    Exporters are the publishing component of Scaphandre. Their primary responsibilities are:

    1. Requesting new metrics from the selected sensor.
    2. Exporting/publishing those metrics to a specific target.

    Common Exporters:

    • Stdout: Prints metrics directly to the standard output (terminal).
    • prometheus: Exposes metrics on an HTTP endpoint for scraping by a Prometheus instance.
    • Qemu: A specialized exporter used on Qemu/KVM hypervisors. It collects metrics related to running virtual machines and writes them to files. This allows a Scaphandre instance running inside a VM to use the PowercapRAPL sensor (with the --vm flag) as if it were running on bare metal.