repository·master·Indexed 43 days ago
https://github.com/openstack/ceilometerAn OpenStack service for collecting metrics and telemetry data. It utilizes notification agents to transform message bus events into samples and polling agents (Compute, Central, and IPMI) to retrieve resource usage from hypervisors and APIs. Features include custom meter definitions via YAML, JSON path extraction, and CLI tools for managing samples, storage upgrades, and service status.
Ceilometer is a data collection service within the OpenStack Telemetry project. It is designed to normalize and transform data across all current OpenStack core components (and future components).
Key use cases for Ceilometer data include:
The dynamic pollster feature enables system administrators to create or update REST API pollsters dynamically without requiring code changes. The subsystem functions by reading YAML configuration files located in the directories specified by the pollsters_definitions_dirs parameter.
Administrators can organize these configurations using either a single file per dynamic pollster or by grouping multiple dynamic pollsters within a single file.
Ceilometer is an infrastructure project designed to collect information regarding OpenStack projects. Its primary role is to provide a single source of truth for data collection that can be used by rating engines to transform events into billable items, a process known as "metering."
In a standard three-step billing process (common in the telco industry), Ceilometer is strictly limited to the first step:
Ceilometer does not perform rating or billing itself. Instead, it collects the data and can push it to various targets using provided publishers.
Ceilometer's Telemetry Data Collection services are responsible for polling metering data from OpenStack services, collecting event and metering data via service notifications, and publishing that data to targets like data stores and message queues.
Note on Storage: Ceilometer's internal storage and API solution is deprecated. For efficient storage and statistical analysis of collected data, Gnocchi is the recommended target.
The Compute agent is responsible for collecting resource usage data for VM instances on individual compute nodes. Because this requires close interaction with the hypervisor, the agent must be installed on every compute node in the deployment.
Key details:
Ceilometer processes two main types of data:
Event Structure: An event generally contains:
event_type: A dotted string (e.g., compute.instance.resize.start).message_id: A UUID for the event.generated: A timestamp of when the event occurred on the source system.traits: A flat mapping of typed key-value pairs (strings, ints, floats, or datetimes) containing the event details.raw (Optional): The full, unindexed notification message for auditing.Ceilometer is designed to normalize cloud data and push it to various targets via a Pipeline Manager (handled by notification agents).
Available Transport/Publisher Options:
gnocchi: Publishes samples/events to the Gnocchi API (Recommended for time-series storage).notifier: Pushes samples to a message queue for external consumption.udp: Publishes samples via UDP packets.http: Targets a REST interface.https: Targets a REST interface over SSL.file: Publishes samples to a specific file location.zaqar: Uses the Zaqar multi-tenant cloud messaging service.prometheus: Publishes samples to a Prometheus Pushgateway.Recommended Workflow: Push data to Gnocchi for efficient time-series storage and resource lifecycle tracking.
Ceilometer uses an agent-based architecture to collect, normalize, and redirect data for metering, monitoring, and alerting. The system relies on an AMQP service for internal communication and for consuming notifications from other OpenStack services.
The architecture is composed of several key agents:
ceilometer-polling: Polls for meter data using registered polling plug-ins (pollsters) across different namespaces. It provides a unified polling interface.ceilometer-agent-notification: Consumes AMQP messages from OpenStack services, normalizes them, and publishes them to configured targets.Deployment Note: While ceilometer-polling can support any namespace, some distributions still use namespace-scoped agents such as ceilometer-agent-central, ceilometer-agent-compute, and ceilometer-agent-ipmi. Generally, all services except for those polling compute or ipmi namespaces are placed on controller nodes.
Ceilometer tracks measurements for resources using three distinct meter types. When building integrations or analyzing data, identify which type applies to your metric:
The Telemetry service is composed of three primary agent types, which communicate via the OpenStack messaging bus:
ceilometer-agent-compute): Deployed on each compute node. It polls for resource utilization statistics. Technically, this is the ceilometer-polling agent running with the --polling-namespace compute parameter.ceilometer-agent-central): Deployed on a central management server. It polls for resource utilization statistics for resources not tied to specific instances or compute nodes. You can scale this service horizontally by starting multiple agents. Technically, this is the ceilometer-polling agent running with the --polling-namespace central parameter.ceilometer-agent-notification): Deployed on central management server(s). It consumes messages from message queues to build event and metering data, then publishes it to defined targets (by default, data is pushed to Gnocchi).To ensure consistency in telemetry data, follow these rules for units and dimensions:
b) or bytes (B).apples, disk, routers, floating IPs).| Dimension | Unit | Abbreviations |
|---|---|---|
| None | N/A | Dimension-less variable |
| Volume | byte | B |
| Time | seconds | s |