DOCSEARCHbyAlgolia
Submit Docs

Yet Another CloudWatch Exporter (YACE)

repository·master·Indexed 22 days ago

https://github.com/prometheus-community/yet-another-cloudwatch-exporter

A Prometheus exporter written in Go that retrieves AWS CloudWatch metrics and exposes them in a Prometheus-compatible format. YACE features automatic resource discovery via tags, mapping of CloudWatch dimensions and tags to Prometheus labels, and support for multi-account monitoring using RoleArns. It supports a wide range of AWS namespaces across compute, networking, databases, and security, and provides internal metrics to track CloudWatch request counts for cost estimation.

Tokens
13K
Snippets
26
Records
56
Agent score
72%

What's inside yet-another-cloudwatch-exporter

  1. Overview of YACE (Yet Another CloudWatch Exporter)

    master

    YACE is a Prometheus exporter designed to scrape AWS CloudWatch metrics. It is written in Go and utilizes the official AWS SDK.

    Key features include:

    • Auto-discovery: Automatically discovers AWS resources via tags, reducing the need for manual ID management.
    • Labeling: Automatically adds AWS tags and CloudWatch dimensions as labels to exported metrics.
    • Flexibility: Supports static metrics (without auto-discovery), regex filtering for resources, and cross-account roles to pull data from multiple AWS accounts.
    • Customization: Supports scraping custom namespace metrics with CloudWatch Dimensions and allows exporting 0 even if CloudWatch returns nil.
    • Observability: Provides structured logging (json and logfmt) and tracks its own CloudWatch request counts to help calculate costs.

  2. How serviceFilters work in the tagging package

    master

    In the tagging package, serviceFilters are used to look up or filter resources for specific CloudWatch namespaces when tag data alone is insufficient for discovery.

    Developers extending the exporter can interact with serviceFilters by:

    • Adding a new service filter implementation for a new AWS service.
    • Modifying the behavior of a ResourceFunc.
    • Modifying the behavior of a FilterFunc.

  3. Understand the YACE clients package architecture

    master

    The clients package provides an abstraction layer over the AWS SDK for Go v2 to hide implementation details. It uses a factory pattern to isolate common interfaces from their specific implementations.

    Key components include:

    • /clients: Contains the Factory interface and the CachingFactory implementation.
    • /clients/account: Handles looking up AWS account information.
    • /clients/cloudwatch: Responsible for gathering CloudWatch metrics data.
    • /clients/tagging: Manages resource discovery via tagging interfaces and service-specific filters.

  5. Run YACE using Docker

    master

    You can run YACE as a Docker container using images from the GitHub Container Registry. The image name is quay.io/prometheuscommunity/yet-another-cloudwatch-exporter and tags follow the vX.Y.Z format (ensure you include the v prefix).

    To run the container locally, you must mount your AWS credentials and your config.yml file into the container and expose port 5000.

    docker run -d --rm \
  -v $PWD/credentials:/home/.aws/credentials \
  -v $PWD/config.yml:/tmp/config.yml \
  -p 5000:5000 \
  --name yace quay.io/prometheuscommunity/yet-another-cloudwatch-exporter:latest

  6. Enable experimental or breaking features via feature flags

    master

    YACE uses feature flags to manage features that are either experimental or represent breaking changes. These features are disabled by default. To enable one or more features, use the -enable-feature CLI flag followed by a comma-separated list of the feature names.

    Note that the behavior of these features may change in future releases.

  7. Install YACE on Kubernetes using Manifests

    master

    To install YACE using standard Kubernetes manifests, you need to create a ConfigMap containing your config.yml and a Deployment that mounts this ConfigMap. Ensure the container argument --config.file=/tmp/config.yml is set and that you use a specific version tag (e.g., vX.Y.Z) for the image.

    ---
apiVersion: v1
kind: ConfigMap
metadata:
  name: yace
data:
  config.yml: |-|
    # (Your config content goes here)
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: yace
spec:
  replicas: 1
  selector:
    matchLabels:
      name: yace
  template:
    metadata:
      labels:
        name: yace
    spec:
      containers:
      - name: yace
        image: quay.io/prometheuscommunity/yet-another-cloudwatch-exporter:vX.Y.Z # release version as tag - Do not forget the version 'v'
        imagePullPolicy: IfNotPresent
        args:
          - "--config.file=/tmp/config.yml"
        ports:
        - name: app
          containerPort: 5000
        volumeMounts:
        - name: config-volume
          mountPath: /tmp
      volumes:
      - name: config-volume
        configMap:
          name: yace

  8. Install the CloudWatch Mixin for Grafana

    master

    The CloudWatch Mixin is a Prometheus Monitoring Mixin that provides pre-defined dashboards for CloudWatch metrics. To install it, you need jsonnet-bundler to manage dependencies and Grizzly to apply the mixin to your Grafana instance.

    Follow these steps:

    1. Install jsonnet-bundler using Go.
    2. Use jb install to download the required dependencies for the mixin.
    3. Install Grizzly using Go.
    4. Apply the mixin to your Grafana instance using grr apply pointing to mixin.libsonnet.
    go install -a github.com/jsonnet-bundler/jsonnet-bundler/cmd/jb@latest

# Install dependencies
jb install

# Install Grizzly and apply the mixin
go install github.com/grafana/grizzly/cmd/grr@latest
grr apply mixin.libsonnet

  12. Set up a local development environment with docker-compose

    master

    You can use the provided docker-compose configuration to launch a complete local development stack. This stack includes:

    • YACE: Configured using the yace-config.yaml file.
    • Prometheus: Pre-configured to scrape metrics from YACE.
    • Grafana: Pre-configured with Prometheus as a datasource (no login required).

    AWS Authentication: The setup mounts your host's ~/.aws directory into the container to reuse your existing AWS credentials. You must specify the target AWS region and profile using environment variables when running the command.

    # Build the YACE docker image
docker-compose build

# Start all docker-compose resource
AWS_REGION=us-east-1 AWS_PROFILE=sandbox docker-compose up -d