Promruval

Promruval organizes its validations into different scopes. Depending on the type of rule you are validating (e.g., a Rule Group, an Alert, or a Recording Rule), only specific validators are available.

• Groups: Validations that apply specifically to the Group scope (e.g., maxRulesPerGroup, hasAllowedEvaluationInterval).
• Universal rule validators: Validations that can be applied to any rule, including Labels and PromQL expression validators.
• Alert validators: Validations specific to Alert rules, covering Labels, Annotations, and other alert-specific properties.
• Recording rules validators: Validations specific to Recording rules.

For a sane starting point, you can refer to default_validation.yaml to see recommended default validations.

Note on Regular Expressions: Unless explicitly stated otherwise, all regular expressions used in configuration are fully anchored (^...$) and default to matching everything (.*).

Promruval uses a YAML or Jsonnet configuration file to define how rules are validated.

Configuration Structure:

• customExcludeAnnotation: (Optional) Overrides the annotation used for disabling rules.
• prometheus: Configuration for connecting to a Prometheus instance:
  • url: The URL of the running Prometheus instance.
  • insecureSkipTlsVerify: (Optional) Skip TLS verification.
  • bearerTokenFile: (Optional) Path to a file containing a bearer token. Alternatively, use the PROMETHEUS_BEARER_TOKEN environment variable.
  • timeout: (Optional) Request timeout.
  • cacheFile: (Optional) File to save Prometheus call cache.
  • maxCacheAge: (Optional) Maximum age of the cache.
  • queryOffset: (Optional) Offset for query evaluation time.
  • queryLookback: (Optional) How far back to look in queries.
  • httpHeaders: (Optional) Custom HTTP headers.
• validationRules: A list of rule sets to apply:
  • name: Unique name for the validation rule set.
  • scope: The target of the validation (Group, Alert, Recording rule, or All rules).
  • validations: A list of specific validation types to perform.
    • type: The validation type (see docs/validations.md).
    • additionalDetails: (Optional) Custom text appended to error messages.
    • params: Parameters for the validation.
    • paramsFromFile: (Optional) Path to a file containing parameters (mutually exclusive with params).
  • onlyIf: (Optional) Conditions that must be met for the rule to apply.

customExcludeAnnotation: my_disable_annotation

prometheus:
  url: https://foo.bar/
  insecureSkipTlsVerify: false
  bearerTokenFile: bearer_token.txt
  timeout: 30s
  cacheFile: .promruval_cache.json
  maxCacheAge: 1h
  queryOffset: 1m
  queryLookback: 20m
  httpHeaders:
    foo: bar

validationRules:
  - name: example-validation
    scope: All rules
    validations:
      - type: hasLabels
        additionalDetails: "We do this because ..."
        params:
          labels: [ "severity" ]

You can restrict which metrics or labels are used within PromQL expressions to prevent unauthorized data access or ensure compatibility with specific environments (like Thanos or KSM).

• expressionDoesNotUseMetrics: Fails if the expression uses metrics matching any provided regexps. Regexps are fully anchored (^...$).
• expressionDoesNotUseLabels: Fails if the expression uses specific labels in matchers, aggregations, or joins.
• expressionUsesOnlyAllowedLabelsForMetricRegexp: Restricts labels used when a specific metric (matched by metricNameRegexp) is present.
• expressionDoesNotUseLabelsForMetricRegexp: Fails if specific labels are used in combination with a metric matching metricNameRegexp.
• expressionUsesOnlyAllowedLabelValuesForMetricRegexp: Ensures that if a metric (matching metricNameRegexp) is used, its label selectors match at least one value in the allowedLabelValues map.

# Example: expressionDoesNotUseMetrics
params:
  metricNameRegexps: [ "foo_bar.*", "foo_baz" ]

# Example: expressionUsesOnlyAllowedLabelsForMetricRegexp
params:
  metricNameRegexp: "kube_pod_labels"
  allowedLabels: [ "pod", "cluster", "app", "team" ]

# Example: expressionUsesOnlyAllowedLabelValuesForMetricRegexp
params:
  metricNameRegexp: "kube_pod_labels"
  allowedLabelValues:
    cluster: ['kube1', 'kube2', 'kube3']
    team: ['sre', 'backend']

Universal rule validators can be applied to All rules, Recording rule, and Alert scopes. These validators check for the presence, absence, or specific patterns of labels on Prometheus rules.

Key validators include:

• hasLabels: Ensures all specified labels are present. If searchInExpr is true, it also checks the rule's expr.
• doesNotHaveLabels: Fails if any of the specified labels are present.
• hasAnyOfLabels: Fails if none of the specified labels are present.
• labelMatchesRegexp: Validates a label against a regex. Use negative: true to fail if it matches.
• labelHasAllowedValue: Ensures a label value is within a whitelist. Supports commaSeparatedValue splitting and ignoreTemplatedValues to skip templated labels.
• nonEmptyLabels: Fails if any label has an empty value.
• exclusiveLabels: Prevents the coexistence of two specific labels (optionally checking their values).

# Example: hasLabels
params:
  labels: [ "foo", "bar" ]
  searchInExpr: true

# Example: labelHasAllowedValue
params:
  label: "foo"
  allowedValues: [ "foo", "bar" ]
  commaSeparatedValue: true
  ignoreTemplatedValues: false

# Example: exclusiveLabels
params:
  firstLabel: "severity"
  firstLabelValue: "critical"
  secondLabel: "page"
  secondLabelValue: "true"

Promruval supports validation for LogQL (Loki) and specialized checks for multi-tenant environments.

LogQL

• expressionIsValidLogQL: Validates LogQL syntax.
• logQlExpressionUsesRangeAggregation: Ensures LogQL expressions used in rules utilize range aggregation functions.

Multi-tenancy (Mimir/Mimir-like)

• hasSourceTenantsForMetrics: Ensures that if a rule uses a metric matching a specific regexp, that metric's tenant is explicitly listed in the rule group's source_tenants configuration. This is useful for ensuring metrics from specific tenants (like KSM or Kafka) are correctly scoped.

Typo Detection

• doesNotContainTypos: Uses Levenshtein distance to identify typos in well-known labels, annotations, or values. You can configure maxLevenshteinDistance or maxDifferenceRatio to define the threshold for what constitutes a typo.

# Example: hasSourceTenantsForMetrics
params:
  defaultTenant: "my-tenant"
  sourceTenants:
    k8s:
      - regexp: "kube_.*|container_.*"
        description: "Metrics from KSM"

# Example: doesNotContainTypos
params:
  maxLevenshteinDistance: 2
  wellKnownAnnotations: ['playbook', 'dashboard', 'title', 'description']
  wellKnownRuleLabels: ['do_not_inhibit']
  wellKnownSeriesLabels: ['pod', 'locality', 'cluster']

Use these validators to ensure Alert rules follow specific formatting and metadata requirements.

Label Validation

• validateLabelTemplates: Ensures label values contain valid Go templates.

Annotation Presence

• hasAnnotations: Fails if the rule is missing any of the specified annotations.
• doesNotHaveAnnotations: Fails if the rule contains any of the specified annotations.
• hasAnyOfAnnotations: Fails if the rule does not contain at least one of the specified annotations.

Annotation Content Validation

• annotationMatchesRegexp: Fails if an annotation value does not match a regex. Use negative: true to fail if it does match.
• annotationHasAllowedValue: Fails if an annotation value is not in the allowedValues list. If the annotation is missing entirely, validation succeeds.
• annotationIsValidURL: Validates that an annotation is a valid URL.
  • If resolveUrl: true, it attempts an HTTP request to the URL and fails on non-200 status or connection failure.
  • If asTemplate: true, it parses the annotation as a Go text template before validation. If both are enabled, the template is executed first, then the resulting URL is resolved.
• annotationIsValidPromQL: Validates that the content of a specific annotation is a valid PromQL expression.
• validateAnnotationTemplates: Ensures annotation values contain valid Go templates.

# Example: Ensure 'playbook' annotation is a valid, resolvable URL
params:
  annotation: "playbook"
  resolveUrl: true
  asTemplate: false