Datadog Terraform Provider

repository·master·Indexed 19 days ago

https://github.com/datadog/terraform-provider-datadog

The Datadog Terraform Provider enables Infrastructure as Code (IaC) workflows to manage Datadog monitoring, dashboards, and platform configurations. It requires Terraform version 1.1.5 or higher. The provider includes tools like generate_import_config.sh for migrating Cloud Cost Management and Tag Pipeline resources, and internal fwutils for implementing write-only secret attributes in Terraform 1.11+.

Tokens
215.6K
Snippets
438
Records
811
Agent score
65%

What's inside terraform-provider-datadog

  1. Use aggregate-augmented composite query variables

    master

    The variables.aggregate_augmented_query block allows you to create composite queries by joining a reference table or an event augment to a base metrics or events query.

    Core Components:

    • data_source: Must be set to aggregate_augmented_query.
    • compute: A list of aggregation steps (e.g., sum, avg, p95) applied to the query.
    • group_by: Defines facets to group results by. Supports sort options.
    • join_condition: Defines how the augment and base queries connect using augment_attribute, base_attribute, and a join_type (inner or left).

    Augmentation Types (Mutually Exclusive):

    • augment_event_query: Uses an event platform-based query (e.g., logs, rum, spans). Requires a search block with a query string.
    • augment_reference_table: Uses a Datadog reference table. Requires table_name and optionally columns and query_filter.
  2. Understand Datadog Org Group Policy Override behavior

    master

    Server-side auto-creation

    Overrides can be created automatically by the Datadog server in two scenarios:

    1. Membership change: When an organization is moved into an org group, the server compares the org's current config to non-GROUP_MANAGED policies. Mismatches trigger auto-created overrides.
    2. Policy update: When a policy is created or updated, the server creates overrides for any member organizations whose configuration does not match the new policy value.

    Auto-created overrides are indistinguishable from user-declared ones. You can manage them via terraform import or by using the datadog_org_group_policy_overrides data source.

    Delete behavior

    Warning: Removing a datadog_org_group_policy_override resource from your Terraform code will cause the target organization's configuration to be reset to match the parent policy's value. The server treats deletion as a command to re-apply the parent policy.

    Best Practice: If you want to stop managing an override via Terraform without resetting the organization's value, use terraform state rm instead of deleting the resource block.

    GROUP_MANAGED tier cascade

    If a parent datadog_org_group_policy is changed to enforcement_tier = "GROUP_MANAGED", the server automatically deletes all overrides for that policy.

    Migration Tip: If you plan to transition a policy to GROUP_MANAGED, you must remove all associated datadog_org_group_policy_override resource blocks in the same Terraform commit to avoid FailedPrecondition errors during the next apply.

  3. Manage the order of Datadog Tag Pipeline Rulesets

    master

    Use the datadog_tag_pipeline_rulesets resource to define the execution order of your Tag Pipeline Rulesets. Rulesets are executed sequentially based on the order of IDs provided in the ruleset_ids argument.

    Behavior for UI-defined Rulesets

    When managing ruleset order via Terraform, you must decide how to handle rulesets that were created manually in the Datadog UI and are not managed by your Terraform configuration:

    1. Preserve (Default): Set override_ui_defined_resources = false. UI-defined rulesets not in your Terraform list will be kept at the end of the execution order.

      • Warning: Terraform will warn if unmanaged rulesets are at the end of the order.
      • Error: Terraform will throw an error if unmanaged rulesets exist in the middle of the order. In this case, you must either import them, set override_ui_defined_resources = true, or manually reorder/delete them in the Datadog UI.
    2. Override: Set override_ui_defined_resources = true. Any ruleset created via the UI that is not explicitly defined in your Terraform configuration will be automatically deleted during terraform apply. This makes Terraform the absolute source of truth.

    resource "datadog_tag_pipeline_rulesets" "order" {
      ruleset_ids = [
        datadog_tag_pipeline_ruleset.first.id,
        datadog_tag_pipeline_ruleset.second.id
      ]
    }
  4. Manage Datadog Logs Custom Pipelines

    master

    The datadog_logs_custom_pipeline resource allows you to create and manage Datadog Logs Pipelines. Each resource defines a complete pipeline including its filters and processors.

    Important: Pipeline Ordering

    Defining a datadog_logs_custom_pipeline resource is not enough to make the pipeline active in the correct sequence. Pipeline execution order is managed by a separate resource: datadog_logs_pipeline_order.

    To ensure your pipeline works correctly:

    1. To add a pipeline: You must explicitly add the new pipeline to the datadog_logs_pipeline_order resource.
    2. To remove a pipeline: You must remove its references from the datadog_logs_pipeline_order resource before or during destruction.

    Failure to manage the datadog_logs_pipeline_order resource will result in the pipeline not being tracked or executed in the intended order.

    resource "datadog_logs_custom_pipeline" "sample_pipeline" {
      filter {
        query = "source:foo"
      }
      name       = "sample pipeline"
      is_enabled = true
    
      processor {
        arithmetic_processor {
          expression         = "(time1 - time2)*1000"
          target             = "my_arithmetic"
          is_replace_missing = true
          name               = "sample arithmetic processor"
          is_enabled         = true
        }
      }
    
      # ... other processors ...
    }
  5. Configure account_filters for AWS CUR

    master

    The account_filters block allows you to control which AWS accounts are included in your Datadog cost analysis. You must choose between including all accounts (with specific exclusions) or including only specific accounts.

    Logic Rules:

    • If include_new_accounts is true: Use excluded_accounts to specify accounts to omit. You cannot use included_accounts in this mode.
    • If include_new_accounts is false: Use included_accounts to specify exactly which accounts to include. You cannot use excluded_accounts in this mode.
    ### Nested Schema for `account_filters`
    
    Optional:
    - `excluded_accounts` (List of String): List of AWS account IDs to exclude. Only used when `include_new_accounts` is `true`.
    - `include_new_accounts` (Boolean): Whether to automatically include new member accounts. When `true`, use `excluded_accounts`. When `false`, use `included_accounts`.
    - `included_accounts` (List of String): List of AWS account IDs to include. Only used when `include_new_accounts` is `false`.
  6. New features in datadog_dashboard_v2

    master

    The datadog_dashboard_v2 resource introduces several improvements over the legacy datadog_dashboard resource:

    • Flexible Widget Time Spans: Supports arbitrary live spans and fixed time ranges via the time block. This is mutually exclusive with the existing live_span enum.
    • Funnel Widget Support: Includes the funnel_definition widget.
    • Toplist Sort Control: Adds a sort block to toplist widget requests to control direction, limit, and sorting criteria.
    • Expanded Formula/Query Support: Consistent support for query/formula blocks across more widgets, including the distribution widget.
    • Correct number_format Serialization: Fixes serialization issues for number_format blocks on non-query-table widgets.
    • Consistent Legacy Queries: audit_query and network_query are now available on all standard request widgets (change, distribution, heatmap, query_value, toplist, and sunburst).
    • API Resilience: Silently ignores unknown fields, preventing import failures when the Datadog API introduces new fields.
  7. Manage env-gated resources

    master

    Some resources are gated behind environment variables to allow private beta testing without publishing them to the Terraform Registry.

    Example: datadog_integration_databricks_account is gated by DD_TERRAFORM_DATABRICKS_INTEGRATION_ENABLED (defaults to false).

    Local Development:

    • Set DD_TERRAFORM_DATABRICKS_INTEGRATION_ENABLED=true to make the resource visible to terraform plan/apply and local acceptance tests.
    • Warning: If you run make docs with this variable set, tfplugindocs will generate documentation for the gated resource. You must revert these changes before pushing, as gated resources are intentionally excluded from public registry documentation.
  8. Manage Datadog Custom Allocation Rule order with datadog_custom_allocation_rules

    master

    The datadog_custom_allocation_rules resource is used to manage the execution order of Datadog Custom Allocation Rules. It accepts a list of rule IDs and determines how rules created outside of Terraform (e.g., via the Datadog UI) are handled.

    Modes of Operation

    1. Preserve Mode (Default): Set override_ui_defined_resources = false. This mode allows unmanaged rules to exist at the end of the order. Terraform will preserve existing rules created outside of Terraform as long as they are positioned at the end of the list. If unmanaged rules are found in positions other than the end, an error will be thrown during the terraform plan phase.

    2. Override Mode: Set override_ui_defined_resources = true. This mode enforces strict management. Any rules created via the UI that are not explicitly defined in your Terraform configuration will be deleted, making Terraform the sole source of truth for both rule existence and ordering.

    resource "datadog_custom_allocation_rules" "override_order" {
      override_ui_defined_resources = true
      rule_ids = [
        datadog_custom_allocation_rule.rule_1.id,
        datadog_custom_allocation_rule.rule_2.id,
        datadog_custom_allocation_rule.rule_3.id
      ]
    }
  9. Manage Datadog Logs Integration Pipelines

    master

    The datadog_logs_integration_pipeline resource manages integration pipelines. These are pipelines automatically installed by Datadog when logs are sent with specific sources. While you typically do not need to maintain or update these pipelines, declaring them as resources allows you to control their execution order by referencing them in a datadog_logs_pipeline_order resource. If you do not need to manage pipeline order, you can omit this resource declaration.

    resource "datadog_logs_integration_pipeline" "python" {
      is_enabled = true
    }
  10. Configure widget request types in datadog_dashboard

    master

    When defining a widget within a datadog_dashboard, you must provide a request block. This block defines the data source for the widget. You must include exactly one of the following query types within the request block:

    • q: A metric query (String).
    • apm_query: An APM-specific query block.
    • log_query: A log-specific query block.
    • rum_query: A Real User Monitoring (RUM) query block.
    • security_query: A security-specific query block.
    • process_query: A process-specific query block.

    Each query type has its own specific schema requirements (e.g., apm_query requires an index).

    # Example structure for a widget request
    
    ```hcl
    widget {
      request {
        q = "avg:system.cpu.idle{host:host0}"
      }
    }
  11. Use formula-based widget queries in datadog_powerpack_v2

    master

    While legacy metric query strings (e.g., q = "avg:system.cpu.user{*}") are still supported in datadog_powerpack_v2, formula-based queries must use the query/formula block structure. This is consistent with the syntax used in datadog_dashboard_v2.

    widget {
      timeseries_definition {
        request {
          formula { formula_expression = "query1 / query2" }
          query {
            metric_query {
              name  = "query1"
              query = "sum:requests.count{*}"
            }
          }
          query {
            metric_query {
              name  = "query2"
              query = "sum:requests.errors{*}"
            }
          }
        }
      }
    }