Terraform Provider Elastic Stack

Factory workflows are label-triggered, issue-scoped agentic workflows in CI. Each "factory" reacts to a specific GitHub label on an issue, performs deterministic pre-activation, and delegates work to an AI agent with bounded outputs (typically one PR and one sticky comment).

Workflow Lifecycle

1. Issue Classification: A classifier reads new issues and applies routing labels (e.g., needs-research, needs-reproduction).
2. Triggering: A maintainer promotes an issue by applying a factory trigger label (e.g., research-factory, code-factory).
3. Pre-activation: The system qualifies the trigger, captures context, suppresses duplicate PRs, sanitizes content, and sets a phase-* label.
4. Agent Execution: An agent uses tools (like elastic-docs MCP or github tools) to perform the task and emit bounded outputs.

Factory Trigger Labels

Note: These labels must exist in the repository settings before they can be used to trigger workflows.

The repository uses a dual-track automation system to move work from GitHub issues to merged Pull Requests. This system combines GitHub Agentic Workflows (AWF) running in CI with OpenSpec, which serves as the canonical source for functional requirements.

The Two Tracks

1. Reactive Track (Human-raised issues): This is the primary path for new features and bugs. It uses an Issue Classifier to route issues to specific 'factories' (like research-factory or reproducer-factory). These flows typically move through OpenSpec (creating proposals and design artifacts) before reaching the implementation phase.
2. Proactive Track (Scheduled scans): This track handles mechanical quality work such as refactors, test coverage gaps, and flaky tests. It uses scheduled scanners that automatically create triaged issues and hand them directly to the code-factory, often skipping the OpenSpec requirement for purely mechanical changes.

Core Components

• GitHub Agentic Workflows (AWF): Located in .github/workflows/, these run in CI. They include 'factories' (reactive to issue labels) and 'continuous quality' workflows (scheduled).
• OpenSpec: Located in openspec/, this holds requirements. Changes are managed via artifacts in openspec/changes/<id>/ (proposals, designs, tasks, delta specs).
• Skills: Located in .agents/skills/, these are used locally on a developer's machine to apply OpenSpec changes.

The examples-validation process ensures that Terraform examples located in examples/resources/ and examples/data-sources/ remain valid and copy-pasteable. It achieves this by running each .tf file in isolation through a PlanOnly step using the provider's acceptance-test harness.

Key validation rules include:

• Plan-time validation: Every example must pass a terraform-plugin-testing PlanOnly step without error severity diagnostics.
• Self-containment: Each .tf file must be able to plan in isolation. It cannot depend on resources, data sources, locals, or variables defined in sibling files within the same directory.
• Error attribution: Failures must be attributed to a single file, with subtest names including the file's path relative to the examples/ root to allow targeting via go test -run.
• Non-mutation: The validation harness only performs planning; it does not apply, update, import, or destroy any resources.

The entitycore substrates (ResourceBase and DataSourceBase) handle the boilerplate for wiring the provider's client factory during the Configure lifecycle method.

Behavior:

1. Conversion: The substrate's Configure implementation calls clients.ConvertProviderDataToFactory to transform the provider data.
2. Error Handling:
   • If the conversion results in error-level diagnostics, the substrate will not assign the new factory. It preserves any previously stored *clients.ProviderClientFactory from a prior successful Configure call.
   • If there are no error-level diagnostics, the substrate assigns the conversion result (which may be nil if providerData is nil), replacing any prior value.
3. Access: Concrete resources and data sources must access the stored factory using the Client() method rather than attempting to use a mutable exported field.

The ecs_mapping attribute allows you to map query columns to Elastic Common Schema (ECS) fields. It uses a nested map structure where each key (the column name) must define exactly one mapping type using a three-way exactly-one-of constraint.

Supported Mapping Types

For every entry in the ecs_mapping map, you must provide exactly one of these keys:

1. field: Maps the column to a specific field name.
   • Example: { "process.name" = { field = "cmdline" } }
2. value: Maps the column to a static scalar string value.
   • Example: { "event.category" = { value = "process" } }
3. values: Maps the column to a static set of string values.
   • Example: { "event.category" = { values = ["process", "network"] } }

Note: Providing more than one of these keys (e.g., both field and value) or providing an empty object will result in a Terraform plan validation error.

ecs_mapping = {
  "process.name" = { field = "cmdline" }
  "event.category" = { value = "process" }
  "network.status" = { values = ["established", "listen"] }
}

Elasticsearch automatically injects default values into Watcher configurations (e.g., adding lang: "painless" to scripts). To prevent these injected defaults from causing unnecessary Terraform diffs, the provider uses semantic equality for the following attributes:

• input
• transform
• condition
• actions

Injected Defaults Handled

When comparing your configuration to the remote state, the provider ignores the following if they were injected by Elasticsearch:

Search Requests

For any search request object, the following are treated as equal if they are absent in your config but present in the API response:

• rest_total_hits_as_int: defaults to true
• search_type: defaults to "query_then_fetch"
• indices: defaults to []

Scripts

• lang: defaults to "painless" if the script object is present.

Logging

• level: defaults to "info" if the logging object is present.

Note: Explicitly setting a value in your configuration (e.g., rest_total_hits_as_int = false) will be preserved and will trigger a change if modified.

The provider manages chunking_settings to ensure that your Terraform plan remains stable even when Elasticsearch applies default values.

Default Values Applied by Provider

When you configure a strategy but omit specific keys, the provider populates the following defaults:

• sentence strategy:
  • strategy = "sentence"
  • max_chunk_size = 250
  • sentence_overlap = 1
• word strategy:
  • max_chunk_size = 250
  • overlap = 100
• none and recursive strategies: No automatic defaults are applied; the provider preserves your exact input shape.

Task Settings Drift Behavior

To prevent perpetual drift caused by server-side defaults, the provider follows these rules for task_settings:

1. It retains only the keys you explicitly defined in your Terraform configuration.
2. It discards any additional keys returned by the Elasticsearch API that were not in your configuration.
3. If a key you did configure has a different value on the server, the provider will detect the drift and show it in the plan.

When configuring vis_config for panels of type = "vis", you must choose between two mutually exclusive methods:

1. By-Reference (by_reference): Used to link to an existing Kibana saved object.

   • Requires ref_id (the reference name in the API).
   • Can include references_json (a normalized JSON array of { id, name, type }) to wire objects like Lens visualizations to the ref_id.
   • Supports structured drilldowns to other dashboards, Discover, or custom URLs.

2. By-Value (by_value): Used for inline Lens visualization configuration.

   • Requires exactly one typed chart kind to be set.
   • For complex raw JSON configurations, use the panel-level config_json instead.

Note on Drilldowns for By-Reference panels:

• Dashboard: Requires dashboard_id and label. Can optionally use_filters or use_time_range.
• Discover: Requires label.
• URL: Requires label, url, and a trigger (e.g., on_click_row, on_click_value, on_open_panel_menu, on_select_range). The Kibana API will reject URL drilldowns if the trigger is omitted.

### Nested Schema for `sections.panels.vis_config`
Optional:
- `by_reference` (Attributes) By-reference `vis` configuration: structured `drilldowns`, `ref_id`, optional `references_json`, and optional `time_range`.
- `by_value` (Attributes) Inline by-value Lens visualization configuration for `type = "vis"` panels (`vis_config`). Exactly one typed chart kind must be set.