tfmigrate

tfmigrate is a state migration tool designed for GitOps workflows using Terraform or OpenTofu. It allows you to define Terraform state operations (like mv, rm, import, or replace-provider) declaratively in HCL files. This enables you to commit state changes to version control, plan them to ensure no infrastructure drift occurs, and apply them safely as part of a CI/CD pipeline.

Key features include:

• GitOps friendly: Write state operations in HCL.
• Monorepo support: Easily split or merge tfstate files by moving resources between directories.
• Dry run migration: Use tfmigrate plan to simulate operations against a temporary local state to verify that terraform plan shows no changes before touching remote state.
• Migration history: Tracks applied migrations to ensure they are executed in sequence.

The xmv command is similar to mv but supports wildcards (*) in the source definition. The matched values can be used in the destination via ordinal numbers ($1, $2, etc.).

To use ordinal numbers when there is ambiguity, you must escape the dollar sign by doubling it (e.g., $${1}).

# Renaming all security groups by appending '2' to the name
migration "state" "test" {
  dir = "dir1"
  actions = [
    "xmv aws_security_group.* aws_security_group.$${1}2",
  ]
}

You can install tfmigrate using Homebrew, by downloading binaries, or building from source.

Homebrew (macOS)

brew install tfmigrate

Download Binaries

Download the latest compiled binaries from the GitHub releases page and add them to your executable path.

Build from Source

Requires a Go 1.26+ development environment:

git clone https://github.com/minamijoyo/tfmigrate
cd tfmigrate/
make install
tfmigrate --version

When using Terragrunt, you must set TFMIGRATE_EXEC_PATH to point to the terragrunt executable. The configuration depends on your Terragrunt version:

For Terragrunt < v0.73.0

TFMIGRATE_EXEC_PATH=terragrunt tfmigrate $OTHEROPTIONS

For Terragrunt >= v0.73.0

TFMIGRATE_EXEC_PATH="terragrunt run --" tfmigrate $OTHEROPTIONS

Using Terragrunt with Dynamic State

If you use Terragrunt's dynamic state generation, your remote_state block must include a generate block to ensure tfmigrate can work with a file-based backend during planning:

remote_state {
  backend = "s3"

  config = {
    bucket = "highway-terraform-state"
    # Other config here
  }

  generate = {
    path      = "backend.tf"
    if_exists = "overwrite_terragrunt"
  }
}

To migrate resources within a single Terraform/OpenTofu directory, create a migration "state" block in an HCL file.

1. Define the migration: Create a file (e.g., state_mv.hcl) with the dir and actions (e.g., mv, rm, import).
2. Plan the migration: Run tfmigrate plan <file> to verify that the migration results in a clean terraform plan (no changes).
3. Apply the migration: Run tfmigrate apply <file> to execute the operations against the remote state.

Example state_mv.hcl:

migration "state" "test" {
  dir = "dir1"
  actions = [
    "mv aws_security_group.foo aws_security_group.foo2",
    "mv aws_security_group.bar aws_security_group.bar2",
  ]
}

tfmigrate apply state_mv.hcl

Migration files define Terraform state operations using HCL2. Files must use either the .hcl extension (for HCL native syntax) or .json (for HCL JSON syntax).

Best Practices:

• Each file must contain exactly one migration block. Multiple blocks are not allowed to prevent partial failure issues.
• While filenames can be arbitrary, tfmigrate applies unapplied migrations in alphabetical order in history mode. To avoid git conflicts and ensure correct ordering, use a timestamp prefix (e.g., 20201114000000_dir1.hcl) instead of simple serial numbers.

# HCL Native Syntax (.hcl)
migration "state" "test" {
  dir = "dir1"
  actions = [
    "mv aws_security_group.foo aws_security_group.foo2",
  ]
}

To move resources between different Terraform/OpenTofu directories (splitting or merging states), use a migration "multi_state" block. This requires specifying both from_dir and to_dir.

Example tfmigrate_multi_state_test.hcl:

migration "multi_state" "test" {
  from_dir = "."
  to_dir   = "dir2"

  actions = [
    "mv aws_security_group.baz aws_security_group.baz2",
  ]
}

Use tfmigrate plan and tfmigrate apply as you would with single-state migrations.

tfmigrate apply tfmigrate_multi_state_test.hcl

A configuration file (defaulting to .tfmigrate.hcl) allows you to define migration directories, backend types, and history storage.

Requirements:

• Must be written in HCL2 (extension .hcl) or HCL JSON (extension .json).
• Must contain exactly one tfmigrate block.
• You can access environment variables within the HCL file using the env variable (e.g., ${env.VAR_NAME}).

Core Attributes:

• migration_dir: Path to migration files. (Default: .)
• is_backend_terraform_cloud: Set to true if using the Terraform cloud block. When true, you must also specify a workspace name in your migration files.

tfmigrate {
  migration_dir = "./tfmigrate"
  is_backend_terraform_cloud = true
  history {
    storage "s3" {
      bucket = "tfmigrate-test"
      key    = "tfmigrate/${env.ENV_NAME}/history.json"
    }
  }
}

You can customize tfmigrate behavior using the following environment variables:

• TFMIGRATE_LOG: Sets the log level. Valid values: TRACE, DEBUG, INFO, WARN, ERROR. (Default: INFO)
• TFMIGRATE_EXEC_PATH: The command used to execute Terraform. Use this to inject wrappers (e.g., direnv exec . terraform) or to switch to OpenTofu (set to tofu). (Default: terraform)
• TFMIGRATE_CONFIG: Path to the configuration file. (Default: .tfmigrate.hcl)

Note: Some history storage implementations may require additional cloud-specific environment variables (e.g., AWS_PROFILE, AWS_REGION).

To see the exact terraform or tofu commands being executed by tfmigrate, set the TFMIGRATE_LOG environment variable to DEBUG.

TFMIGRATE_LOG=DEBUG tfmigrate plan <migration_file>