DOCSEARCHbyAlgolia
Submit Docs

driftctl Documentation

repository·main·Indexed 25 days ago

https://github.com/snyk/driftctl

driftctl is an open-source CLI tool used to detect infrastructure drift by comparing actual cloud provider resources against Terraform state files. It helps DevSecOps teams identify unmanaged or manually changed resources across AWS, GitHub, Azure, and GCP. The tool provides capabilities to scan, analyze, and ignore resources, and supports custom middlewares for resource reconciliation and noise filtering.

Tokens
8.7K
Snippets
18
Records
55
Agent score
80%

What's inside driftctl

  1. Overview of driftctl

    main

    driftctl is an open-source CLI tool designed to measure Infrastructure as Code (IaC) coverage and track infrastructure drift. It works by scanning cloud provider resources and mapping them against your IaC code (specifically Terraform) to identify discrepancies.

    Key Capabilities:

    • Scan: Scans cloud providers to map resources against IaC code.
    • Analyze: Analyzes diffs to warn about drift and unmanaged resources.
    • Ignore: Allows users to specify resources to be ignored in scans.
    • Output: Supports multiple output formats.

    Supported Technologies:

    • IaC: Terraform
    • Cloud Providers: AWS, GitHub, Azure, GCP

    :warning: Note: This tool is in beta and may undergo breaking changes. The project is currently in maintenance mode.

  2. Understand the acceptance testing workflow

    main

    The driftctl acceptance test framework follows this execution lifecycle:

    1. OnStart: Executes arbitrary code before the test begins.
    2. terraform apply: Applies the provided Terraform configuration.
    3. Check Loop: For each defined AccCheck:
      • PreExec: Executes code before the scan (e.g., manually modifying a resource to create drift).
      • driftctl scan: Runs the driftctl command.
      • check results: Validates the scan output against assertions.
      • PostExec: Executes code after the scan.
    4. OnEnd: Executes code after all checks are complete.
    5. terraform destroy: Cleans up the Terraform-managed resources.

    Warning: While driftctl handles Terraform resource removal, you are responsible for removing any unmanaged resources created during the PreExec step.

  3. Understand driftctl core concepts and terminology

    main

    To work with the driftctl codebase, it is important to understand how it retrieves data and the terminology used for its components:

    Data Retrieval

    • Resource Listing: Performed using cloud provider SDKs.
    • Resource Details: Retrieved by calling Terraform providers via gRPC.

    Terminology

    • Remote: A representation of a cloud provider.
    • Resource: An abstract representation of a cloud provider resource (e.g., an S3 bucket or an EC2 instance).
    • Enumerator: A component used to list resources of a specific type from a given remote. There should be exactly one Enumerator per resource type.

  4. Declare a new remote provider

    main

    To add a new remote provider (e.g., a cloud provider like AWS, GitHub, GCP, or Azure), follow these steps:

    1. Create the provider directory: Create a new directory at pkg/remote/<provider name>.
    2. Define the remote name: Create an init.go file in the new directory and define a constant for the remote name. Use the +tf suffix to indicate that Terraform is used to retrieve resource details.
    3. Implement the Init function: Create an Init function to initialize the provider, resource enumerators, and the provider library. This function should handle default versions, create the Terraform provider, initialize a cache, and add the provider to the providerLibrary.
    4. Implement the provider representation: Create a provider.go file containing a struct that composes with terraform.TerraformProvider. Implement a constructor (e.g., New<Provider>TerraformProvider) that uses tf.NewProviderInstaller to retrieve the provider and configures the terraform.NewTerraformProvider with a GetProviderConfig callback.
    5. Annotate configuration: Ensure any configuration structs used in GetProviderConfig are annotated with cty tags so they can be passed to the Terraform provider.
    // Example remote name constant in pkg/remote/<provider>/init.go
const RemoteAWSTerraform = "aws+tf"

// Example configuration with cty tags
type githubConfig struct {
	Token        string
	Owner        string `cty:"owner"` 
	Organization string
}

  5. Configure credentials for acceptance testing

    main

    Acceptance tests require cloud provider credentials. For best results, use two distinct sets of credentials:

    1. Read/Write access: Required for Terraform actions (apply, destroy, etc.).
    2. Read-only access: Required for the driftctl scan.

    You can override environment variables for specific lifecycle stages (like PreExec or PostExec) by adding an ACC_ prefix to the environment variable name.

    For AWS, use ACC_AWS_PROFILE to override the named profile used for Terraform operations while keeping AWS_PROFILE for the driftctl scan.

    $ ACC_AWS_PROFILE=read-write-profile AWS_PROFILE=read-only-profile DRIFTCTL_ACC=true go test -run=TestAcc_ ./pkg/resource/aws/aws_instance_test.go

  6. Define a new resource type

    main

    To add a new resource, create a file at pkg/resource/<providername>/<resourcetype>.go. This file must define a string constant for the resource type identifier and an initialization function to register metadata with the resource.SchemaRepositoryInterface.

    After defining the resource, you must:

    1. Register the initialization function in pkg/resource/<providername>/metadatas.go.
    2. Add the resource type to the supportedTypes map in pkg/resource/resource_types.go to ensure compatibility with the Terraform state reader.
    const AwsIamRoleResourceType = "aws_iam_role"

func initAwsIAMRoleMetaData(resourceSchemaRepository resource.SchemaRepositoryInterface) {
	// force_detach_policies should not be compared so it will be removed before the comparison
	resourceSchemaRepository.SetNormalizeFunc(AwsIamRoleResourceType, func(res *resource.Resource) {
		val := res.Attrs
		val.SafeDelete([]string{"force_detach_policies"})
	})
}

  7. Update golden files for assertions

    main

    The project uses the golden file pattern to assert on results. If you modify code that affects these assertions (e.g., changing an S3 bucket policy), you can update the existing golden files using the --update flag with the go test command.

    Warning: Updating golden files may trigger calls to external services. If you are using mocked AWS responses in JSON golden files, ensure the proper resources are configured on the AWS side before running the update.

    $ go test ./pkg/remote/aws/ --update s3_bucket_policy_no_policy

  8. Run driftctl acceptance tests

    main

    Acceptance tests in driftctl are designed to apply Terraform code and then run a series of Checks to verify results using JSON output.

    To run acceptance tests, ensure the test function name is prefixed with TestAcc_ and set the DRIFTCTL_ACC=true environment variable. You can target specific test files using the standard go test command.

    $ DRIFTCTL_ACC=true go test -run=TestAcc_ ./pkg/resource/aws/aws_instance_test.go

  9. Use middlewares for resource reconciliation and noise filtering

    main

    Middlewares can be used for several specific reconciliation tasks:

    1. Matching: Helping driftctl match IaC resources to their corresponding remote resources.
    2. Noise Filtering: Removing noise caused by provider-default resources.
    3. Transformation: Transforming resource types or structures (e.g., converting specific attachment resources into a generic attachment resource).
    4. Edge Case Handling: Addressing specific provider behaviors or ID mismatches.

  10. Register a new remote provider in driftctl

    main

    After implementing the provider logic, you must register it within the core driftctl logic:

    1. Update supportedRemotes: In pkg/remote/remote.go, add your new remote constant to the supportedRemotes slice.
    2. Update Activate: In pkg/remote/remote.go, add a new case to the switch statement in the Activate function to call your provider's Init function.

  11. Prepare driftctl to support new resources for a provider

    main

    Once a provider is registered, you can add support for specific resources:

    1. Create resource directory: Create a directory at pkg/resource/<provider name>.
    2. Initialize metadata: Create a metadatas.go file in that directory and implement the InitResourcesMetadata(resourceSchemaRepository resource.SchemaRepositoryInterface) function.
    3. Link metadata initialization: Call InitResourcesMetadata from the Init function in your provider's pkg/remote/<provider>/init.go file.
    4. Generate test schemas: Use the TestCreateNewSchema function located in test/terraform/schemas_test.go to generate a schema file for use with the mocked provider in tests.

  12. Install testing tools and run tests

    main

    To run the driftctl test suite, you must first install the required tools using make install-tools. Once installed, you can execute the tests using make test. The project uses gotestsum to wrap the standard go test command.

    Note: While unit tests are required for PRs, acceptance tests are optional.

    $ make install-tools
$ make test