DOCSEARCHbyAlgolia
Submit Docs

Flux Source Controller

repository·main·Indexed 42 days ago

https://github.com/fluxcd/source-controller

A Kubernetes operator that automates the acquisition and validation of artifacts from external sources including Git, OCI, Helm, and S3-compatible buckets. It implements the source.toolkit.fluxcd.io API (v1, v1beta1, v1beta2, v1alpha1) to provide GitOps workflows with resources such as GitRepository, OCIRepository, HelmRepository, HelmChart, and Bucket.

Tokens
63.7K
Snippets
177
Records
257
Agent score
53%

What's inside Flux Source Controller

  1. Overview of the source-controller

    main

    The source-controller is a Kubernetes operator specialized in acquiring artifacts from external sources. It is a core component of the GitOps toolkit and implements the source.toolkit.fluxcd.io API.

    Key Capabilities:

    • Authentication: Supports SSH, user/password, API tokens, and Workload Identity.
    • Validation: Validates source authenticity using PGP, Cosign, or Notation.
    • Change Detection: Detects source changes based on update policies (e.g., semver).
    • Fetching: Fetches resources on-demand or on a schedule.
    • Packaging: Packages fetched resources into standard formats like tar.gz or yaml.
    • Addressing: Makes artifacts addressable by source identifiers such as sha, version, or ts (timestamp).
    • Availability: Makes artifacts available in-cluster for other components and notifies interested parties via status conditions, events, and hooks.
    • Event Reaction: Reacts to push events from Git, Helm, and OCI artifacts (integrated with notification-controller).

  2. Overview of the source.toolkit.fluxcd.io/v1 API

    main
    The source.toolkit.fluxcd.io/v1 API is the specification used to define the desired state sources for Kubernetes clusters within the Flux ecosystem. It provides a standardized way to declare where configuration, Helm charts, or container images are located so that other controllers can consume them.

  3. Overview of source.toolkit.fluxcd.io/v1alpha1 API

    main
    The source.toolkit.fluxcd.io/v1alpha1 API is used to define the desired state sources for Kubernetes clusters. It provides a standardized way to declare external sources of truth (like Git repositories, Helm repositories, or cloud buckets) that other Flux controllers can then consume to synchronize cluster state.

  4. Overview of source.toolkit.fluxcd.io/v1beta1 API

    main
    The source.toolkit.fluxcd.io/v1beta1 API is used to define the desired state sources for Kubernetes clusters. It provides a standardized way to declare external sources (like Git repositories, Helm repositories, Helm charts, or cloud buckets) so that other Flux controllers can consume them to synchronize cluster state.

  5. Source Controller Overview

    main
    The Source Controller is a component of Flux that manages the retrieval and synchronization of external sources (such as Git repositories or S3 buckets) into Kubernetes. It provides the underlying mechanism to fetch artifacts and make them available for other Flux controllers (like the Kustomize controller) to consume.

  6. Use the source.toolkit.fluxcd.io/v1beta2 API

    main

    The source.toolkit.fluxcd.io/v1beta2 API is used to define the desired state sources for Kubernetes clusters. It provides a standardized way to declare external sources (like Git repositories, Helm charts, or OCI registries) that Flux controllers can then consume to synchronize cluster state.

    Supported source kinds include:

    • GitRepository
    • OCIRepository
    • HelmRepository
    • HelmChart
    • Bucket
    apiVersion: source.toolkit.fluxcd.io/v1beta2
kind: GitRepository
metadata:
  name: example-repo
spec:
  interval: 1m
  url: https://github.com/org/repo

  8. What is a Bucket source?

    main

    The Bucket API defines a Source that produces an Artifact from objects stored in S3-compatible storage solutions. This includes Amazon S3, Google Cloud Storage, Minio, Alibaba Cloud OSS, and others.

    When a Bucket source is active, the source-controller:

    1. Periodically checks the specified object storage bucket.
    2. Compiles a list of object keys and their etags (filtering via default ignore rules).
    3. Calculates a digest (defaulting to SHA256) of this list to use as the Artifact revision.
    4. If the revision has changed, it fetches all objects and archives them into a .tar.gz tarball.
    5. Reports the resulting Artifact in the .status.artifact field.

  10. Use OCIRepository to fetch artifacts from OCI registries

    main

    The OCIRepository API is a Source type used to produce an Artifact from an OCI (Open Container Initiative) repository. The source-controller periodically checks the specified OCI URL for changes. When a change in the resolved digest is detected, the controller pulls the data and archives it as a .tar.gz Artifact within the cluster.

    Key behaviors:

    • Polling: The controller checks the repository at the interval defined in .spec.interval.
    • Resolution: It resolves the provided reference (e.g., a tag) to a specific SHA256 digest.
    • Artifact Creation: A new Artifact is created whenever the current digest differs from the latest fetched digest. The resolved tag and digest are reported in .status.artifact.revision.
    apiVersion: source.toolkit.fluxcd.io/v1
kind: OCIRepository
metadata:
  name: podinfo
  namespace: default
spec:
  interval: 5m0s
  url: oci://ghcr.io/stefanprodan/manifests/podinfo
  ref:
    tag: latest

  11. Understand GitRepository lifecycle conditions

    main

    The GitRepository uses Kubernetes Conditions to communicate its state. Monitoring these conditions is the recommended way to determine if a source is ready or failing.

    Reconciling State

    A GitRepository is marked as Reconciling when it is fetching state, a new generation is detected, or a new revision is being resolved.

    • Condition Type: Reconciling (status: True)
    • Reasons: Progressing or ProgressingWithRetry (if retrying)
    • Outdated Artifact: If reconciliation is due to a new revision, an additional condition ArtifactOutdated (status: True, reason: NewRevision) is added.

    Ready State

    A GitRepository is Ready when it has a valid artifact in storage and is up-to-date with the remote repository.

    • Condition Type: Ready (status: True, reason: Succeeded)
    • Storage Status: When the artifact is successfully archived, ArtifactInStorage (status: True, reason: Succeeded) is also set.

    Failed State

    If reconciliation fails (e.g., due to network issues, invalid credentials, or missing secrets), the Ready condition is set to False and one of the following error conditions is added:

    • FetchFailed (e.g., AuthenticationFailed or GitOperationFailed)
    • IncludeUnavailable
    • StorageOperationFailed
    • SourceVerifiedCondition (status: False, reason: Failed) if Git commit signature verification fails.

  12. Configure the Bucket provider

    main

    The .spec.provider field determines how the source-controller communicates with the object storage. If not specified, it defaults to generic.

    Supported providers:

    • generic: For any Amazon S3 compatible storage (e.g., Minio, GCS, Wasabi). Requires a secretRef with accesskey and secretkey.
    • aws: For Amazon S3. Supports EC2 instance metadata, static credentials via secretRef, or Workload Identity.
    • azure: For Azure Blob Storage. Supports Managed Identity, Service Principals (via secret), Shared Key, or SAS Tokens.
    • gcp: For Google Cloud Storage. Supports static service account keys via secretRef or Workload Identity.