Tekton Pipelines-as-Code

repository·main·Indexed 41 days ago

https://github.com/tektoncd/pipelines-as-code

An opinionated CI/CD framework for Tekton that enables managing pipelines as code by storing them in Git repositories. It automates PipelineRun lifecycles based on Git events, supports multi-provider integration (GitHub, GitLab, Bitbucket, Forgejo), and provides ChatOps capabilities via PR comments such as /test, /retest, and /cancel.

Tokens
95.2K
Snippets
290
Records
423
Agent score
51%

What's inside Pipelines-as-Code

  1. Overview of Pipelines-as-Code

    main

    Pipelines-as-Code is an opinionated CI/CD system built on Tekton. It allows you to define CI/CD workflows using Tekton PipelineRuns and Tasks stored directly within your application's source code repository. This approach enables versioning, code reviews, and collaboration on pipeline definitions using standard Git workflows.

    Key capabilities include:

    • Automatic Triggers: Automatically triggers PipelineRuns for Pull Requests and Push events from SCM providers.
    • GitOps via Comments: Control pipelines directly from Pull Request comments using commands like /retest, /test, and /cancel.
    • Automatic Task Resolution: Automatically discovers and resolves Task definitions from local files, Artifact Hub, or remote URLs.
    • Event Filtering: Configure separate pipelines for different Git event types (e.g., specific pipelines for push to main vs. pull_request events).
  2. Overview of the tkn pac CLI

    main

    The tkn pac command-line tool is a plugin for the Tekton CLI (tkn). It is used to install, configure, and manage Pipelines-as-Code (PAC) resources directly from your terminal.

    Key capabilities include:

    • Setup & Configuration: bootstrap to install PAC with a GitHub App, and webhook to manage Git provider secrets.
    • Resource Management: create and delete Repository Custom Resources (CRs), and list or describe them to check status.
    • Development & Debugging: generate to scaffold starter PipelineRuns, cel to evaluate Common Expression Language (CEL) expressions against webhook payloads locally, and resolve to simulate server-side PipelineRun processing locally.
    • Observability: logs to stream PipelineRun logs and info to view installation details and globbing patterns.
  3. Overview of Tekton Pipelines-as-Code

    main

    Tekton Pipelines-as-Code is an opinionated CI/CD framework for Tekton that enables defining and running pipelines directly from a Git repository.

    Key features include:

    • Git-native workflow: Pipelines are stored in a .tekton/ directory within your repository and are versioned alongside your source code.
    • Multi-provider support: Works with GitHub (Apps & Webhooks), GitLab, Bitbucket (Cloud & Data Center), and Forgejo.
    • ChatOps capabilities: Manage runs using PR comments like /test, /retest, /cancel, or by using [skip ci].
    • Inline resolver: Automatically bundles remote tasks from Artifact Hub before they are submitted to the cluster.
    • Automated housekeeping: Automatically prunes old PipelineRuns and cancels superseded runs when new pushes occur.
  4. Core workflows in Pipelines-as-Code

    main

    Pipelines-as-Code provides several core workflows for managing CI/CD via GitOps and Kubernetes resources. Key tasks include:

    • Authoring PipelineRuns: Creating pipelines, defining CEL variables, and managing GitHub tokens.
    • Configuring the Repository CR: Setting up repository connections, concurrency limits, and comment settings.
    • Event Matching: Defining when pipelines trigger using on-event, on-target-branch, path filters, CEL expressions, and labels.
    • Issuing GitOps Commands: Using slash commands like /retest, /test, and /cancel to control pipelines.
    • Monitoring Statuses: Tracking PipelineRun status and failure detection.
    • Validating Resources: Using OpenAPI schemas to validate Repository CRs in your IDE.
    • AI/LLM Analysis: Utilizing automated pipeline analysis via LLMs.
  5. Explore Pipelines-as-Code API resources

    main

    The Pipelines-as-Code API is structured around Custom Resources (CRs) and configuration objects. To manage your pipelines, you will primarily interact with the following resources:

    • Repository CR: The central custom resource used to link Pipelines-as-Code to a specific Git repository.
    • Repository Spec: Defines the detailed configuration and desired state for a Repository resource.
    • ConfigMap Reference: Contains global settings that apply across all managed repositories.
    • Settings Reference: Contains repository-specific settings, including authorization, provider-specific behaviors, and AI analysis configurations.
  6. Overview of Pipelines-as-Code Core Components

    main

    Pipelines-as-Code (PAC) consists of three primary components that manage the lifecycle of Git-driven CI/CD:

    1. Controller (pipelines-as-code-controller): The central brain. It receives webhooks, validates permissions, resolves pipeline definitions from the .tekton/ directory, creates PipelineRun resources, and manages concurrency/queuing. It also handles GitOps commands like /test, /retest, and /cancel.
    2. Watcher (pipelines-as-code-watcher): The feedback loop. It monitors PipelineRun status changes to report progress back to Git providers (via checks, comments, or badges) and manages the cleanup of old PipelineRun resources based on retention policies.
    3. Webhook Handler (pipelines-as-code-webhook): The entry point. Embedded in the controller, it receives HTTP webhook requests, validates signatures, and routes events to the controller.
  7. Understand the Pipelines-as-Code configuration layers

    main

    Pipelines-as-Code uses a two-layer configuration model to manage settings. This allows administrators to set cluster-wide defaults while allowing repository owners to apply specific overrides.

    LayerResourceScope
    Globalpipelines-as-code ConfigMapCluster-wide defaults, set by administrators
    Per-repositoryRepository CR spec.settingsPer-repository overrides, set by repository owners

    Configuration Inheritance: When a setting is defined in both layers, the value in the Repository CR spec.settings takes precedence. Global settings are used as fallbacks whenever a repository-specific override is absent.

  8. Use the cel: prefix for advanced CEL expressions

    main

    For complex logic, string manipulation, or conditional evaluations, use the cel: prefix. This tells Pipelines-as-Code to evaluate the content as a Common Expression Language (CEL) expression rather than a simple property lookup.

    When using cel:, you have access to the following data sources:

    • body: The full webhook payload.
    • headers: HTTP request headers.
    • files: Information about changed files, including files.all, files.added, files.deleted, files.modified, and files.renamed.
    • pac: Standard Pipelines-as-Code parameters such as pac.revision, pac.target_branch, and pac.source_branch.

    Note: If a cel: expression contains a syntax error or fails to evaluate, Pipelines-as-Code returns an empty string, allowing the PipelineRun to continue.

    params:
      - name: dynamic-param
        value: "{{ cel: body.action == 'opened' ? 'new' : 'existing' }}"
  9. Access the GitHub App installation token

    main

    Pipelines-as-Code automatically generates a temporary GitHub App installation token for each PipelineRun. This token allows your pipeline to interact with the GitHub API (e.g., posting comments or updating statuses).

    The token is stored in a temporary secret named after the value of the {{ git_auth_secret }} variable. Within that secret, the token is located under the key git-provider-token.

  10. Parameter expansion precedence and rules

    main

    When defining custom parameters in the spec.params list, Pipelines-as-Code follows these rules:

    • Value vs Secret: If both value and secret_ref are defined, value takes precedence.
    • Missing Values: If both value and secret_ref are omitted, and no GitOps command overrides the parameter, the parameter remains unexpanded (it will appear as {{ param }} in the PipelineRun).
    • Missing Name: If the name field is omitted from a params entry, the parameter is ignored.
    • Duplicates: If multiple params entries share the same name, the last one defined is used.
    • No Default: Defining a parameter without a value or secret_ref (e.g., - name: start_time) means it will only expand if a value is supplied via a GitOps command.
  11. How GitLab fork merge requests and status reporting work

    main

    When a Merge Request (MR) comes from a forked repository to an upstream repository, Pipelines-as-Code uses a three-layer fallback mechanism to report build status:

    1. Source Project (Fork): Attempts to set the commit status directly on the fork. This requires a token with write access to the fork repository.
    2. Target Project (Upstream): If the first step fails, it attempts to set the status on the upstream project. This requires a token with CI pipeline access to the upstream project.
    3. Merge Request Comments: As a final fallback, it posts the status as a comment on the MR. This requires MR write permissions and is designed to always succeed if the user has permission to comment.

    If you encounter issues where status checks are not appearing, ensure your token has the appropriate permissions for the fork or upstream repository.

  12. Configure authentication models for Git providers

    main

    Pipelines-as-Code supports two authentication models based on your Git integration:

    1. GitHub App authentication: Automatically authenticates as a GitHub App installation, generates short-lived scoped tokens for each repository, injects them into PipelineRuns as secrets, and handles automatic token refreshing.

    2. Webhook authentication (e.g., GitLab, Bitbucket): Requires manual configuration of a personal access token (or API token) and a webhook secret for request validation. These credentials must be stored in Kubernetes secrets and referenced by the Repository Custom Resource (CR).