DOCSEARCHbyAlgolia
Submit Docs

Flyte 2 Documentation

repository·main·Indexed 27 days ago

https://github.com/flyteorg/flyte

Flyte 2 is a platform for reliably orchestrating ML pipelines, models, and agents using a pure Python SDK. It includes tools for task execution, model serving, and local development. The documentation covers deployment via Helm charts (flyte-binary, flyte-devbox, and flyteconnector), configuration for storage providers and databases, and the setup of the Data Proxy service for managing storage backends like S3, GCS, and RustFS.

Tokens
21.3K
Snippets
67
Records
115
Agent score
93%

What's inside Flyte

  1. Overview of flytestdlib components

    main

    The flytestdlib library provides shared Go components for building robust applications. Key modules include:

    • config: Enables strongly typed configuration using Go structs, including parsing, validation, and watching for changes.
    • cli/pflags: Tool to generate pflags for all fields in a given struct.
    • storage: Abstract storage library supporting S3, Azure, GCS (via stow), in-memory storage for testing, and native protobuf support.
    • contextutils: A wrapper around Go's context to set and get known keys.
    • logger: A configurable, taggable, and context-aware wrapper around logrus.
    • profutils: Starts an HTTP server providing /metrics (Prometheus), /healthcheck, and /version endpoints.
    • promutils: Provides a Scope instance for convenient construction and scoping of Prometheus metrics per component.
    • atomic: A wrapper around sync.atomic providing types like AtomicInt32.
    • sets: Strongly typed and convenient interface sets.
    • utils and version: General utility and versioning modules.

  2. Overview of Flyte Manager Unified Binary

    main

    The Flyte Manager is a single unified binary that runs all Flyte services in a single process. This simplifies deployment and management by consolidating the following services:

    • Runs Service: Manages workflow runs and action state.
    • Executor/Operator: Reconciles and transitions TaskAction CRs through their lifecycle.
    • Actions Service: Serves action metadata and lifecycle APIs, including enqueueing TaskAction CRs.
    • DataProxy Service: Proxies signed-URL and blob access for task I/O.
    • Events Service: Ingests and fans out task/run events.
    • Cache Service: Backs task output caching and lookups.
    • App Service: Hosts the Flyte UI/app and routes to internal services.
    • Secret Service: Manages secret references used by tasks.

    Key features include a shared PostgreSQL backend, automatic Kubernetes detection via kubeconfig, unified configuration, and HTTP/2 support (Buf Connect compatible).

  4. Overview of Flyte CoPilot

    main
    Flyte CoPilot is a tool that provides a sidecar capable of understanding the Flyte Metadata Format (as specified in FlyteIDL). It enables the execution of arbitrary containers within Flyte by managing the downloading and uploading of metadata and other configured data. It is distributed as a binary named flyte-copilot which operates in two distinct modes: downloader and sidecar.

  5. Overview of Queue, Run, and State Services

    main

    Flyte's core backend functionality is divided into three primary gRPC services implemented using buf connect:

    • QueueService: Manages the execution queue for actions.
    • RunService: Manages workflow runs and their entire lifecycle.
    • StateService: Manages state persistence for actions.

    Each service is backed by a PostgreSQL database and uses pgx for database operations.

  7. Understand the Docker Image Build Performance and Caching

    main

    The Flyte Docker image is optimized for speed using a multi-stage build strategy and a multi-layer caching hierarchy. This reduces build times from ~15 minutes (cold build) to ~2-3 minutes (warm build).

    Build Strategy

    • Multi-Stage Parallelism: The build process runs Go, Node, Python, and Buf stages in parallel to download tools simultaneously.
    • Cache Mounts: The Dockerfile utilizes BuildKit cache mounts for package managers to persist dependencies across builds:
      • apt: /var/cache/apt
      • go mod: /root/go/pkg/mod
      • cargo: /root/.cargo/registry

    Cache Hierarchy

    When building, the system attempts to pull layers in the following order:

    1. buildcache tag (dedicated cache image)
    2. Current PR tag (if it exists)
    3. v2 tag (stable baseline)
    4. GitHub Actions (GHA) cache
    5. Build from scratch

  8. Understand Settings API inheritance and scope rules

    main

    Flyte settings follow a hierarchical scope: org → domain → project. Settings resolve top-down through this hierarchy.

    Setting States

    All settings use a SettingValue message. The state is determined by the field set:

    • state: INHERIT (or empty): Delegates to the parent scope.
    • state: UNSET: Explicitly clears the setting and stops inheritance.
    • stringValue, intValue, listValue, or mapValue: Implies a VALUE state.

    Resolution Rules

    • GetSettings: Resolves inheritance top-down and annotates each setting with a scopeLevel.
    • Map Settings: Settings like environmentVariables are additive. Parent entries are merged with child entries; child entries win on key conflicts. The scopeLevel reflects the most specific scope in the resolution chain.
    • GetSettingsForEdit: Returns a levels array (broadest to most specific) containing the raw stored state at each level, including descriptions but without scopeLevel.

  9. Understand Golang GitHub Actions image publishing behavior

    main

    The Golang GitHub Actions workflows automatically push Docker images to two different repositories in the GitHub registry:

    1. docker.pkg.github.com/flyteorg/<repo>/<package_name>: The primary image repository.
    2. docker.pkg.github.com/flyteorg/<repo>/<package_name>-stages: A repository used to cache build stages to accelerate subsequent iterative builds.

    Workflow Triggers:

    • Pull Requests: Builds and pushes images tagged with the specific commit SHA.
    • Master Merges: Bumps the patch version of the release tag, then builds and pushes images tagged with the version, the commit SHA, and latest.

  10. Enable Golang GitHub Actions workflows

    main

    To enable the Golang GitHub Actions workflows in your project, follow these two steps:

    1. Add flyteorg/github_workflows to your boilerplate/update.cfg file.
    2. Define a GitHub secret named package_name. This should be the name used for publishing (e.g., flytepropeller).

    Note for Forks: If you are working on a fork, include the fork prefix in your package_name (e.g., myfork/flytepropeller).

  11. Generate Flyte 2 client libraries

    main
    To generate the protocol buffer-based client libraries for all supported languages (Go, TypeScript, Python, and Rust), use the make gen command. This command updates Buf dependencies, formats/lints proto files, generates code and Go mocks, and runs go mod tidy.
    make gen

  12. Run common development commands via Make

    main

    Use the following Make targets to perform common development tasks within the Docker environment:

    • make gen: Generate Protocol Buffers.
    • make docker-shell: Start an interactive shell inside the Docker container.
    • make docker-pull: Pull the latest CI image.
    • make build-crate: Build a Rust crate.
    make gen
make docker-shell