Argo Workflows

Instead of implementing REST API calls and request/response types manually, you can use client libraries to interact with the Argo API. These libraries often handle authentication and provide language-specific abstractions.

Supported libraries include:

• Go SDK: Fully-featured client with two approaches (Kubernetes Client and Argo Server Client).
• Java SDK: Community-supported library auto-generated via OpenAPI.
• Hera (Python SDK): Recommended for Python users; allows turning Python functions into script templates and writing workflows using Pythonic syntax.
• Juno (TypeScript SDK): Community-supported workflow generator that provides types and validation to reduce string-based configuration.

The ArtifactServiceApi provides methods to retrieve artifacts (files) from Argo Workflows. It supports fetching general artifacts, input artifacts, and output artifacts using various identifiers such as namespace, name, node ID, or UID.

Available Methods:

• artifactServiceGetArtifactFile: Get a specific artifact file.
• artifactServiceGetInputArtifact: Get an input artifact.
• artifactServiceGetInputArtifactByUID: Get an input artifact using its UID.
• artifactServiceGetOutputArtifact: Get an output artifact.
• artifactServiceGetOutputArtifactByUID: Get an output artifact using its UID.

The Argo UI is a web-based interface designed for managing the Argo Workflow engine. It provides a visual way to interact with workflows running in your Kubernetes cluster without using the CLI or YAML manifests directly.

Key capabilities include:

• Workflow Monitoring: View both live (running) and completed Argo Workflows.
• Workflow Creation: Create new Argo Workflow templates directly through the interface.
• Cron Workflow Management: View and create Argo Cron Workflows for scheduled execution.
• Observability: Access and view container logs for individual steps within a workflow.

Argo Workflows is an open-source, container-native workflow engine designed for orchestrating parallel jobs on Kubernetes. It is implemented as a Kubernetes Custom Resource Definition (CRD).

Key capabilities include:

• Defining workflows where each step is a container.
• Modeling multi-step workflows as a sequence of tasks or as a Directed Acyclic Graph (DAG) to capture task dependencies.
• Running compute-intensive jobs (e.g., machine learning or data processing) at scale on Kubernetes.

The EventServiceApi provides endpoints for interacting with workflow event bindings and receiving events. All URIs are relative to http://localhost:2746.

Available methods:

• eventServiceListWorkflowEventBindings: Retrieves workflow event bindings for a specific namespace.
• eventServiceReceiveEvent: Sends an event to a specific namespace and discriminator.

GET /api/v1/workflow-event-bindings/{namespace}
POST /api/v1/events/{namespace}/{discriminator}

The Java SDK is designed to access the Argo Server API. While you can use the provided models to interact with Kubernetes APIs, you will need to implement the logic to communicate with the Kubernetes API server yourself.

The SDK exposes several specialized services for different Argo components. Detailed API documentation for each service can be found in the following locations within the SDK repository:

• Workflow service: Manage and interact with workflows.
• Pipeline service: Manage and interact with pipelines.
• Event service: Handle event-related operations.
• Sensor service: Manage sensors.
• Event source service: Manage event sources.
• Info service: Retrieve general information/metadata.

Argo Workflows is used by a wide range of organizations and integrated into several major projects.

Official Projects Using Argo Workflows

• Couler
• Hera Workflows
• Kubeflow
• Metaflow
• Onepanel
• SQLFlow
• BisQue
• Tator
• Juno

Official Organizations Using Argo Workflows

Many large-scale organizations use Argo Workflows, including Adobe, Alibaba Cloud, ByteDance, Canva, Cisco, Datadog, Google, IBM, NVIDIA, Oracle, Red Hat, Salesforce, and VMware, among many others.

Argo Workflows allows you to manage synchronization limits (semaphores) programmatically using the Argo Server API or CLI. This is useful for adjusting limits dynamically without redeploying ConfigMaps or for automating infrastructure setup via CI/CD.

There are two modes of operation:

1. ConfigMap-based limits: Always available. Limits are stored in Kubernetes ConfigMaps. Access is controlled via standard Kubernetes RBAC for ConfigMaps.
2. Database-based limits: Requires a shared database (PostgreSQL, MySQL, or MariaDB) and specific configuration. This mode allows multiple workflow controllers (e.g., across different clusters) to share synchronization state. Access is controlled via Argo Server using workflow permissions as a proxy.

Argo Workflows provides two types of metrics to monitor system and workload state:

1. Controller Metrics: Built-in metrics emitted by the Argo controller that describe the internal state of the controller itself.
2. Custom Metrics: User-defined metrics that can be emitted at the Workflow or Template level to track specific workload behaviors.

Custom metrics are useful for:

• Monitoring the duration of a Workflow or Template to trigger alerts when thresholds are exceeded.
• Tracking failure rates of specific Workflow or Template types over time.
• Reporting domain-specific data, such as model training scores or internal application error rates.

Metrics can be collected using the OpenTelemetry protocol or via Prometheus-compatible scraping.