Apache OpenWhisk Documentation

repository·master·Indexed 27 days ago

https://github.com/apache/openwhisk

A serverless functions platform for building cloud applications. OpenWhisk enables developers to create serverless APIs from functions, compose them into workflows, and connect events via rules and triggers. Documentation covers standalone and Kubernetes deployment, the wsk CLI, the Scala core API (including FullyQualifiedEntityName and ByteSize), Etcd integration for leader election and key-value operations, and HTTP error response formats.

Tokens
52.9K
Snippets
125
Records
268
Agent score
92%

What's inside Apache OpenWhisk

  1. Overview of OpenWhisk Actions

    master
    Actions are stateless functions that run on the OpenWhisk platform. They are typically invoked in response to an event (such as an API call or a database change) and produce an observable output. Actions can be implemented using various supported languages and runtimes, as binary-compatible executables, or as Docker containers. You can manage actions using the OpenWhisk CLI (wsk) or the REST API.
  2. Overview of Apache OpenWhisk Modules

    master

    Apache OpenWhisk is composed of several specialized modules categorized by their role in the ecosystem, including core platform components, client libraries, runtimes, deployment tools, and integration packages.

    Main Components

    • openwhisk: The core open source serverless cloud platform.
    • openwhisk-apigateway: The API Gateway service that exposes actions as REST interfaces.
    • openwhisk-catalog: A curated catalog of packages for interfacing with event producers and consumers.
    • openwhisk-cli: The official Command Line Interface (CLI).
    • openwhisk-composer / openwhisk-composer-python: High-level programming models (JavaScript and Python) for composing serverless functions.
    • openwhisk-wskdeploy: Utility for deploying and managing OpenWhisk projects and packages.

    Client Libraries

    • openwhisk-client-go: Go client library.
    • openwhisk-client-js: JavaScript client library.

    Runtimes

    OpenWhisk supports various execution environments via specific runtime modules:

    • Deno, Docker (blackbox runtimes), .Net, Go, Java (and other JVM languages), NodeJS, PHP, Python, Ruby, Rust, and Swift.

    Deployment and Packages

    • openwhisk-deploy-kube: Supports deploying OpenWhisk on Kubernetes and OpenShift clusters.
    • Integration Packages: Includes packages for Alarms (periodic/time-based), Cloudant/CouchDB, Kafka/Message Hub, and Bluemix Push Notifications.

    Development Tools

    • openwhisk-devtools: Tools for building and deploying.
    • openwhisk-intellij-plugin: IntelliJ IDEA plugin.
    • openwhisk-vscode-extension: VSCode extension.
    • openwhisk-wskdebug: Debugging and live development tool.
  3. Understand ActionLoop proxy upload handling

    master

    When a user uploads an action via the wsk CLI, the ActionLoop proxy prepares a work folder containing src and bin directories. The proxy handles different file types as follows:

    • Executable file: Stored as bin/exec and executed directly.
    • Non-executable/Non-zip file: Stored as src/exec, then the compilation script is invoked.
    • ZIP file: Unzipped into the src folder.
      • If src/exec exists and is an executable, src is renamed to bin and bin/exec is executed.
      • If src/exec is missing or not an executable, the compilation script is invoked.

    An executable for ActionLoop must be either a Linux binary (ELF) or a script starting with a shebang (e.g., #!/bin/bash).

  4. Understand the OpenWhisk internal processing flow

    master

    OpenWhisk follows a RESTful design where commands (like those from the wsk CLI) are translated into HTTP requests. The flow of an action invocation is as follows:

    1. nginx: Acts as the entry point, handling SSL termination and forwarding HTTP calls.
    2. Controller: A Scala-based REST API implementation. It disambiguates requests (e.g., translating a POST to an action into an 'invocation') and manages the Load Balancer.
    3. CouchDB (Authentication/Authorization): The Controller verifies user credentials and privileges against the subjects database.
    4. CouchDB (Action Retrieval): The Controller loads the action details (code, default parameters, resource restrictions) from the whisks database.
    5. Load Balancer: Part of the Controller, it selects an available Invoker based on health status.
    6. Kafka: Acts as a distributed messaging buffer between the Controller and Invoker to ensure reliability and handle heavy loads. The Controller publishes the invocation message to Kafka.
    7. Invoker: The core component that executes the action. It uses Docker to create an isolated container, injects the code, runs it, captures the result/logs, and destroys the container.
    8. CouchDB (Results): The Invoker stores the final result and logs in the activations database under a unique ActivationId.
  5. Architectural patterns for implementing OpenWhisk feeds

    master

    OpenWhisk allows users to expose event producer services as feeds within a package. There are three primary architectural patterns for implementing a feed:

    1. Hooks: The easiest method for low-frequency feeds. It uses a webhook facility from an external service to POST directly to an OpenWhisk trigger URL. No persistent external service is required.
    2. Polling: An OpenWhisk action is configured to poll an endpoint periodically. This is easy to build but limited by the polling interval and is suitable for low-volume/low-latency requirements.
    3. Connections: The highest performance option for high-volume feeds. It requires a separate, persistent provider service outside of OpenWhisk that maintains long-polling or push notification connections to the source and proxies events to OpenWhisk via its REST API.
  6. Requirements for OpenWhisk Actions

    master

    To function as an OpenWhisk action, a function must meet these criteria:

    • Input/Output: The function must accept a dictionary (JSON object) as input and produce a dictionary as output. Keys are strings, and values are any valid JSON types.
    • Entry Point: The function must be named main or be explicitly exported. If using the wsk CLI, you can specify a different entry point using the --main flag.
    • Statelessness: Functions should be stateless or idempotent. While containers may be reused (warm starts), there is no guarantee that state will persist across different invocations.
  7. Understand OpenWhisk metric types

    master

    OpenWhisk provides two distinct categories of metrics:

    1. System Metrics: Focus on system performance. These are intended for OpenWhisk providers or operators and can be sent to Kamon or written to log files in logmarker format.
    2. User Metrics (Events): Focus on action performance. These are sent to Kafka as events and are intended for end-users for purposes such as billing or auditing.

    Note: User metrics are not directly exposed via an API; consuming them requires a separate micro-service based on a Kafka Consumer to process the data.

  8. Understand the Function Pulling Container Scheduler (FPCScheduler)

    master

    The FPCScheduler is a proposed high-performance scheduler for OpenWhisk designed to address performance bottlenecks in the default ShardingPoolBalancer.

    Key improvements include:

    • Container-centric scheduling: Instead of scheduling requests to invokers, containers pull activation requests from dedicated action queues, maximizing container reuse.
    • Distributed resource management: It allows schedulers to create containers on any invoker with sufficient resources, preventing the resource partitioning issues found in the standard controller-to-invoker model.
    • Dedicated Action Queues: Each action has its own MemoryQueue to prevent interference between different actions.
    • ETCD-backed coordination: Uses ETCD for transactions, health checks (via leases), and sharing cluster-wide information like scheduler/queue endpoints and throttling data.
  9. Understand OpenWhisk core concepts

    master

    OpenWhisk is an event-driven compute platform (Serverless/FaaS) that executes code in response to events or direct invocations.

    Key concepts include:

    • Actions: Small snippets of code (JavaScript, Swift, etc.) or custom binary code in Docker containers. They are executed on-demand.
    • Triggers: Channels for events (e.g., database changes, IoT sensor readings, HTTP requests). Triggers fire and cause actions to execute.
    • Rules: Logic that maps a specific trigger to a specific action.
    • Chains: A sequence of actions where the output of one action is passed as the input to the next.
    • Packages: Bundles of Feeds (code that configures external event sources to fire triggers) and Actions (reusable logic).
    • Feeds: Specialized code used to connect external services (like Cloudant or GitHub) to OpenWhisk triggers.
  10. Understand OpenWhisk entity naming and hierarchy

    master

    OpenWhisk uses a hierarchical structure for its entities (actions, triggers, rules, packages, and namespaces).

    Hierarchy

    • Namespaces: The top-level container. The /whisk.system namespace is reserved for system-distributed entities.
    • Packages: Optional containers within a namespace. Packages can contain actions and feeds, but cannot be nested.
    • Entities: Actions, triggers, and rules belong to a namespace and can optionally belong to a package.

    Fully Qualified Names

    The format for a fully qualified name is /namespaceName[/packageName]/entityName.

    Fully qualified nameAliasNamespacePackageName
    /whisk.system/cloudant/read/whisk.systemcloudantread
    /myOrg/video/transcodevideo/transcode/myOrgvideotranscode
    /myOrg/filterfilter/myOrgfilter

    If you are in your default namespace, you can omit the namespace prefix in your commands.

  11. Use OpenWhisk-enabled services from the catalog

    master
    OpenWhisk provides a catalog of pre-built packages in the /whisk.system namespace. These packages allow you to easily integrate external services and capabilities into your applications without writing custom integration code. You can browse these packages using the command line tool to find specific integrations like Slack, GitHub, or Watson services.