Globus Compute

The Gladier toolkit is designed to accelerate the development of flows using Globus Compute. It provides:

• Input Validation: Validates inputs before starting a flow.
• Automatic Re-registration: Re-registers functions automatically when they are modified.
• Flow Generation: Capabilities to automatically generate flow definitions.

The globus-compute-endpoint provides several primary lifecycle and management functions for deploying and managing endpoints:

• init: Performs a one-time initialization to set up global options for the deployment site.
• start: Starts an endpoint by its name. If no valid configuration exists, it provides a default configuration for the user to edit. If the endpoint is configured but not yet registered, it performs the registration.
• stop: Stops a running endpoint by its name.
• list: Lists all available endpoints currently known to the system.

A Globus subscription is an organization-level plan (purchased by an institution, research group, or company) that provides additional capabilities across the Globus platform. It is not a personal account and applies to all users within that organization.

For Globus Compute specifically, core functionality—such as using Compute endpoints and submitting tasks—does not require a subscription; it only requires authentication via Globus Auth.

A Globus Compute Endpoint acts as a conduit for executing functions on computing resources. It manages site-specific interactions to provide a unified API for running code across diverse environments (laptops, clusters, HPC systems). The architecture consists of three main process types:

1. Manager endpoint process (MEP): Renders configuration templates to launch and manage user endpoint processes.
2. User endpoint process (UEP): Establishes secure communication with Globus Compute web services to receive tasks, launch worker processes, and publish results.
3. Worker process: Executes the actual submitted tasks on target computing resources (e.g., HPC cluster nodes) based on the rendered configuration template.

If you are transitioning from AWS Lambda to Globus Compute, note these key differences:

In Compute, functions are Python or Bash source code registered with the service. They are not tied to a specific execution context. Instead, users choose an endpoint (the runtime environment) to execute the code. Endpoints are identified by a UUID assigned during registration.

Globus Compute functions support standard Python *args and **kwargs.

Constraint: All data passed to or returned from a function must be serializable (e.g., via Pickle) and must stay within the service's size and rate limits.

def hello(firstname, lastname):
    """Say hello to someone."""
    return 'Hello {} {}'.format(firstname, lastname)

func_id = gcc.register_function(hello)
task_id = gcc.run("Bob", "Smith", endpoint_id=tutorial_endpoint, function_id=func_id)

try:
    print(gcc.get_result(task_id))
except Exception as e:
    print("Exception: {}".format(e))

Multi-user Compute Endpoints (running as root) use Identity Mapping to translate a Globus Auth identity set into a local POSIX user account. This allows the endpoint to use the standard Unix fork() -> drop privileges -> exec() workflow, ensuring that task processing is performed under the permissions of the specific local user. This enables standard Unix administrative user controls to be enforced on the endpoint host.

For multi-user endpoints, an identity mapping configuration is required to perform this translation. Refer to the configuration documentation for specific mapping examples.

Globus Compute templates have access to a set of reserved variables that cannot be overridden by users or administrators. These are accessed through the _GC object.

Note: While parent_config, user_runtime, and mapped_identity are available as top-level variables for legacy reasons, using the _GC prefix is the recommended approach and top-level access is deprecated.

Key reserved variables include:

• _GC.parent_config: Configuration values of the parent manager endpoint.
• _GC.env: Environment variables available to the user's runtime (e.g., _GC.env.HOME).
• _GC.user_runtime: Information about the user's Python and platform environment.
• _GC.mapped_identity: Information about the user's mapped identity (UID, GID, groups, etc.).