plombery

Plombery is a Python-based task scheduler designed for running and monitoring recurring Python scripts. It provides a user-friendly web interface and a REST API for management and integration.

Key features include:

• Task Scheduling: Powered by APScheduler, supporting Interval, Cron, and Date triggers.
• Web UI: A built-in interface for monitoring and manual execution (no HTML/JS/CSS required).
• Python-Centric: Pipelines and tasks are defined entirely in pure Python.
• Parameterization: Pipelines can be parameterized using Pydantic models.
• Observability: Debugging capabilities via logs and output data exploration.
• Security: Secured via OAuth2.
• Alerting: Pipeline monitoring and alerting capabilities.

Plombery is a Python-based pipeline orchestration tool that provides a web interface for managing and running tasks. Key capabilities include:

• Python Pipelines: Define pipelines and tasks using pure Python functions.
• Parametrized Pipelines: Use Pydantic models to define parameters, which Plombery automatically converts into web forms for manual execution.
• Task Scheduling: Built on APScheduler, supporting Interval, Cron, and Date triggers.
• Built-in Web Interface: A ready-to-use UI for manual pipeline runs, exploring logs, and inspecting output data without requiring HTML/JS/CSS knowledge.
• REST API: Provides endpoints for advanced integrations, such as triggering pipeline runs via HTTP requests.
• Security: Supports optional OAuth2 authentication.
• Monitoring: Includes alerting capabilities to notify users when tasks fail.

A trigger is the entrypoint used to run a pipeline. Triggers can be based on schedules or external events and can have specific input parameters associated with them.

By default, if you do not add any triggers when registering a pipeline, Plombery adds a pipeline trigger. This provides:

1. A manual run button in the Web UI (on the home page and pipeline page).
2. An HTTP trigger (an HTTP POST endpoint) that allows you to run the pipeline programmatically.

If your pipeline uses a Pydantic BaseModel for input parameters, the manual run button will automatically generate a form in the UI to customize those parameters. When using the HTTP trigger, you pass these parameters as a JSON body in your request.

Pipelines can be made configurable by passing a Pydantic BaseModel to the params argument in register_pipeline.

When a pipeline has defined parameters:

1. Manual Execution: The UI will automatically generate a form based on the Pydantic model, allowing users to customize inputs via a dialog.
2. HTTP Trigger: You can pass the parameters as a JSON body in your HTTP request to trigger the pipeline with specific values.

from pydantic import BaseModel
from plombery import register_pipeline, task

class InputParams(BaseModel):
    some_value: int

@task
def my_task():
    pass

register_pipeline(
    id="param_pipeline",
    tasks=[my_task],
    params=InputParams
)

When you define params within a Trigger object, those parameters are immutable.

Unlike the default pipeline trigger (which opens a dialog to customize parameters), clicking the Run trigger button for a specific Trigger instance will execute immediately without a confirmation dialog or parameter customization. This design ensures that a trigger (which might be run by a schedule without human intervention) behaves consistently whether it is triggered by the scheduler, the UI, or an HTTP call.

Requirements:

• The params dictionary in the Trigger must satisfy all mandatory fields defined in your pipeline's Pydantic BaseModel.
• You can omit optional fields (those with default values in the BaseModel), but you cannot omit required ones.

To build with Plombery, you need to understand its four core abstractions:

• Task: The fundamental building block. It is a Python function that performs a specific job (e.g., querying a database or calling an API).
• Pipeline: A sequence of one or more Tasks. Pipelines can be executed manually, via a schedule, or through other triggers.
• Trigger: The entrypoint for a pipeline. Triggers define how a pipeline starts (e.g., a specific schedule, a webhook, or a button click in the Web UI).
• Pipeline Run: The specific instance or result of a pipeline being executed.

Plombery uses the Apprise library to handle notification delivery. To send notifications to different services, provide an Apprise-compatible URI in the channels list.

Common examples include:

• Email: mailto://myuser:mypass@gmail.com
• MS Teams: msteams://TokenA/TokenB/TokenC/
• AWS SES: ses://user@domain/AccessKeyID/AccessSecretKey/RegionName/email1/

For a complete list of supported services and their URI formats, refer to the official Apprise documentation.

Plombery is ideal if you need to run and monitor recurring Python scripts with a simple web interface and REST API.

Do NOT use Plombery if:

• You require high scalability across a distributed system.
• You want a no-code tool or prefer not to use Python.

Plombery executes tasks sequentially. The return value of a task is treated as its output data and is passed to subsequent tasks as positional arguments. You can name these arguments in the next task's signature to receive the output from specific previous tasks.

@task
def task_1():
  return 1

@task
def task_2(from_1):
  # from_1 will be 1
  return from_1 + 1

@task
def task_3(from_1, from_2):
  # from_1 and from_2 receive outputs from previous tasks
  return from_1 + from_2

The microsoft_tenant_id configuration is optional and its necessity depends on your Azure Application Registration type:

• Required: If you configured your Azure Application Registration to allow users only from your tenant (the "Accounts in this organizational directory only" option).
• Omit: If you allow other account types, the common tenant will be used in the OAuth endpoints automatically.

To start building HTML emails with utility-first CSS using this Maizzle starter, clone the template using degit, install the dependencies via npm, and then use the provided scripts for development or production builds.

# Clone the starter
npx degit maizzle/maizzle my-project

# Install dependencies
cd my-project
npm install

# Start local development
npm run dev

# Build emails for production
npm run build