tokio-cron-scheduler

repository·main·Indexed 20 days ago

https://github.com/mvniekerk/tokio-cron-scheduler

A cron scheduler for the Tokio runtime that supports scheduling tasks using cron-like annotations, fixed durations, or one-shot executions. It features persistent storage backends via PostgreSQL and NATS to ensure job metadata and notifications survive restarts, as well as support for natural language schedules and custom lifecycle notifications.

Tokens
12.8K
Snippets
38
Records
58
Agent score
73%

What's inside tokio-cron-scheduler

  1. Understand the storage and code traits in version 0.6

    main

    Version 0.6 uses a modular approach to job execution and persistence through several key traits:

    Storage Traits

    • MetaDataStore: Required by the scheduler to schedule jobs.
    • NotificationStore: Required by the scheduler to run notifications on job start, scheduled, stop, or removals.

    Code Traits

    • ToCode: A generic trait that provides a PinnedGetFuture for a given UUID.
    • JobCode: A trait (specializing ToCode) that provides the runnable closures for the scheduler. The default implementation is SimpleJobCode.
    • NotificationCode: A trait (specializing ToCode) that provides the runnable notification closures. The default implementation is SimpleNotificationCode.
  2. How job creation and deletion work

    main

    Jobs and their associated notifications are managed through specific creation and deletion workflows that ensure metadata and runnable code remain synchronized.

    Creating a Job

    1. A creation request is sent to a queue.
    2. A JobCreator generates the next tick, saves the job metadata to MetadataStorage, and updates the list of active GUIDs.
    3. The process completes once the job GUID is produced to a 'done' queue.

    Deleting a Job

    Deleting a job is a multi-step process to ensure no orphaned notifications remain:

    1. Metadata Deletion: The job's metadata is removed from MetadataStorage.
    2. Code Cleanup: The job's runnable code is removed from JobStorage.
    3. Notification Cleanup: The system identifies all notifications associated with the Job ID in NotifyStorage and deletes them individually, ensuring the NotifyCode is also cleaned up.
  3. How job activity and notifications work

    main

    The tokio-cron-scheduler operates through a decoupled architecture involving a scheduler, runners, and notification systems. Understanding this lifecycle helps in debugging job execution and state changes.

    1. Scheduling Lifecycle

    • The Scheduler polls MetadataStorage every second to retrieve job GUIDs.
    • It filters jobs based on their next scheduled tick and updates the metadata with the new next_tick and last_tick timestamps.
    • Once a job is ready, its GUID is sent to the JobActivationQueue and a SCHEDULED state is sent to the NotifyQueue.

    2. Job Execution Lifecycle

    • A Runner picks up the job GUID from the JobActivationQueue.
    • It retrieves the actual runnable code from JobStorage using the GUID.
    • The runner emits state notifications: STARTED before execution and DONE after execution.

    3. Notification Lifecycle

    • The NotifyQueue receives a GUID and a job state (e.g., STARTED, DONE).
    • A NotifyRunner picks up the message, retrieves notification metadata from NotifyStorage, and executes the associated notification code retrieved from NotifyCode.
  4. Schedule jobs using cron expressions

    main

    Jobs can be scheduled using any ToString implementation that follows the cron format. The scheduler uses the croner library for parsing.

    Cron Format: sec min hour day_of_month month day_of_week (Year is optional).

    Key Syntax Rules:

    • Timezone: By default, time is specified in UTC. To use a specific timezone, use the _tz suffix in job creation calls (e.g., Job::new_async_tz).
    • Multiple Values: Use commas (e.g., 5,8,10).
    • Ranges: Use dashes (e.g., 5-10).
    • Days of Week: Use abbreviations or full names (e.g., Sun,Sat).

    Example Jobs:

    • 0 2,14,26 * * * *: Executes on the 2nd, 14th, and 26th minute of every hour.
    • 0 0 * 5-10 * *: Executes once per hour on days 5 through 10 of the month.
    • 0 0 6 * * Sun,Sat: Executes at 6 am on Sunday and Saturday.
    // Example of a basic cron job
    sched.add(
        Job::new("1/10 * * * * *", |_uuid, _l| {
            println!("I run every 10 seconds");
        })?
    ).await?;
  5. How notification management works

    main

    Notifications are lifecycle hooks attached to jobs that trigger specific code when a job enters a certain state.

    Creating a Notification

    • A request containing the Job GUID, Notify GUID, and Job State is processed.
    • The NotificationCreator retrieves existing notification data, updates the job metadata with the new notification information, and saves it to NotifyStorage.

    Deleting a Notification

    • A request containing the Notify GUID and Job State is processed.
    • The NotificationDeleter retrieves the metadata, updates it to reflect the deletion, and saves the changes to NotifyStorage.
  6. Implement custom storage for metadata and notifications

    main

    The JobScheduler can use custom storage implementations by implementing the MetadataStore and NotificationStore traits.

    Available Storage Options:

    • Volatile (Default): Uses in-memory HashMaps (SimpleMetadataStore and SimpleNotificationStore).
    • Nats: Persistent storage using Nats (NatsMetadataStore and NatsNotificationStore). Requires nats_storage feature.
    • PostgreSQL: Persistent storage using Postgres (PostgresMetadataStore and PostgresNotificationStore). Requires postgres_storage feature.
  7. Set up NATS with Jetstream for persistent storage

    main

    To use NATS as a persistent storage backend for tokio-cron-scheduler, you must run a NATS instance with Jetstream enabled. You can quickly spin up a compatible instance using Docker with the following command:

    docker run --rm -it -p 4222:4222 -p 6222:6222 -p 7222:7222 -p 8222:8222 nats -js -DV
  8. Migrate from versions 0.4/0.5 to 0.6

    main

    Version 0.6 introduced a significant architectural change. If you are upgrading from 0.4 or 0.5, the primary change for most users is that creating or removing job notifications now requires passing a reference to the scheduler as the first parameter.

    Key architectural shifts include:

    • The JobStore trait has been replaced by two specialized traits: MetadataStore and NotificationStore.
    • JobSchedulerWithoutSync and its implementation SimpleJobScheduler have been removed.
    • JobScheduler::new_with_scheduler() has been replaced by JobScheduler::new_with_storage_and_code().
  9. Use the JobBuilder API for advanced configuration

    main

    For fine-grained control, such as setting specific timezones, use the JobBuilder API. This is useful when you need to work with chrono-tz timezones.

    Note: chrono-tz is not a direct dependency; you must add it to your Cargo.toml to use it with JobBuilder.

    async fn tz_job() {
        let job = JobBuilder::new()
            .with_timezone(chrono_tz::Africa::Johannesburg)
            .with_cron_job_type()
            .with_schedule("*/2 * * * *")
            .unwrap()
            .with_run_async(Box::new(|uuid, mut l| {
                Box::pin(async move {
                    // Job logic here
                })
            }))
            .build()
            .unwrap();
    }
  10. Run PostgreSQL via Docker for testing

    main

    To use PostgreSQL persistent storage, you need a running PostgreSQL instance. You can quickly spin up a local instance using Docker with the following command:

    docker run --rm -it -p 5432:5432 -e POSTGRES_USER="postgres" -e POSTGRES_PASSWORD="" -e POSTGRES_HOST_AUTH_METHOD="trust" postgres:14.1
  11. Configure PostgreSQL connectivity via environment variables

    main

    You can configure the connection to your PostgreSQL instance using environment variables. If POSTGRES_URL is provided, it follows the standard postgres crate configuration and all other individual connection variables will be ignored.

    Variable          | Default   | Description
    -------------------|-----------|--------------------------------------------------------------------------------------------------------------------------------------------
    POSTGRES_URL      |           | URL as per docs. Other DB connection setup variables ignored if set.
    POSTGRES_HOST     | localhost | Host to connect to
    POSTGRES_PORT     | 5432      | Port to connect to
    POSTGRES_DB       | postgres  | Database name
    POSTGRES_USERNAME | postgres  | Username
    POSTGRES_PASSWORD |           | Password
    POSTGRES_APP_NAME |           | Application name to register on PostgreSQL server