repository·master·Indexed 23 days ago
https://github.com/kagkarlsson/db-schedulerA lightweight, persistent, and cluster-friendly task scheduler for Java applications. It uses a relational database to manage task state, ensuring exactly-once execution in distributed environments. It supports recurring and one-time tasks, high throughput (2k-10k executions per second), and is compatible with PostgreSQL, SQL Server, MySQL, MariaDB, Oracle, SQLite, and HSQLDB. Requires Java 17 or higher since v16.x.
db-scheduler supports tasks that do not follow a recurring pattern:
Tasks.oneTime(..)): Tasks with a single execution time. You can encode data into the instanceId or store arbitrary binary data in a separate field (defaulting to Java serialization).Tasks.custom(..)): Provides full control over task behavior. Use this if you need to decide whether to reschedule or remove a task based on the result of its execution.If you are deciding between db-scheduler and Quartz, consider the following design goals:
db-scheduler aims to be simple to use and non-invasive while still solving persistence and cluster coordination.db-scheduler is designed for applications that want to avoid adding a large number of tables (e.g., 11 tables) to their existing database schema.db-scheduler uses your existing RDBMS for both persistence and coordination, following the KISS (Keep It Simple, Stupid) principle by leveraging the most common type of shared state available to most applications.The scheduler manages task health through heartbeats and registration checks:
RecurringTask is typically rescheduled to now() when this happens.deleteUnresolvedAfter period.You can define a priority for executions to control the order in which due tasks are fetched from the database (order by priority desc, execution_time asc).
Important Requirements:
.enablePriority() in the Scheduler builder.priority column (defined as SMALLINT).(execution_time asc, priority desc).Per Instance: Use the TaskInstance.Builder when scheduling a specific execution.
Per Task Type: Set a default priority for all instances of a specific task using .defaultPriority(Priority) on the task builder.
// Set priority for a specific instance
scheduler.schedule(
MY_TASK
.instance("1")
.priority(100)
.scheduledTo(Instant.now()));
// Set default priority for a task type
Tasks.recurring("my-task", FixedDelay.ofSeconds(5))
.defaultPriority(Priority.LOW)
.execute(...);A recurring task runs regularly according to a Schedule. When an execution finishes, the scheduler consults the Schedule to determine the next execution time and creates a new task-execution record.
There are two types of recurring tasks:
Static Recurring Tasks (Tasks.recurring(..)):
Schedule is defined in code.Schedule is updated in code, the scheduler updates the next execution time.Dynamic Recurring Tasks (Tasks.recurringWithPersistentSchedule(..)):
Schedule is persisted in the database (task_data field).SchedulerClient to create or update instances.db-scheduler offers two primary polling strategies that affect how executions are retrieved from the database and how many SQL statements are executed per batch.
fetch (Default)This is the original strategy. It works by selecting a batch of due executions and then attempting to update each one to mark it as picked=true for the current scheduler instance. This can result in 'misses' if competing schedulers have already picked the same execution.
select + (2 * batch-size) updates (excluding misses).SKIP LOCKED support might be unavailable.lock-and-fetchThis strategy utilizes SELECT FOR UPDATE ... SKIP LOCKED to fetch executions that are already pre-locked by the scheduler instance. This eliminates the need for a separate update step to 'pick' the execution and prevents misses from competing schedulers.
select for update .. skip locked + (1 * batch-size) updates (no misses).SKIP LOCKED.The scheduler uses different strategies to fetch tasks from the database. The default is fetch.
fetch (Default): Uses the .pollUsingFetch(double, double) method. Fetched executions are not locked; the scheduler competes with other instances for the lock when executing. Use this for normal usage.
lock-and-fetch: Uses SELECT FOR UPDATE ... SKIP LOCKED for lower overhead and higher throughput. Use this if you are running >1000 executions/s. It is configured via .pollUsingLockAndFetch(double, double). Fetched executions are already locked for the current scheduler instance, saving an UPDATE statement.
Supported Databases for lock-and-fetch: PostgreSQL, SQL Server, and MySQL v8+.
Both methods use two double parameters representing fractions of threads:
lowerLimitFractionOfThreads * nr-of-threads.Recommended settings:
.pollUsingLockAndFetch(0.5, 1.0).pollUsingLockAndFetch(1.0, 4.0)When using db-scheduler, keep the following in mind:
Schedule will be executed. The scheduler picks the nearest future time after the previous task finishes.SchedulerClient methods (schedule, cancel, reschedule) use a new Connection from the provided DataSource. To ensure these actions are part of an existing transaction, provide a TransactionAwareDataSourceProxy (e.g., from Spring).pollingInterval (default 10s). You can trigger an immediate check using scheduler.triggerCheckForDueExecutions() or enable immediateExecution() on the Builder.To get started, define a RecurringTask using Tasks.recurring, then instantiate the Scheduler using a DataSource, register the task, and call start(). The task will be automatically scheduled on startup if it does not already exist in the database.
RecurringTask<Void> hourlyTask = Tasks.recurring("my-hourly-task", FixedDelay.ofHours(1))
.execute((inst, ctx) -> {
System.out.println("Executed!");
});
final Scheduler scheduler = Scheduler
.create(dataSource)
.startTasks(hourlyTask)
.build();
// hourlyTask is automatically scheduled on startup if not already started (i.e. exists in the db)
scheduler.start();For Spring Boot applications, use the db-scheduler-spring-boot-starter to simplify wiring.
DataSource with the schema initialized.db-scheduler-spring-boot-starter.db-scheduler-spring-boot-4-starter (version 16.11.0).Task instances as Spring beans. Recurring tasks will be automatically picked up and started by the scheduler.db-scheduler health indicator.<!-- Maven dependency for Spring Boot 2.x -->
<dependency>
<groupId>com.github.kagkarlsson</groupId>
<artifactId>db-scheduler-spring-boot-4-starter</artifactId>
<version>16.11.0</version>
</dependency>To use db-scheduler, you must meet the following requirements:
scheduled_tasks table created in your schema.