ShedLock Documentation

repository·master·Indexed 26 days ago

https://github.com/lukas-krecan/shedlock

ShedLock is a library that ensures scheduled tasks are executed at most once at a time in a distributed environment using an external data store for coordination. It provides integrations for Spring and Micronaut, and supports multiple lock providers including JDBC, MongoDB, Redis, ZooKeeper, DynamoDB, and NATS JetStream. It is a locking mechanism rather than a distributed scheduler, designed for tasks that can be safely executed repeatedly but not in parallel.

Tokens
5.9K
Snippets
14
Records
28
Agent score
38%

What's inside ShedLock

  1. Overview of ShedLock

    master

    ShedLock ensures that scheduled tasks are executed at most once at the same time across multiple nodes or threads. When a task starts on one node, it acquires a lock that prevents the same task from executing on other nodes.

    Important behaviors:

    • Skipping, not waiting: If a task is already running on one node, execution on other nodes is simply skipped; it does not wait for the lock to be released.
    • Not a scheduler: ShedLock is a locking mechanism, not a distributed scheduler. It is intended for tasks that can be safely executed repeatedly but should not run in parallel.
    • Clock synchronization: ShedLock uses time-based locks and assumes that clocks on all participating nodes are synchronized.
    • External coordination: It requires an external store (e.g., JDBC, Mongo, Redis, ZooKeeper) to manage locks.
  2. ShedLock Architecture and Components

    master

    ShedLock is composed of three distinct parts:

    1. Core: The fundamental locking mechanism.
    2. Integration: Connects the core to your application via Spring AOP, Micronaut AOP, or manual code.
    3. Lock provider: Interfaces with external storage (SQL databases, Mongo, Redis, etc.) to persist and coordinate locks.
  3. Use multiple LockProviders in Spring

    master

    Since version 6.0.0, you can define multiple LockProvider beans in your Spring application context. Use the @LockProviderToUse("lockProviderBeanName") annotation on a method, class, or package to disambiguate which provider to use.

    Note: If the annotation is missing, execution fails at runtime.

  4. Configure InMemoryLockProvider for testing

    master

    For unit or integration tests where a real database is not required, use the in-memory implementation.

    1. Add the dependency: shedlock-provider-inmemory with <scope>test</scope>.
    2. Configure the Bean: Create a bean of type InMemoryLockProvider.
    <!-- Dependency -->
    <dependency>
        <groupId>net.javacrumbs.shedlock</groupId>
        <artifactId>shedlock-provider-inmemory</artifactId>
        <version>7.7.0</version>
        <scope>test</scope>
    </dependency>
    
    <!-- Configuration -->
    @Bean
    public LockProvider lockProvider() {
        return new InMemoryLockProvider();
    }
  5. Implement a Multi-tenancy LockProvider

    master

    For multi-tenancy use cases, you can implement a LockProvider that routes requests to different providers based on a tenant identifier. This is achieved by overriding lock to resolve the tenant name from the LockConfiguration and then delegating to the appropriate provider.

    private static abstract class MultiTenancyLockProvider implements LockProvider {
        private final ConcurrentHashMap<String, LockProvider> providers = new ConcurrentHashMap<>();
    
        @Override
        public Optional<SimpleLock> lock(LockConfiguration lockConfiguration) {
            String tenantName = getTenantName(lockConfiguration);
            return providers.computeIfAbsent(tenantName, this::createLockProvider).lock(lockConfiguration);
        }
    
        protected abstract LockProvider createLockProvider(String tenantName);
    
        protected abstract String getTenantName(LockConfiguration lockConfiguration);
    }
  6. Use KeepAliveLockProvider

    master

    The KeepAliveLockProvider keeps a lock alive by periodically extending it. It wraps an existing LockProvider and extends the lock in the middle of the lockAtMostFor interval.

    Important Considerations:

    • Use moderately; it adds complexity and makes the flow harder to reason about.
    • The minimal supported lockAtMostFor time is 30s.
    • A single-threaded scheduler is sufficient for the extension tasks.
    @Bean
    public LockProvider lockProvider(...) {
        return new KeepAliveLockProvider(new XyzProvider(...), scheduler);
    }
  7. Enable Micrometer metrics for ShedLock (Spring)

    master

    To publish lock execution metrics via Micrometer in Spring, add the shedlock-micrometer dependency and define a MicrometerLockingTaskExecutorListener bean. ShedLock will automatically wire it in.

    1. Add Dependency:
    <dependency>
        <groupId>net.javacrumbs.shedlock</groupId>
        <artifactId>shedlock-micrometer</artifactId>
        <version>${shedlock.version}</version>
    </dependency>
    1. Define Listener Bean:
    @Bean
    public LockingTaskExecutorListener micrometerLockingTaskExecutorListener(MeterRegistry meterRegistry) {
        return new MicrometerLockingTaskExecutorListener(meterRegistry);
    }
    1. Pre-register metrics (Optional, to avoid missing data in dashboards):
    @Bean
    public LockingTaskExecutorListener micrometerLockingTaskExecutorListener(MeterRegistry meterRegistry) {
        MicrometerLockingTaskExecutorListener listener = new MicrometerLockingTaskExecutorListener(meterRegistry);
        listener.registerMetricsFor("myLock1", "myLock2");
        return listener;
    }

    Available Metrics (tagged with lock.name):

    MeterTypeDescription
    shedlock.lock.attemptsCounterTotal lock acquisition attempts
    shedlock.lock.acquiredCounterSuccessful lock acquisitions
    shedlock.lock.not.acquiredCounterFailed lock acquisitions (lock held elsewhere)
    shedlock.execution.durationTimerTask execution time
    shedlock.execution.activeGaugeNumber of currently executing tasks
  8. Configure DynamoDBLockProvider

    master

    To use AWS DynamoDB for locking:

    1. Add the dependency: shedlock-provider-dynamodb2 (requires AWS SDK v2).
    2. Create the table: The lock table must be created externally with _id as the partition key.
    3. Configure the Bean: Pass the DynamoDbClient and a table name (defaults to "Shedlock").
    <!-- Dependency -->
    <dependency>
        <groupId>net.javacrumbs.shedlock</groupId>
        <artifactId>shedlock-provider-dynamodb2</artifactId>
        <version>7.7.0</version>
    </dependency>
    
    <!-- Configuration -->
    @Bean
    public LockProvider lockProvider(software.amazon.awssdk.services.dynamodb.DynamoDbClient dynamoDB) {
        return new DynamoDBLockProvider(dynamoDB, "Shedlock");
    }
  9. Customize LockProvider behavior

    master

    You can wrap an existing LockProvider to add custom logic, such as performing actions after a lock is successfully obtained. Implement the LockProvider interface and delegate the lock call to your underlying provider.

    public class MyLockProvider implements LockProvider {
        private final LockProvider delegate;
    
        public MyLockProvider(LockProvider delegate) {
            this.delegate = delegate;
        }
    
        @Override
        public Optional<SimpleLock> lock(LockConfiguration lockConfiguration) {
            Optional<SimpleLock> lock = delegate.lock(lockConfiguration);
            if (lock.isPresent()) {
                // do something
            }
            return lock;
        }
    }
  10. Integrate ShedLock with Micronaut

    master

    Since version 4.0.0, ShedLock supports Micronaut.

    1. Add Dependency:
    <dependency>
        <groupId>net.javacrumbs.shedlock</groupId>
        <artifactId>shedlock-micronaut4</artifactId>
        <version>7.7.0</version>
    </dependency>
    1. Configure Defaults (in application.yml):
    shedlock:
      defaults:
        lock-at-most-for: 1m
    1. Configure LockProvider:
    @Singleton
    public LockProvider lockProvider() {
        ... select and configure your lock provider
    }
    1. Annotate Tasks:
    @Scheduled(fixedDelay = "1s")
    @SchedulerLock(name = "myTask")
    public void myTask() {
        assertLocked();
        ...
    }
  11. Configure MongoLockProvider

    master

    To use MongoDB for locking:

    1. Add the dependency: shedlock-provider-mongo.
    2. Configure the Bean: Pass the MongoClient and the target database name.

    Requirements:

    • MongoDB >= 2.4
    • mongo-java-driver >= 3.7.0

    For reactive applications, use shedlock-provider-mongo-reactivestreams with MongoDB >= 4.x and mongodb-driver-reactivestreams 1.x.

    <!-- Standard Mongo -->
    @Bean
    public LockProvider lockProvider(MongoClient mongo) {
        return new MongoLockProvider(mongo.getDatabase(databaseName));
    }
    
    <!-- Reactive Mongo -->
    @Bean
    public LockProvider lockProvider(MongoClient mongo) {
        return new ReactiveStreamsMongoLockProvider(mongo.getDatabase(databaseName));
    }
  12. Configure JdbcTemplateLockProvider

    master

    To use a relational database via JDBC, follow these steps:

    1. Create the lock table: Use the SQL schema appropriate for your database (MySQL, Postgres, Oracle, MS SQL, or DB2). The name column must be the primary key.
    2. Add the dependency: Include shedlock-provider-jdbc-template.
    3. Configure the Bean: Create a JdbcTemplateLockProvider bean. It is strongly recommended to use .usingDbTime() to ensure the lock uses the database server's UTC clock, preventing issues caused by unsynchronized application server clocks.
    <!-- SQL Example (MySQL/MariaDB) -->
    CREATE TABLE shedlock(name VARCHAR(64) NOT NULL, lock_until TIMESTAMP(3) NOT NULL,
        locked_at TIMESTAMP(3) NOT NULL DEFAULT CURRENT_TIMESTAMP(3), locked_by VARCHAR(255) NOT NULL, PRIMARY KEY (name));
    
    <!-- Java Configuration -->
    @Bean
    public LockProvider lockProvider(DataSource dataSource) {
        return new JdbcTemplateLockProvider(
            JdbcTemplateLockProvider.Configuration.builder()
            .withJdbcTemplate(new JdbcTemplate(dataSource))
            .usingDbTime()
            .build()
        );
    }