filelock

Choose a lock type based on your filesystem capabilities and synchronization requirements:

When using soft locks, you must choose between mutual exclusion and progress based on your system's failure model:

1. StrictSoftFileLock (Fail-Closed):

   • Provides mutual exclusion without a native lock.
   • Behavior: It never breaks a claim based on age. If a process crashes while holding the lock, the lock remains held until an operator manually clears it using force_break(claim.name).
   • Use case: When you prefer the system to hang rather than risk two processes overlapping (e.g., Terraform state locks).

2. SoftFileLease (Fail-Open):

   • Behavior: The claim expires after a lease_duration. A peer can take over if the holder wedges. You can provide an on_compromise callback to stop work when the lease is lost.
   • Use case: When you prioritize progress and can handle a process resuming after its lease has expired (e.g., by fencing the resource).

Clearing a Strict lock:

stale = StrictSoftFileLock("work.lock")
for claim in stale.claims:
    stale.force_break(claim.name)

from filelock import StrictSoftFileLock

lock = StrictSoftFileLock("work.lock")
with lock:
    do_exclusive_work()

# Manual recovery after a crash:
stale = StrictSoftFileLock("work.lock")
for claim in stale.claims:
    stale.force_break(claim.name)

If you are on a filesystem without native locking but require strict mutual exclusion, use StrictSoftFileLock.

Unlike standard SoftFileLock which uses a single shared marker, StrictSoftFileLock uses immutable owner claims via hard links. This prevents multiple contenders from both reading themselves as the lowest claim.

Important considerations:

• Crash Recovery: It does not automatically infer that an owner has died. An orphaned or crashed claim will block entry until manually cleared using force_break().
• Compatibility: It uses a <path>.filelock/claims directory. It is compatible with legacy SoftFileLock holders, but mixing them (e.g., by deleting lock files or using break_lock()) can void mutual exclusion.
• Requirements: Requires filesystem support for coherent directory reads and atomic no-replace hard links (e.g., Linux open(2) pattern or NTFS hard links).

# To clear a stuck lock after a crash:
lock.force_break()

Since filelock provides advisory locking, it only protects processes that explicitly use it. To ensure data integrity even if a lock is bypassed (e.g., by a different tool or a network mount error), design your system so that the lock is an optimization rather than the sole source of correctness.

Use atomic filesystem operations like os.replace to ensure readers always see a complete file. In this model, losing the lock results in duplicated work (e.g., two processes downloading the same file) rather than data corruption.

import os
    import tempfile
    from pathlib import Path

from filelock import FileLock, Timeout

def populate(target: Path, produce) -> None:
        fd, tmp = tempfile.mkstemp(dir=target.parent)
        try:
            with os.fdopen(fd, "wb") as handle:
                handle.write(produce())
            os.chmod(tmp, 0o644)  # mkstemp creates 0600; widen before publishing
            os.replace(tmp, target)  # atomic: readers see the old file or the new one, never a partial one
        except BaseException:
            os.unlink(tmp)
            raise

def get(target: Path, produce) -> bytes:
        lock = FileLock(f"{target}.lock")
        try:
            with lock.acquire(timeout=30):
                if not target.exists():
                    populate(target, produce)
        except Timeout:
            if not target.exists():  # the lock only saved duplicate work; do the work anyway
                populate(target, produce)
        return target.read_bytes()

When using soft locks (locks that rely on marker files rather than kernel primitives), you must choose between two distinct contracts. Do not mix these contracts on the same file path, as they use different record formats and can interfere with each other.

StrictSoftFileLock

StrictSoftFileLock prioritizes safety and mutual exclusion. It never automatically reclaims a marker. If a holder crashes, the marker remains, and subsequent contenders will wait indefinitely rather than risking an overlap. To recover from a crashed holder, you must manually call force_break().

SoftFileLease

SoftFileLease prioritizes progress over strict exclusion. It uses a lease mechanism where a holder periodically refreshes its claim. If a holder becomes stale (exceeds lease_duration), a peer can take the marker. This means a holder and its successor might overlap briefly. Use the on_compromise callback to handle cases where your lease is lost.

Comparison Table

from filelock import SoftFileLease, StrictSoftFileLock

# Strict mode: no peer enters while this holder lives
with StrictSoftFileLock("work.lock", timeout=30):
        pass

def stop_working(compromise):
        print("lost the claim:", compromise.reason)

# Lease mode: a peer may enter 60s after the last refresh
with SoftFileLease("work.lock", lease_duration=60, on_compromise=stop_working):
        pass