portalocker

The behavior of file locks depends heavily on your operating system, which affects how your application interacts with other processes:

• POSIX (Linux/macOS): Advisory Locking. Locks are a convention. The kernel only enforces them between processes that explicitly request a lock. An unrelated process that simply opens and writes to the file will not be blocked and will succeed immediately. To ensure safety, every participant accessing the file must use a locking mechanism.
• Windows: Mandatory Locking. The OS enforces locks. An unrelated process attempting to read or write an overlapping range of a locked file will fail with a PermissionError, even if that process never requested a lock.

Note: Linux removed support for mandatory locking in kernel 5.15. Always treat POSIX locks as advisory.

# POSIX Example: Process B succeeds even if Process A holds an exclusive lock
import fcntl

# Process A
fh = open('data.txt', 'r+')
fcntl.flock(fh, fcntl.LOCK_EX)

# Process B (No lock requested)
with open('data.txt', 'r+') as other:
    other.write('clobbered')  # Succeeds on POSIX

When a RedisLock is revoked (e.g., due to a connection error or subscription loss), the holder can detect this loss through four distinct channels. Note that since version 4.2.0, the subscription uses a dedicated connection with a zero-retry policy, meaning the first read error after revocation is terminal.

Loss Detection Channels:

1. RedisLock.lost property: Becomes True and remains True through RedisLock.release(). It is reset only by a subsequent RedisLock.acquire().
2. RedisLock.ensure_held() method: Raises portalocker.exceptions.LockLostError. This is useful for long critical sections to periodically check if the lock is still valid. It checks local state and does not perform a new network request.
3. with block exit: If a lock is used as a context manager, the __exit__ method will raise portalocker.exceptions.LockLostError if the lock was lost during the block's execution.
4. on_lost callback: A function passed to the RedisLock constructor that fires exactly once per loss on a background keep-alive thread.

Important Notes:

• Callback constraints: Keep on_lost callbacks short. Do not acquire application locks inside them. Exceptions raised in the callback are logged, not propagated.
• Interrupt behavior: By default, a loss interrupts the main thread with a KeyboardInterrupt. In portalocker 5.0.0+, interrupt_on_lost defaults to False. Use the interrupt_on_lost parameter to explicitly control this behavior.

import portalocker

# Using the on_lost callback and ensure_held
def my_callback():
    print("Lock was lost!")

lock = portalocker.RedisLock(
    'my_channel',
    connection=redis_conn,
    on_lost=my_callback,
    interrupt_on_lost=False
)

lock.acquire()
try:
    # Perform work
    lock.ensure_held()
except portalocker.LockLostError as e:
    print(f"Lost lock on {e.channel}")

Whole-file locking

portalocker is designed to lock whole files, not specific byte ranges. Even though underlying implementations (like Windows msvcrt or POSIX fcntl.lockf) might support ranges, portalocker anchors all locks at byte 0 to ensure that any two processes attempting to lock the same file will always contend.

Implementation Details

• POSIX: Uses fcntl.lockf with defaults start=0, whence=SEEK_SET, and len=0 (locking to the end of the file).
• Windows (msvcrt): Locks 64 KiB (0x10000) starting at byte 0 (it seeks to 0 before locking and restores the position after).
• Windows (Win32): Uses LockFileEx with an offset of 0 and a fixed large range.

Limitations

• No Concurrent Region Access: You cannot use portalocker to allow two processes to work on different regions of the same large file concurrently.
• Workaround: If you require byte-range locking, you must call fcntl.lockf directly with explicit start and len arguments (note: this is POSIX-only).
• Mental Model: Treat the lock as a token that names the file, rather than a mechanism protecting a specific range of bytes.

import portalocker

# Two locks on the same file will always contend, regardless of seek position
first = portalocker.Lock('example.lock', 'a', fail_when_locked=True)
first.acquire()

second = portalocker.Lock('example.lock', 'a', fail_when_locked=True)
try:
    second.acquire()
except portalocker.AlreadyLocked:
    print('contended, as expected')

first.release()

When using portalocker, keep these technical details in mind:

Unix/Linux

Unix file locks are advisory. All participating processes must explicitly acquire the lock to respect it.

Network Filesystems (NFS/etc.)

Network filesystems have unique locking and buffering behaviors. To ensure data integrity before releasing a lock, you may need to manually flush and sync the file handle:

fh.flush()
os.fsync(fh.fileno())

PidFileLock Behavior

By default, a PidFileLock context is for inspection. The body of the context runs even if another process holds the lock; it returns the holder's PID (or None if the current process acquired it). To ensure the body only runs after successful acquisition, use .fail_closed().

RedisLock Behavior

RedisLock uses a pub/sub subscription. Ownership is released when the subscription is removed. If a network partition occurs, the holder may observe a loss of the lock. For advanced usage, see documentation regarding lost, ensure_held(), health checks, and fencing tokens.