DOCSEARCHbyAlgolia
Submit Docs

python-atomicwrites

repository·master·Indexed 43 days ago

https://github.com/untitaker/python-atomicwrites

A high-level API for performing atomic file writes across POSIX and Windows platforms to prevent data corruption. Provides the `atomic_write` function, a low-level API including `replace_atomic`, `move_atomic`, and the `AtomicWriter` class. Note: This package is currently unmaintained.

Tokens
831
Snippets
2
Records
6
Agent score
6%

What's inside atomicwrites

  2. How atomic writes work in python-atomicwrites

    master

    The library ensures atomicity by following these steps:

    1. Temporary File Creation: A temporary file is created in the same directory as the target path to ensure it resides on the same filesystem.
    2. Atomic Move:
      • POSIX: Uses rename if overwrite=True. If overwrite=False, it uses a combination of link and unlink.
      • Windows: Uses MoveFileEx via ctypes with appropriate flags.
    3. Durability (fsync):
      • POSIX: Invokes fsync on the temporary file (to flush content/metadata) and on the parent directory (to flush the filename). On OS X, fcntl with F_FULLFSYNC is used.
      • Windows: Uses _commit.

    Important Notes:

    • File Permissions: Because a new file is moved into place, the permissions of the original target file may change. The library does not automatically manage chmod to preserve permissions.
    • Race Window: When using link/unlink on POSIX, there is a brief window where the file may appear under both the temporary name and the target name.

  3. Handle filesystem errors in atomicwrites

    master

    All filesystem-related errors raised by atomicwrites are subclasses of OSError.

    • UNIX systems: Errors are thrown directly from Python standard library calls.
    • Windows systems: Errors are thrown from Python's ctypes implementation.

    In both cases, you can inspect the errno attribute on the caught exception to identify the specific error code, which maps to the standard errno module.

  4. Use atomic_write() for safe file writing

    master

    The atomic_write function is the primary high-level API for writing data to a file atomically. It ensures that the file is either fully written or not written at all, preventing data corruption in case of crashes or power failures.

    from atomicwrites import atomic_write

with atomic_write('foo.txt', mode='w') as f:
    f.write('some data')

  5. Perform atomic file writes with atomic_write()

    master

    Use the atomic_write function to ensure that a file is written completely or not at all. This is achieved by writing to a temporary file in the same directory and then moving it to the target location.

    By default, atomic_write will fail if the target file already exists. To allow overwriting the existing file, set the overwrite=True parameter.

    from atomicwrites import atomic_write

# To overwrite an existing file:
with atomic_write('foo.txt', overwrite=True) as f:
    f.write('Hello world.')

# To fail if the file already exists (default behavior):
with atomic_write('foo.txt') as f:
    f.write('Hello world.')

  6. Use the Low-level API for atomic operations

    master

    For more granular control, atomicwrites provides a low-level API including:

    • replace_atomic: Performs an atomic replacement of a file.
    • move_atomic: Performs an atomic move operation.
    • AtomicWriter: A class providing manual control over the atomic writing process.