pyfakefs Documentation

repository·main·Indexed 20 days ago

https://github.com/pytest-dev/pyfakefs

pyfakefs implements a fake, in-memory file system by mocking Python's file system modules, allowing tests to perform file I/O without touching the physical disk. It integrates with pytest via the 'fs' fixture, unittest via 'fake_filesystem_unittest.TestCase', and provides a Patcher context manager and @patchfs decorator. Key features include real-to-fake mapping, OS emulation (Linux, macOS, Windows), disk usage tracking, and the ability to pause/resume patching. Note: it does not work with Python libraries that use C extensions to access the file system.

Tokens
32.4K
Snippets
93
Records
150
Agent score
73%

What's inside pyfakefs

  1. Overview of pyfakefs features

    main

    pyfakefs provides a memory-based fake filesystem that mocks Python's filesystem modules. Key features include:

    • Transparent Operation: Code works on the fake filesystem without modification.
    • Framework Support: Direct support for pytest (via the fs fixture) and unittest, but compatible with other frameworks.
    • Isolation: Each test starts with an empty filesystem (except for OS temporary directories). Real files are not modified even when mapping real files into the fake system.
    • Configurability:
      • Filesystem Size: Can be configured arbitrarily.
      • OS Simulation: Can simulate Linux, macOS, or Windows regardless of the host OS.
      • User Simulation: Can simulate running as root or a non-root user.
    • Pause/Resume: Ability to temporarily use the real filesystem during a test step.
  2. Key features of pyfakefs

    main

    Beyond automatically mocking core Python file-system modules, pyfakefs includes these advanced capabilities:

    • Real-to-Fake Mapping: Map files and directories from the actual host file system into the fake in-memory file system.
    • Size Tracking: Configure and track the size of the fake file system.
    • Pause/Resume Patching: Temporarily pause the patching to allow a test step to interact with the real file system.
    • OS Emulation: Limited emulation of different operating systems (Linux, macOS, or Windows).
    • Non-root Emulation: Configure the environment to behave as if running as a non-root user, even when the test process is running under root.
  3. Handle global pathlib.Path objects in tests

    main

    Global pathlib.Path objects created at the module level (outside of tests) are instantiated using the real filesystem. Because pyfakefs replaces pathlib.Path with FakePath during tests, these two types will not compare as equal, even if they point to the same location.

    Solutions:

    1. Use strings for globals: Store paths as strings or use os.path functions for global constants, then convert them to Path objects inside the test.
    2. Reload modules: If you cannot change the production code, use the modules_to_reload option to reload the module containing the global Path objects so they are re-instantiated as FakePath objects.
    import pathlib
    import os
    
    # RECOMMENDED: Use strings or os.path for globals
    FILE_PATH = os.path.join(os.path.dirname(os.path.abspath(__file__)), "file.csv")
    # OR
    FILE_PATH = str(pathlib.Path(__file__).parent / "file.csv")
    
    def test_path_equality(fs):
        # This creates a FakePath inside the test context
        fake_file_path = pathlib.Path(FILE_PATH)
        assert fake_file_path.exists()
  4. Limitations of pyfakefs

    main

    When choosing pyfakefs, be aware of the following constraints:

    • C Libraries: It cannot mock filesystem access performed by C libraries (other than os and io).
    • MRO for File Objects: It does not retain the MRO for file objects; avoid relying on isinstance checks for file objects (though io.BufferedIOBase and io.TextIOBase checks still work).
    • Threading: It is not guaranteed to work correctly in multi-threaded environments, specifically regarding concurrent write access.
    • Patching Conflicts: It may not work correctly if filesystem functions are also being patched by other tools like unittest.mock.patch.
    • Python Implementations: Primarily tested and supported on CPython and newer PyPy versions.
    • Framework Specifics: Does not work correctly with behave if filesystem modules are imported globally in steps/environment files (workaround: import locally inside test steps).
  5. Understand nested filesystem fixtures and Patcher limitations

    main

    pyfakefs does not support nested fake filesystems. It uses reference counting on a single instance. If you attempt to create a new fake filesystem (via a new Patcher or a nested fs fixture) while one is already active:

    1. Arguments are ignored: Any custom arguments passed to the nested Patcher or fixture will be ignored.
    2. Reference counting: Only the reference count increases. When the nested context exits, only the count decreases; no changes are reverted.
    3. Warnings: A warning is issued by pyfakefs when nested instantiation is detected.

    This commonly happens when using module-scoped fixtures (like fs_module) and then requesting the standard fs fixture within individual tests.

  6. Understand the pyfakefs Python version support policy

    main

    pyfakefs follows these policies regarding Python version support:

    • New Python Versions: Support for new Python versions is typically added during the Python release beta phase, with official support following shortly after the final release.
    • End-of-Life (EOL) Versions: Support for EOL Python versions is removed once the CI (GitHub Actions) no longer provides those versions (typically several months after the official EOL).
    • Legacy Patches: If support is removed earlier (for example, during the transition to version 6), patches for previous versions can be provided upon request.
  7. Why some modules do not work with pyfakefs

    main

    pyfakefs works by patching specific modules: os, os.path, pathlib, the built-in open and io.open functions, and shutil.disk_usage. Other modules like shutil (except disk_usage), tempfile, glob, and zipfile work because they rely on these patched functions.

    A module might fail if:

    1. It uses a file system function that is not or is incorrectly patched (e.g., rare or new functions in Python libraries).
    2. It executes file system functions during module import (these require modules_to_reload).
    3. It uses OS-specific functions not in Python libraries (these can be patched via modules_to_patch or unittest.patch).
    4. It uses C libraries to access the file system (e.g., sqlite3, lxml). These cannot be patched and will always access the real filesystem.
  8. Use pytest scoped fixtures: fs_class, fs_module, and fs_session

    main

    For different test scopes, pyfakefs provides fs_class, fs_module, and fs_session fixtures.

    Important Behavior:

    • If a scoped fixture is active, any other fs fixture used within that scope will act only as a reference to the active fake filesystem rather than setting up/tearing down a new one.
    • Changes made to the fake filesystem in one test will persist until the end of the fixture's scope (e.g., the end of the class or module).
    • To prevent side effects, patching is automatically paused between individual tests, even if the scoped fixture remains active.
  9. Simulate different operating systems

    main

    By default, pyfakefs assumes the file system of the host OS. You can simulate Linux, macOS, or Windows by setting the os attribute of pyfakefs.FakeFilesystem to one of the OSType constants.

    Important: Changing the os attribute resets the fake file system. You must set the operating system type before adding any files to the system.

    When you change fs.os, pyfakefs automatically updates several internal attributes, including:

    • is_windows_fs: True for Windows (NTFS).
    • is_macos: True for macOS (HFS+) if is_windows_fs is False.
    • is_case_sensitive: True for Linux, False for Windows and macOS by default.
    • path_separator: \ for Windows, / for Posix.
    • alternative_path_separator: / for Windows, None for Posix.
    import os
    from pyfakefs.fake_filesystem import OSType
    
    def test_windows_paths(fs):
        fs.os = OSType.WINDOWS
        assert r"C:\foo\bar" == os.path.join("C:\\", "foo", "bar")
        assert os.path.splitdrive(r"C:\foo\bar") == ("C:", r"\foo\bar")
        assert os.path.ismount("C:")
  10. Understand OS temporary directories in pyfakefs

    main

    pyfakefs does not fake the tempfile module. To ensure tempfile.gettempdir() returns a valid value, a temporary directory is always present at the start of a new fake filesystem:

    • POSIX: A directory named /tmp is present.
    • Windows: C:\Users\<user>\AppData\Local\Temp is present.

    On macOS and Linux, if the actual system temp path is not /tmp, pyfakefs creates a symlink from /tmp to the actual temp directory in the fake filesystem. The size of this link is ignored when calculating fake filesystem size.

    import os
    
    
    def test_something(fs):
        # the temp directory is always present at test start
        assert len(os.listdir("/")) == 1
  11. Patch using fake_filesystem_unittest.TestCase

    main

    If you are using the Python unittest framework, inherit from pyfakefs.fake_filesystem_unittest.TestCase.

    To activate the fake filesystem, call self.setUpPyfakefs() inside your setUp() method. You do not need to call self.tearDownPyfakefs() in your tearDown() method; it is handled automatically. You can access the fake filesystem via self.fs or self.fake_fs().

    import os
    from pyfakefs.fake_filesystem_unittest import TestCase
    
    class ExampleTestCase(TestCase):
        def setUp(self):
            self.setUpPyfakefs()
    
        def test_create_file(self):
            file_path = "/test/file.txt"
            self.assertFalse(os.path.exists(file_path))
            self.fs.create_file(file_path)
            self.assertTrue(os.path.exists(file_path))
  12. Run pyfakefs unit tests

    main

    If you are contributing to pyfakefs, you can run the test suite using pytest, unittest, tox, or Docker.

    Using pytest or unittest

    cd pyfakefs/
    export PYTHONPATH=$PWD
    
    # Run all tests with pytest
    python -m pytest pyfakefs
    
    # Run all tests except pytest-specific ones with unittest
    python -m pyfakefs.tests.all_tests

    Using tox

    To run tests against all supported Python versions locally:

    tox

    Using Docker

    Build and run the tests in a container based on the latest Ubuntu:

    # Build the container
    docker build -t pyfakefs .
    
    # Run the tests
    docker run -t pyfakefs