vmtouch Documentation

repository·master·Indexed 23 days ago

https://github.com/hoytech/vmtouch

A tool for diagnosing and controlling the file system cache on Unix and Unix-like systems. Includes instructions for installation from source and tuning memory locking limits on Linux, FreeBSD, and Solaris, as well as specific configuration guidance for S3FS and 9P filesystems.

Tokens
765
Snippets
2
Records
6
Agent score
34%

What's inside vmtouch

  1. Using vmtouch with S3FS (Amazon S3 FUSE driver)

    master

    When using S3FS on Linux, vmtouch cannot meaningfully operate on the S3 mount point itself because S3FS does not use the standard page cache. Instead, it uses a local cache directory.

    To use vmtouch effectively with S3FS:

    1. Target the files within the S3FS cache directory rather than the mount point.
    2. The default cache directory is $HOME/.fuse-s3fs-cache/.
    3. You can change this location using the cachedir mount option.
    4. Use the preserve_cache mount option if you want the cache to persist between mountings.
  2. Adjust memory locking limits on Solaris

    master

    On Solaris, memory locking limits are managed on a per-project or per-zone basis using resource-controls(5).

    • Use prctl or zonecfg to manipulate and observe limits.
    • The PRIV_PROC_LOCK_MEMORY privilege controls whether a process is allowed to lock memory.

    Example of checking the project.max-locked-memory limit using prctl:

    $ prctl -n project.max-locked-memory -i process $$
    process: 4690: bash
    NAME    PRIVILEGE       VALUE    FLAG   ACTION                       RECIPIENT
    project.max-locked-memory
            usage               0B
            system    16.0EB    max   deny                                 -
  3. Adjust memory locking limits on FreeBSD

    master

    To allow vmtouch to lock memory on FreeBSD, check the following:

    • security.bsd.unprivileged_mlock: This sysctl determines if unprivileged users are permitted to lock memory.
    • System limits: Monitor vm.max_wired and vm.stats.vm.v_wire_count for system-wide memory wiring statistics.
  4. Install vmtouch from source

    master

    To install vmtouch on a Unix-like system, clone the repository, build it using make, and install it to the system using sudo make install.

    $ git clone https://github.com/hoytech/vmtouch.git
    $ cd vmtouch
    $ make
    $ sudo make install
  5. Using vmtouch with 9P filesystems

    master
    The 9P2000 filesystem driver on Linux does not use the filesystem cache by default. To enable vmtouch functionality, you must mount the filesystem with the fscache option. Once fscache is enabled, vmtouch should operate as normal.
  6. Adjust memory locking limits on Linux

    master

    Because vmtouch uses the mlock(2) system call to lock files into memory, you may encounter OS-level restrictions. On Linux, you can manage these via:

    1. locked memory rlimit: Limits the amount of memory a process can lock.
      • Use ulimit -l to view/set the limit (requires super-user).
      • Unprivileged processes can have this limit raised by editing /etc/security/limits.conf.
      • Processes with CAP_IPC_LOCK capability are not restricted by this limit.
    2. vm.max_map_count: A sysctl parameter that controls the maximum number of Virtual Memory Areas (VMAs). Since vmtouch requires one VMA per file, this parameter limits the total number of files a single vmtouch process can lock.