LXCFS Documentation

repository·main·Indexed 22 days ago

LXCFS is a FUSE-based filesystem that provides container-aware versions of /proc and /sys files. It allows Linux containers to see resource usage, such as CPU, memory, and swap, relative to their own limits rather than the host's. The documentation covers building from source using meson, running the daemon, integrating with LXC and Docker, and managing shared library upgrades.

Tokens: 1.8K
Snippets: 6
Records: 8
Agent score: 29%

What's inside LXCFS

What is LXCFS and how does it work?
main
LXCFS is a FUSE filesystem designed to make Linux containers feel more like virtual machines. It provides container-aware versions of crucial procfs and sysfs files. Instead of showing host-level information, LXCFS ensures that files like /proc/uptime, /proc/meminfo, and /proc/stat reflect the container's specific resource usage and lifecycle.
Key files managed by LXCFS include:
- /proc/cpuinfo
- /proc/diskstats
- /proc/meminfo
- /proc/stat
- /proc/swaps
- /proc/uptime
- /proc/slabinfo
- /proc/pressure/io, /proc/pressure/cpu, /proc/pressure/memory
- /sys/devices/system/cpu/online
https://github.com/lxc/lxcfs/blob/main/README.md
Understand LXCFS swap handling
main
LXCFS swap reporting is subject to kernel limitations and configuration.
Key constraints:
- Accounting: SWAP is only reported if SWAP accounting is enabled on the host (requires swapaccount=1 kernel boot option and/or CONFIG_MEMCG_SWAP). If not enabled, no SWAP is reported.
- Size Reporting: Because container swap usage can exceed the delta between RAM and RAM+SWAP limits, LXCFS reports the SWAP size as the smaller of the RAM+SWAP limit or the host SWAP device itself. This prevents applications from assuming more swap is available than actually exists.
- Swappiness 0: If swappiness is set to 0 and there is no usage, no SWAP is reported. If there is usage, a SWAP device equal to the size of the usage (100% full) is reported to accurately reflect memory consumption without misleading the application about available capacity.
https://github.com/lxc/lxcfs/blob/main/README.md

Build LXCFS from source

main

To build LXCFS, you must have fuse3 and the FUSE development headers installed on your system. The project uses the meson build system.

Standard Build

git clone git://github.com/lxc/lxcfs
cd lxcfs
meson setup -Dinit-script=systemd --prefix=/usr build/
meson compile -C build/
sudo meson install -C build/

Build with Sanitizers

To enable AddressSanitizer (ASAN) and UndefinedBehaviorSanitizer (UBSAN), use the -Db_sanitize option during setup:

meson setup -Dinit-script=systemd --prefix=/usr build/ -Db_sanitize=address,undefined
meson compile -C build/

meson setup -Dinit-script=systemd --prefix=/usr build/ -Db_sanitize=address,undefined
meson compile -C build/

Configure LXCFS with LXC
main
For systemd-based containers using LXC:
1. If using LXC 1.1, it should work automatically.
2. For other versions, copy the lxc.mount.hook and lxc.reboot.hook files (generated during build) to /usr/share/lxcfs and ensure they are executable.
3. Add the following configuration to your container config:
```
lxc.mount.auto = cgroup:mixed
lxc.autodev = 1
lxc.kmsg = 0
lxc.include = /usr/share/lxc/config/common.conf.d/00-lxcfs.conf
```
```
lxc.mount.auto = cgroup:mixed
lxc.autodev = 1
lxc.kmsg = 0
lxc.include = /usr/share/lxc/config/common.conf.d/00-lxcfs.conf
```
https://github.com/lxc/lxcfs/blob/main/README.md
Run LXCFS
main
The recommended way to start the LXCFS daemon is to create a directory for its data and then run the binary pointing to that directory:
```
sudo mkdir -p /var/lib/lxcfs
sudo lxcfs /var/lib/lxcfs
```
If you want to set all values in meminfo that refer to swap to 0, use the -u flag:
```
sudo lxcfs -u /var/lib/lxcfs
```
```
sudo mkdir -p /var/lib/lxcfs
sudo lxcfs /var/lib/lxcfs
```
https://github.com/lxc/lxcfs/blob/main/README.md
Upgrade LXCFS without restarting containers
main
LXCFS is designed to allow upgrading the shared library liblxcfs without restarting the lxcfs binary or the containers using it. The binary detects a new library version and reloads it using dlclose(3) and dlopen(3).
Note for musl users: Because musl does not guarantee that destructors are run on dlclose(3), users on musl systems should perform a complete restart of the lxcfs process and all associated containers.
To force a reload of the shared library on glibc systems:
1. Delete the old library file.
2. Copy the new library file into place.
3. Send SIGUSR1 to the lxcfs process.
```
rm /usr/lib64/lxcfs/liblxcfs.so # MUST to delete the old library file first
cp liblxcfs.so /usr/lib64/lxcfs/liblxcfs.so # to place new library file
kill -s USR1 $(pidof lxcfs) # reload
```
https://github.com/lxc/lxcfs/blob/main/README.md

Collect a core dump for LXCFS troubleshooting

main

If LXCFS crashes, you can collect a core dump by temporarily modifying the kernel's core_pattern.

Steps to collect a core dump (as root):

Back up the current core_pattern.
Set a new core_pattern that pipes the dump through gzip into /var/crash.
Wait for the crash and check /var/crash for a file like core-lxcfs.<pid>.gz.
Restore the original core_pattern.

# 1. Save old value
cat /proc/sys/kernel/core_pattern > /root/core_pattern.old_value.bak

# 2. Set new pattern
echo '|/bin/sh -c $@ -- eval exec gzip --fast > /var/crash/core-%e.%p.gz' > /proc/sys/kernel/core_pattern

# 3. (Wait for crash and check /var/crash)

# 4. Restore
cat /root/core_pattern.old_value.bak > /proc/sys/kernel/core_pattern

cat /proc/sys/kernel/core_pattern > /root/core_pattern.old_value.bak
echo '|/bin/sh -c $@ -- eval exec gzip --fast > /var/crash/core-%e.%p.gz' > /proc/sys/kernel/core_pattern
# ... wait for crash ...
cat /root/core_pattern.old_value.bak > /proc/sys/kernel/core_pattern

Use LXCFS with Docker

main

To use LXCFS in Docker, you must bind mount the container-aware files from the LXCFS directory into the container's /proc and /sys paths.

Example command:

docker run -it -m 256m --memory-swap 256m \
      -v /var/lib/lxcfs/proc/cpuinfo:/proc/cpuinfo:rw \
      -v /var/lib/lxcfs/proc/diskstats:/proc/diskstats:rw \
      -v /var/lib/lxcfs/proc/meminfo:/proc/meminfo:rw \
      -v /var/lib/lxcfs/proc/stat:/proc/stat:rw \
      -v /var/lib/lxcfs/proc/swaps:/proc/swaps:rw \
      -v /var/lib/lxcfs/proc/uptime:/proc/uptime:rw \
      -v /var/lib/lxcfs/proc/slabinfo:/proc/slabinfo:rw \
      -v /var/lib/lxcfs/proc/pressure/io:/proc/pressure/io:rw \
      -v /var/lib/lxcfs/proc/pressure/cpu:/proc/pressure/cpu:rw \
      -v /var/lib/lxcfs/proc/pressure/memory:/proc/pressure/memory:rw \
      -v /var/lib/lxcfs/sys/devices/system/cpu:/sys/devices/system/cpu:rw \
      ubuntu:18.04 /bin/bash

docker run -it -m 256m --memory-swap 256m \
      -v /var/lib/lxcfs/proc/cpuinfo:/proc/cpuinfo:rw \
      -v /var/lib/lxcfs/proc/diskstats:/proc/diskstats:rw \
      -v /var/lib/lxcfs/proc/meminfo:/proc/meminfo:rw \
      -v /var/lib/lxcfs/proc/stat:/proc/stat:rw \
      -v /var/lib/lxcfs/proc/swaps:/proc/swaps:rw \
      -v /var/lib/lxcfs/proc/uptime:/proc/uptime:rw \
      -v /var/lib/lxcfs/proc/slabinfo:/proc/slabinfo:rw \
      -v /var/lib/lxcfs/proc/pressure/io:/proc/pressure/io:rw \
      -v /var/lib/lxcfs/proc/pressure/cpu:/proc/pressure/cpu:rw \
      -v /var/lib/lxcfs/proc/pressure/memory:/proc/pressure/memory:rw \
      -v /var/lib/lxcfs/sys/devices/system/cpu:/sys/devices/system/cpu:rw \
      ubuntu:18.04 /bin/bash