DOCSEARCHbyAlgolia
Submit Docs

LeilFS Documentation

website·Indexed 33 days ago

https://docs.leil.io/

Documentation for LeilFS version 5.11.0, a POSIX filesystem featuring a master and chunkserver architecture. Includes guides on installation, container deployment, CLI reference, mount options via FUSE3, NFS-Ganesha integration, and administration of quotas, snapshots, and trash policies.

Tokens
26.7K
Snippets
134
Records
195
Agent score
60%

What's inside LeilFS

  1. Overview of SaunaFS Metaloggers

    A metalogger is a pure, independent backup of cluster metadata. It connects to the master to receive a copy of every changelog entry and periodic full downloads of the metadata image. Unlike a shadow master, a metalogger does not participate in consensus and cannot become the master. For maximum reliability, metaloggers should be run on a host separate from the master to ensure that a master-host failure does not destroy the backup.

  2. Overview of LeilFS configuration files

    LeilFS configuration files are located in /etc/saunafs/ and are read by their respective components at startup. Most changes require a service restart, although some can be reloaded using SIGHUP.

    FileComponentPurpose
    leil-master.cfgMaster metadata serverPersonality, ports, chunk management
    leil-chunkserver.cfgChunkserverMaster connection, storage, performance
    leil-mount.cfgClient mount defaultsMaster address, password, mount options
    leil-exports.cfgExport access controlClient mount permissions and restrictions
    leil-goals.cfgReplication goals / ECNamed replication and erasure-coding policies
    leil-hdd.cfgChunkserver disk listStorage directories, metadata/data separation
    leil-topology.cfgNetwork topologyRack-awareness for replication decisions
    leil-metalogger.cfgMetaloggerMaster connection, metadata backup retention
    leil-uraft.cfguRaft HA daemonConsensus nodes, election timing, floating IP

  3. Overview of LeilFS replication modes

    LeilFS (v5.11.0) provides two primary replication modes. Replication settings are applied at the chunk level rather than the node level to ensure balanced data distribution across servers. To ensure system repairability during server failures, it is recommended to maintain at least one extra chunk server beyond the number of copies required by your configured goals.

    • Simple Goal Setup: Specifies a fixed number of copies for each file or directory chunk across chunk servers.
    • EC Mode (Erasure Coding): An advanced mode using configurable data and parity copies. It supports up to 32 data and parity chunks, allowing clients to write quasi-parallel to chunk servers for higher performance.

  4. LeilFS architectural overview

    LeilFS is a distributed POSIX file system inspired by the Google File System. It uses a chunk-based storage architecture where files are segmented into 64 MiB chunks, which are further subdivided into 64 KiB blocks. Each block includes a 4-byte CRC (Cyclic Redundancy Check) for data integrity. The system consists of three main components: Metadata Servers (Master, Shadows, Metaloggers), Data Servers (Chunkservers), and Clients (supporting multiple operating systems and NFS).

  5. LeilFS hardware and deployment guidelines

    Follow these fundamental guidelines when deploying LeilFS services:

    • Master Server: Ideally run on a separate machine equipped with an SSD.
    • Chunk Server: Must have at least one dedicated disk.
    • Metalogger: Optional. Avoid installing on the same machine as the Master server, though it can coexist with a Chunk server.
    • Data Safety: Use Shadow Master servers to provide failover and enhance data safety.
    • Prerequisites: Ensure all servers have proper network settings and suitable kernel settings before setup.

  6. List active mounts and metadata servers

    Use list-mounts to see information about all connected mounts (including goal and trash time limits with --verbose) and list-metadataservers to check the status of active metadata servers.
    list-mounts <master ip> <master port> [--porcelain] [--verbose]
list-metadataservers <master ip> <master port> [--porcelain]

  7. LeilFS Core Architecture Glossary

    Key architectural terms for LeilFS v5.11.0:

    • Chunk: The atomic unit of storage in LeilFS, representing a file segment up to 64 MiB.
    • Chunkserver: A storage node that holds chunk data on local disks.
    • Master: The single active metadata server per cluster.
    • Shadow master: A hot-standby metadata server that receives real-time changelogs and can be promoted to master via uRaft or manually.
    • Metalogger: A daemon that archives metadata changelogs as an independent backup; it cannot become a master.
    • uRaft: The Raft consensus implementation used for master election and automatic failover.
    • Inode: A metadata record representing a file or directory in the namespace.
    • Topology: Network rack/datacenter assignments used for failure-domain-aware chunk placement and nearest-replica reads.

  8. Understand LeilFS quota limits

    LeilFS supports quotas on both disk space (bytes) and inode count (number of files and directories). Quotas can be applied per user, per group, or per directory subtree. Each quota consists of two thresholds for both space and inodes:

    • Soft limit: May be exceeded temporarily; crossing it is reported but not blocked.
    • Hard limit: Cannot be exceeded; writes that would cross this limit fail.

  9. Understand LeilFS Trash Policy and Default Behavior

    Files deleted from a LeilFS mount are moved to a special trash directory rather than being immediately erased. They remain there for a configurable period called trashtime before their chunks are physically deallocated.

    Key behaviors:

    • Default Trashtime: 94,620 seconds (~26 hours, 17 minutes).
    • Granularity: Trashtime is stored per inode; every file or directory can have its own policy.
    • Maximum Value: 32-bit unsigned integer (approx. 136 years).
    • Inheritance: Objects inherit trashtime from their parent directory only at creation time. Subsequent changes to the parent do not affect existing children unless a recursive update is performed.

  10. Prerequisites for a minimal LeilFS setup

    To perform a minimal LeilFS installation for testing on a single machine, you must set up three components: a master server, a chunkserver, and a client.

    Important Caveat: Running the LeilFS master on localhost is considered experimental and may lead to unexpected behavior. It is recommended to use a dedicated IP address for the master server instead of localhost.

  11. Install LeilFS on Ubuntu

    LeilFS supports Ubuntu 22.04 and 24.04. Installation requires importing the public GPG key and adding the appropriate repository to the apt sources. If your version of apt requires ASCII format keyrings (via sqv), you must export the keyring to an .asc file.
    # 1. Ensure GPG is setup
gpg2 --list-keys

# 2. Import the public key
gpg --no-default-keyring \
    --keyring /usr/share/keyrings/saunafs-archive-keyring.gpg \
    --keyserver hkps://keyserver.ubuntu.com \
    --receive-keys 0xA80B96E2C79457D4

# 3. Add repository (Ubuntu 24.04)
sudo tee /etc/apt/sources.list.d/saunafs.list <<EOF
deb [arch=amd64 signed-by=/usr/share/keyrings/saunafs-archive-keyring.gpg] https://repo.leil.io/repository/saunafs-ubuntu-24.04/ noble main
EOF

# OR Add repository (Ubuntu 22.04)
sudo tee /etc/apt/sources.list.d/saunafs.list <<EOF
deb [arch=amd64 signed-by=/usr/share/keyrings/saunafs-archive-keyring.gpg] https://repo.leil.io/repository/saunafs-ubuntu-22.04/ jammy main
EOF

# 4. Update and install
sudo apt update

  12. Remove or replace a damaged storage device

    To remove a device (or handle a failed one), delete or comment out the corresponding line in leil-hdd.cfg. The system will treat the device as missing and will begin replicating data from other devices to restore chunks after a delay defined by OPERATIONS_DELAY_INIT in leil-master.cfg. Note that for large datasets, this method is faster than a graceful move but puts more strain on the overall system.