Lmod Documentation

The Lmod codebase is organized into several key directories that separate core logic, shell integration, and administrative tools:

• src/: The core of Lmod. Contains all Lua source files implementing Lmod's functionality (e.g., lmod.in.lua, MainControl.lua, Var.lua, modfuncs.lua).
  • src/MC_*.lua: Specific implementations for different modes of the MainControl program (MCP).
• shells/: Lua modules used by the Output Formatter to provide shell-specific formatting (e.g., Bash.lua, Csh.lua, Zsh.lua).
• libexec/: Contains helper scripts and main executable entry points like lmod and ml. These are typically wrappers that invoke the Lua interpreter with specific entry point files (like lmod.in.lua or ml_cmd.in.lua).
• init/: Shell initialization scripts (Bash, Csh, Zsh, Fish, etc.) that define the module shell function or alias used by end-users.
• lmodadmin/: Administrative scripts, such as update_lmod_system_cache_files.
• etc/: Global configuration files, including lmod_config.lua (site-wide settings) and lmodrc.lua (system-wide module RC file).
• docs/: Documentation source files (reStructuredText).
• rt/: The regression testing suite used to verify functionality and stability across different shells.
• tools/: Auxiliary Lua modules or scripts for development and debugging.

Lmod (version 8.7.54+) provides a complete solution for tracking module usage by using syslog to capture usage events and collecting that data into a database. This allows administrators to monitor which modules are being used (or ignored) by users across a cluster.

Key Architecture

In a typical cluster setup:

1. Nodes: Each compute node sends module usage messages via syslog to a central machine.
2. Central Machine (Master): A central machine collects these syslog messages and writes them to a dedicated log file containing only tracking data.
3. Database: The tracking data is then ingested from the log file into a database (e.g., MySQL 8.0+).

Version Note (Gen 1 vs Gen 2)

This guide describes the Gen 2 database implementation. Gen 2 is significantly more efficient, storing much less data (up to 100x less in some environments) and making it easier to prune old records. If you are migrating from an older version, you must follow the conversion process from gen_1 to gen_2.

The System Spider Cache is a mechanism used by Lmod to speed up module avail and module spider commands. Instead of walking the entire directory tree in MODULEPATH to find modulefiles, Lmod reads a pre-built cache file.

Key behaviors:

• Performance: Reading a single cache file is significantly faster than directory walking.
• Scope: The cache only contains information about system modules. Personal module files are always found via standard directory walking and are not affected by the cache.
• Limitations: The cache does not store the actual contents of modulefiles. Lmod still reads and evaluates the real modulefile during module load, module show, and similar commands to ensure correctness.
• Safety: If the cache is out-of-date, Lmod will not find the modules listed in it. To prevent users from being unable to load modules, you can enable cached loads, but only if you ensure the cache is kept up-to-date.

The mcp (Main Control Program) is a central, stateful object that orchestrates Lmod's behavior. It is an instance of the MainControl class (defined in src/MainControl.lua). The mcp object interprets modulefile commands and dispatches them to specific implementations based on its current operational mode.

Creation and Modes

An mcp object is typically created using the factory method MainControl.build("mode"). The mode determines how commands like setenv() or prepend_path() are executed:

• load: Handles module load or module add. Commands like setenv() apply environment changes.
• unload: Handles module unload or module rm. Commands are interpreted in reverse (e.g., setenv() unsets the variable).
• show / access: Handles module show, display, help, or whatis. Commands typically print what they would do instead of executing changes.
• spider: Used for module spider cache generation. Evaluation is limited to metadata extraction.
• checkSyntax: Validates modulefiles without applying changes.
• computeHash: Used in collection management to detect modulefile changes.
• mgrLoad: Used for restoring collections; load() and depends_on() commands are ignored.
• refresh: Used for module refresh to redefine aliases and shell functions.
• quiet: Suppresses most operations; used for internal tasks.

-- Typical creation pattern
mcp = MainControl.build("load")

The MarkdownDetector uses a scoring system to determine if a string should be treated as Markdown. This prevents plain text (like environment variable dumps) from being incorrectly formatted.

Detection Rules:

• Minimum Score: A score of 3 or more is required to treat content as Markdown.
• Minimum Length: Content under 30 characters is never treated as Markdown.
• Scoring Indicators:
  • ATX Headers: Lines matching ^#+%s+%S.
  • Setext Headers: Lines matching ^===+$ or ^---+$ with a non-empty line above them.
  • Lists: Lines matching ^[-*+]%s+%S or ^%d+\.%s+%S (allowing 0–3 spaces of indentation).
  • Other indicators: Emphasis, code blocks, links, images, and general structure.

Implementation Note: The detector uses splitLines() which relies on string.find() rather than string.gmatch() to ensure consistent behavior across different Lua versions (5.1–5.4).

Lmod searches for modulefiles in the directories listed in the MODULEPATH environment variable.

Modulefile Formats

• Lua modules: Files with the .lua extension are interpreted as Lua code.
• TCL modules: All other files are assumed to be TCL. Crucial: TCL modulefiles MUST start with the string #%Module or they will be silently ignored.
• Module Naming: The name of the module is determined by the file or directory name under a MODULEPATH entry.
  • Name/Version (N/V): A common pattern where a directory represents the module name and files within it represent versions (e.g., ucc/8.1.lua).
  • Meta-modules: A file like StdEnv.lua located directly in a MODULEPATH directory acts as a module with no version, typically used to load other modules.

Priority Rules

• If both a .lua and a non-.lua version file exist for the same module, the .lua file takes precedence.

## Example Directory Structure
/opt/apps/modulefiles
├── StdEnv.lua
├── ucc/
│   ├── 8.1.lua
│   └── 8.2.lua
└── xyz/
    └── 10.1.lua

# Reported as:
StdEnv ucc/8.1 ucc/8.2 (D) xyz/10.1