DOCSEARCHbyAlgolia
Submit Docs

earlyoom

repository·master·Indexed 26 days ago

https://github.com/rfjakob/earlyoom

A user-space daemon that monitors available memory and swap to prevent system unresponsiveness. It intervenes earlier than the kernel's OOM killer by sending SIGTERM/SIGKILL to the largest processes when thresholds are breached. Features include configurable memory/swap thresholds, process filtering via regex (--prefer, --avoid, --ignore), custom pre- and post-kill scripts, and the oomstat tool for real-time memory and pressure monitoring.

Tokens
3.4K
Snippets
9
Records
17
Agent score
88%

What's inside earlyoom

  1. Install earlyoom

    master

    You can install earlyoom using your distribution's package manager or by compiling from source.

    Using Package Managers

    • Debian 10+ / Ubuntu 18.04+: sudo apt install earlyoom
    • Fedora / RHEL 8 (with EPEL): sudo dnf install earlyoom (then sudo systemctl enable --now earlyoom)
    • Arch Linux: sudo pacpac -S earlyoom (then sudo systemctl enable --now earlyoom)

    Compiling from Source

    git clone https://github.com/rfjakob/earlyoom.git
cd earlyoom
make

    To register earlyoom as a system service after compiling:

    • For systemd: sudo make install
    • For non-systemd: sudo make install-initscript
    sudo apt install earlyoom

  2. Use oomstat to monitor memory and pressure

    master

    Use oomstat to monitor system memory availability, free swap, and memory pressure (PSI) in real-time. The tool prints updates every 100ms.

    Output Columns:

    • Time (s): Elapsed time in seconds.
    • MemAvail (MiB): Available memory from /proc/meminfo.
    • SwapFree (MiB): Free swap space.
    • PSI (Memory Pressure): Memory pressure statistics from /proc/pressure/memory, including:
      • some avg10
      • full avg10
      • some %
      • full %
    $ ./oomstat

  3. Configure earlyoom priority in systemd

    master

    The -p flag (which sets niceness to -20 and oom_score_adj to -100) does not work when earlyoom is run through its default systemd service. To achieve high priority, you must edit the systemd service directly.

    Run sudo systemctl edit earlyoom and add the following lines:

    [Service]
OOMScoreAdjust=-100
Nice=-20

  4. Configure earlyoom via /etc/default/earlyoom

    master

    If running earlyoom as a system service (systemd or init.d), you can configure it using the file /etc/default/earlyoom. This file accepts the same arguments as the command line.

    Example configuration:

    EARLYOOM_ARGS="-m 5 -r 60 --avoid '(^|/)(init|Xorg|ssh)$' --prefer '(^|/)(java|chromium)$'"

    After modifying the file, restart the service to apply changes:

    systemctl restart earlyoom

    Note: This configuration file does not affect earlyoom instances running outside of systemd/init.d.

    EARLYOOM_ARGS="-m 5 -r 60 --avoid '(^|/)(init|Xorg|ssh)$' --prefer '(^|/)(java|chromium)$'"

  5. Configure memory and swap thresholds

    master

    earlyoom monitors available memory and swap to prevent system unresponsiveness. You can configure the minimum thresholds for both memory and swap using percentages or absolute sizes.

    By default, earlyoom sends SIGTERM when both memory and swap fall below the specified threshold, and SIGKILL when they fall below a second threshold (defaults to half of the primary threshold).

    Memory Thresholds:

    • -m PERCENT[,KILL_PERCENT]: Set available memory minimum as a percentage of user mem total (default 10%).
    • -M SIZE[,KILL_SIZE]: Set available memory minimum in KiB.

    Swap Thresholds:

    • -s PERCENT[,KILL_PERCENT]: Set free swap minimum as a percentage of total (default 10%).
    • -S SIZE[,KILL_SIZE]: Set free swap minimum in KiB.

    Notes:

    • If both -m and -M are provided, the lower value is used.
    • If both -s and -S are provided, the lower value is used.
    • To ignore swap usage and only trigger based on memory, use -s 100.

  6. Check earlyoom logs

    master

    If running as a systemd service, you can view the status and recent logs using:

    systemctl status earlyoom

    To parse logs specifically for killed processes (to trigger actions like sending emails), use journalctl:

    sudo journalctl -u earlyoom | grep sending

    For older versions of earlyoom, use:

    sudo journalctl -u earlyoom | grep -iE "(sending|killing)"
    sudo journalctl -u earlyoom | grep sending

  7. Execute scripts on process termination

    master

    You can run custom scripts when earlyoom kills a process. You can choose to run the script either before or after the process is killed.

    Options:

    • -P /PATH/TO/SCRIPT: Run the script beforehand. There is a 200ms delay after spawning the script before the process is killed to allow it to gather information.
    • -N /PATH/TO/SCRIPT: Run the script afterwards.

    Environment Variables available in the script:

    • EARLYOOM_PID: Process PID
    • EARLYOOM_NAME: Process name (truncated to 16 bytes from /proc/PID/comm)
    • EARLYOOM_CMDLINE: Process cmdline (truncated to 256 bytes from /proc/PID/cmdline)
    • EARLYOOM_UID: UID of the user running the process

    Warning: EARLYOOM_NAME can contain spaces, newlines, or special characters. Ensure your script handles these safely.

  8. Reference: earlyoom exit statuses

    master

    Exit codes returned by earlyoom.

    0: Successful program execution.
1: Other error
2: Switch conflict.
4: Could not cd to /proc
5: Could not open proc
7: Could not open /proc/sysrq-trigger
13: Unknown options.
14: Wrong parameters for other options.
15: Wrong parameters for memory threshold.
16: Wrong parameters for swap threshold.
102: Could not open /proc/meminfo
103: Could not read /proc/meminfo
104: Could not find a specific entry in /proc/meminfo
105: Could not convert number when parse the contents of /proc/meminfo

  9. Enable notifications and custom scripts

    master

    earlyoom supports notifications and script execution when a process is killed:

    • -n: Enables d-bus notifications. Requires systembus-notify to be running in your GUI session to see them.
    • -N /PATH/TO/SCRIPT: Calls a script after the OOM kill. The script receives EARLYOOM_PID, EARLYOOM_UID, and EARLYOOM_NAME as environment variables.
    • -P /PATH/TO/SCRIPT: Calls a script before the OOM kill.

    Warning: In --dryrun mode, scripts will be executed in rapid succession; ensure your scripts implement rate-limiting.

    -n
-N /PATH/TO/SCRIPT
-P /PATH/TO/SCRIPT

  11. Command line options reference

    master

    Full list of available command line flags for earlyoom:

    FlagDescription
    -m PERCENT[,KILL_PERCENT]Set available memory minimum to PERCENT of total (default 10 %). Sends SIGTERM at PERCENT, SIGKILL at KILL_PERCENT (default PERCENT/2).
    -s PERCENT[,KILL_PERCENT]Set free swap minimum to PERCENT of total (default 10 %). Both memory and swap must be below minimum to act.
    -M SIZE[,KILL_SIZE]Set available memory minimum to SIZE KiB.
    -S SIZE[,KILL_SIZE]Set free swap minimum to SIZE KiB.
    -nEnable d-bus notifications.
    -N /PATH/TO/SCRIPTCall script after oom kill.
    -P /PATH/TO/SCRIPTCall script before oom kill.
    -gKill all processes within a process group.
    -d, --debugEnable debugging messages.
    -vPrint version information and exit.
    -r INTERVALMemory report interval in seconds (default 1), set to 0 to disable.
    -pSet niceness of earlyoom to -20 and oom_score_adj to -100.
    --ignore-root-userDo not kill processes owned by root.
    --sort-by-rssFind process with the largest rss (default oom_score).
    --prefer REGEXPrefer to kill processes matching REGEX.
    --avoid REGEXAvoid killing processes matching REGEX.
    --ignore REGEXIgnore processes matching REGEX.
    --dryrunDry run (do not kill any processes).
    --syslogUse syslog instead of std streams.
    -h, --helpShow this help text.
    -m PERCENT[,KILL_PERCENT]
-s PERCENT[,KILL_PERCENT]
-M SIZE[,KILL_SIZE]
-S SIZE[,KILL_SIZE]
-n
-N /PATH/TO/SCRIPT
-P /PATH/TO/SCRIPT
-g
-d, --debug
-v
-r INTERVAL
-p
--ignore-root-user
--sort-by-rss
--prefer REGEX
--avoid REGEX
--ignore REGEX
--dryrun
--syslog
-h, --help

  12. Set preferred and avoided processes

    master

    You can influence which processes are targeted by using regex patterns:

    • --prefer REGEX: Specifies processes that earlyoom should prefer killing.
    • --avoid REGEX: Specifies processes that earlyoom should avoid killing.
    • --ignore REGEX: Specifies processes that earlyoom should ignore entirely.

    To find the victim based on the largest Resident Set Size (RSS) instead of the kernel's oom_score, use the --sort-by-rss flag.

    --prefer REGEX
--avoid REGEX
--ignore REGEX
--sort-by-rss