DOCSEARCHbyAlgolia
Submit Docs

Pueue Documentation

repository·main·Indexed 27 days ago

https://github.com/nukesor/pueue

A command-line task management tool for the sequential and parallel execution of shell commands. It consists of a CLI client (pueue), a background daemon (pueued) to ensure tasks persist after terminal sessions close, and a shared Rust library (pueue-lib) for programmatic interaction and custom front-end development.

Tokens
14.3K
Snippets
23
Records
110
Agent score
89%

What's inside Pueue

  1. Overview of the Pueue Daemon components

    main

    The daemon consists of two primary functional areas:

    • Request Handling (daemon::network): Manages communication with clients. It includes specific handlers for all Pueue subcommands located in daemon::network::message_handler.
    • Process Handling (daemon::process_handler): Responsible for the actual management and execution of system processes.

    Both components interact with a single source of truth via a SharedState handle, which is an Arc<Mutex<State>> wrapping the central State struct.

  2. Overview of Pueue-lib

    main

    Pueue-lib is the shared library used by both the Pueue client and daemon. It provides the necessary components for third-party applications to communicate with, manipulate, or monitor the Pueue daemon, or to build custom front-ends.

    Key components include:

    • State: Represents the current daemon state as exposed to clients.
    • Data Models: Includes Task, TaskResult, and TaskStatus.
    • Settings: The Settings struct used by both clients and the daemon.
    • Network Communication: Provides async and blocking code, including Request and Response message types, and helper functions like network::send_request and network::receive_response.
    • Client Implementation: A reference Client implementation (available via the client feature) which implements Client::send_request and Client::receive_response.

  3. Understand the Pueue project structure

    main

    Pueue is divided into three main components:

    1. pueue (Client): The CLI tool used by end-users to interact with the daemon.
    2. pueued (Daemon): The background service that manages task scheduling and process execution.
    3. pueue-lib: A shared library used by both the client and the daemon. It contains the communication protocol, settings parsing, and core data structures such as state, task, and message.

  4. Interface with Pueue programmatically via Rust

    main

    If you need to interact with Pueue using Rust code, use the pueue_lib crate instead of the internal library files found in the main repository. Documentation for the library can be found on docs.rs.

    https://docs.rs/pueue-lib/latest/pueue_lib/

  5. Install Pueue from source

    main

    To build Pueue from source, clone the repository and use cargo build. This requires the current stable Rust version. The resulting binaries pueue and pueued will be located in target/release/.

    git clone git@github.com:Nukesor/pueue
cd pueue
cargo build --release --locked --path ./pueue

  6. Cross-compile Pueue using `cross`

    main

    You can compile and test Pueue for different architectures using cargo-cross.

    Prerequisites

    1. Install cargo-cross.
    2. Install qemu on your host system:
      • Arch-Linux: Install qemu-user-static-binfmt.
      • Ubuntu: Install binfmt-support and qemu-user-static.

    Usage

    Run build or test commands against your desired target infrastructure using the --target flag.

  7. Configure Pueue-lib features

    main

    Pueue-lib uses Cargo features to manage its dependencies and functionality. For a minimal setup, you should disable default features and enable only the specific modules you need.

    • client: Adds a Client and/or BlockingClient implementation. This requires the network and/or network_blocking features to be active.
    • network: Adds asynchronous network and protocol functions.
    • network_blocking: Adds blocking std network and protocol functions.
    • log: Adds functions for reading Pueue's local log files.
    • settings: Provides the Settings struct used by both the daemon and client.

  8. Understand the Daemon's main execution loops

    main

    The daemon runs two main loops in parallel using a multi-threaded tokio async runtime via try_join!:

    Task Handler Main Loop

    Located in daemon::task_handlers::run, this loop manages the core scheduling logic:

    • Scheduling and starting new tasks when slots become available.
    • Handling finished tasks and cleaning up processes.
    • Enqueueing delayed tasks once their enqueue_at time is reached.
    • Managing callback processes.
    • Performing task dependency checks (e.g., marking tasks as failed if dependencies fail).
    • Handling reset and shutdown logic.

    Message Handler Main Loop

    Located in daemon::network::socket::accept_incoming, this loop manages client connectivity:

    • Listening on the daemon's socket (Unix/TCP).
    • Accepting connections and spawning tokio tasks for each connection via handle_incoming.
    • Performing authorization (secret & certificate).
    • Deserializing incoming messages and routing them through handle_message.

  10. Install Pueue via Cargo

    main

    Install Pueue using Cargo. This requires the current stable Rust version. The binary will be installed to $CARGO_HOME/bin/pueue (defaulting to ~/.cargo/bin/pueue).

    cargo install --locked pueue

  11. Manage task states: Stash, Enqueue, and Start

    main

    Control when tasks are executed using the following subcommands:

    • pueue stash: Move tasks to a Stashed state so they won't start automatically. Use --group or --all to stash multiple tasks.
    • pueue enqueue: Move stashed tasks into the active queue. Use --delay to schedule when they should be enqueued.
    • pueue start: Resume operation of tasks. Use --group or --all to resume groups. Note that pueue start <task_id> can force-start tasks, ignoring parallelism limits and dependencies.
    • pueue restart: Restart failed or successful tasks. Use --in-place to reuse the existing task (overwriting logs) or --not-in-place to create a new identical task.

  12. Edit tasks using the Pueue CLI

    main

    You can edit one or more tasks using the Pueue client. When you initiate an edit, the selected tasks are Locked on the daemon side, preventing them from being started or manipulated until the editing process is complete or aborted.

    Editing Modes

    Pueue supports two different editing modes, which can be configured in your settings:

    1. TOML Mode (EditMode::Toml): Creates a single temporary tasks.toml file containing all selected tasks. This is convenient but requires you to follow TOML syntax and escaping rules to avoid deserialization errors.
    2. Files Mode (EditMode::Files): Creates a temporary directory containing a subfolder for each task. Each task folder contains individual files for its editable properties (command, path, label, and priority). This mode is safer as it avoids complex file format parsing.

    Requirements

    • The $EDITOR environment variable must be set in your shell so Pueue knows which text editor to launch.
    • In Files Mode, the following files are used per task:
      • command: The task's original command.
      • path: The task's working directory.
      • label: The task's label (an empty file results in no label).
      • priority: The task's priority (must be a valid integer).

    Error Handling

    If an error occurs during editing (e.g., the editor exits with a non-zero code, or you provide invalid data like an empty command or a non-existent path), the client attempts to notify the daemon to restore the tasks from their Locked state to their previous state.