Poe the Poet

repository·main·Indexed 24 days ago

https://github.com/nat-n/poethepoet

A batteries-included task runner designed to work seamlessly with Python dependency managers like Poetry and uv. It allows developers to define complex task sequences, shell scripts, and Python functions as manageable CLI commands, supporting advanced variable resolution, environment projection, and task composition via Sequence, Parallel, or Graph models.

Tokens
68.1K
Snippets
206
Records
328
Agent score
84%

What's inside poethepoet

  1. How Literal types are translated to JSON Schema

    main

    When translating a LiteralType, the behavior depends on the diversity of the values within the literal:

    1. Homogeneous Literals: If all values in the Literal map to the same JSON type (e.g., Literal["a", "b"]), the resulting schema includes both the type and the enum keys.
    2. Heterogeneous Literals: If the values span multiple JSON types (e.g., Literal[True, "yes"]), the type key is omitted, and only the enum key is emitted to allow for multi-type validation.

    Example behavior:

    • Literal["a", "b"] $\rightarrow$ {"type": "string", "enum": ["a", "b"]}
    • Literal[1, "a"] $\rightarrow$ {"enum": [1, "a"]}
  2. Understand the Poe the Poet execution flow

    main

    When you invoke a task via the CLI, Poe the Poet follows a specific lifecycle:

    1. Configuration Loading: PoeConfig.load() parses your pyproject.toml or poe_tasks.* files.
    2. CLI Parsing: PoeUi.build_parser() handles command-line arguments and help generation.
    3. Task Instantiation: RunContext.execute_task() uses a TaskSpecFactory to create the specific task instance from your configuration.
    4. Task Execution: PoeTask._handle_run() executes the task-specific logic.
    5. Environment Execution: PoeExecutor.execute() runs the task within the appropriate environment (e.g., a poetry or uv virtualenv) via a subprocess.
  3. Control argument placement using `$POE_EXTRA_ARGS`

    main
    By default, free arguments passed to a cmd task are appended to the end of the command. If you need to place these arguments in a specific position (for example, if fixed options must follow the arguments), you can explicitly reference $POE_EXTRA_ARGS within your command string. When referenced, Poe will expand it in place and will not append the arguments to the end of the command automatically.
  4. Understand the poethepoet.schema package architecture

    main

    The schema generation logic is split into several specialized modules:

    • generator.py: The orchestrator that walks ProjectConfig.ConfigOptions and registries to assemble the root schema.
    • context.py: Manages the central state, including the $defs registry and $ref lifting.
    • translate.py: Handles the translation of TypeAnnotation subclasses into structural JSON Schema.
    • fragments.py: Contains cross-cutting JSON Schema fragments that aren't owned by a single class (e.g., task definitions, executor tagged unions, or environment variable polymorphism).
  5. Naming requirements for tasks and groups

    main

    When defining tasks and groups in your Poe configuration, the keys must follow specific naming patterns enforced by the JSON schema and runtime validations:

    Task Names

    Task names must follow the pattern ^[A-Za-z_][\w\-:+]*$.

    • Accepted: Names starting with a letter or underscore, followed by alphanumeric characters, colons (:), underscores (_), dashes (-), or plus signs (+). Examples: my_task, Task-1.
    • Rejected: Names starting with a digit (e.g., 1bad), containing spaces (e.g., bad name), or starting with punctuation.

    Group Names

    Group names must match the pattern defined by _GROUP_NAME_PATTERN in the project's partition configuration.

  6. Use shell tasks for script-like execution

    main

    Shell tasks are executed inside a new shell and are interpreted as shell scripts rather than single commands. This allows you to use full shell syntax, including pipes, command substitution, and background processes.

    By default, Poe attempts to find a POSIX shell in the following order: sh, bash, or zsh. On Windows, it looks for Git Bash or attempts to find it via the PATH.

    Example of using background processes in shell tasks:

    [tool.poe.tasks.pfwd]
    shell = """
      ssh -N -L 0.0.0.0:8080:$STAGING:8080 $STAGING &
      ssh -N -L 0.0.0.0:5432:$STAGINGDB:5432 $STAGINGDB &
    """
    
    [tool.poe.tasks.pfwdstop]
    shell = """kill $(pgrep -f "ssh -N -L .*:(8080|5432)")"""
    [tool.poe.tasks.pfwd]
    shell = """
      ssh -N -L 0.0.0.0:8080:$STAGING:8080 $STAGING &
      ssh -N -L 0.0.0.0:5432:$STAGINGDB:5432 $STAGINGDB &
    """
  7. How boolean arguments are projected to the environment

    main

    Boolean arguments follow specific rules when being projected from typed Python values to string-based environment variables:

    • True: Projects to the environment value "True".
    • False: Remains available as the Python boolean False for expr and script tasks, but its environment projection is unset (removed from the environment).

    This ensures consistent falsey behavior in shells while maintaining typed access for Python-based tasks.

  8. How private variables are handled

    main

    Variables introduced by Poe-managed sources (like env, envfile, uses, or args) that start with an underscore _ and contain no uppercase characters are treated as private.

    Behavior of Private Variables:

    • They are available for configuration-time interpolation.
    • They are inherited by child tasks.
    • They can be remapped to public variables using task-level env.
    • Crucially: They are filtered out and not passed to the subprocess environment.

    Note: Host environment variables are not automatically treated as private.

  9. Compose tasks using Sequence, Parallel, or Graph

    main

    You can compose multiple tasks using three primary models:

    1. Sequence: Runs tasks in a specific order. You can optionally configure it to ignore failures.
    2. Parallel: Runs tasks concurrently. Output is typically color-coded to distinguish between tasks.
    3. Graph: Executes tasks as a Directed Acyclic Graph (DAG) based on the deps key. Graph tasks can capture stdout from dependencies and make it available to downstream tasks.
  10. Define and run parallel tasks

    main

    A Parallel task is defined by an array of other tasks to be run concurrently. By default, strings in the array are interpreted as references to other tasks. Subtask outputs are forwarded to the console with a prefix identifying the task.

    [tool.poe.tasks]
    check.parallel = ["mypy", "pylint"]