DOCSEARCHbyAlgolia
Submit Docs

Hydra Documentation

repository·main·Indexed 27 days ago

https://github.com/facebookresearch/hydra

Hydra is a framework for configuring complex applications, widely used in machine learning and deep learning workflows to manage hierarchical configurations. It includes tools like configen for generating Structured Configs and a variety of plugins for hyperparameter optimization (Ax, Optuna, Nevergrad) and job launching (Ray, Submitit, Joblib, RQ). The framework supports stable version 1.3 and development version 1.4.

Tokens
53.6K
Snippets
182
Records
315
Agent score
94%

What's inside Hydra

  1. Introduction to Structured Configs in Hydra

    main

    Structured Configs allow you to use Python dataclasses to define your configuration structure and types. This provides runtime type checking during configuration composition or mutation, and enables static type checking when using tools like mypy or PyCharm.

    Supported Types

    • Primitive types: int, bool, float, str, Enums, bytes, pathlib.Path.
    • Nesting: Structured Configs can be nested within each other.
    • Containers: List and Dict containing primitives, other Structured Configs, or nested containers.
    • Optional fields.

    Limitations

    • Union types are only partially supported (refer to OmegaConf documentation for details).
    • User-defined methods on dataclasses are not supported.

    Primary Usage Patterns

    1. As a config: Using dataclasses in place of traditional YAML configuration files.
    2. As a config schema: Using dataclasses to validate the structure and types of existing configuration files (recommended for complex use cases).

    Both patterns maintain full compatibility with Hydra features like config composition and command-line overrides.

  2. Understand Hydra Configuration Terminology

    main

    Hydra uses several core concepts to build and manage configurations:

    • Input Configs: The building blocks (YAML files or Python objects) used to construct the final configuration.
    • Primary Config: The specific input config designated via @hydra.main() or the Compose API.
    • Output Config: The final, composed configuration resulting from combining Input Configs and Overrides.
    • Overrides: Strings used to update, add, or delete config values or defaults list options. These can be passed via the command line or the Compose API.
    • Defaults List: A special list within an Input Config that instructs Hydra on how to compose the configuration (e.g., selecting which config group options to use).
    • Config Group: A directory in the Config Search Path containing Input Configs. Groups can be nested using forward slashes (e.g., group/subgroup).
    • Config Group Option: An individual Input Config within a Config Group.
    • Package: The path to a node in the configuration tree. By default, a Config Group Option's package is derived from its group (e.g., configs in mi6/agent have the package mi6.agent).
    • Config Search Path: An ordered list of paths searched to find configuration files, similar to PYTHONPATH.

  4. Understand Hydra's current synchronous execution model

    main

    Hydra's current launcher and sweeper model is synchronous and batch-oriented. The lifecycle follows this contract:

    1. Receive a batch of job overrides.
    2. Launch the jobs.
    3. Wait for the jobs to finish.
    4. Return completed job results.

    This model is suitable for local and simple parallel execution but is limited for distributed or queue-based systems (like Slurm, Ray, or Airflow) because it conflates job submission with result collection and lacks a way to represent intermediate states (e.g., 'submitted' or 'running' without being 'complete').

  5. Use configen to automatically generate Structured Configs

    main

    Configen is a tool that automatically generates Python dataclasses (Structured Configs) from existing classes. These generated configs can be used with hydra.utils.instantiate() to provide type safety and validation for Hydra configurations.

    Instead of manually writing dataclasses for every class in your project, you can provide a list of modules and classes to configen, and it will produce the corresponding _target_ based dataclasses.

  6. How the Hydra tab completion accelerator works

    main

    The hydra-completion-accelerator provides a persistent service to avoid the overhead of restarting the Python interpreter and re-running imports/registrations for every tab completion request.

    Workflow

    1. Installation: The user installs the accelerator via Reploy.
    2. Registration: When a user activates Hydra tab completion for an application, Hydra automatically detects the accelerator and registers the application with it.
    3. Execution: The accelerator starts the application normally. When the application reaches the wrapped hydra.main function, Hydra enters completion-service mode.
    4. Service Mode: In this mode, Hydra does not execute the user's task function. Instead, it initializes composition, signals readiness, and serves completion requests to the accelerator.
    5. Request Handling: The accelerator routes shell completion requests to the initialized application worker. The worker retains expensive state (imports, plugins, resolvers, search paths) and applies command-line overrides in an isolated request state to ensure one query does not affect another.

  8. Understand Hydra Sweepers and Launchers

    main

    Hydra separates the logic of what to run from how to run it:

    • Sweeper: Determines the combinations of parameters to be tested. The default is BasicSweeper, but plugins like Ax Sweeper can perform automated optimization.
    • Launcher: Determines how the jobs are executed. By default, Hydra runs jobs locally and serially. Plugins like JobLib Launcher allow for parallel execution on local machines, while other plugins can launch jobs on remote clusters.

  9. Understand how Hydra determines config packages

    main

    A package determines where the content of an input config is placed in the final output configuration.

    By default, the package is derived from the config's directory structure (its Config Group). For example, a file at server/db/mysql.yaml has a default package of server.db.

    Priority for determining the final package:

    1. The package specified in the Defaults List (relative to the package of the including config).
    2. The package specified via the @package directive (absolute).
    3. The default package (derived from the file path).