Gin Config

repository·master·Indexed 24 days ago

https://github.com/google/gin-config

A lightweight dependency injection framework for Python that allows developers to manage complex, nested configurations for functions and classes via .gin files. It features support for configurable references, scoping, and specialized modules for TensorFlow and PyTorch, making it ideal for machine learning experiments. It also includes a Gin-Fiddle bridge for converting configurations into Fiddle objects.

Tokens
10.6K
Snippets
28
Records
43
Agent score
73%

What's inside gin-config

  1. Handle reevaluation of Configs inside Partials

    master

    In Gin, parameters are injected at call time, meaning evaluated references are re-evaluated every time the function is called. In Fiddle, an fdl.Config is evaluated exactly once during fdl.build. If that config is a parameter of an fdl.Partial, subsequent calls to the partial will reuse the same value.

    To preserve Gin's re-evaluation semantics, as_config and as_partial automatically detect if an fdl.Config is inside an fdl.Partial. If so, they use ReevaluatedConfig and PartialWithReevaluations to ensure parameters are re-evaluated on every call.

    If you know your configuration does not rely on this behavior (e.g., partials are only called once), you can disable it by setting reevaluate_configs_inside_partials=False to improve performance or simplify the object graph.

  2. Handle naming collisions with modules

    master

    If multiple configurable functions or classes share the same name, you can disambiguate them by prepending the module path in your .gin file. You can use as much of the module path as necessary to make the name unique.

    Example: If some_configurable exists in both a.b.c.configurables and x.y.z.configurables, use:

    a.b.c.configurables.some_configurable.param = 'value'

    You can also customize the module name used for disambiguation during registration using the module keyword argument in @gin.configurable or @gin.external_configurable. Alternatively, you can specify both a custom name and a custom module by combining them into a single string.

    When registering external functions (like those from TensorFlow), it is recommended to specify a module name that matches typical Python usage to make configuration more intuitive.

    # Customizing the module name for disambiguation
    @gin.configurable(module='custom.module.spec')
    def my_network(images, num_outputs, num_layers=3, weight_decay=1e-4):
      ...
    
    # Customizing both name and module
    @gin.configurable('custom.module.spec.supernet')
    def my_network(images, num_outputs, num_layers=3, weight_decay=1e-4):
      ...
    
    # Registering external functions with a preferred module name
    gin.external_configurable(tf.nn.relu, module='tf.nn')
    gin.external_configurable(tf.nn.relu, 'tf.nn.relu')
  3. Use configurable references with the '@' symbol

    master

    Gin allows you to pass other configurable objects as parameters using the @ prefix. This enables complex dependency injection.

    Two types of references:

    1. Instance Reference (@Name()): Adding parentheses calls the configurable object. The result of the call (e.g., a new instance) is passed as the value.
      • Warning: Evaluated references are called every time they are required. To share an instance, use singletons.
    2. Class/Object Reference (@Name): Omitting parentheses passes the uncalled object (e.g., the class itself) to the parameter.

    Example:

    # In a Gin config file:
    train_model.network_fn = @DNN()  # Passes a new instance of DNN
    train_model.optimizer = @MomentumOptimizer  # Passes the MomentumOptimizer class
    @gin.configurable
    class DNN(object):
      def __init__(self, num_units=(1024, 1024)):
        ...
    
    @gin.configurable(denylist=['data'])
    def train_model(network_fn, data, learning_rate, optimizer):
      ...
    
    # Gin configuration:
    train_model.network_fn = @DNN()
    train_model.optimizer = @MomentumOptimizer
    train_model.learning_rate = 0.001
    
    DNN.num_units = (2048, 2048, 2048)
    MomentumOptimizer.momentum = 0.9
  4. Share object instances using singletons

    master

    To ensure multiple bindings receive the exact same instance of an object, use the gin.singleton configurable function.

    1. Define a scope for the singleton.
    2. Bind the singleton's constructor to a callable.
    3. Reference the singleton using @scope/gin.singleton().

    Subsequent calls to the same singleton scope will return the cached instance.

    # In config.gin:
    shared_object_name/gin.singleton.constructor = @callable
    
    some_function.shared_object = @shared_object_name/gin.singleton()
    another_function.shared_object = @shared_object_name/gin.singleton()

    Using macros with singletons:

    # In config.gin:
    SHARED_OBJECT = @shared_object_name/gin.singleton()
    shared_object_name/gin.singleton.constructor = @callable
    
    some_function.shared_object = %SHARED_OBJECT
    another_function.shared_object = %SHARED_OBJECT
  5. Use macros to share values in Gin

    master

    Macros allow you to define a value once and reuse it across multiple bindings, reducing maintenance.

    In a Gin config file, a binding without an argument name (e.g., name = value) is interpreted as a macro. You can reference this macro using the % syntax.

    Important Note on Instances: When using a macro to refer to an evaluated configurable reference (like @some_scope/some_fun()), each reference to the macro implies a separate call to the underlying function. If you need to share the exact same instance of an object, use gin.singleton instead.

    # Standard macro usage via @gin.configurable
    @gin.configurable
    def macro(value):
      return value
    # In config.gin:
    num_layers = 10
    network.num_layers = %num_layers
  6. Use configurable references with @ syntax

    master

    Gin allows you to pass other configurable functions, classes, or instances as parameters using configurable references.

    • Reference a function/class: Use the @ prefix followed by the name (e.g., @tf.nn.tanh or @MyClass).
    • Evaluate a reference (pass an instance): Use the @name() syntax. This instructs Gin to call the function or constructor with its own Gin-configured parameters just before passing the result to the target parameter. Note that evaluated references are not cached; a new instance is created for every call to the parent function.
    # Inside "config.gin"
    
    # Passing a function/class reference
    dnn.activation_fn = @tf.nn.tanh
    train_fn.optimizer_cls = @tf.train.GradientDescentOptimizer
    
    # Passing an evaluated instance (calling the constructor)
    build_model.network_fn = @DNN()
  7. Use scoping to provide different bindings for multiple invocations

    master

    When a configurable function is called multiple times and requires different parameter values for each call, use Gin's scoping mechanism.

    In a .gin file, you can prefix a configurable reference or a parameter binding with a scope name followed by a /. This allows you to create separate 'buckets' of configuration for different instances of the same function.

    Example: To give two different optimizers different learning rates:

    # In the .gin file
    gan_trainer.generator_optimizer = @generator/GradientDescentOptimizer
    gan_trainer.discriminator_optimizer = @discriminator/GradientDescentOptimizer
    
    generator/GradientDescentOptimizer.learning_rate = 0.01
    discriminator/GradientDescentOptimizer.learning_rate = 0.001
    # In the .gin file
    gan_trainer.generator_optimizer = @generator/GradientDescentOptimizer
    gan_trainer.discriminator_optimizer = @discriminator/GradientDescentOptimizer
    
    generator/GradientDescentOptimizer.learning_rate = 0.01
    discriminator/GradientDescentOptimizer.learning_rate = 0.001
  8. Configure the same function differently using scopes

    master

    If you need to use the same function or class with different parameters in the same project (e.g., a generator and a discriminator in a GAN), use scopes.

    A scope provides a unique namespace for a set of bindings. In .gin files, the scope name precedes the function name, separated by a forward slash: scope_name/function_name.

    Inheritance: Parameters set on the unscoped (root) function name are inherited by all scoped variants unless explicitly overridden. This inheritance works hierarchically (e.g., a/b/func inherits from a/func).

    # Inside "config.gin"
    
    # Bind scoped references
    build_model.generator_network_fn = @generator/dnn
    build_model.discriminator_network_fn = @discriminator/dnn
    
    # Configure scoped parameters
    generator/dnn.layer_sizes = (128, 256)
    generator/dnn.num_outputs = 784
    
    discriminator/dnn.layer_sizes = (512, 256)
    discriminator/dnn.num_outputs = 1
    
    # This unscoped parameter is inherited by both scopes
    dnn.activation_fn = @tf.nn.tanh
  9. Use configurable references with @name

    master

    Gin allows you to pass other configurable functions or classes as parameters using the @name syntax (known as configurable references).

    • Passing a reference: Use @function_name or @module.function_name to pass the object itself.
    • Evaluating a reference: Use @name() to pass the result of calling the function or class constructor. Note that evaluated references are not cached; a new instance is created for each call to the parent function.
    # Inside "config.gin"
    
    # Passing a reference (the function/class itself)
    dnn.activation_fn = @tf.nn.tanh
    train_fn.optimizer_cls = @tf.train.GradientDescentOptimizer
    
    # Passing an evaluated reference (the result of a call)
    build_model.network_fn = @DNN()
  10. Configure TensorFlow objects and save configs with gin.tf

    master

    The gin.tf package provides specialized tools for TensorFlow users.

    Making TF objects configurable

    Import gin.tf.external_configurables to make standard TensorFlow optimizers, learning rate decays, and losses available as configurable references in your .gin files.

    import gin.tf.external_configurables

    Saving operative config to files and TensorBoard

    Use gin.tf.GinConfigSaverHook to automatically save the operative configuration to a file and log it to the "Text" tab in TensorBoard. This hook should be added to a tf.train.MonitoredSession.

    Note: In distributed training, ensure the hook runs only on the chief worker.

    import gin.tf.external_configurables
    import gin.tf
    import tensorflow as tf
    
    # ... setup code ...
    
    # Construct the hook
    config_saver = gin.tf.GinConfigSaverHook(output_dir, summarize_config=True)
    
    # Add as a chief-only hook to MonitoredTrainingSession
    with tf.train.MonitoredTrainingSession(
        ..., 
        chief_only_hooks=[config_saver], 
        ...) as sess:
        # Training loop
        ...
  11. Run experiments with multiple Gin files and CLI bindings

    master

    For complex experiments, you can combine multiple .gin files and pass individual parameter overrides via command-line flags.

    A common pattern using absl flags is to define multi_string flags for files and parameters, then parse them using gin.parse_config_files_and_bindings.

  12. Import modules and include other Gin files

    master

    Importing Python modules in Gin

    To make dependencies explicit, you can import Python modules directly within a .gin file using standard Python syntax. This ensures the modules are imported and their configurables are registered when the Gin file is parsed.

    import some.module.spec

    Including other Gin files

    You can split large configurations into smaller components using the include statement. The included file is parsed as if its content were literally pasted at the location of the include statement.

    include 'path/to/another/file.gin'
    import some.module.spec
    
    include 'path/to/another/file.gin'