DOCSEARCHbyAlgolia
Submit Docs

Evolutionary Model Merge

repository·main·Indexed 23 days ago

https://github.com/sakanaai/evolutionary-model-merge

Official repository and package for Evolutionary Optimization of Model Merging Recipes by SakanaAI. Version 0.1.0 provides tools to reproduce evaluation results, including the evomerge API for model management, evaluation logic, and utilities for dynamic object instantiation and configuration loading.

Tokens
706
Snippets
1
Records
7
Agent score
79%

What's inside evomerge

  1. Reproduce the Evolutionary Model Merge evaluation

    main

    To reproduce the evaluation results presented in the SakanaAI paper, follow these steps to set up the environment and run the evaluation script.

    1. Clone the repository

    git clone https://github.com/SakanaAI/evolutionary-model-merge.git
cd evolutionary-model-merge

    2. Set up the fastext Model

    The evaluation uses fastext for language detection.

    1. Download lid.176.ftz from the fasttext language identification page.
    2. Place the file in the current directory, or set the LID176FTZ_PATH environment variable to the specific path of the file.

    3. Install dependencies

    Install the package in editable mode:

    pip install -e .

    Note: The authors used Python 3.10.12 and CUDA 12.3 for their tests.

    4. Run the evaluation

    Execute the evaluate.py script by providing a path to a configuration file. The configurations used in the paper are located in the configs directory.

    python evaluate.py --config_path {path-to-config}
    python evaluate.py --config_path {path-to-config}

  3. Access the evomerge public API

    main

    The evomerge package exposes its primary functionality through the top-level namespace. It re-exports all symbols from the following submodules:

    • .eval: Evaluation logic and metrics.
    • .models: Model definitions and management.
    • .utils: Utility functions for the evolutionary merging process.

    To use the package, import the desired components directly from evomerge.

  4. Instantiate objects from configuration dictionaries

    main

    The instantiate_from_config(config) function allows you to create Python objects (classes or functions) based on a configuration dictionary.

    To work, the dictionary must contain a target key specifying the full import path (e.g., module.submodule.ClassName). Optional parameters for the object's constructor should be provided in a params dictionary.

    Special sentinel values __is_first_stage__ and __is_unconditional__ return None.

  6. Get a class or function from a string path

    main

    Use get_obj_from_str(string, reload=False, invalidate_cache=True) to dynamically import a class or function using its full dot-notation string (e.g., package.module.ClassName).

    • string: The full path to the object.
    • reload: If True, reloads the module before retrieving the object.
    • invalidate_cache: If True, calls importlib.invalidate_caches().