DOCSEARCHbyAlgolia
Submit Docs

GaLore (Gradient Low-Rank Projection)

repository·master·Indexed 23 days ago

https://github.com/jiaweizzhao/galore

A memory-efficient low-rank training strategy that enables full-parameter learning with lower memory requirements than standard methods. The library provides the galore-torch optimizer for integration into existing training workflows.

Tokens
120
Snippets
1
Records
1
Agent score
33%

What's inside GaLore

  1. Install GaLore optimizer

    master

    You can install the GaLore optimizer via pip or from source.

    Via pip

    pip install galore-torch

    From source

    git clone git@github.com:jiaweizzhao/GaLore.git
cd GaLore
pip install -e .

    Install experiment dependencies

    If you are running benchmarks or experiments, install the required dependencies:

    pip install -r exp_requirements.txt

    Note: Experiments are tested on Python 3.8 with PyTorch 2.1.

    pip install galore-torch