DOCSEARCHbyAlgolia
Submit Docs

GigaWorld-0 Documentation

repository·main·Indexed 23 days ago

https://github.com/open-gigaai/giga-world-0

A unified world model framework serving as a data engine for Vision-Language-Action (VLA) learning in embodied AI. It includes GigaWorld-0-Video for large-scale video generation and GigaWorld-0-3D for geometrically consistent and physically realistic modeling. The framework supports foundation model downloads, data packing, LoRA training, and multi-GPU inference.

Tokens
1.1K
Snippets
4
Records
5
Agent score
32%

What's inside GigaWorld-0

  1. Prepare video data for training

    main

    To prepare data, organize your video files and text prompts in a directory where each .mp4 file has a corresponding .txt file with the same base name:

    raw_data/
├── 0.mp4                # Video file 0
├── 0.txt                # Prompt for video file 0
├── 1.mp4                # Video file 1
├── 1.txt                # Prompt for video file 1
...

    Then, use scripts/pack_data.py to pack the data and extract prompt embeddings.

    python scripts/pack_data.py \
  --video-dir /path/to/raw_data/ \
  --save-dir /path/to/packed_data/

  2. Train GigaWorld-0-Video

    main

    Run training using scripts/train.py with a configuration file. Ensure data is already packed and prompt embeddings are extracted.

    To perform LoRA Training, update your configuration file with the following settings:

    • config.train_mode.train_mode='lora'
    • config.train_mode.lora_rank=64
    python scripts/train.py --config configs.giga_world_0_video.config

  3. Download GigaWorld-0 models

    main

    Use the scripts/download.py script to download the foundation models, including the text encoder and VAE.

    Available model names:

    • video_pretrain: GigaWorld-0-Video-Pretrain-2b (61x480x640 foundation model).
    • video_gr1: GigaWorld-0-Video-GR1-2b (93x480x768 fine-tuned with GR1 dataset).
    python scripts/download.py --model-name video_pretrain --save-dir /path/to/giga_world_0_video_pretrain/
python scripts/download.py --model-name video_gr1 --save-dir /path/to/giga_world_0_video_gr1/

  4. Run inference with GigaWorld-0-Video

    main

    Use scripts/inference.py to generate videos from text prompts. You can run inference on a single GPU, multiple GPUs, or using LoRA weights.

    Single GPU Inference

    python scripts/inference.py \
  --data-path /path/to/packed_test_data/ \
  --save-dir /path/to/vis_results/ \
  --transformer-model-path /path/to/your_transformer/ \
  --text-encoder-model-path /path/to/giga_world_0_video/text_encoder/ \
  --vae-model-path /path/to/giga_world_0/vae/ \
  --gpu_ids 0

    Multi-GPU Inference

    Specify multiple IDs in --gpu_ids for faster inference or larger batches.

    python scripts/inference.py \
  --data-path /path/to/packed_test_data/ \
  --save-dir /path/to/vis_results/ \
  --transformer-model-path /path/to/your_transformer/ \
  --text-encoder-model-path /path/to/giga_world_0_video/text_encoder/ \
  --vae-model-path /path/to/giga_world_0_video/vae/ \
  --gpu_ids 0 1 2 3 4 5 6 7

    LoRA Inference

    Include the --lora-model-path argument to load fine-tuned LoRA weights.

    python scripts/inference.py \
  --data-path /path/to/packed_test_data/ \
  --save-dir /path/to/vis_results/ \
  --transformer-model-path /path/to/giga_world_0_video/transformer/ \
  --text-encoder-model-path /path/to/giga_world_0_video/text_encoder/ \
  --vae-model-path /path/to/giga_world_0_video/vae/ \
  --lora-model-path /path/to/your_lora/ \
  --gpu_ids 0

  5. Install GigaWorld-0

    main

    GigaWorld-0 depends on giga-train, giga-datasets, and giga-models. It is recommended to use a fresh conda environment for installation.

    Follow these steps to set up the environment:

    1. Create and activate a conda environment with Python 3.11.10.
    2. Install giga-train, giga-datasets, and natten via pip.
    3. Clone and install giga-models in editable mode.
    4. Clone the giga-world-0 repository.
    conda create -n giga_world_0 python=3.11.10 -y
conda activate giga_world_0

pip3 install giga-train
pip3 install giga-datasets
pip3 install natten

git clone https://github.com/open-gigaai/giga-models.git
cd giga-models
pip3 install -e .

git clone git@github.com:open-gigaai/giga-world-0.git