DOCSEARCHbyAlgolia
Submit Docs

OpenLLaMA Documentation

repository·main·Indexed 27 days ago

https://github.com/openlm-research/open_llama

An open-source reproduction of Meta's LLaMA models released under the Apache 2.0 license. OpenLLaMA provides 3B, 7B, and 13B models trained on 1T tokens, available in PyTorch (Hugging Face Transformers) and JAX (EasyLM) formats. It serves as a drop-in replacement for LLaMA in existing LLM workflows, with v1 and v2 versions available.

Tokens
1.1K
Snippets
3
Records
7
Agent score
43%

What's inside OpenLLaMA

  1. Overview of OpenLLaMA

    main
    OpenLLaMA is an open-source reproduction of Meta AI's LLaMA, released under the Apache 2.0 license. It provides a series of 3B, 7B, and 13B models trained on 1T tokens. The weights are available in PyTorch format (for Hugging Face Transformers) and JAX format (for EasyLM). OpenLLaMA models are designed as drop-in replacements for the original LLaMA in existing implementations.

  3. Evaluate OpenLLaMA with LM-Eval-Harness

    main

    To evaluate OpenLLaMA using lm-eval-harness, you must ensure the fast tokenizer is disabled to obtain correct results. When configuring the model in lm-eval-harness, pass use_fast=False to the tokenizer initialization.

    tokenizer = self.AUTO_TOKENIZER_CLASS.from_pretrained(
    pretrained if tokenizer is None else tokenizer,
    revision=revision + ("/" + subfolder if subfolder is not None else ""),
    use_fast=False
)

  4. Load OpenLLaMA weights with EasyLM

    main
    OpenLLaMA weights are available in JAX format specifically for use with the EasyLM framework. Unlike the original LLaMA, OpenLLaMA's tokenizer and weights are trained from scratch, so you do not need to obtain the original LLaMA tokenizer or weights. Refer to the EasyLM LLaMA documentation for specific implementation details.

  5. Load OpenLLaMA weights with Hugging Face Transformers

    main

    You can load OpenLLaMA weights directly from the Hugging Face Hub using the transformers library.

    CRITICAL: Avoid using the Hugging Face 'fast' tokenizer, as auto-converted fast tokenizers may produce incorrect tokenizations. Instead, use the LlamaTokenizer class directly or pass use_fast=False to the AutoTokenizer class.

    import torch
from transformers import LlamaTokenizer, LlamaForCausalLM

## v2 models
model_path = 'openlm-research/open_llama_3b_v2'
# model_path = 'openlm-research/open_llama_7b_v2'

## v1 models
# model_path = 'openlm-research/open_llama_3b'
# model_path = 'openlm-research/open_llama_7b'
# model_path = 'openlm-research/open_llama_13b'

tokenizer = LlamaTokenizer.from_pretrained(model_path)
model = LlamaForCausalLM.from_pretrained(
    model_path, torch_dtype=torch.float16, device_map='auto',
)

prompt = 'Q: What is the largest animal?\nA:'
input_ids = tokenizer(prompt, return_tensors="pt").input_ids

generation_output = model.generate(
    input_ids=input_ids, max_new_tokens=32
)
print(tokenizer.decode(generation_output[0]))

  6. OpenLLaMA Model Versions and Paths

    main

    OpenLLaMA is released in v1 and v2 versions. v2 models use a different data mixture (Falcon refined-web, StarCoder, and RedPajama subsets) and are generally better than v1.

    Note on Code Generation: For code-related tasks (e.g., HumanEval), use v2 models. The v1 tokenizer merges multiple empty spaces, which makes it unsuitable for code generation tasks.

  7. Cite OpenLLaMA in research

    main

    If you use OpenLLaMA in your research or applications, please cite the project using the following BibTeX entry:

    @software{openlm2023openllama,
  author = {Geng, Xinyang and Liu, Hao},
  title = {OpenLLaMA: An Open Reproduction of LLaMA},
  month = May,
  year = 2023,
  url = {https://github.com/openlm-research/open_llama}
}