Llama Models

The Llama 3.1 collection consists of multilingual, auto-regressive large language models (LLMs) available in three sizes: 8B, 70B, and 405B.

Model Types

• Pretrained models: Designed for various natural language generation tasks and can be adapted for specific uses.
• Instruction tuned models: Optimized for multilingual dialogue and assistant-like chat using supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF).

Key Specifications

• Context Length: 128k tokens for all sizes.
• Architecture: Optimized transformer architecture using Grouped-Query Attention (GQA) for improved inference scalability.
• Input Modality: Multilingual Text.
• Output Modality: Multilingual Text and code.
• Knowledge Cutoff: December 2023.
• Supported Languages: English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai.

Llama 4 is a collection of natively multimodal AI models using a mixture-of-experts (MoE) architecture. They support text and image inputs, with outputs in multilingual text and code.

Key models in the series:

• Llama 4 Scout (17Bx16E): 17B activated parameters (109B total), 10M context length.
• Llama 4 Maverick (17Bx128E): 17B activated parameters (400B total), 1M context length.

Supported Languages: Arabic, English, French, German, Hindi, Indonesian, Italian, Portuguese, Spanish, Tagalog, Thai, and Vietnamese.

Llama 3.2-Vision is a collection of multimodal large language models (LLMs) designed for image reasoning. The collection includes both pretrained and instruction-tuned models in 11B and 90B parameter sizes. These models accept text and image inputs and generate text outputs.

Key Specifications

• Input Modalities: Text + Image
• Output Modalities: Text
• Context Length: 128k tokens
• Architecture: Built on Llama 3.1 with a separately trained vision adapter using cross-attention layers.
• Knowledge Cutoff: December 2023
• Supported Languages:
  • Text-only tasks: English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai.
  • Image + Text tasks: English is the only officially supported language.

Llama 3.3 is a 70B parameter multilingual large language model (LLM) featuring an optimized transformer architecture with Grouped-Query Attention (GQA). The instruction-tuned version is optimized for multilingual dialogue and assistant-like chat.

Key Specifications:

• Parameters: 70B
• Context Length: 128k tokens
• Input Modalities: Multilingual Text
• Output Modalities: Multilingual Text and code
• Knowledge Cutoff: December 2023
• Supported Languages: English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai.
• Architecture Features: Uses Grouped-Query Attention (GQA) for improved inference scalability.

Llama 3 is a family of auto-regressive large language models (LLMs) developed by Meta. The models use an optimized transformer architecture with Grouped-Query Attention (GQA) for improved inference efficiency and a 128K token vocabulary.

Model Variants

• 8B Model: 8k context length, knowledge cutoff March 2023.
• 70B Model: 8k context length, knowledge cutoff December 2023.

Capabilities

• Input: Text only.
• Output: Text and code only.
• Variants: Available in both pretrained (for natural language generation tasks) and instruction tuned (optimized for dialogue and assistant-like chat) versions.

Llama 2 is a family of auto-regressive large language models using an optimized transformer architecture. The family includes pretrained models for general natural language generation and fine-tuned Llama-2-Chat models optimized for dialogue.

Key Specifications:

• Parameter Sizes: 7B, 13B, and 70B.
• Input/Output: Text only.
• Context Length: 4k tokens.
• 70B Model Feature: Uses Grouped-Query Attention (GQA) for improved inference scalability.
• Training Data: Pretrained on 2 trillion tokens (cutoff September 2022). Fine-tuning data includes instruction datasets and over one million human-annotated examples (up to July 2023).

When providing multiple images to a Llama 4 model, each image must be encapsulated within its own set of image tags. The structure follows a pattern of <|image_start|> followed by a series of <|patch|> tokens (representing the image data), separated by <|tile_x_separator|> and <|tile_y_separator|> tokens, and ending with <|image_end|>.

To include multiple images, repeat this entire block for each image within the user header section.

Llama 4 is a new technology with unpredictable outputs. Before deploying any applications using Llama 4 models, developers must perform safety testing and tuning tailored to their specific use cases. Because testing cannot cover all possible scenarios, the model may produce inaccurate or objectionable responses.

For guidance on implementing protections, refer to:

• Developer Use Guide: AI Protections
• Llama Protections solutions
• Official Llama resources at https://llama.meta.com/docs/get-started/

Because model outputs cannot be predicted in advance and may be inaccurate, biased, or objectionable, developers must perform safety testing tailored to their specific application.

Recommended evaluation steps:

• Build dedicated evaluation datasets: Create datasets specific to your use case rather than relying solely on general benchmarks.
• Use Scaled Evaluations: Combine Llama models with Purple Llama safeguards to filter input prompts and output responses.
• Perform Red Teaming: Conduct adversarial prompting exercises to discover risks in your specific implementation.

Llama 3.3 models should not be deployed in isolation. For agentic systems or applications requiring high reliability, developers should implement an overall AI system with additional safety guardrails. Meta provides several specialized safeguards that should be deployed alongside Llama models:

• Llama Guard 3: For input prompt and output response filtering.
• Prompt Guard: For protecting against prompt injection and other input risks.
• Code Shield: For securing code-related outputs.

Reference implementations containing these safeguards by default can be found in the meta-llama/llama-agentic-system repository.