repository·master·Indexed 24 days ago
https://github.com/moonshotai/kimi-k2.5Kimi K2.5 is an open-source, native multimodal agentic model featuring a Mixture-of-Experts (MoE) architecture with 1T total parameters and 32B activated parameters. Designed for complex reasoning, visual-to-code generation, and autonomous agentic tasks, it supports both 'thinking' and 'instant' modes. The model features a 256K context length and integrates vision and language understanding via the MoonViT encoder. Documentation covers deployment using vLLM, SGLang, and KTransformers, as well as LoRA SFT fine-tuning via LLaMA-Factory.
Kimi K2.5 provides the following core capabilities:
Kimi-K2.5 supports two primary modes: Thinking mode (for deep reasoning) and Instant mode (for faster responses).
| Parameter | Thinking Mode | Instant Mode |
|---|---|---|
temperature | 1.0 | 0.6 |
top_p | 0.95 | 0.95 |
The method to enable Instant mode depends on whether you are using the official Moonshot API or a third-party deployment (like vLLM/SGLang):
extra_body={'thinking': {'type': 'disabled'}}.extra_body={'chat_template_kwargs': {'thinking': False}}.For specific benchmarks like HLE with tools and all Agentic Search tasks, Kimi K2.5 is equipped with the following tools:
BrowseComp (which uses a discard-all strategy), no context management is applied. Tasks exceeding the supported context length are counted as failed.For CPU+GPU heterogeneous inference, use KTransformers combined with SGLang. This setup allows leveraging both CPU and GPU resources for high-performance inference.
Example Command:
python -m sglang.launch_server \
--model path/to/Kimi-K2.5/ \
--kt-amx-weight-path path/to/Kimi-K2.5/ \
--kt-cpuinfer 64 \
--kt-threadpool-count 2 \
--kt-num-gpu-experts 180 \
--kt-amx-method AMXINT4 \
--trust-remote-code \
--mem-fraction-static 0.98 \
--chunked-prefill-size 16384 \
--max-running-requests 48 \
--max-total-tokens 50000 \
--tensor-parallel-size 8 \
--enable-p2p-check \
--disable-shared-experts-fusionYou can perform LoRA SFT (Supervised Fine-Tuning) on Kimi-K2.5 using the llamafactory-cli with KTransformers enabled via the USE_KT=1 environment variable.
LoRA SFT Training:
USE_KT=1 llamafactory-cli train examples/train_lora/kimik2_lora_sft_kt.yamlPost-Training Tasks: After performing LoRA SFT, you can use the following commands to interact with the model:
llamafactory-cli chat examples/inference/kimik2_lora_sft_kt.yamlllamafactory-cli api examples/inference/kimik2_lora_sft_kt.yaml# For LoRA SFT
USE_KT=1 llamafactory-cli train examples/train_lora/kimik2_lora_sft_kt.yaml
# For Chat with model after LoRA SFT
llamafactory-cli chat examples/inference/kimik2_lora_sft_kt.yaml
# For API with model after LoRA SFT
llamafactory-cli api examples/inference/kimik2_lora_sft_kt.yamlKimi-K2.5 can be deployed using several inference engines. Recommended engines include:
Requirements:
transformers library must be version 4.57.1 or higher.For detailed deployment instructions, refer to the Model Deployment Guide.
To deploy Kimi-K2.5 with vLLM, you must use the nightly vLLM wheel to access the kimi_k2 tool-call and reasoning parsers.
Installation:
Install the nightly vLLM wheel using uv:
uv pip install -U vllm \
--torch-backend=auto \
--extra-index-url https://wheels.vllm.ai/nightlyServing the model:
Use the vllm serve command. For a single node with Tensor Parallelism 8 (TP8) on H200 hardware, use the following configuration. Note that the kimi_k2 parser flags are mandatory for correct functionality.
Required Flags:
--tool-call-parser kimi_k2: Enables tool calling functionality.--reasoning-parser kimi_k2: Enables correct processing of the model's default thinking/reasoning mode.vllm serve $MODEL_PATH -tp 8 --mm-encoder-tp-mode data --trust-remote-code --tool-call-parser kimi_k2 --reasoning-parser kimi_k2To deploy Kimi-K2.5 with SGLang, install the latest main branch from GitHub and the specific nvidia-cudnn-cu12 version.
Installation:
pip install "sglang @ git+https://github.com/sgl-project/sglang.git#subdirectory=python"
pip install nvidia-cudnn-cu12==9.16.0.29Serving the model:
Use the sglang serve command. For TP8 on H200 in a single node, ensure you include the kimi_k2 parser flags to support tool usage and reasoning content.
Required Flags:
--tool-call-parser kimi_k2: Required for tool usage.--reasoning-parser kimi_k2: Required for correctly processing reasoning content.sglang serve --model-path $MODEL_PATH --tp 8 --trust-remote-code --tool-call-parser kimi_k2 --reasoning-parser kimi_k2To use Thinking mode, perform a standard chat completion call. In Thinking mode, the model provides reasoning content which can be accessed via the reasoning_content field in the message object.
import openai
def simple_chat(client: openai.OpenAI, model_name: str):
messages = [
{'role': 'system', 'content': 'You are Kimi, an AI assistant created by Moonshot AI.'},
{
'role': 'user',
'content': [
{'type': 'text', 'text': 'which one is bigger, 9.11 or 9.9? think carefully.'}
],
},
]
response = client.chat.completions.create(
model=model_name, messages=messages, stream=False, max_tokens=4096
)
# Access reasoning content
print(f'reasoning content: {response.choices[0].message.reasoning_content}')
# Access final response
print(f'response: {response.choices[0].message.content}')Kimi-K2.5 supports multimodal inputs including images and video. Content should be passed as base64 encoded strings within the message content array using image_url or video_url types.
Note: Chat with video content is currently an experimental feature and is only supported in the official Moonshot API.
import openai
import base64
import requests
# Image Input Example
def chat_with_image(client: openai.OpenAI, model_name: str):
url = 'https://huggingface.co/moonshotai/Kimi-K2.5/resolve/main/figures/kimi-logo.png'
image_base64 = base64.b64encode(requests.get(url).content).decode()
messages = [
{
'role': 'user',
'content': [
{'type': 'text', 'text': 'Describe this image in detail.'},
{'type': 'image_url', 'image_url': {'url': f'data:image/png;base64, {image_base64}'}},
],
}
]
return client.chat.completions.create(model=model_name, messages=messages)
# Video Input Example
def chat_with_video(client: openai.OpenAI, model_name: str):
url = 'https://huggingface.co/moonshotai/Kimi-K2.5/resolve/main/figures/demo_video.mp4'
video_base64 = base64.b64encode(requests.get(url).content).decode()
messages = [
{
"role": "user",
"content": [
{"type": "text","text": "Describe the video in detail."},
{"type": "video_url", "video_url": {"url": f"data:video/mp4;base64,{video_base64}"}},
],
}
]
return client.chat.completions.create(model=model_name, messages=messages)Kimi K2.5 is evaluated across several key dimensions including Reasoning & Knowledge, Image & Video, Coding, Long Context, and Agentic Search. The model is specifically tested in 'Thinking' mode.