Function calling allows the model to request tool execution. The workflow is:
- Request: Send user prompt and a list of
TOOLS (OpenAI format) to the server. - Model Output: The model returns a tool call instruction (e.g., wrapped in
<tool_call> tags). - Execution: Parse the instruction, execute the local function, and append the result to the message history with the
role: "tool". - Final Synthesis: Send the updated message history back to the model to get the final natural language answer.
Note: The demo implementation is optimized for Qwen3 series models which use <tool_call> XML-like tags. Other models may require different parsing logic.
from chat_api_flask import RKLLMClient, TOOLS, parse_tool_calls, execute_tool_calls
client = RKLLMClient(base_url="http://x.x.x.x:8080")
messages = [
{"role": "system", "content": "You are Qwen..."},
{"role": "user", "content": "What's the temperature in San Francisco now?"},
]
# Step 1: Get tool calls from model
resp = client.chat(messages=messages, tools=TOOLS, stream=False)
tool_calls = parse_tool_calls(resp["choices"][0]["message"]["content"])
# Step 2: Execute tools and update messages
assistant_msg, tool_msgs = execute_tool_calls(tool_calls)
messages.append(assistant_msg)
messages.extend(tool_msgs)
# Step 3: Get final answer
resp = client.chat(messages=messages, tools=None, stream=False)
print("A:", resp["choices"][0]["message"]["content"])
