deep-chat

Use the qwen property within directConnection to connect directly to Alibaba Cloud's Qwen API. You can provide a boolean true for default settings or an object to configure specific model parameters.

Supported models include Qwen LMs, Qwen-VL, Qwen-Coder, Qwen-Omni, and Qwen-Math.

Configuration Options:

• key: Your Alibaba Cloud API key.
• model: The name of the Qwen model (default: qwen-plus).
• temperature: Controls randomness (0.0-2.0).
• max_tokens: Maximum number of tokens to generate.
• top_p: Nucleus sampling diversity (0.0-1.0).
• frequency_penalty: Reduces repetition (-2.0 to 2.0).
• presence_penalty: Controls token repetition (-2.0 to 2.0).
• stop: Sequences where generation stops.
• system_prompt: Defines the model's role/objective.
• tools: Array of QwenTool declarations.
• tool_choice: Controls tool usage ("auto" | "none" | {type: "function", function: {name: string}}).
• function_handler: Callback for handling tool calls.

<deep-chat
  directConnection='{
    "qwen": {
      "key": "placeholder key",
      "system_prompt": "You are a helpful assistant.",
      "temperature": 0.7
    }
  }'
></deep-chat>

Gemini supports function calling. To implement this, you must provide two things in the gemini configuration:

1. tools (GeminiTool): An array of objects containing functionDeclarations. Each declaration includes a name, description, and parameters (defined using JSON Schema).
2. function_handler (FunctionHandler): A function that receives functionsDetails (information about which tools to call) and returns either:
   • An array of objects { response: string }[] containing the results for each tool call to be fed back to the model.
   • An object { text: string } to immediately display text in the chat.

Example Implementation:

chatElementRef.directConnection = {
  gemini: {
    tools: [
      {
        functionDeclarations: [
          {
            name: 'get_current_weather',
            description: 'Get the current weather in a given location',
            parameters: {
              type: 'object',
              properties: {
                location: {
                  type: 'string',
                  description: 'The city and state, e.g. San Francisco, CA',
                },
                unit: {type: 'string', enum: ['celsius', 'fahrenheit']},
              },
              required: ['location'],
            },
          },
        ],
      },
    ],
    function_handler: (functionsDetails) => {
      return functionsDetails.map((functionDetails) => {
        return {
          response: getCurrentWeather(functionDetails.arguments),
        };
      });
    },
    key: 'placeholder-key',
  },
};

The webModel feature allows you to run a chat model entirely within the user's browser without connecting to any external services. This provides privacy and reduces server costs.

Setup

1. Integrate the deep-chat-web-llm module into your project.
2. Configure the webModel property on the <deep-chat> component.

Configuration Options

You can set webModel to true for default settings or provide an object to customize the behavior:

• model (string): The name of the model to be used. (Default: "Llama-3.2-1B-Instruct-q4f16_1-MLC")
• instruction (string): Directs how the model should respond.
• urls (WebModelUrls): Defines the endpoints to retrieve the web model assets.
• load (WebModelLoad): Defines how and when the model is loaded.
• introMessage (WebModelIntro): Configuration for the introductory web model message.
• worker (Worker): A Web Worker that can be used to enhance rendering performance.

<deep-chat webModel="true"></deep-chat>