LLM.js

When instantiating LLM or calling the LLM function, you can provide a configuration object to control the behavior of the request.

Key Configuration Options:

• service: The LLM provider. Supported values: openai, anthropic, google, xai, groq, deepseek, ollama.
• apiKey: API key for the service. If omitted, the library attempts to read it from environment variables.
• model: The specific model name (e.g., gpt-4o). If not provided, it is auto-detected from the service.
• stream: Boolean to enable real-time streaming responses.
• extended: Boolean to return a detailed response including usage, costs, and metadata.
• think: Boolean to enable reasoning/thinking mode for supported models.
• temperature: Controls randomness (0 = deterministic, 2 = very creative).
• max_tokens: Maximum response length.
• parser: A function or preset to transform response content (e.g., LLM.parsers.json).
• tools: An array of functions the model can call.
• max_thinking_tokens: Maximum tokens allocated for the thinking process.

const llm = new LLM(input, {
  service: "openai",        // LLM service provider
  apiKey: "sk-123",          // apiKey
  model: "gpt-4o",          // Specific model
  max_tokens: 1000,         // Maximum response length
  temperature: 0.7,         // "Creativity" (0-2)
  stream: true,             // Enable streaming
  extended: true,           // Extended responses with metadata
  messages: [],             // message history
  think: true,              // Enable thinking mode
  parser: LLM.parsers.json, // Content parser
  tools: [...],             // Available tools
  max_thinking_tokens: 500, // Max tokens for thinking
});

LLM.js provides a unified interface to interact with various local and remote providers. You can switch providers by specifying the service and model in the options object.

Note: Local models (via ollama) track token usage, but the cost is always reported as $0.

// Defaults to Ollama (local)
await LLM("the color of the sky is");

// OpenAI
await LLM("the color of the sky is", { model: "gpt-4o-mini", service: "openai" });

// Anthropic
await LLM("the color of the sky is", { model: "claude-3-5-sonnet-latest", service: "anthropic" });

// Google
await LLM("the color of the sky is", { model: "gemini-1.5-pro", service: "google" });

// xAI
await LLM("the color of the sky is", { service: "xai", model: "grok-beta" });

// DeepSeek with thinking
await LLM("solve this puzzle", { service: "deepseek", model: "deepseek-reasoner", think: true });

// Ollama (local)
await LLM("the color of the sky is", { model: "llama3.2:3b", service: "ollama" });

Enable LLMs to call custom functions by providing a tools array in the options. Each tool follows a schema defining its name, description, and input_schema (JSON Schema).

Passing tools automatically enables extended mode. Tool calls are automatically added to the message history. In streaming mode, you can listen for type: "tool_calls" chunks.

const getCurrentWeather = {
  name: "get_current_weather",
  description: "Get the current weather for a city",
  input_schema: {
    type: "object",
    properties: {
      city: { type: "string", description: "The name of the city" }
    },
    required: ["city"]
  }
};

const response = await LLM("What's the weather in Tokyo?", {
  tools: [getCurrentWeather],
});

// tool use automatically enables extended mode
console.log(response.tool_calls); 
// [{ id: "call_123", name: "get_current_weather", input: { city: "Tokyo" } }]

For complete control over the API interaction, subclass LLM and implement the required parse methods. This is necessary for handling custom response formats for content, tools, thinking, models, options, token usage, and cost usage. You must also implement streaming variants (e.g., parseContentChunk) if you support streaming.

class Custom extends LLM {
    static readonly service: ServiceName = "secretAGI";
    static DEFAULT_BASE_URL: string = "http://localhost:9876";
    static DEFAULT_MODEL: string = "gpt-999";
    static isLocal: boolean = false; // don't track pricing
    static isBearerAuth: boolean = false;

    get chatUrl() { return `${this.baseUrl}/chat` }
    get modelsUrl() { return `${this.baseUrl}/models` }

    parseContent(data: any): string { ... }
    parseTools(data: any): ToolCall[] { ... }
    parseThinking(data: any): string { ... }
    parseModel(model: any): Model { ... }
    parseOptions(options: Options): Options { ... }
    parseTokenUsage(usage: any): InputOutputTokens | null { ... }
    parseUsage(tokenUsage: InputOutputTokens): Usage { ... }

    // streaming methods
    parseToolsChunk(chunk: any): ToolCall[] { return this.parseTools(chunk) }
    parseContentChunk(chunk: any): string { return this.parseContent(chunk) }
    parseThinkingChunk(chunk: any): string { return this.parseThinking(chunk) }
}