musistudio/llms

The LLM API transformation server acts as a middleware to standardize requests and responses between different LLM providers (such as Anthropic, Gemini, and Deepseek). It uses a modular transformer system to bridge the gap between provider-specific API formats and a standardized internal format.

Data Flow

1. Request: Incoming provider-specific requests are transformed into a unified format using a transformer, which is then processed by the server.
2. Response: The server's unified response is transformed back into the provider's specific format. For providers supporting real-time streaming, the server handles chunked data conversion to maintain a standardized format.

To support a new LLM provider, you must implement a dedicated transformer class. Each transformer is responsible for mapping between the provider's specific API and the server's unified formats (UnifiedChatRequest and UnifiedChatResponse).

A transformer must implement the following methods:

• transformRequestIn: Converts the provider's request format to the unified format.
• transformResponseIn: Converts the provider's response format to the unified format.
• transformRequestOut: Converts the unified request format to the provider's format.
• transformResponseOut: Converts the unified response format back to the provider's format.
• endPoint: A property specifying the API endpoint for the provider (e.g., "/v1/messages" for Anthropic).

To use this project, you can install dependencies and run the server in development or production modes.

Installation

Install dependencies using npm or pnpm:

npm install
# or
pnpm install

Development

Run the server in development mode with hot-reloading (uses nodemon + tsx for src/server.ts):

npm run dev

Build

Build the project for production. This outputs to dist/cjs and dist/esm:

npm run build

Testing

Run the test suite:

npm test

The LLMProvider interface defines how a specific LLM provider is configured within the system. This is used to map provider-specific details to the unified interface.

Fields:

• name: The name of the provider.
• baseUrl: The API endpoint.
• apiKey: The authentication key.
• models: A list of supported model strings.
• transformer: An optional mapping of model names to an array of Transformer instances used to convert data between formats.

const myProvider: LLMProvider = {
  name: "custom-provider",
  baseUrl: "https://api.example.com",
  apiKey: "sk-xxx",
  models: ["model-a", "model-b"],
  transformer: {
    "model-a": { use: [/* Transformer instances */] }
  }
};

When instantiating the Server class, you can provide a ServerOptions object. This object extends FastifyServerOptions and includes an optional initialConfig field.

Key configuration behaviors:

• initialConfig: An AppConfig object used to seed the ConfigService.
• logger: If not provided, it defaults to true (enabling Fastify's default logger).
• port & host: These are extracted from the ConfigService during .start(). If not explicitly set in the config, they default to 3000 and 127.0.0.1 respectively.

interface ServerOptions extends FastifyServerOptions {
  initialConfig?: AppConfig;
}

When registering a new transformer, you should provide a constructor that follows the TransformerConstructor type. This allows the system to instantiate your transformer with optional TransformerOptions. You can also provide a static TransformerName property on the class to identify it.

export type TransformerConstructor = {
  new (options?: TransformerOptions): Transformer;
  TransformerName?: string;
};

The @musistudio/llms package provides a central entrypoint via its default export that contains all available Transformer classes. You can import this default object to access specific transformers for different LLM providers (like Anthropic, OpenAI, Gemini, etc.) or for specific functional transformations (like reasoning, sampling, or token management).

import transformers from './transformer';

// Example: Accessing a specific provider transformer
const transformer = new transformers.AnthropicTransformer();

// Example: Accessing a functional transformer
const maxTokenTransformer = new transformers.MaxTokenTransformer();

Transforms a UnifiedChatRequest into an AnthropicChatRequest. This handles the conversion of system messages, message roles, and the complex Anthropic content block structure (text vs tool_use vs tool_result).

import { convertToAnthropic } from './utils/converter';

const anthropicRequest = convertToAnthropic(unifiedRequest);