DOCSEARCHbyAlgolia
Submit Docs

LM Studio Python SDK

repository·main·Indexed 21 days ago

https://github.com/lmstudio-ai/lmstudio-python

A programmatic interface for interacting with LM Studio instances to perform text completions, chat-based interactions, and agentic workflows using local LLMs. The SDK provides both a synchronous convenience API and an AsyncClient for managing model lifecycles, streaming predictions, generating embeddings, and implementing tool-use plugins.

Tokens
12.8K
Snippets
43
Records
56
Agent score
72%

What's inside lmstudio

  1. Understand LM Studio SDK versioning

    main

    The SDK follows a 3-part X.Y.Z versioning scheme similar to semantic versioning, but with specific triggers for each part:

    • X (Major): Incremented when minimum versions of significant dependencies (like Python or LM Studio) are updated, or when deprecated features are removed.
    • Y (Minor): Incremented when new features are added or notable changes (like new Python support) are introduced. New deprecation warnings may appear here.
    • Z (Patch): Incremented for bug fixes that do not introduce other changes (e.g., adding exceptions/warnings for previously undetected edge cases).

    Development versions may include a .devN suffix.

  2. Understand the LM Studio Python SDK Data Model source

    main

    The Python data model class definitions are automatically generated from lmstudio-js Zod schema files. Because these classes are derived from the JavaScript SDK's schema, they represent the authoritative messaging protocol for the SDK.

    Important for developers: Do not attempt to manually modify the class definitions in the src/lmstudio/_sdk_models/ directory. Any changes to the protocol or data structures must be made in the lmstudio-js repository first and then exported to the Python SDK via the automated code generation process to ensure consistency across language implementations.

  4. Export JSON Schemas from lmstudio-js

    main

    You can generate Python models from the lmstudio-js JSON Schemas by running a specific tox command. This process uses sync-sdk-schema.py to synchronize schemas and write the resulting models to the lmstudio package source.

    Output Location: The generated models are written to ../src/lmstudio/_sdk_models/*.py relative to the sdk-schema directory.

    tox -e sync-sdk-schema

  5. Run the Wikipedia plugin local development instance

    main

    To run a local development instance of the lmstudio/wikipedia plugin, use the pdm package manager to execute the lmstudio.plugin module in development mode, pointing to the plugin directory. This example showcases how to implement asynchronous tool definitions for searching Wikipedia and retrieving specific pages.

    pdm run python -m lmstudio.plugin --dev examples/plugins/wikipedia

  6. Run the dice-tool plugin local development instance

    main

    To run a local development instance of the dice-tool plugin, use the pdm package manager to execute the lmstudio.plugin module in development mode, pointing it to the plugin directory.

    This example demonstrates synchronous tool definitions for random number generation (rolling dice) and includes a demonstration of tool call status updates.

    pdm run python -m lmstudio.plugin --dev examples/plugins/dice-tool

  7. How AsyncClient connection management works

    main

    Unlike the synchronous API, the AsyncClient does not implicitly connect the websocket when you send a request. To avoid violating principles of structured concurrency (as the websocket manages background tasks for pings and keepalives), you must explicitly manage the connection lifecycle.

    It is recommended to use the client within an async with block or explicitly call await client.connect() before performing operations. The connection is cleaned up when you call await client.disconnect() or exit the context manager.

  8. Manage chat history with the Chat class

    main

    The Chat class is a helper used to track and manage LLM interactions, including system prompts, user messages, assistant responses, and tool calls. It maintains a sequence of messages that represent the conversation context.

    Key capabilities:

    • Initialization: Create a new chat from scratch or reconstruct one from existing history data.
    • Message Management: Add system prompts, user messages (including multi-part content like text and images), assistant responses (including tool calls), and tool results.
    • Cloning: Create deep copies of a chat instance to branch conversations without affecting the original.

    Note on terminology: In the context of the Chat history API, "prompt" specifically refers to system prompts used for behavioral directives. This differs from "completion prompts" used in text completion or "chat prompts" used in template application.

    from lmstudio import Chat

# Initialize a new chat with a system prompt
chat = Chat(initial_prompt="You are a helpful assistant.")

# Add a user message
chat.add_user_message("Hello, how are you?")

# Add an assistant response
chat.add_assistant_response("I am doing well, thank you!")

  9. How Prompt Preprocessor hooks work

    main

    The Prompt Preprocessor system allows plugins to intercept and modify user messages before they are sent to the model.

    Lifecycle

    1. Request: The server sends a PromptPreprocessingRequest via a websocket channel (setPromptPreprocessor endpoint).
    2. Execution: The PromptPreprocessorController is instantiated with the request context (task ID, PCI, and tokens). Your hook_impl is then invoked.
    3. Status Reporting: While running, the hook can use the controller to create and update UI status blocks, providing real-time feedback to the user.
    4. Abort Handling: If the server sends an abort signal, the plugin receives a PromptPreprocessingAbortEvent. The SDK handles the cancellation of the running hook task.
    5. Response:
      • If the hook returns a modified UserMessage or UserMessageDict, the server receives a PromptPreprocessingCompleteDict containing the new content.
      • If the hook returns None, the original message is used.
      • If the hook raises an exception, the server receives a PromptPreprocessingErrorDict containing error details and a stack trace.

  10. How DevPluginClient manages plugin registration

    main

    The DevPluginClient is used to facilitate the communication between a development plugin and the LM Studio server. It uses a specialized DevPluginRegistrationEndpoint to register the plugin and retrieve the required credentials (client_id and client_key).

    When using the register_dev_plugin context manager, the client:

    1. Sends a registration request containing the plugin's manifest (owner, name, and type).
    2. Waits for the server to respond with a ready event containing the clientIdentifier and clientPasskey.
    3. Yields these credentials to the caller.
    4. Automatically sends an end message to the server upon exiting the context to deregister the plugin.

  11. Implement LM Studio plugin hooks

    main

    LM Studio plugins are implemented by defining specific hook functions within a src/plugin.py file. These hooks allow the plugin to intercept and extend various parts of the LM Studio lifecycle.

    Supported Hooks

    • preprocess_prompt: Intercepts and modifies prompts before they are sent to the model.
    • generate_tokens: Intercepts the token generation process.
    • list_provided_tools: Provides a list of tools available to the model.

    Configuration Schemas

    Plugins can define configuration schemas to allow users to customize behavior via the LM Studio UI. You should define two types of schemas in your plugin.py namespace:

    • ConfigSchema: Defines plugin-specific configuration.
    • GlobalConfigSchema: Defines global configuration settings.

    Both must be subclasses of BaseConfigSchema from the lmstudio.plugin.config_schemas module.

  12. How LM Studio handles JSON schema and key casing

    main

    The LM Studio SDK implements specific logic to ensure compatibility with the LM Studio API:

    1. Schema Flattening: The LM Studio API does not support JSON schemas containing top-level $ref keys pointing to named subschemas. The _to_json_schema utility (used by BaseModel.model_json_schema) automatically resolves these references and extracts the target schema fields into the top-level definition.
    2. Case Conversion: The SDK automatically converts Python snake_case keys to camelCase for API communication. It includes specific overrides for known edge cases like useFp16ForKvCache (converting use_fp16_for_kv_cache to useFp16ForKVCache).
    3. Null Handling: Fields that are None are omitted from the resulting JSON rather than being sent as null, adhering to the SDK's omit_defaults=True configuration.