LMQL (Language Model Query Language)

The LMQL documentation is organized into four main sections:

• Language: Practical guides and examples demonstrating LMQL's core language capabilities, including the language reference.
• Model Support: Overviews of supported model backends and instructions for integrating them into your workflow.
• Library: Documentation for integrating LMQL in Python and an overview of the LMQL standard library (e.g., Chat, Output, Actions, etc.).
• Development: Information on extending LMQL, contributing to the project, and setting up a development environment.

LMQL provides various resources for learning and implementation:

• Documentation: The primary technical documentation is located in the docs/ directory and is written in Markdown.
• Blog: Latest updates and posts can be found in the blog/ directory.
• Feature Highlights: High-level feature overviews are available in the features/ directory.
• Public Assets: Logos and custom JavaScript are stored in public/.

The Language Model Transport Protocol (LMTP) is a lightweight, transport-agnostic protocol designed for streaming language model output between peers, processes, backends, and frontend applications.

It follows a decoupled architecture where model loading and inference occur in a long-lived separate process, while clients can be short-lived, language-agnostic, and start/stop quickly.

Communication occurs via two asynchronous channels:

1. Inference Process -> Client: Sends interleaved streams of generated tokens. Order within a single stream is preserved.
2. Client -> Inference Process: Sends new generation requests. The process responds with a token stream once the request is scheduled.

LMQL constraints are applied eagerly during generation using token masking. This prevents the model from generating tokens that violate the constraints, saving computation compared to post-processing validation.

Key characteristics:

• Expressiveness: Limited to the validation of context-free languages because constraints are decided on a token-by-token basis.
• Eager Evaluation: Constraints trigger as soon as a violation is determined to be definitive (final), preventing the generation of invalid output.
• Soundness: LMQL uses final/follow semantics to provide a soundness guarantee with respect to token masking.

LMQL supports two syntax variants:

1. Standard Syntax: The modern, minimalistic approach that integrates with a standard Python environment. It uses query strings as top-level expressions. This is the recommended syntax for continued development.
2. Standalone Syntax: A legacy, more static syntax used for standalone LMQL use-cases. While still supported, users are advised to migrate to the modern syntax.

Both variants are compatible and can be used interchangeably.

LMQL (Language Model Query Language) implements Language Model Programming (LMP), which generalizes prompting by combining text prompting with scripting and control flow. Key features include:

• Constraints: You can specify constraints over the language model output.
• Efficiency: LMQL leverages constraints and control flow to generate an efficient inference procedure that minimizes expensive calls to the underlying language model.
• Cost Savings: By optimizing inference, LMQL can reduce computation and costs (reported 26-85% savings) compared to standard prompting methods.

Streaming decorators are applied during the decoding process. The decorator is called for every intermediate value of a variable as it is being generated. This is useful for showing partial responses in chat applications or command-line interfaces. Use the @lmql.decorators.streaming decorator to define them.

from lmql.runtime.program_state import ProgramState
import lmql

@lmql.decorators.streaming
def stream(value: str, context: ProgramState):
    """Decorator to stream the variable value"""
    print("VALUE", [value])

"Enumerate the alphabet without spaces:[@stream TEST]"