DOCSEARCHbyAlgolia
Submit Docs

LangDiff

repository·main·Indexed 43 days ago

https://github.com/globalaiplatform/langdiff

A Python and TypeScript library for streaming structured LLM outputs to frontends. It provides intelligent partial parsing of JSON streams, type-safe event handlers for streaming progress, and automatic JSON Patch generation (RFC 6902) to efficiently synchronize application state with UIs.

Tokens
20.8K
Snippets
74
Records
85
Agent score
53%

What's inside LangDiff

  1. Overview of the LangDiff Tracker Module

    main

    The Tracker module enables change tracking to generate JSON Patch diffs. This is useful for implementing progressive UIs where you only want to transmit or apply the differences between states.

    Core Classes:

    • ChangeTracker: The abstract base class for all tracking implementations.
    • JSONPatchChangeTracker: Standard implementation for JSON Patch tracking.
    • EfficientJSONPatchChangeTracker: An enhanced version that supports optimized append operations.

    Utility Functions:

    • track_change(): Wraps an existing object to enable automatic change tracking.
    • apply_change(): Applies JSON Patch operations to a target object.

  2. Overview of the LangDiff Parser Module

    main

    The Parser module is designed for processing token streams into structured, streaming-aware data types. It allows you to handle JSON-like structures (Objects, Lists, Strings, Atoms) as they are being generated by an LLM.

    Core Streaming Types:

    • Object: Represents a streaming JSON object.
    • List: Represents a streaming JSON array.
    • String: Represents a streaming string value.
    • Atom: Represents atomic values like numbers, booleans, or null.
    • Parser: The engine that processes token streams and triggers the associated callbacks.

    Key Capabilities:

    • Event Callbacks: You can hook into the lifecycle of streaming data using on_start and on_complete for all types. Object provides on_update, while List and String provide on_append for incremental updates.

  3. Understand the LangDiff Parser Module

    main

    The Parser Module is designed for processing token streams into structured data. It uses streaming-aware data types that allow you to react to data as it arrives rather than waiting for the full payload.

    Core Classes

    • StreamingObject: Base class for streaming JSON objects.
    • StreamingList: Represents a streaming JSON array.
    • StreamingString: Represents a streaming string value.
    • Atom: Represents atomic values like numbers, booleans, or null.
    • Parser: The engine that processes token streams and triggers the callbacks defined in your schema.

    Key Features

    • Event Callbacks: Every streaming type supports onStart, onAppend, and onComplete callbacks to handle data incrementally.
    • Zod Integration: You can bridge LangDiff with Zod using fromZod(zodSchema) to create a streaming schema from an existing Zod schema, or toZod() to convert a LangDiff schema back to Zod.

  4. Define streaming schemas with class-based models

    main

    LangDiff uses class-based models to define schemas for streaming structured LLM outputs. You extend the ld.Object class and implement the _initializeFields method to register fields using LangDiff types like ld.String or ld.List. This allows you to receive granular, type-safe callbacks as tokens stream in.

    import * as ld from '@langdiff/langdiff';

class ArticleResponse extends ld.Object {
  title!: ld.String;
  sections!: ld.List<ld.String>;

  protected _initializeFields(): void {
    this.addField('title', new ld.String());
    this.addField('sections', new ld.List(ld.String));
  }
}

  5. Track changes and generate JSON Patch diffs

    main

    LangDiff can instrument existing objects and arrays to track mutations. Instead of sending entire objects to the frontend, LangDiff automatically generates JSON Patch diffs (RFC 6902) to allow for efficient state synchronization. This is useful for updating UI components incrementally as the LLM generates data.

    // Example of how mutations result in JSON Patch output:
// If you modify a tracked UI object:
// ui.body.push(`<ins>${chunk}</ins>`);

// The generated output looks like:
// {"op": "add", "path": "/body/-", "value": "<ins>Hell</ins>"}

  6. Define streaming schemas with LangDiff models

    main

    To enable intelligent partial parsing, define your streaming structure using LangDiff model classes like ld.Object, ld.List, and ld.String. These classes handle the internal progression of the JSON structure as tokens arrive.

    Example schema definition:

    import langdiff as ld

class ArticleGenerationResponse(ld.Object):
    section_titles: ld.List[ld.String]
    section_contents: ld.List[ld.String]

  7. Track changes with JSON Patch generation

    main

    LangDiff can instrument existing Pydantic models, Python dicts, lists, or objects to track mutations. It automatically generates JSON Patch diffs (RFC 6902) which can be used for efficient frontend synchronization.

    Example output format for tracked changes:

    {"op": "add", "path": "/body", "value": "<ins>Hell</ins>"}
{"op": "replace", "path": "/body/0", "value": "Hell"}

  8. Use StreamingString, StreamingList, and StreamingObject for real-time updates

    main

    When you call .create() on a schema, you receive instances of core streaming classes that allow you to react to data as it arrives:

    StreamingString

    Used for incremental string building.

    • onStart(callback): Called when streaming begins.
    • onAppend(callback): Called with each new chunk: (chunk: string) => void.
    • onComplete(callback): Called when complete: (value: string | null) => void.

    StreamingList<T>

    Used for arrays receiving items incrementally.

    • onStart(callback): Called when streaming begins.
    • onAppend(callback): Called when a new item is added: (item: T, index: number) => void.
    • onComplete(callback): Called when complete: (value: T[]) => void.

    StreamingObject<T>

    Used for objects with streaming field values.

    • onStart(callback): Called when streaming begins.
    • onUpdate(callback): Called on any field update: (value: Record<string, any>) => void.
    • onComplete(callback): Called when complete: (value: T) => void.

    Atom<T>

    Used for non-streaming values (numbers, booleans, etc.) that arrive as complete units.

    • onStart(callback): Called when the value is received.
    • onComplete(callback): Called with the final value: (value: T) => void.

  9. Handle streaming events with the LangDiff Event System

    main

    LangDiff provides an event-driven system to react to data as it streams. You can register handlers using decorators on the fields of your model instance.

    Common Events

    • on_start(): Called when streaming begins for a value.
    • on_append(): Called as new data is appended.
      • For String types, the argument is the chunk: str.
      • For List types, the argument is (item: ld.String, index: int).
    • on_update(): Called when an Object is updated. The argument is data: dict.
    • on_complete(): Called when a value is fully received (e.g., the final string or the final object state).
    @response.title.on_start
def on_title_start():
    print("Title streaming started")

@response.content.on_append
def on_content_chunk(chunk: str):
    print(f"New content: {chunk}")

@response.items.on_append
def on_item_append(item: ld.String, index: int):
    print(f"New item at index {index}")

@response.on_update
def on_object_update(data: dict):
    print(f"Object updated: {data}")

@response.title.on_complete
def on_title_complete(final_title: str):
    print(f"Title completed: {final_title}")

  10. Define schemas for streaming structured outputs

    main

    LangDiff uses Pydantic-style models to define schemas. You can define structured outputs using ld.Object, ld.String, and ld.List. These models support granular, type-safe callbacks that trigger as tokens stream in.

    import langdiff as ld

class ArticleResponse(ld.Object):
    title: ld.String
    sections: ld.List[ld.String]

  11. Use streaming callbacks with LangDiff models

    main

    LangDiff provides granular, type-safe event handlers to respond to streaming progress. You can attach handlers to list elements or string contents using decorators like @on_append and @on_complete.

    Common event handlers:

    • on_append: Triggered when new content is added to a field or a new item is added to a list.
    • on_complete: Triggered when a field or item is fully parsed.

    Example of attaching handlers to a model instance:

    response = ArticleGenerationResponse()

@response.section_titles.on_append
def on_section_title_append(title: ld.String, index: int):
    # Triggered when a new title is added to the list
    @title.on_append
    def on_title_append(chunk: str):
        # Triggered as characters are added to the specific title
        pass

  12. Choose a ChangeTracker implementation

    main

    LangDiff provides different tracker implementations depending on your performance and compliance needs:

    1. JSONPatchChangeTracker: Generates standard, RFC 6902 compliant JSON Patch operations (add, remove, replace, etc.).
    2. EfficientJSONPatchChangeTracker (Default): Extends the standard tracker by adding optimized detection for string appends. This is highly recommended for streaming LLM outputs to avoid sending the entire string on every new token.
    3. ChangeTracker: The abstract base class you can extend to implement custom tracking logic.
    import { trackChange, JSONPatchChangeTracker, EfficientJSONPatchChangeTracker } from '@langdiff/langdiff';

// Standard compliance
const [p1, buf1] = trackChange({ name: "" }, JSONPatchChangeTracker);

// Optimized for streaming (Default)
const [p2, buf2] = trackChange({ name: "" }, EfficientJSONPatchChangeTracker);