DOCSEARCHbyAlgolia
Submit Docs

TypeChat Documentation

repository·main·Indexed 27 days ago

https://github.com/microsoft/typechat

TypeChat is a library for building natural language interfaces using types and schema engineering instead of manual prompt engineering. It automates prompt construction, response validation, and error repair to ensure LLM outputs conform to specific data structures. Available for TypeScript/JavaScript via npm and Python (typechat-py v0.0.4), it provides tools like TypeChatJsonTranslator and TypeChatValidator to convert natural language into structured JSON.

Tokens
11K
Snippets
27
Records
66
Agent score
93%

What's inside TypeChat

  1. Overview of TypeChat

    main

    TypeChat is a library designed to build natural language interfaces using types instead of traditional prompt engineering. It uses "schema engineering" to define intents through types (such as interfaces or discriminated unions).

    TypeChat automates the following workflow:

    1. Prompt Construction: Generates a prompt for the LLM based on your defined types.
    2. Validation & Repair: Validates that the LLM response conforms to the schema. If validation fails, it attempts to repair the output through additional LLM interaction.
    3. Summarization: Succinctly summarizes the instance (without using an LLM) to confirm alignment with user intent.

  2. Understand the TypeChat workflow

    main

    TypeChat enables the creation of natural language interfaces by using types to represent your application's domain. The workflow follows these steps:

    1. Prompt Construction: TypeChat constructs a prompt for the LLM using your defined types.
    2. Validation & Repair: It validates the LLM response against your schema. If the response does not conform, TypeChat automatically attempts to repair the output through further interaction with the language model.
    3. Summarization: It provides a succinct summary of the resulting instance (without using an LLM) to allow for confirmation that the output aligns with user intent.

  3. How TypeChat uses TypeScript for validation

    main

    TypeChat uses TypeScript types as the specification language for LLM responses.

    When a response is received, TypeChat utilizes the TypeScript compiler API to validate the data against your provided types. If validation fails, TypeChat sends a 'repair prompt' back to the model that includes specific diagnostics from the TypeScript compiler to guide the model toward a correctly typed response.

  4. Understand the core concept of TypeChat

    main
    TypeChat allows you to build natural language interfaces by using TypeScript types as schemas to guide Large Language Model (LLM) responses. Instead of manually parsing natural language or writing complex JSON templates, you define TypeScript interfaces that represent the structured data your application expects. TypeChat then handles the prompt construction, response validation, and automatic repair of non-conforming outputs to ensure the LLM returns well-typed data.

  6. Use TypeScript types to guide LLM responses

    main

    To use TypeChat, define TypeScript types (such as interfaces or discriminated unions) that represent the intents or data structures your application needs. TypeChat uses these types to:

    1. Construct prompts: It automatically generates a prompt for the LLM that includes your type definitions.
    2. Validate responses: It checks if the LLM's JSON response conforms to your defined schema.
    3. Repair output: If the response is invalid, TypeChat interacts with the LLM again to attempt to fix the non-conforming output.
    4. Summarize: It provides a succinct summary of the resulting instance to confirm alignment with user intent (without using an LLM).
    interface Response {
     data: Array<{ venue: string, description: string }>;
}

  7. Define a Schema using Dataclasses or TypedDict

    main

    To guide the language model's response, define a schema using a Python @dataclass or TypedDict.

    • Use @dataclass if you want to access attributes using dot notation (e.g., result.sentiment).
    • Use TypedDict if you prefer dictionary-style access (e.g., result["sentiment"]).

    You can use typing.Literal to restrict values to specific strings, and typing.Annotated or Doc to add metadata/comments to attributes.

    from dataclasses import dataclass
from typing import Literal

@dataclass
class Sentiment:
    """
    The following is a schema definition for determining the sentiment of a some user input.
    """
    sentiment: Literal["negative", "neutral", "positive"]

  8. Apply Schema Engineering best practices for Response Models

    main

    TypeChat uses 'Schema Engineering' instead of prompt engineering. You define TypeScript types (Response Models) to constrain LLM responses. These types act as a bridge between natural language and your application logic.

    To maximize success, follow these best practices when defining your TypeScript schemas:

    • Keep it simple: Use primitives, arrays, and objects.
    • JSON compatibility: Only use types representable as JSON (do not use classes).
    • Structure: Make data structures as flat and regular as possible; avoid deep inheritance hierarchies.
    • Documentation: Include natural language comments on types and properties to describe intent.
    • Type constraints: Restrict use of generics; avoid conditional, mapped, and indexed access types.
    • Flexibility: Allow LLMs to 'color outside the lines' (e.g., use string instead of literal types).
    • Include an escape hatch: Always provide a way for the LLM to handle requests outside the intended domain to prevent hallucinations.

  10. Use an escape hatch to suppress LLM hallucinations

    main

    When a Response Model is too narrow, LLMs may hallucinate answers to fit the schema (e.g., turning an out-of-domain request into a valid but incorrect object).

    To prevent this, include an 'escape hatch' in your schema—such as an unknown category or a specific property for unhandled requests. This allows the LLM to route non-domain requests into a known bucket, which suppresses hallucinations and allows your application to identify when a request wasn't understood.

  12. Use TypeChat to retrieve structured AI responses

    main

    TypeChat allows you to turn user intent into structured, type-safe JSON by combining a human prompt with a TypeScript schema.

    To implement this:

    1. Define a TypeScript interface for your expected response.
    2. Create a language model using createLanguageModel.
    3. Create a translator using createJsonTranslator, passing the model, the schema file content, and the name of the interface.
    4. Use the translator's .translate() method to process requests. The result contains either the typed data or a message describing the failure.
    import * as fs from "fs";
import * as path from "path";
import dotenv from "dotenv";
import * as typechat from "typechat";
import { SentimentResponse } from "./sentimentSchema";

// Load environment variables.
dotenv.config({ path: path.join(__dirname, "../.env") });

// Create a language model based on the environment variables.
const model = typechat.createLanguageModel(process.env);

// Load up the contents of our "Response" schema.
const schema = fs.readFileSync(path.join(__dirname, "sentimentSchema.ts"), "utf8");
const translator = typechat.createJsonTranslator<SentimentResponse>(model, schema, "SentimentResponse");

// Process requests interactively.
typechat.processRequests("😀> ", /*inputFile*/ undefined, async (request) => {
    const response = await translator.translate(request);
    if (!response.success) {
        console.log(response.message);
        return;
    }
    console.log(`The sentiment is ${response.data.sentiment}`);
});