Instructor JS

When designing Zod schemas for use with Instructor, follow these principles to ensure high-quality LLM outputs:

• Modularity: Build self-contained schemas that can be reused in different parts of your data model.
• Self-Description: Use .describe() on fields to provide explicit instructions to the LLM about what that field should contain.
• Optionality: Use .optional() or z.union([schema, z.undefined()]) for fields that may not be present.
• Standardization: Use z.enum for fixed sets of values. Including an 'OTHER' option helps handle ambiguity.
• Dynamic Data: Use z.record(z.string()) to capture arbitrary key-value pairs that don't fit a strict schema.
• Entity Relationships: Use explicit identifiers (like IDs) to define relationships between objects.
• Contextual Logic: Include a 'chain of thought' field to allow the LLM to reason before providing a final answer.

Instructor enables structured data extraction by combining OpenAI's function calling API with Zod for schema validation and type inference.

To use it, initialize the Instructor function by passing an existing OpenAI client and a mode (e.g., "FUNCTIONS"). You then define your desired data structure using a Zod schema. When calling client.chat.completions.create, provide the schema in the response_model option. The returned object will be automatically validated against the schema and will have the correct TypeScript type inferred from the Zod schema.

import Instructor from "@instructor-ai/instructor";
import OpenAI from "openai";
import { z } from "zod";

const oai = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY ?? undefined,
  organization: process.env.OPENAI_ORG_ID ?? undefined
});

const client = Instructor({
  client: oai,
  mode: "FUNCTIONS"
});

const UserSchema = z.object({
  // Description will be used in the prompt
  age: z.number().describe("The age of the user"), 
  name: z.string()
});

// User will be of type z.infer<typeof UserSchema>
const user = await client.chat.completions.create({
  messages: [{ role: "user", content: "Jason Liu is 30 years old" }],
  model: "gpt-3.5-turbo",
  response_model: { 
    schema: UserSchema, 
    name: "User"
  }
});

console.log(user);
// { age: 30, name: "Jason Liu" }

To use Together AI, set the TOGETHER_API_KEY environment variable and patch an OpenAI client pointing to Together AI's endpoint. Use mode: "TOOLS" for this provider.

export TOGETHER_API_KEY="your-api-key"

import { z } from "zod";
import Instructor from "@instructor-js/instructor";
import OpenAI from "openai";

const client = Instructor({
  client: new OpenAI({
    apiKey: process.env.TOGETHER_API_KEY,
    base_url: "https://api.together.xyz/v1",
  }),
  mode: "TOOLS",
});

const UserExtractSchema = z.object({
  name: z.string(),
  age: z.number(),
});

const user = await client.chat.completions.create({
  model: "mistralai/Mixtral-8x7B-Instruct-v0.1",
  response_model: { schema: UserExtractSchema, name: "UserExtract" },
  messages: [
    { role: "user", content: "Extract jason is 25 years old" },
  ],
});

console.assert(user instanceof UserExtractSchema, "Should be instance of UserExtract");

To maintain simplicity and minimize technical debt, follow this four-step pattern when integrating instructor into your TypeScript codebase:

1. Define a Schema: Use Zod to create your structured data definition.
2. Add Logic to Schema: Define validators and methods directly on your Zod schema.
3. Encapsulate LLM Logic: Wrap the instructor call inside a dedicated function that returns your schema type.
4. Perform Computations: Use the resulting typed data for computations, either via standalone functions or by calling methods defined on the schema itself.

// 1. Define a Schema
const StructuredData = z.object({
  // ... schema definition
});

// 2. Define validators and methods on your schema
// (e.g., using z.preprocess or custom methods)

// 3. Encapsulate all your LLM logic into a function
async function extract(input: string): Promise<z.infer<typeof StructuredData>> {
  // instructor call logic here
}

// 4. Define typed computations against your data
function compute(data: z.infer<typeof StructuredData>) {
  // ...
}
// OR call methods on your schema
// data.compute();