repository·main·Indexed 11 days ago
https://github.com/567-labs/instructorA library for obtaining reliable, structured JSON from any LLM using Pydantic for schema definition, validation, and type safety. Version 1.16.0 supports in-memory and persistent caching via AutoCache and DiskCache, a CLI for managing batch jobs, and integration with providers like OpenAI, Anthropic, and Google.
Instructor supports a wide range of providers, including:
The Instructor CLI allows you to manage fine-tuning jobs on OpenAI directly from your terminal. You can use it to create new fine-tuning jobs, view existing jobs, and manage files.
Note: The CLI is currently under development and does not support all OpenAI API features. If a specific feature is missing, you may need to use the Python library directly.
Instructor's functionality is organized into several key domains:
Lists, Arrays, Union Types, Enums, and TypedDicts.Fields for attributes, Alias for renaming, and handling Missing (optional) values.Stream Partial for partial responses or Stream Iterable for collections of objects.Retrying for automatic error recovery and Validators for custom logic.Caching, Prompt Caching, and tracking Usage Tokens.FastAPI, TypeAdapter, and Distillation for production workflows.While specific capabilities vary by provider, all Instructor integrations support a core set of features:
Partial or Iterable modes.The Instructor repository is organized into several key directories that separate the core library from its supporting tools:
instructor/: Contains the core library, including clients, adapters, and utilities required to achieve structured outputs from LLMs.cli/: Contains the command-line interface used for job management and usage tracking.docs/: Contains the source files for the project documentation (built with MkDocs).examples/: Provides practical examples and cookbooks to demonstrate various usage patterns.tests/: Contains the test suite and evaluation scripts used to verify library functionality.The Instructor V2 architecture is organized into core logic and provider-specific implementations.
instructor.v2)ModeHandler: Abstract base class for mode implementations.mode_registry: The central registry for mode handlers.RequestHandler: Protocol for handling requests.ReaskHandler: Protocol for handling re-asking/retry logic.ResponseParser: Protocol for parsing LLM responses.from_anthropic: Factory function for Anthropic clients.instructor.v2.core: Contains Protocols, Registry, decorators.py (@register_mode_handler), exceptions.py, handler.py, patch.py, and retry.py.instructor.v2.providers.anthropic: Contains the Anthropic implementation, including the from_anthropic factory and specific handlers (e.g., TOOLS, JSON).# Example of available top-level exports
from instructor.v2 import ModeHandler, mode_registry, RequestHandler, ReaskHandler, ResponseParser, from_anthropicSelf-refine is an iterative prompting approach where an LLM is used to perform three distinct roles in a loop:
This loop repeats until a predefined stopping condition is met (e.g., the LLM signals it is 'done' or a maximum number of iterations is reached). This pattern is useful for tasks requiring high precision, such as code generation or complex reasoning.
graph TD
A[Generate initial response]:::blue --> B[Generate feedback]:::orange
B --> C{Stopping<br>condition<br>met?}:::orange
C -->|No| D[Refine response]:::orange
C -->|Yes| E[Final output]:::green
D --> BSelf-Ask is a prompting strategy designed to solve the compositionality gap in LLMs. The compositionality gap occurs when a model can solve individual sub-components of a problem but fails to synthesize them into the correct final answer.
By using a structured response_model with instructor, you can force the model to perform a chain-of-thought style reasoning within a single prompt by explicitly generating and answering intermediate questions before arriving at the final conclusion.
Universal Self Prompting (USP) is a two-stage prompting technique designed to improve LLM performance by using unlabeled data to generate high-quality exemplars (few-shot examples) and a scoring function to select them.
Unlike standard few-shot prompting where examples are manually curated, USP automates the selection process through two stages:
This method allows the model to self-adapt to the specific task and distribution of the data it is processing.
Consistency Based Self Adaptive Prompting (COSP) is an ensembling technique designed to improve LLM output quality by automatically generating high-quality few-shot examples to include in the final prompt.
Since these examples are generated from questions without ground truth labels, COSP uses two metrics to select the best ones:
The process follows two main steps:
k examples with the lowest scores.k examples to the target prompt, sample the model multiple times, and use a majority vote to determine the final answer.CitationMixin is a Pydantic mixin that adds citation validation to your models. It introduces a substring_quotes field (a list of strings) which contains quotes from the source text.
context={"context": source_text}.Prompt Caching is a feature that allows you to cache portions of your prompt to optimize performance for multiple API calls that share the same context. Using prompt caching helps reduce costs and improves response times by minimizing redundant processing of large, static prompt segments.
To maximize cache hits, use prefix matching: ensure that common instructions or static context (like a system prompt) are placed at the beginning of the prompt, and move all variable parts (like user queries or dynamic data) to the end of the message.