Instructor Elixir

Instructor bridges the gap between unstructured LLM text and structured Software 1.0 data by using Ecto schemas as the source of truth.

Instead of manually parsing LLM strings, you define an Ecto.Schema. Instructor automatically converts this schema into a JSON schema and instructs the LLM to conform to it (using techniques like function calling or BNF grammar sampling). The resulting output is then treated as standard Ecto data, allowing you to use familiar casting and validation patterns.

This approach allows for bidirectional interoperability: you can turn data into text for the LLM, and turn LLM text back into structured data for your application.

defmodule Recipe do
  use Ecto.Schema

  @primary_key false
  embedded_schema do 
    field :title, :string
    field :cook_time, :integer
    field :steps, {:array, :string}
    embeds_many :ingredients, Ingredients, primary_key: false do
      field :name, :string
      field :quantity, :decimal
      field :unit, :string
    end
  end
end

Instructor allows you to extract structured data from LLMs using Ecto schemas. Instead of manually defining JSON schemas, you use standard Ecto schemas and provide semantic guidance via a @llm_doc attribute.

To implement a structured response model:

1. Define an embedded_schema using Ecto.Schema.
2. Add a @llm_doc string attribute to the module to explain the fields to the LLM.
3. Implement a validate_changeset/1 function to ensure semantic correctness (e.g., checking number ranges).
4. Use the use Instructor macro (or similar validation macros provided by the library) to register the validation logic.
5. Call Instructor.chat_completion/1 passing the schema module as the response_model parameter.

# Example of a structured schema
defmodule SpamPrediction do
  use Ecto.Schema
  use Validator # Note: Ensure your specific version uses the correct macro for validation registration

  @llm_doc """
  ## Field Descriptions:
  - class: Whether or not the email is spam.
  - reason: A short, less than 10 word rationalization for the classification.
  - score: A confidence score between 0.0 and 1.0 for the classification.
  """
  @primary_key false
  embedded_schema do
    field(:class, Ecto.Enum, values: [:spam, :not_spam])
    field(:reason, :string)
    field(:score, :float)
  end

  @impl true
  def validate_changeset(changeset) do
    changeset
    |> Ecto.Changeset.validate_number(:score,
      greater_than_or_equal_to: 0.0,
      less_than_or_equal_to: 1.0
    )
  end
end

When using Entra ID, you can implement a GenServer to manage the lifecycle of an access token. The pattern involves:

1. Starting a GenServer with credentials (tenant_id, client_id, client_secret, scope).
2. Fetching the token via an OAuth2 request to https://login.microsoftonline.com/{tenant_id}/oauth2/v2.0/token.
3. Using Process.send_after/3 to schedule a refresh before the current token expires.
4. Exposing a function get_token_func!/4 that returns a closure (0-arity function) which calls the GenServer to retrieve the latest token.

# Example implementation of a token refresher
defmodule AzureServicePrincipalTokenRefresher do
  use GenServer

  # ... implementation details ...

  def get_token_func!(tenant_id, client_id, client_secret, scope) do
    {:ok, pid} = __MODULE__.start_link(tenant_id, client_id, client_secret, scope)

    fn ->
      case __MODULE__.get_access_token(pid) do
        {:ok, access_token} -> access_token
        {:error, error} -> raise "Could not fetch Microsoft Entra ID token: #{inspect(error)}"
      end
    end
  end

  # ... rest of GenServer logic ...
end

For Microsoft Entra ID (formerly Azure AD) authentication, set auth_mode to :bearer. This uses the Authorization: Bearer header. Because Entra ID access tokens expire, you should not provide a static string for api_key. Instead, provide a 0-arity function that returns the current valid token.

It is recommended to use a background process (like a GenServer) to fetch and refresh the token automatically so that the api_key function always returns a fresh token.

# Using a token refresher function for dynamic credentials
config: [
  instructor: [
    adapter: Instructor.Adapters.OpenAI,
    openai: [
      auth_mode: :bearer, 
      api_key: AzureServicePrincipalTokenRefresher.get_token_func!(
          System.get_env("LB_AZURE_ENTRA_TENANT_ID"),
          System.get_env("LB_AZURE_OPENAI_CLIENT_ID"),
          System.get_env("LB_AZURE_OPENAI_CLIENT_SECRET"),
          "https://cognitiveservices.azure.com/.default"
      ),
      api_url: azure_openai_endpoint,
      api_path: azure_openai_api_path
    ]
  ]
]

By default, Instructor returns the raw output from the LLM. You can implement the Instructor.Validator behaviour in your response schemas to critique the response using standard Ecto changeset validations.

When you provide a validate_changeset/1 or validate_changeset/2 function, you can use the :max_retries parameter in Instructor.chat_completion/1 to automatically retry the completion until it passes your custom validation logic.

defmodule Instructor.Demos.SpamPrediction do
  use Ecto.Schema
  use Instructor.Validator # Implements the validation hook

  @primary_key false
  schema "spam_prediction" do
    field :class, :string
    field :score, :float
  end

  @impl true
  def validate_changeset(changeset) do
    changeset
    |> Ecto.Changeset.validate_number(:score, less_than_or_equal_to: 1.0)
  end
end

# Usage with retries:
Instructor.chat_completion(
  model: "gpt-4o-mini",
  response_model: Instructor.Demos.SpamPrediction,
  max_retries: 1,
  messages: [%{role: "user", content: "..."}]
)