PromptingTools.jl

RAGTools provides a modular pipeline for Retrieval-Augmented Generation. The primary functions are:

• build_index(documents; ...): Creates a ChunkIndex from a list of documents.
• airag(index; question): An end-to-end function that performs both retrieval and generation, returning an AIMessage.
• retrieve(index, question): Retrieves relevant chunks from the index for a given question. Returns a RAGResult.
• generate!(index, result): Takes a RAGResult (from retrieve) and generates an answer from the retrieved chunks.
• annotate_support: Highlights which parts of an answer are supported by the index documents. This is applied automatically when using pprint(result).
• build_qa_evals: Generates question-answer pairs from a corpus for evaluating RAG models.

PromptingTools.jl is designed to unify prompt engineering across different backends. Key capabilities include:

• Prompt Templating: Use placeholders to manage complex prompts easily.
• Observability: Automatic serialization and tracing of AI conversations.
• Fine-tuning Support: Export conversations into ShareGPT-compatible formats.
• Debugging: Code evaluation and automatic error localization for LLM debugging.
• Agentic Workflows: The AgentTools module supports lazy ai* calls with states, automatic code feedback, and Monte-Carlo tree search-based auto-fixing.
• RAG Support: Retrieval-Augmented Generation workflows are supported via the companion package RAGTools.jl.

Prompt files can be plain text or include optional TOML frontmatter for metadata. If frontmatter is used, the fields are accessible via the .meta property of the loaded prompt object.

Example File Structure:

---
title = "Expert System Prompt"
version = "1.0"
author = "Team Name"
description = "A system prompt for expert assistance"
---
You are a {role}. Be {style} and helpful.

If no frontmatter is provided, the file is treated as plain text and the filename is used as the title.

There are two primary ways to maintain conversation state:

1. Using the ai macros

The ai!"" macro (with an exclamation mark) automatically uses the last response as the context for the next turn.

2. Using the aigenerate function

Pass the conversation keyword argument to include previous messages. To capture the history, you must set return_all=true.

# Using macros
ai"Hi! I'm John"
ai!"What's my name?"

# Using aigenerate
conv = aigenerate("Hi! I'm John"; return_all=true)
response = aigenerate("What's my name?"; return_all=true, conversation=conv)

The package is built around three core pillars that allow it to interface with various LLM providers (OpenAI, Anthropic, Ollama, etc.) using a unified approach:

1. Prompt Schemas: Subtypes of AbstractPromptSchema that define how prompts are structured, separated, and rendered for specific APIs (e.g., OpenAISchema). Task functions use these schemas for dispatch.
2. Messages: Representations of a conversation. Use SystemMessage for instructions, UserMessage for inputs, and AIMessage for model responses. A conversation is typically a collection of these messages.
3. Task-oriented functions: High-level functions starting with ai... (e.g., aigenerate, aiembed, aiclassify) that perform specific tasks. These functions are dispatched based on the provided AbstractPromptSchema.

The Responses API is designed for agentic workflows and reasoning models. Unlike the standard Chat Completions API, it supports server-side state management, built-in tools (web search, file search, code interpreter), and better caching efficiency (40-80% improvement). To use this API, you must use OpenAIResponseSchema() instead of OpenAISchema().

using PromptingTools
const PT = PromptingTools

# Use OpenAIResponseSchema to target the Responses API
schema = PT.OpenAIResponseSchema()

msg = aigenerate(schema, "What is the capital of France?"; model="gpt-5-mini")

PromptingTools.jl operates using five core abstractions that manage the flow from user input to AI response:

1. API/Model Providers: The interface to the LLM (e.g., OpenAI API, or local providers like Ollama/Llama.cpp).
2. Schemas: Objects of type AbstractPromptSchema that determine how messages are formatted and which provider is used (e.g., OpenAISchema).
3. Prompts: The information (questions, context, or commands) sent to the model.
4. Messages: The basic unit of communication (e.g., UserMessage, AIMessage).
5. Prompt Templates: Re-usable prompts (AITemplate) with placeholders (e.g., {{name}}) that allow for dynamic input injection.

The typical execution flow is: render $\rightarrow$ UserMessage(s) $\rightarrow$ render $\rightarrow$ Provider API $\rightarrow$ AIMessage.

To avoid manually passing the schema every time you call aigenerate or aiembed, you can register a model name in the PT.MODEL_REGISTRY. You can also create aliases in PT.MODEL_ALIASES for easier access.

Supported functions for these calls: aigenerate and aiembed.

# Register a model with a specific schema
PT.register_model!(; name = "my-model", schema = PT.CustomOpenAISchema())

# Create an alias
PT.MODEL_ALIASES["shortname"] = "my-model"

# Use the registered model or alias directly
msg = aigenerate("Hello!"; model = "shortname")
# Or using the macro syntax
ai"Hello!"shortname