Raif (Ruby AI Framework)

Raif is a Rails engine designed to integrate AI-powered features into Ruby on Rails applications. It provides high-level abstractions for managing AI workflows, including tasks, conversations, and agents.

Raif is provider-agnostic and supports several major LLM providers:

• OpenAI
• Anthropic Claude
• AWS Bedrock
• OpenRouter
• Google AI
• xAI

Within Prompt Studio, you can initiate a batch run. This re-executes a task against a set of existing records using the current prompt template and a selected LLM. This is useful for testing how prompt or model updates perform across a large dataset.

Each batch run stores its items and outputs for later inspection. You can optionally score batch runs using an LLM judge. Raif includes the following judge types:

• Raif::Evals::LlmJudges::Binary: Pass/fail judgments against a specific criterion.
• Raif::Evals::LlmJudges::Scored: Numeric scoring based on a provided rubric.
• Raif::Evals::LlmJudges::Comparative: Compares model outputs against a reference.
• Raif::Evals::LlmJudges::Summarization: Scores summaries using a built-in rubric.

Raif uses ERB (Embedded Ruby) templates to define LLM prompts, separating prompt content from Ruby logic. This allows you to use Rails view helpers, partials, and instance methods directly within your prompts.

There are two primary template types:

• .prompt.erb: The main LLM prompt (used for tasks).
• .system_prompt.erb: A system prompt override (can be used for tasks, conversations, or agents).

Templates are automatically resolved based on the class name of the task, agent, or conversation.

rails generate raif:task SummarizeDocument --response-format html

It is important to distinguish between these two OpenRouter controls:

• data_collection: Relates to training and non-transient storage. Setting open_router_data_collection = "deny" ensures you only route to providers that do not collect data for training. A provider that holds prompts for a fixed window (retention) but does not train on them still satisfies the "deny" requirement.
• zdr (Zero Data Retention): A much stricter filter. Setting open_router_zdr = true restricts routing exclusively to endpoints that have a formal Zero Data Retention policy.

Warning: Many models do not have ZDR endpoints. Enabling open_router_zdr may result in no-endpoints errors for models that do not support it. Always verify your model's capabilities before enabling ZDR.

Within an eval block, you can use several methods to validate LLM outputs:

• expect "description" do ... end: A standard assertion that returns true or false. You can pass result_metadata to attach extra data to the result.
• expect_llm_judge_score(...): Uses an LLM to grade the output based on a rubric.

Example of an LLM judge usage:

expect_llm_judge_score(
  task.parsed_response,
  scoring_rubric: Raif::Evals::ScoringRubric.clarity,
  min_passing_score: 4,
  result_metadata: { compression_ratio: 0.5 }
)

# Example of a full eval block structure
eval "description" do
  # ... setup code ...
  expect "description" do
    # ... assertion ...
  end
end

Prompt Studio allows you to inspect how changes to your prompt templates affect real-world data. It provides a side-by-side comparison of the current prompt template versus the version that was originally used when a specific record was created.

You can access Prompt Studio for different record types at:

• /raif/admin/prompt_studio/tasks
• /raif/admin/prompt_studio/conversations
• /raif/admin/prompt_studio/agents

Raif includes a built-in mechanism to handle malformed LLM tool calls. If the LLM produces a tool call that fails validation (unknown tool name or invalid arguments based on the schema), Raif will:

1. Append a synthetic feedback message to the request.
2. Re-prompt the model with the error details.

This loop repeats up to Raif.config.conversation_entry_max_retries times (default is 2). If all attempts fail, the entry is marked as failed!.

To disable this behavior, set max_retries to 0 in your Raif configuration.

Raif.configure do |config|
  config.conversation_entry_max_retries = 0 # Disables the repair loop
end

Rows are not deleted immediately upon reaching the retention period. They must meet specific safety criteria:

Model Completions

• Created before the retention cutoff.
• Quiescent (not updated recently).
• Not part of a non-terminal model completion batch.
• Terminal rows only: A valid Raif::InferenceCostEvent must exist and be at least as fresh as the completion.
• Citations (if any) must have been copied to the Raif::ConversationEntry source.

Tasks

• Created before the retention cutoff.
• Quiescent (not updated recently).
• Completion Gate: No Raif::ModelCompletion row still names it as its source (completions are archived first).
• Prompt Studio Gate: No raif_prompt_studio_batch_run_items row references it (these rows are never deleted, so tasks touched by Prompt Studio are retained indefinitely).

When interacting with an LLM via Raif, you can define the desired output format. Supported formats are text, json, and html. You can specify this format in several contexts:

• During direct chat sessions.
• When creating a Task.
• When creating a Conversation.