Ash AI

repository·main·Indexed 41 days ago

https://github.com/ash-project/ash_ai

An Elixir library that integrates Large Language Models (LLMs) with the Ash Framework. It enables developers to expose Ash actions as tools for AI agents, implement prompt-backed and evaluation actions, and build Model Context Protocol (MCP) servers. It also provides comprehensive support for vectorization, including embedding model implementation, vector similarity searches using vector_cosine_distance/2, and asynchronous embedding updates via ash_oban.

Tokens
25K
Snippets
76
Records
99
Agent score
52%

What's inside Ash AI

  1. Understand Ash AI core concepts

    main

    Ash AI is an extension for the Ash framework that integrates AI capabilities directly into Ash resources. Its core capabilities include:

    • Vectorization: Converting text attributes into vector embeddings for semantic search.
    • AI Tools: Exposing Ash actions as tools that LLMs can call.
    • Prompt-backed Actions: Creating actions where the logic is implemented by an LLM.
    • Evaluation Actions: Asking typed questions of System One models.
    • MCP Server: Exposing tools via the Model Context Protocol.
  2. Understand Read Tool Pagination Results

    main

    Read tools (without get_by) mirror Ash.read/2 behavior. If the action is paginated, the tool returns a page instead of a bare array.

    • Offset Pagination: Returns results, limit, offset, has_more, and next_offset (which is null on the last page).
    • Keyset Pagination: Returns results, limit, has_more, start_keyset, and end_keyset. To move through pages, pass end_keyset as the after argument or start_keyset as the before argument.
    • Defaults: Actions supporting both kinds of pagination use keyset pagination by default. Passing a positive offset switches to offset pagination.
    • Count: The count field is included only if the action is configured with countable: :by_default. Otherwise, the LLM can request result_type: "count".

    When using AshAi.ToolLoop.run/2, the raw result in the on_tool_end callback is the Ash.Page.Offset or Ash.Page.Keyset struct for paginated actions.

  3. Understand AI Tool data access rules

    main

    Tools follow specific data access rules to ensure security:

    1. Filtering/Sorting/Aggregation: Only attributes marked with public?: true can be used.
    2. Arguments: Only action arguments marked with public?: true are exposed to tools.
    3. Response data: Public attributes are returned by default. To include private attributes or relationships in a tool response, you must explicitly use the load option in the tool definition.

    Example of loading private data:

    tools do
      # Returns only public attributes
      tool :read_posts, MyApp.Blog.Post, :read
      
      # Returns public attributes plus loaded fields (including private ones)
      tool :read_posts_with_all_details, MyApp.Blog.Post, :read do
        load [:author, :internal_notes]
      end
    end
  4. Choose a vectorization strategy

    main

    Ash AI provides three strategies for updating embeddings:

    1. :after_action (Default): Updates embeddings synchronously after every create/update action. Not recommended for production as it can significantly slow down your application.
    2. :ash_oban: Updates embeddings asynchronously using ash_oban. This is the recommended approach for production. It requires the ash_oban extension and a configured Oban trigger.
    3. :manual: Does not update embeddings automatically. It generates an :ash_ai_update_embeddings action that you can call manually. You can disable this action generation using define_update_action_for_manual_strategy? false.
  5. Configure prompt formats in Ash AI

    main

    The prompt option in run prompt(...) supports several formats:

    1. EEx String: Uses @input and @context. Example: "Analyze: <%= @input.arguments.text %>".
    2. System/User Tuple: Both parts can be EEx templates. Example: {"System prompt", "User prompt: <%= @input.arguments.text %>"}.
    3. ReqLLM.Context: The canonical form for complex prompts (e.g., multi-modal). Requires import ReqLLM.Context.
    4. List of Messages: A list of ReqLLM.Message structs or %{role: "user", content: "..."} maps. Static strings in these lists are EEx-templated.
    5. Function: fn input, context -> ... end. Note: Content returned by a function is used verbatim and is not EEx-evaluated, which is a security best practice for user-supplied text.
    import ReqLLM.Context
    
    # Using ReqLLM.Context for multi-modal prompts
    run prompt(fn input, _ctx ->
      ReqLLM.Context.new([
        system("You are an OCR expert"),
        user([
          ReqLLM.Message.ContentPart.text("Extract the text from this image"),
          ReqLLM.Message.ContentPart.image_url(input.arguments.image_url)
        ])
      ])
    end)
  6. Configure Tool Data Access and Privacy

    main

    When exposing Ash actions as tools, access is governed by attribute visibility and the load option:

    • Filtering/Sorting/Aggregation: Only public attributes (public?: true) can be used.
    • Single-record lookup: Use get_by on read tools to look up one record by filterable fields.
    • Arguments: Only public action arguments are exposed.
    • Response data: Public attributes are returned by default.
    • Loading data: Use the load option to include relationships, calculations, or additional attributes (including private ones) in responses.

    Key Distinction: Private attributes cannot be used for filtering, sorting, or aggregation, but they CAN be included in responses when using the load option.

    tools do
      # Returns only public attributes
      tool :read_posts, MyApp.Blog.Post, :read
    
      # Returns one post by ID instead of a list of results
      tool :get_post_by_id, MyApp.Blog.Post, :read, get_by: :id
    
      # Returns public attributes AND loaded relationships/calculations
      # Note: loaded fields can include private attributes
      tool :read_posts_with_details, MyApp.Blog.Post, :read,
        load: [:author, :comment_count, :internal_notes]
    end
  7. Implement asynchronous vectorization with :ash_oban

    main

    To use the :ash_oban strategy, ensure your resource uses both AshAi and AshOban extensions. You must define an ash_oban_trigger_name and configure an oban trigger block within the resource to handle the :ash_ai_update_embeddings action. Finally, ensure the corresponding queue is added to your Oban configuration in config.exs.

    defmodule MyApp.Artist do
      use Ash.Resource, extensions: [AshAi, AshOban]
    
      vectorize do
        full_text do
          text(fn record -> record.name end)
        end
    
        strategy :ash_oban
        ash_oban_trigger_name :my_vectorize_trigger
        embedding_model MyApp.OpenAiEmbeddingModel
      end
    
      oban do
        triggers do
          trigger :my_vectorize_trigger do
            action :ash_ai_update_embeddings
            queue :artist_vectorizer
            worker_module_name __MODULE__.AshOban.Worker.UpdateEmbeddings
            scheduler_module_name __MODULE__.AshOban.Scheduler.UpdateEmbeddings
          end
        end
      end
    end
  8. Generate a Chat feature with `mix ash_ai.gen.chat`

    main

    The mix ash_ai.gen.chat command is an experimental tool that generates a complete chat feature for Ash & Phoenix applications. It uses ash_oban and ash_postgres for persistence and AshAi.ToolLoop.stream/2 for streaming responses.

    Requirements:

    • A user resource must exist (specify a custom one with --user if it's not <YourApp>.Accounts.User).
    • Tailwind CSS and DaisyUI must be installed for the UI.
    • An LLM API key (e.g., OPENAI_API_KEY) must be set in your environment.

    Usage:

    mix ash_ai.gen.chat --live

    Use the --live flag to generate LiveView components.

  9. Handle multitenancy in AshAi Tool Loops

    main

    When using Ash domains with multitenancy, you must provide the correct tenant (and usually actor) to AshAi.ToolLoop functions. This ensures that tool executions—such as reads, writes, and generic actions—are scoped correctly to the specific tenant.

    AshAi.ToolLoop.run/2 and AshAi.ToolLoop.stream/2 accept the :tenant option. If omitted, tool calls will behave as if no tenant is present, which may fail if your resources require a tenant context.

    Callbacks like on_tool_start receive an AshAi.ToolStartEvent containing the :tenant, allowing you to tie tool runs to your request context for logging or auditing.

    AshAi.ToolLoop.stream(messages,
      otp_app: :my_app,
      tools: true,
      actor: current_user,
      tenant: current_tenant
    )
  10. Write New Live LLM Tests

    main

    To write integration tests that interact with real LLM providers, use the AshAi.LiveLLMCase module. You should tag your tests with :live_llm and a provider-specific tag (e.g., live_llm: :openai) to allow for granular execution.

    Use the provided module attributes to reference the models used in the test suite, ensuring your tests remain compatible with the project's testing infrastructure.

    defmodule AshAi.LiveLLM.MyFeatureTest do
      use AshAi.LiveLLMCase, async: true
    
      describe "OpenAI feature" do
        @tag :live_llm
        @tag live_llm: :openai
        test "my test" do
          require_provider!(:openai)
          
          # Test code using @openai_model
        end
      end
    
      describe "Anthropic feature" do
        @tag :live_llm
        @tag live_llm: :anthropic
        test "my test" do
          require_provider!(:anthropic)
          
          # Test code using @anthropic_model
        end
      end
    end
  11. Use MCP tools to discover and retrieve package usage rules

    main

    The ash_ai development MCP server exposes tools that allow AI coding assistants (like Claude Code) to automatically discover and read usage-rules.md files from your project's Elixir dependencies. This helps the AI follow package-specific best practices and avoid common pitfalls.

    Two primary tools are available via the MCP server:

    1. list_packages_with_rules: Returns a list of package names in your current project that contain a usage-rules.md file.
    2. get_usage_rules: Retrieves the actual markdown content of the usage rules for a specific list of packages.

    These tools are read-only and use Mix.Project.deps_paths() to locate the files on your file system.

    # Conceptual API usage via MCP tools:
    
    # 1. List available packages with rules
    # Returns: ["ash", "ash_postgres", "igniter", ...]
    list_packages_with_rules()
    
    # 2. Get rules for specific packages
    # Argument: packages: ["ash", "igniter"]
    # Returns: [%{package: "ash", rules: "...markdown content..."}, ...]
    get_usage_rules(packages: ["ash", "igniter"])
  12. Configure multitenancy for generated chat via mix ash_ai.gen.chat

    main

    The responders generated by mix ash_ai.gen.chat pass the tenant: option through to AshAi.ToolLoop.stream. For data reads, the generated code uses scope: context where context is an Ash.Scope.

    To ensure generated chat works correctly in a multitenant application:

    1. Populate the tenant and actor within the context (Ash change/request context) you pass to the generated code.
    2. In LiveView or controllers, ensure the context is properly populated according to your application's multitenancy strategy.
    3. Note that while templates might only show actor: by default, you should explicitly add tenant: to generated calls and any custom reads/writes.