Elixir LangChain

Elixir LangChain supports a wide variety of chat models and providers, including:

• Anthropic Claude: Includes extended thinking support and AWS Bedrock support.
• AWS Bedrock Mantle: OpenAI-compatible gateway for Bedrock-hosted models.
• OpenAI: Via Chat Completions API and the newer Responses API (with WebSocket support).
• Cloudflare Workers AI: Via ChatOpenAI (OpenAI-compatible).
• xAI Grok: Grok-4, Grok-3-mini, Grok-4 Heavy, etc.
• Google: Gemini AI models and Vertex AI.
• DeepSeek: Supports prompt caching.
• Mistral: Mistral AI models.
• Perplexity: Perplexity AI models.
• Ollama: Locally hosted open-source models.
• Bumblebee: Self-hosted models via Nx (Llama, Mistral, Zephyr).
• orq.ai: Deployments API.
• ReqLLM: Multi-provider adapter (Anthropic, OpenAI, Gemini, Grok, Ollama, AWS Bedrock, etc.).

LangChain is a framework for building data-aware and agentic applications. It is built on two primary abstractions:

1. Components: Modular abstractions for working with language models (and other integrations). These are easy to use individually or as part of the larger framework.
2. Off-the-shelf chains: Structured assemblies of components designed to accomplish specific high-level tasks.

Developers can use pre-built chains for quick starts or compose custom chains using individual components for more complex use cases.

When building agent systems, you can use LangChain.Trajectory to evaluate the reasoning process (the sequence of tool calls) rather than just the final answer. A trajectory captures the structured sequence of tool calls produced during an LLMChain run, which is useful for regression testing, cost control, and debugging.

To capture a trajectory, run your chain and then call Trajectory.from_chain(chain).

alias LangChain.Trajectory

{:ok, chain} =
  LLMChain.new!(%{llm: llm})
  |> LLMChain.add_tools(my_tools)
  |> LLMChain.add_message(Message.new_user!("What's the weather in Paris?"))
  |> LLMChain.run(mode: :while_needs_response)

trajectory = Trajectory.from_chain(chain)
trajectory.tool_calls
#=> [%{name: "search", arguments: %{"query" => "weather paris"}},
#    %{name: "get_forecast", arguments: %{"city" => "Paris"}}]

LangChain provides two independent layers for monitoring LLM applications:

1. LangChain.Telemetry: A vendor-neutral layer that emits standard :telemetry events for every LLM call, chain execution, and tool call. This has no extra dependencies and can be used with any :telemetry handler (e.g., Logger, PromEx, Telemetry.Metrics).

2. LangChain.OpenTelemetry: An optional integration that translates telemetry events into OpenTelemetry spans and metrics. This follows a subset of the GenAI Semantic Conventions (v1.40+) and is intended for distributed tracing backends like Langfuse, Honeycomb, Grafana Tempo, or Jaeger.

Note that the OpenTelemetry layer is built on top of the Telemetry layer; the core events apply to both.

Pre-filling an assistant's response (starting an assistant message with a specific prefix to guide the model's output) behaves differently depending on the provider:

• Anthropic Claude 3: Responds well to pre-filled assistant responses and this pattern is officially encouraged.
• OpenAI (ChatGPT 3.5 and 4): Does not reliably complete pre-filled responses. For example, if you instruct the model to use an <answer>{{ANSWER}}</answer> template and start the assistant message with <answer>, it may fail to include the closing </answer> tag.

To ensure your OpenTelemetry spans correctly identify which agent is running, provide the agent's name and ID in your custom_context.

If you do not provide these, the gen_ai.agent.name attribute will fall back to the generic chain type (e.g., "llm_chain"), which prevents effective grouping of traces by specific agents.

# Example of setting agent identity in custom_context
custom_context = %{
  agent_name: "customer_support_agent",
  agent_id: "agent_123"
}

Token usage reporting varies across providers:

• GoogleAI: Returns token usage for each MessageDelta. Note that GoogleAI generates incremental values with each message.
• OpenAI ChatGPT, Anthropic Claude, and Bumblebee: Return token usage information only at the end of the interaction.

The central module for interacting with LLMs in this library is LangChain.Chains.LLMChain. Most other components (chat models, messages, tools) are designed to be passed into an LLMChain to orchestrate the conversation flow. To use it, initialize a chain with an LLM, add messages, and call run().

# Conceptual pattern
{:ok, chain} = 
  LLMChain.new!(%{llm: chat_model})
  |> LLMChain.add_message(Message.new_user!("Hello"))
  |> LLMChain.run()

When streaming tool calls, the library automatically provides a display_text field for each ToolCall. This allows the UI to show human-friendly labels (e.g., "Reading file" instead of file_read) immediately as the tool is identified.

Resolution Logic:

1. If Function.display_text is defined on the tool, it is used.
2. Otherwise, Utils.humanize_tool_name/1 is used (e.g., "file_read" becomes "File read").

Streaming Behavior:

• The :on_llm_new_delta callback is enriched with display_text via rewrap_callbacks_for_model. This ensures the UI can show tool names the moment they appear in the stream.
• The post-streaming path (apply_deltas) also independently sets display_text as an idempotent guard.