AIChat

AIChat supports two primary interaction modes:

1. CMD Mode: Direct command-line usage for quick queries or piping data. Example: aichat hello or cat data.txt | aichat.

2. REPL Mode: An interactive chat environment featuring tab autocompletion, multi-line input, history search, and configurable keybindings.

You can install AIChat using various package managers depending on your environment:

• Rust (Cargo): cargo install aichat
• Homebrew/Linuxbrew: brew install aichat
• Pacman: pacman -S aichat
• Scoop (Windows): scoop install aichat
• Termux (Android): pkg install aichat

Alternatively, download pre-built binaries for macOS, Linux, and Windows from the GitHub Releases page and add the binary to your $PATH.

AIChat can act as a lightweight local server providing Chat Completions, Embeddings, and Rerank APIs. It also hosts a web-based LLM Playground and LLM Arena for side-by-side model comparison.

Start the server using: aichat --serve

Available endpoints at http://127.0.0.1:8000/:

• /v1/chat/completions
• /v1/embeddings
• /v1/rerank
• /playground
• /arena

AIChat accepts multiple input forms in both CMD and REPL modes. Use the -f flag in CMD mode or the .file command in REPL mode.

aichat -f dir/ -f data.txt explain

AIChat operates in three primary modes based on the provided arguments:

1. CMD Mode: Used when providing text or files directly. It executes a single prompt and exits.
2. REPL Mode: Triggered when no text or files are provided. It starts an interactive chat session.
3. Serve Mode: Triggered by the --serve <ADDR> flag. It starts a server at the specified address.

The --execute flag enables Shell Assistant mode. AIChat generates a shell command based on your prompt, then provides an interactive menu to:

• e (execute): Run the generated command in your shell.
• r (revise): Enter a revision to regenerate the command.
• d (describe): Use a specific role to explain the generated command.
• c (copy): Copy the command to the clipboard.
• q (quit): Exit the assistant.

AIChat accepts input in two ways:

1. Direct text arguments: Pass text directly after the flags.
2. Standard Input (stdin): Pipe text into the command. If the input is not a terminal (e.g., from a pipe), AIChat will read the entire stdin content.

If both direct text and stdin are provided, AIChat combines them. When using --macro, it joins them using a -- separator.

Once the AIChat server is running via aichat --serve, you can test the Chat Completions endpoint using curl.

curl -X POST -H "Content-Type: application/json" -d '{
  "model":"claude:claude-3-5-sonnet-20240620",
  "messages":[{"role":"user","content":"hello"}], 
  "stream":true
}' http://127.0.0.1:8000/v1/chat/completions

The eval_tool_calls(config: &GlobalConfig, mut calls: Vec<ToolCall>) function processes a batch of tool calls.

It includes safety mechanisms:

• Deduplication: It calls ToolCall::dedup to remove redundant calls (based on id).
• Infinite Loop Protection: If deduplication results in an empty list (indicating the same calls are being repeated), it returns an error: "The request was aborted because an infinite loop of function calls was detected."
• Null Handling: If a tool returns a null result, it is treated as "DONE" internally, but if all calls in a batch return null, the resulting output vector is empty.

The loader utility provides several ways to ingest content:

• load_file: Loads a local file. It attempts to find a specific loader command based on the file extension; if none is found, it falls back to reading the file as plain text.
• load_url: Fetches content from a URL. It uses available loaders to process the response and attaches the extension to the __extension__ metadata key.
• load_recursive_url: Performs a recursive crawl of a website. If a specific recursive loader is configured, it uses that; otherwise, it uses a built-in crawler. The output is a list of LoadedDocument objects, each with the __extension__ metadata set to md.

The ToolCall::eval(&self, config: &GlobalConfig) method executes a function call requested by an LLM.

It performs the following:

1. Resolves the command name and arguments based on whether the function is defined in an Agent or the global Config.
2. Parses the arguments (which can be a JSON object or a JSON string).
3. Runs the command as a system process using run_llm_function.
4. Captures the output. If the output is not valid JSON, it wraps the raw string in a JSON object: {"output": "<contents>"}.

Note: The execution environment is augmented with a specific PATH containing function binary directories and an LLM_OUTPUT environment variable pointing to a temporary file where the tool is expected to write its results.