DOCSEARCHbyAlgolia
Submit Docs

Local OpenAI LLM for Home Assistant

repository·master·Indexed 42 days ago

https://github.com/skye-harris/hass_local_openai_llm

A custom Home Assistant integration that connects to any OpenAI-compatible LLM API, including local providers like Ollama, llama.cpp, LocalAI, and vLLM, or cloud services like DeepSeek and Gemini. Features include streaming responses, response cleaning, advanced AI task support, and experimental Retrieval Augmented Generation (RAG) using a local Weaviate vector database.

Tokens
3.2K
Snippets
4
Records
16
Agent score
5%

What's inside Local OpenAI LLM

  1. Overview of Local OpenAI LLM Integration

    master

    The Local OpenAI LLM integration allows Home Assistant to communicate with any OpenAI-compatible LLM service. This includes cloud providers and local hosting solutions.

    Supported Services include:

    • DeepSeek
    • Google (Gemini)
    • llama.cpp
    • LM Studio
    • LocalAI
    • Ollama
    • OpenRouter
    • Requesty
    • Scaleway
    • vLLM

    Key Features:

    • Streaming: Supports streamed LLM responses and TTS streaming for Conversation Agents.
    • Response Cleaning: Automatically strips <think> tags and provides an option to strip emojis.
    • Advanced AI Task Support: Supports image inputs, image generation, and tools for 'Generate Data' actions.
    • Context Management: Includes options to trim conversation history to manage context windows.
    • Enhanced Capabilities: Supports parallel tool calling, chat template arguments, and experimental Retrieval Augmented Generation (RAG).

  2. Configure Chat Template Arguments and Request Body Parameters

    master

    You can pass custom parameters to your LLM requests using two distinct methods:

    Chat Template Arguments

    Supplies key/value pairs to the chat_template_kwargs request parameter. This is used to provide custom arguments to your model's chat template.

    • Values: Support Jinja2 templates for complex data structures.
    • Note: Arguments are model-specific; check your model's documentation for available keys.

    Request Body Parameters

    Supplies key/value pairs that are merged directly into the top-level chat completion request body.

    • Values: Support Jinja2 templates.
    • Note: You cannot configure internally managed parameters or parameters that have dedicated configuration options in the UI.

  3. How Retrieval Augmented Generation (RAG) with Weaviate works

    master

    Retrieval Augmented Generation (RAG) allows you to pre-feed your LLM with contextually relevant data based on the user's current input. This integration connects your Home Assistant Agent to a Weaviate vector database.

    How it functions:

    1. When a user sends a message, the integration queries the Weaviate database using that message.
    2. Relevant data found in the database is injected into the current conversation as context.
    3. The LLM uses this context to generate a more informed response.

    Key Constraints:

    • Not general memory: Data is only provided if it matches the current user input. It does not act as a long-term conversational memory.
    • Turn-based context: Search results are used for the current user/assistant turn (including multi-turn tool usages) but are not carried forward to subsequent turns.
    • Data Structure: Objects in Weaviate are stored as two parts:
      • query: The text used for vectorization and searching.
      • content: The actual information fed to the LLM (includes the query text for context).
    • Scope: Only the current generation's user message is queried; prior messages are not included in the search.

  4. Manage Weaviate data with the included WebApp

    master

    Since self-hosted Weaviate lacks a built-in front-end, this repository includes a NodeJS-based WebApp located in the /weaviate directory. This tool is purpose-built for this integration to view and manage your object classes.

    Setup:

    • The WebApp is included in the provided docker-compose.yml.
    • It is exposed on port 9090 by default.

    Capabilities:

    • Connect to your server and list available object classes.
    • View entry data within classes.
    • Add new entry data to a class.
    • Perform vector and hybrid searches.

  5. Configure the Local OpenAI LLM integration

    master

    After installing the integration, you must configure it via the Home Assistant UI:

    1. Navigate to Settings → Devices & Services.
    2. Click Add Integration.
    3. Search for Local OpenAI LLM.
    4. Follow the setup wizard to provide your server details.

    Key Configuration Requirements:

    • Server URL: Must be a fully qualified URL pointing to an OpenAI-compatible API (e.g., http://<ip>:port/v1).
    • Server Type: Selecting a specific server type (e.g., llama.cpp, vLLM, DeepSeek Cloud) exposes specialized configuration options for that provider.
    • Context Size: For Home Assistant Assist to function correctly with tools and entities, it is strongly recommended to use a context size of at least 10k. This must be configured on your inference server directly, as it is not configurable via the OpenAI-compatible API.

  6. Install Local OpenAI LLM via HACS

    master

    The recommended way to install this integration is via HACS (Home Assistant Community Store). This ensures you can easily receive updates.

    1. Open HACS in your Home Assistant instance.
    2. Search for Local OpenAI LLM and click Install.
    3. Restart Home Assistant.

    If you cannot find the integration in the HACS search, add it manually as a custom repository:

    • Repository URL: https://github.com/skye-harris/hass_local_openai_llm
    • Category: Integration
    https://github.com/skye-harris/hass_local_openai_llm

  7. Configure Weaviate for RAG

    master

    To use RAG, you must set up a local Weaviate instance and configure it within Home Assistant.

    1. Install Weaviate

    It is recommended to install Weaviate locally. A pre-made docker-compose.yml is available in the weaviate directory of this repository for easy setup. Note: Weaviate Cloud is not supported.

    2. Configure the LLM Server Entity

    In Home Assistant, go to your LLM Server entity configuration (not the Agent entity) and provide:

    • Server Address: The address of your Weaviate instance.
    • API Key: Use homeassistant if you are using the provided docker-compose.yml.

    3. (Optional) Configure AI Agent Entities

    You can override default Weaviate settings on a per-agent basis in the AI Agent entity configuration:

    • Object class name: Defaults to Homeassistant. Change this to use a different data store for specific agents.
    • Maximum number of results to use: Defaults to 2.
    • Result score threshold: Defaults to 0.9.
    • Hybrid search alpha: Defaults to 0.5. This balances hybrid scoring between 0 (fully text-matched) and 1 (fully vectorized) matching.

  9. Configure DeepSeek Cloud reasoning effort

    master

    When the Server Type is set to DeepSeek Cloud, you can control the model's thinking behavior via the Reasoning Effort option:

    • Disabled (default): No thinking tokens are generated.
    • High: Enables thinking with standard reasoning effort.
    • Max: Enables thinking with maximum reasoning effort.

    Note: On Home Assistant versions 2026.4+, thinking content returned by the model is fed back into the conversation as reasoning content.

  10. Configure vLLM thinking token budget

    master

    When using the vLLM server type, you can cap the number of tokens spent on reasoning using the thinking_token_budget parameter.

    • Empty (default): Uses the server's default budget.
    • 0: Closes the reasoning block immediately (model answers without thinking).
    • Any higher value: Limits reasoning to that specific number of tokens.

    Note: This only limits length; it does not enable/disable thinking. Enabling/disabling is controlled by the model's chat template (e.g., via enable_thinking Chat Template Argument).

  11. Configure llama.cpp server settings

    master

    When using the llama.cpp server type, the following options are available:

    • Enable thinking: Passes enable_thinking=true via chat_template_kwargs. This overrides any manual enable_thinking setting in Chat Template Arguments.
    • Include prior thinking: Controls if reasoning content from previous turns is sent in new requests.
      • Enabled (default): Passes thinking_content as reasoning_content in the next request.
      • Disabled: Strips prior thinking context (use this for models like Gemma 4 that reject prior reasoning context).
    • Slot ID: Pins requests to a specific llama.cpp server slot for prompt-cache reuse. Leave empty to use any available slot.
    • Use loaded model: Prioritizes models already in memory to minimize latency. If the configured model isn't loaded, it picks the first loaded model supporting the required modality (e.g., images).

  12. Experimental: Inject Date/Time context into Conversation Agent

    master

    To ground the model with temporal context, you can inject the current date and time into the prompt using one of the following experimental methods:

    • Tool Result: Inserts date/time as a Tool Call Result message before the user message. This is the recommended method for reliability.
    • Assistant: Inserts date/time as an Assistant message before the user message. Use if Tool Result fails.
    • User: Inserts date/time as a User message. Use only as a last resort, as some models may repeat the date/time back to the user.
    • Disabled: Removes the injection entirely.