DOCSEARCHbyAlgolia
Submit Docs

AnythingLLM

repository·master·Indexed 13 days ago

https://github.com/mintplex-labs/anything-llm

An all-in-one, private AI application for turning documents into chatbots using RAG, automating workflows with AI agents, and managing multiple workspaces. Version 1.15.0 supports local and cloud-based LLMs, embedding models, and vector databases. Includes deployment guides for Kubernetes via Helm and Red Hat OpenShift, as well as an auto-translator tool for locale management using Ollama.

Tokens
55.9K
Snippets
182
Records
253
Agent score
98%

What's inside AnythingLLM

  1. What is AnythingLLM?

    master
    AnythingLLM is an all-in-one AI application designed to build private, fully-featured ChatGPT-like environments. It allows users to connect local or cloud LLMs, ingest documents for RAG (Retrieval-Augmented Generation), and utilize built-in AI agents. It is designed to be hyper-configurable, multi-user ready, and runs locally by default with minimal setup friction.

  2. Overview of AnythingLLM

    master

    AnythingLLM is a full-stack AI application designed to transform any document, resource (URLs, audio, video), or text snippet into context for Large Language Models (LLMs) to use during chat sessions. It is highly configurable and supports multi-user management with granular permissions.

    Key capabilities include:

    • Contextual Chat: Chat with your documents using various LLMs and vector databases.
    • AI Agents: Built-in no-code agent builder and workspace-level agents capable of web browsing and code execution.
    • MCP Compatibility: Fully compatible with the Model Context Protocol (MCP).
    • Multi-user Support: Manage multiple users and access permissions (available in the Docker version).
    • Customizable Embeddings: Support for various document types (PDF, TXT, DOCX, etc.) with a streamlined ingestion pipeline.
    • Embeddable Chat: Ability to create custom embeddable chat windows for websites.

  3. Explore AnythingLLM ecosystem products

    master

    Beyond the core AnythingLLM application, Mintplex Labs provides several specialized products to extend its functionality across different platforms:

    • AnythingLLM Mobile: A mobile application for using AnythingLLM on mobile devices.
    • AnythingLLM Browser Extension: A browser extension to integrate AnythingLLM directly into your web browsing experience.
    • AnythingLLM Embed: A widget designed to allow you to embed AnythingLLM capabilities directly into your own websites.

  4. Use native audio/video transcription

    master

    AnythingLLM can transcribe audio tracks from uploaded audio and video source documents using a locally running ONNX whisper-small model (built by Xenova).

    Because this model runs locally on the CPU, transcription times will increase with larger file sizes. Once transcription is complete, the text can be embedded into your workspace like any other document. Alternatively, you can use external providers like OpenAI Whisper via API key.

  5. Format of documents in the documents cache folder

    master

    The server/storage/documents/ directory acts as a temporary cache for files collected by the collector/.

    Important Guidelines:

    • Do not add files manually to this folder; use the frontend application to manage files instead.
    • Data Partitioning: Data should be partitioned based on its collection method. This ensures files are added to the correct namespace during the vectorization process.
    • File Format: All files must be in .json format.
    • Schema Requirements:
      • pageContent (Required): The main text content of the document.
      • Metadata: Any additional keys included in the JSON object will be treated as metadata for the document when it is inserted into the vector database.
      • published (Reserved): This key is reserved for storing timestamps.
    {
  "pageContent": "The actual text content to be vectorized",
  "metadata_key": "metadata_value",
  "published": "2026-08-10T00:00:00Z"
}

  6. Use an existing table as a vector database

    master

    You can use an existing PostgreSQL table as a vector database, but it must conform to the expected schema.

    Critical Requirement: The embedding column's VECTOR(XXXX) dimensions must match the dimensions of the embedder configured in AnythingLLM.

    • The default embedding model uses 384 dimensions.
    • If using a custom embedder, ensure the VECTOR dimension in your table matches that model's output exactly.

  8. Understand Data Sovereignty and Air-Gapped Operation

    master

    AnythingLLM follows a local-first architecture. For self-hosted deployments (Docker, Desktop, or Source):

    • Data Isolation: Mintplex Labs Inc. has no access to your documents, chat histories, workspace settings, or embeddings. All data is stored on your own provisioned infrastructure.
    • Air-Gapping: You can operate AnythingLLM in a strictly air-gapped environment with no internet connectivity. To achieve this, you must use local providers for LLMs and Vector databases, such as:
      • Ollama
      • LocalAI
      • LanceDB
    • Model Downloads: By default, the application may attempt to download critical path models (e.g., default embedder and reranking ONNX models) from a hosted CDN as a fallback. For air-gapped installations, you must download these models manually or configure an alternative provider.

  9. Understand the AnythingLLM Storage structure

    master

    The AnythingLLM storage directory is used for local disk storage of ready-to-embed documents, vector-cached embeddings, the LanceDB instance (if enabled), and the local SQLite database.

    A healthy storage directory should contain the following subdirectories and files:

    • documents
    • lancedb (only if using LanceDB)
    • vector-cache
    • anythingllm.db (the SQLite database file)

  10. Connect to services running on the host machine from Docker

    master

    When running AnythingLLM inside a Docker container, you cannot use localhost or 127.0.0.1 to connect to services (like Ollama, Chroma, or LMStudio) running on your host machine. You must use a special hostname to route traffic to the host.

    Connection Rules

    • Windows/macOS: Use http://host.docker.internal:xxxx (where xxxx is the service port).
    • Linux:
      1. Add --add-host=host.docker.internal:host-gateway to your docker run command.
      2. Use http://host.docker.internal:xxxx in the application.
      3. If host.docker.internal fails, use http://172.17.0.1:xxxx instead.

    Example: If Ollama is running on your host at http://127.0.0.1:11434, enter http://host.docker.internal:11434 in the AnythingLLM connection settings.

  11. Core features of AnythingLLM

    master

    AnythingLLM provides several advanced capabilities for AI interaction and automation:

    • Dynamic Model Routing: Automatically route chats to the best provider and model based on user-defined rules.
    • Memories: Supports both automatic and user-managed memories to help LLMs retain important information about users or specific workspaces.
    • Scheduled Tasks: Run recurring prompts or tasks on a cron schedule using agent capabilities.
    • Intelligent Skill Selection: Reduces token usage (up to 80%) by enabling unlimited tools while selecting only the necessary ones per query.
    • No-code AI Agent Builder: Create agentic workflows without writing code.
    • MCP-compatibility: Supports the Model Context Protocol.
    • Multi-modal Support: Works with both closed and open-source LLMs for multi-modal tasks.
    • Multi-user Support: (Docker version only) Provides instance-wide permissioning and user management.
    • Custom Embeddable Chat Widget: (Docker version only) Allows embedding a chat interface into external websites.

  12. Understand AnythingLLM Telemetry and Privacy

    master

    AnythingLLM uses PostHog to collect anonymous usage data. The goal is to understand feature usage and prioritize improvements without compromising user privacy.

    Collected Data Points

    The following anonymous details may be collected:

    • Installation Method: Whether using Docker or the Desktop version.
    • Document Activity: Timestamps of when documents are added or removed (the actual content of documents is never collected).
    • Vector Database Type: To identify popular databases for future support.
    • LLM Type: To identify popular models for future support.
    • Event Triggers: General 'send' events to track daily active users.

    Privacy Guarantees

    • No Content Collection: Chat content and document contents are never tracked.
    • No PII: IP addresses and other personally identifiable information (PII) are not collected.
    • No Third-Party Sharing: Anonymous data is never shared with third parties.