DOCSEARCHbyAlgolia
Submit Docs

Paperless-AI

repository·main·Indexed 27 days ago

https://github.com/clusterzx/paperless-ai

An AI-powered extension for Paperless-ngx that provides automatic document classification, smart tagging, and semantic search using Retrieval-Augmented Generation (RAG). It supports a wide range of OpenAI-compatible APIs and local LLM providers such as Ollama, OpenAI, DeepSeek.ai, and Gemini. Key features include automated assignment of titles and correspondents, a RAG-based AI chat for natural language queries against document archives, and a web interface for manual AI tagging.

Tokens
7K
Snippets
12
Records
46
Agent score
88%

What's inside paperless-ai

  1. Overview of Paperless-AI

    main

    Paperless-AI is an AI-powered extension for Paperless-ngx that provides automatic document classification, smart tagging, and semantic search. It uses Retrieval-Augmented Generation (RAG) to allow users to perform natural language queries against their document archive (e.g., "What was the amount of the last electricity bill?").

    Key capabilities include:

    • Automated Processing: Automatically detects new documents and assigns titles, tags, document types, and correspondents.
    • RAG-Based AI Chat: Semantic search and Q&A using document context.
    • Manual Processing: A web interface for manual AI tagging via the /manual endpoint.
    • Smart Tagging & Rules: Ability to define rules to filter which documents are processed and apply tags automatically.

  2. Run Python RAG Service separately

    main

    If you prefer to run the RAG service independently of the main application, follow these steps:

    1. Install Python dependencies: pip install -r requirements.txt.
    2. Start the service using main.py. Use the --initialize flag to build the document index on startup.

    Command:

    python main.py --host 127.0.0.1 --port 8000 --initialize
    python main.py --host 127.0.0.1 --port 8000 --initialize

  3. Run Paperless-AI and RAG Service together (Recommended)

    main

    To run the full development environment locally without Docker, follow these steps to install dependencies, configure credentials, and launch both the Node.js application and the Python RAG service simultaneously.

    1. Install Dependencies

    Install both Node.js and Python requirements:

    npm install
pip install -r requirements.txt

    2. Configure Credentials

    Create or update a .env file in the data directory with your Paperless-NGX credentials:

    PAPERLESS_API_URL=https://your-paperless-ngx-instance
PAPERLESS_API_TOKEN=your-api-token

    3. Launch Services

    On Linux/macOS, make the startup script executable and run it:

    chmod +x start-services.sh
./start-services.sh
    # Install Node.js dependencies
npm install

# Install Python dependencies
pip install -r requirements.txt

# Make the script executable first (Linux/macOS)
chmod +x start-services.sh

# Run the services
./start-services.sh

  4. Configure Paperless-AI Node.js application for RAG

    main

    To connect the Paperless-AI Node.js application to a running RAG service, you must set the RAG_SERVICE_URL and RAG_SERVICE_ENABLED environment variables before starting the application with npm run dev.

    Linux/macOS

    export RAG_SERVICE_URL=http://localhost:8000
export RAG_SERVICE_ENABLED=true
npm run dev

    Windows (PowerShell)

    $env:RAG_SERVICE_URL="http://localhost:8000"
$env:RAG_SERVICE_ENABLED="true"
npm run dev

    Windows (Command Prompt)

    set RAG_SERVICE_URL=http://localhost:8000
set RAG_SERVICE_ENABLED=true
npm run dev
    export RAG_SERVICE_URL=http://localhost:8000
export RAG_SERVICE_ENABLED=true
npm run dev

  5. Document API routes using JSDoc/Swagger

    main

    All API routes in Paperless-AI must be documented using a JSDoc comment block with the @swagger tag, following the OpenAPI 3.0.0 specification. This block must be placed immediately before the route handler function.

    Key requirements:

    • The route path in the documentation must match the Express route handler exactly.
    • Use curly braces for path parameters: /path/{paramName}.
    • Define one HTTP method per documentation block; document multiple methods for the same path separately.
    • Use a single-line summary and a multi-line description (using the | pipe symbol).
    /**
 * @swagger
 * /path/to/endpoint:
 *   get:
 *     summary: Brief description of what this endpoint does
 *     description: |
 *       Detailed explanation of the endpoint functionality.
 *       This should cover what the endpoint does, how it works,
 *       and any important behaviors users should know about.
 */
router.get('/path/to/endpoint', async (req, res) => {

  6. Set up Local Development Environment

    main

    To contribute to or develop Paperless-AI locally, follow these steps to install dependencies and run the test suite:

    1. Install dependencies using npm.
    2. Run the development/test mode.
    # Install dependencies
npm install

# Start development/test mode
npm run test

  7. Install Paperless-AI

    main

    Paperless-AI supports Docker installation for easy deployment with health monitoring, auto-restart, and persistent volumes.

    Important Note for First-time Installation: After completing the initial setup (configuring API keys and preferences), you must restart the container to allow the system to build the RAG (Retrieval-Augmented Generation) index. This restart is not required for subsequent updates.

  8. Configure Paperless-AI via Docker Compose

    main

    You can deploy Paperless-AI using Docker Compose. The service uses the clusterzx/paperless-ai image and requires specific environment variables for port configuration and RAG (Retrieval-Augmented Generation) service integration.

    By default, the application listens on port 3000. Data is persisted in a named volume paperless-ai_data mapped to /app/data inside the container.

    services:
  paperless-ai:
    image: clusterzx/paperless-ai
    container_name: paperless-ai
    network_mode: bridge
    restart: unless-stopped
    cap_drop:
      - ALL
    security_opt:
      - no-new-privileges=true
    environment:
      - PUID=1000
      - PGID=1000
      - PAPERLESS_AI_PORT=${PAPERLESS_AI_PORT:-3000}
      - RAG_SERVICE_URL=http://localhost:8000
      - RAG_SERVICE_ENABLED=true
    ports:
      - "3000:${PAPERLESS_AI_PORT:-3000}"
    volumes:
      - paperless-ai_data:/app/data

volumes:
  paperless-ai_data:

  9. Configure Paperless-NGX API via .env

    main

    The system requires Paperless-NGX credentials to fetch documents. If the .env file is missing from the data/ directory, the system will create an example one.

    Required Environment Variables:

    • PAPERLESS_URL: The base URL of your Paperless-NGX instance.
    • PAPERLESS_API_TOKEN: Your Paperless-NGX API token.

    File Location: data/.env

    # Example .env content
PAPERLESS_URL=https://your-paperless-instance
PAPERLESS_API_TOKEN=your-api-token

  10. Configure Paperless-AI environment variables

    main

    Paperless-AI requires specific environment variables to connect to your Paperless-NGX instance. It looks for these in a .env file located in the data/ directory or the local directory.

    Required variables:

    • PAPERLESS_API_URL (or PAPERLESS_URL, PAPERLESS_NGX_URL, or PAPERLESS_HOST): The base URL of your Paperless instance.
    • PAPERLESS_TOKEN (or PAPERLESS_API_TOKEN, or PAPERLESS_APIKEY): Your Paperless API token.

    Note: If the URL ends with /api, the system will automatically strip it to ensure correct path construction.

  11. Troubleshoot RAG Service issues

    main

    Missing Documents

    If documents are not appearing in searches, verify that the indexing process has completed. You can check the current status at: http://localhost:8000/indexing/status

    Connection Errors

    • Verify that your Paperless-NGX credentials (PAPERLESS_API_URL and PAPERLESS_API_TOKEN) are correct.
    • Ensure the Paperless-NGX instance is reachable from your development environment.

    Port Conflicts

    If port 8000 is already in use:

    1. Start the Python service with a different port using the --port flag.
    2. Update your RAG_SERVICE_URL environment variable in the Node.js application to match the new port.

  12. Configure PM2 deployment for paperless-ai

    main

    The project uses an ecosystem.config.js file for managing the application process via PM2. This configuration defines how the server.js script is executed, its restart behavior, and its environment settings.

    module.exports = {
  apps: [{
    name: 'paperless-ai',
    script: 'server.js',
    instances: 1,
    autorestart: true,
    watch: false,
    max_memory_restart: '1G',
    env: {
      NODE_ENV: 'production'
    },
    exp_backoff_restart_delay: 100
  }]
};