paperless-gpt

The OCR system is built on a provider-based abstraction layer, allowing different engines to be swapped in.

Provider Interface

All OCR implementations must follow a common interface that includes:

• Methods for image processing.
• Configuration via a standardized Config struct.
• Standardized resource management.

Supported Provider Types

• LLM Provider: Uses vision models (e.g., OpenAI or Ollama) for OCR. It uses Base64 encoding for OpenAI requests and a binary format for Ollama requests. It relies on template-based OCR prompts.
• Google Document AI Provider: An enterprise-grade option that supports MIME type validation, regional endpoints, and configuration via environment variables.

OCR Processing Pipeline

Document Tagged $\rightarrow$ OCR Provider Selected $\rightarrow$ Image Processing $\rightarrow$ Text Extraction $\rightarrow$ Content Update

The Mistral OCR provider is optimized for text extraction and returns markdown-formatted text that preserves document structure (headers, lists, tables, and columns).

Constraints:

• Maximum file size: 50MB
• Maximum page count: 1,000 pages
• Supported formats: PDF, JPEG, PNG

OCR operations in paperless-gpt are treated as first-class, persisted entities called OCR Runs. Instead of using a manual tag-driven queue, the system uses an interactive Playground for single-document testing and an Activity log to track both manual and unattended Auto-OCR executions.

Each OCR Run record includes:

• Trigger type: Whether it was manual or auto.
• Run options: The specific settings used for that execution.
• Per-page results: Detailed outcomes for each page.
• Status and timing: Execution metadata.
• Searchable-PDF outcome: Whether the PDF was attached, replaced, skipped, or failed.

This model allows users to inspect results page-by-page, tune prompts in the Playground, and verify what Auto-OCR has done (especially when using replace-original) via the Activity log.

The metadata suggestion process follows a specific lifecycle within paperless-ngx:

1. Suggestion Queue: Documents marked with a specific filter tag in paperless-ngx enter this queue.
2. Suggestion Job: An asynchronous process that generates metadata for a batch of documents in the queue.
3. Suggestion: An AI-proposed value for a specific metadata field (e.g., title, tags, correspondent, document type, created date, or custom fields).
4. Review: The user inspects field diffs (comparing current values vs. suggested values) and makes decisions.
5. Apply: Writing accepted changes to paperless-ngx. This action removes the filter tag, moving the document out of the Suggestion Queue. Every applied change is recorded and undoable.
6. Skip: A per-document decision to apply nothing. The document retains its filter tag and remains in the Suggestion Queue.

You can customize how the LLM processes data by modifying template files. The system uses specific template files for different tasks:

• title_prompt.tmpl: For generating document titles.
• tag_prompt.tmpl: For generating tags.
• ocr_prompt.tmpl: For OCR-related processing.
• correspondent_prompt.tmpl: For identifying correspondents.

When these templates are executed, the following variables are available for use within the templates:

• Language
• Content
• AvailableTags
• OriginalTags
• Title
• AvailableCorrespondents
• BlackList

The system follows a microservices architecture designed to work alongside an existing paperless-ngx installation.

• paperless-gpt: An AI processing service written in Go.
• paperless-ngx: The external document management system.
• Communication: The services interact via a REST API.
• Deployment: Typically deployed using Docker containers.

paperless-gpt manages configuration through a structured system to ensure transparency:

• Env Registry: The authoritative source of truth for all environment variables. It defines the name, category, default value, secret status, and meaning of every variable.
• Active Configuration: A read-only diagnostic view used to verify the effective value of every setting. It shows the Source of the value and its meaning. Secrets are masked (shown only as set or not set).
• Source: Indicates where a setting's value originates:
  • env: Set directly in the environment.
  • saved: A value saved via the UI that shadows the environment variable.
  • default: The fallback value defined in the registry.

When using the automatic custom field generation feature, you can choose how the AI interacts with existing data on a document. You must enable this feature in the settings and select at least one custom field for it to function.

Available write modes:

• Append: The safest option. It only adds new fields that do not already exist on the document. It will never overwrite an existing field, even if it is empty.
• Update: Adds new fields and overwrites existing fields with new suggestions. Fields on the document that do not have a new suggestion are left untouched.
• Replace: Deletes all existing custom fields on the document and replaces them entirely with the suggested fields.

WARNING: No Built-in Authentication

paperless-gpt has no built-in authentication. Its web UI and /api/* endpoints are open to anyone who can reach the port. By default, it listens on all interfaces (LISTEN_INTERFACE defaults to :8080).

Risks:

• Anyone reaching the port can rewrite documents in your connected paperless-ngx instance.
• Unauthorized users can trigger LLM/OCR jobs using your API keys.
• Settings can be changed without credentials.

Mitigation Strategies:

• Do not expose it directly to the internet or untrusted networks.
• Use a reverse proxy with authentication (e.g., Authelia, Authentik, or Basic Auth).
• Restrict access to a VPN or Tailscale network.

The E2E tests use TestContainers to spin up a complete, isolated environment on a dedicated Docker network.

Container Stack

1. PostgreSQL: Database for paperless-ngx.
2. Redis: Cache and task queue for paperless-ngx.
3. Paperless-ngx: The document management system.
4. Paperless-gpt: The AI enhancement service (the application under test).

Test Data

• Documents: Located in fixtures/ or ../../tests/pdf/.
• Tags: Uses predefined tags: paperless-gpt, paperless-gpt-ocr-auto, and paperless-gpt-ocr-complete.
• Authentication: Uses admin:admin credentials for API authentication.