ChatGPT Retrieval Plugin

The ChatGPT Retrieval Plugin is a standalone retrieval backend designed for Retrieval-Augmented Generation (RAG). It enables models to perform semantic search and retrieval of personal or organizational documents (files, notes, emails, etc.) by using OpenAI's embeddings models to store document chunks in a vector database.

It can be used with:

• ChatGPT Custom GPTs (via Actions)
• Function Calling with the Chat Completions API
• Assistants API
• ChatGPT Plugins model (deprecated)

Developers can self-host the plugin using Docker on platforms like Fly.io, Heroku, Render, or Azure Container Apps. It is particularly useful when you need more granular control over your retrieval system (e.g., embedding chunk length, specific embedding models, or self-hosting data) than ChatGPT's native file upload feature provides.

The process_jsonl.py script is a utility designed to ingest a JSONL file dump of documents and store them in a vector database along with metadata.

Key Features:

• Batch Upsert: Processes documents as a generator and performs batch upserts to the database.
• PII Screening: Optionally uses a language model to detect and skip documents containing personally identifiable information (PII).
• Automated Metadata Extraction: Optionally uses a language model to extract additional metadata directly from the document text.
• Customization: You can override the PII detection logic in services/pii_detection.py or the metadata extraction logic in services/extract_metadata.py to suit specific requirements.

The input file must be a newline-delimited JSON (JSONL) file where each line is a valid JSON object representing a document.

Supported Fields:

• text (Required): The content of the document.
• id (Optional): The document identifier. If omitted, a random UUID will be generated.
• source (Optional): Metadata field.
• source_id (Optional): Metadata field.
• url (Optional): Metadata field.
• created_at (Optional): Metadata field.
• author (Optional): Metadata field.

All optional fields provided in the JSON object will be used to populate the document's metadata in the vector database.

You can configure the indexing strategy via the LLAMA_INDEX_TYPE environment variable. By default, the system uses GPTVectorStoreIndex, which stores document chunks in memory and retrieves top-k nodes via embedding similarity.

Different index types (such as tree, keyword table, or knowledge graph) are optimized for different data and query use-cases. Refer to the LlamaIndex documentation for the full list of accepted identifiers.

Users should be aware of the following limitations when using the plugin:

• Keyword Search: Standard embeddings might not capture exact keyword matches effectively. For better keyword performance, consider vector databases that support hybrid search (e.g., Elasticsearch, Pinecone, Weaviate, or Azure Cognitive Search).
• Sensitive Data: The plugin does not automatically filter sensitive data; developers are responsible for ensuring data privacy compliance.
• Scalability: Performance depends on the chosen vector database provider and dataset size.
• Metadata Extraction Accuracy: The optional metadata extraction relies on LLMs and may not always be accurate.
• PII Detection Reliability: The PII detection feature is not foolproof and should be used with caution.

The process_zip.py script uses specific service modules for PII screening and metadata extraction. You can modify these functions to suit your specific use case or security requirements:

• PII Detection: Customize the logic in services/pii_detection.py (specifically the screen_text_for_pii function).
• Metadata Extraction: Customize the logic in services/extract_metadata.py (specifically the extract_metadata_from_document function).

The plugin can be integrated into the Chat Completions API or the Assistants API using function calling. This allows the model to intelligently decide when to call query, fetch, or upsert based on the conversation context.

• Chat Completions API: Define the plugin's endpoints as tools. Models like gpt-3.5-turbo-0125 and gpt-4-turbo-preview are trained to detect when a function should be called and respond with the appropriate JSON arguments.
• Assistants API: Use the same function definitions as tools. This allows you to build custom AI assistants that leverage your specific vector database.
• Parallel Function Calling: Both APIs support parallel function calling, meaning a model can perform multiple tasks (e.g., querying a document and then saving a new snippet via upsert) in a single message.

The Retrieval Plugin can be used with the Assistants API via function calling in tool use.

When to use this instead of native Assistants retrieval: While the Assistants API has native file retrieval, you should use the Retrieval Plugin with function calling if you require granular control over your retrieval system, such as:

• Specifying embedding chunk lengths.
• Choosing specific embedding models or sizes.
• Customizing the retrieval logic/parameters.

To reduce app size and improve performance before deploying, remove unused vector database provider packages from your pyproject.toml file. You can find specific guidance on which packages to remove for each provider in the removing-unused-dependencies.md guide.

Note that you do not need to run poetry lock or poetry install manually after editing pyproject.toml; the provided Dockerfile uses a requirements.txt file generated via poetry export to install only the necessary dependencies.