DataStax RAGStack

RAGStack AI is a curated stack of open-source software designed to ease the implementation of the Retrieval-Augmented Generation (RAG) pattern in production-ready applications. It is optimized for use with Astra DB or Apache Cassandra as a vector store.

Key features include:

• Compatibility: Uses a tested lineup of open-source versions (like LangChain) to ensure performance, security, and compatibility.
• Ecosystem Integration: Fully compatible with the LangChain ecosystem and LangSmith for monitoring AI deployments.
• Advanced Techniques: Supports advanced RAG patterns such as RAG fusion and FLARE.
• Scalability: Leverages Astra DB for high performance, scalability, and semantic caching.

RAGStack is a comprehensive GenAI stack designed to simplify Retrieval Augmented Generation (RAG) in GenAI applications. It provides a curated and tested combination of open-source tools, including LangChain and CassIO.

Key features include:

• Compatibility: Fully compatible with LangSmith for monitoring AI deployments.
• Vector Store: Leverages Astra DB Serverless (Vector) database for high-performance semantic search, question answering, and semantic caching.
• Reliability: Components are selected for specific version lineups and undergo extensive testing for compatibility, performance, and security to ensure production readiness.

RAGStack Knowledge Store is a proof-of-concept implementation for storing knowledge graphs in Astra DB / Cassandra and retrieving them via graph traversal.

Key features include:

• CassandraGraphStore: A component designed to write LangChain GraphDocuments to Cassandra tables.
• Triple Retrieval: Code to create a runnable for retrieving knowledge triples directly from Cassandra.

Note: This is currently a proof-of-concept. Planned improvements include benchmarking, evaluation, and schema changes.

RAGStack provides specialized support for various technologies and advanced RAG techniques:

• Langflow: Visual orchestration for RAG workflows.
• ColBERT: Late interaction models for improved retrieval.
• Graph Libraries: Support for Knowledge Graphs and Graph Stores.
• Advanced RAG: Techniques for indexing, querying, and evaluating complex RAG systems.

RAGStack provides several Python packages depending on your deployment needs and preferred frameworks:

• ragstack-ai: An all-in-one package containing all supported RAGStack components. Best for convenience, but may be heavier than required.
• ragstack-ai-langchain: Optimized for users integrating with the LangChain framework.
• ragstack-ai-llamaindex: Optimized for users integrating with the LlamaIndex framework.
• ragstack-ai-langflow: Optimized for users integrating with the LangFlow framework.
• ragstack-ai-colbert: Contains the implementation of ColBERT retrieval.

ParentDocumentRAG improves retrieval accuracy by using a two-tier chunking strategy:

1. Parent Chunks: Documents are split into larger chunks (e.g., 512 tokens).
2. Child Chunks: Parent chunks are further split into smaller chunks (e.g., 128 tokens).

Embeddings are generated and stored for the child chunks to ensure precise semantic matching. However, during the retrieval step, each matched child chunk is replaced by its larger parent chunk before being sent to the LLM. This provides the model with significantly more context for its answer.

Pros: Provides richer context to the LLM, often improving answer quality. Cons: Increased token usage and higher costs.

The ragstack-ai-langflow package provides a curated set of dependencies designed to make Langflow compatible with RAGStack.

Key features include:

• Compatibility: Components are tested for performance, security, and compatibility with RAGStack.
• Integrations: Includes all of Langflow's built-in integrations, plus support for Astra DB and all integrations supported by ragstack-ai-langchain.
• Visual Prototyping: Allows you to build complex AI flows (e.g., combining Langchain components, Llama models, Astra vector stores, and Redis caches) using a visual interface.

FLARE is a multi-query RAG technique that uses an iterative loop between the LLM and an AI agent. Instead of a single retrieval step, the process works as follows:

1. The LLM is called with custom instructions to identify key phrases that require more information.
2. The system uses logprobs returned from the LLM to identify uncertain tokens that indicate a gap in knowledge.
3. The agent performs additional retrievals to fill these gaps.
4. The loop terminates once the LLM has sufficient context to generate a final response without gaps.

For a practical implementation, refer to the Langchain example.