repository·main·Indexed 25 days ago
https://github.com/devflowinc/trieveAn all-in-one search and Retrieval-Augmented Generation (RAG) platform providing semantic vector search, hybrid search, and recommendation engines. Trieve supports managed cloud usage and self-hosting. The ecosystem includes a CLI for dataset management and RAG configuration, a Docusaurus search theme (@trieve/docusaurus-search-theme), an MCP server for AI agents (trieve-mcp-server), and n8n nodes for workflow integration.
The Trieve Web Pixel Extension is a Shopify app extension designed to manage and process behavioral data within a secure sandbox environment. It allows developers to subscribe to customer events without requiring users to manually add tracking code.
Key benefits include:
Trieve is an all-in-one solution for search, recommendations, and RAG (Retrieval-Augmented Generation). Key capabilities include:
Trieve implements a lightweight hallucination detection system designed for RAG (Retrieval-Augmented Generation) workflows. Instead of using a slow 'LLM-as-a-judge' approach, it utilizes Named Entity Recognition (NER) to identify and compare critical information between the generated AI completion and the retrieved reference text.
Key focus areas for detection:
Benefits:
Trieve uses a usage-based pricing model designed to support scaling without the friction of large, restrictive tiers. The model includes a base platform fee and charges based on specific resource consumption such as storage, ingestion, and search operations.
| Product | Free Tier | Cost |
|---|---|---|
| Users | First 5 Users free | $5 / User |
| Platform Fee | N/A | $5 / mo |
| Chunk Storage | 1000 Chunks (11 MB) | $132 / 1M chunks ($12.07 / GB) |
| File Storage | 10 GB | $0.046 / GB |
| Datasets | 2 datasets | $0.05 / dataset |
| Write Tokens | First 3M tokens / mo free | $0.028 / 1M tokens |
| Search Tokens | First 3M tokens / mo free | $0.028 / 1M tokens |
| File OCR | First 100 / mo free | $0.01 / Page |
| Web Crawling | First 10 pages / mo free | $0.00086 / Page Crawled |
| Bytes Ingested | First 1 GB | $2 / GB ingested |
| Message Tokens | First 263,000 tokens / mo free | $3.528 / 1M tokens |
| Analytic Events | First 1M events / mo free | $0.0001 / event |
Trieve's typo correction system is built for high performance using Rust, BKTrees, Redis, and ClickHouse. The pipeline consists of several distributed workers:
word-id-cronjob scrolls through search indices and adds chunk IDs to a Redis queue.word-worker processes these chunks, splits text into words, and ingests them into ClickHouse.bktree-worker pulls the completed dictionary from ClickHouse and constructs a Burkhard-Keller Tree (BKTree).lazy_static! to avoid the 300μs+ Redis latency on subsequent searches.Depending on your operating system, install the following dependencies to set up a local development environment:
sudo apt install curl gcc g++ make pkg-config python3 python3-pip libpq-dev libssl-dev opensslsudo pacman -S base-devel postgresql-libs# Install Xcode command line tools
xcode-select --install
# Install Homebrew if not already installed
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
# Install required packages
brew install pkg-config opensslTo prevent incorrect corrections of legitimate English words that might be missing from a specific dataset's dictionary, use a multi-step identification process:
HashSet of ~400,000 English words.PREFIX_TRIE and SUFFIX_TRIE to identify common prefixes (e.g., 'anti', 'pre') or suffixes (e.g., 'able', 'ing'). Strip them and check if the remaining root is in the English corpus.fn is_likely_english_word(word: &str) -> bool {
if ENGLISH_WORDS.contains(&word.to_lowercase()) {
return true;
}
// Check for prefix
if let Some(prefix_len) = PREFIX_TRIE.longest_prefix(word) {
if ENGLISH_WORDS.contains(&word[prefix_len..].to_lowercase()) {
return true;
}
}
// Check for suffix
if let Some(suffix_len) = SUFFIX_TRIE.longest_suffix(word) {
if ENGLISH_WORDS.contains(&word[..word.len() - suffix_len].to_lowercase()) {
return true;
}
}
// Check for compound words
if word.contains('-') {
let parts: Vec<&str> = word.split('-').collect();
if parts
.iter()
.all(|part| ENGLISH_WORDS.contains(part.to_lowercase()))
{
return true;
}
}
false
}