DOCSEARCHbyAlgolia
Submit Docs

EmbedJs Documentation

repository·main·Indexed 20 days ago

https://github.com/llm-tools/embedjs

An open-source Node.js framework for building Retrieval-Augmented Generation (RAG) applications. EmbedJs automates data chunking, embedding generation, and vector database management. It includes a RAGApplicationBuilder for configuring LLM models, vector databases, and data loaders, as well as specialized extensions for Cohere and TwelveLabs video understanding.

Tokens
67.5K
Snippets
253
Records
300
Agent score
66%

What's inside embedjs

  1. Overview of @llm-tools/embedjs-cohere

    main
    The @llm-tools/embedjs-cohere package is an extension for the core embedjs library. It provides specialized functionality and integration support for Cohere's embedding models within the embedjs ecosystem. For core usage patterns, installation of the main library, and general API details, refer to the primary @llm-tools/embedjs documentation.

  2. Overview of supported LLM providers in EmbedJs

    main

    EmbedJs provides built-in support for integrating various Large Language Model (LLM) providers through a unified interface. This allows you to switch between different model providers with minimal code changes.

    Supported providers include:

    • OpenAI
    • Azure OpenAI
    • Anthropic
    • Ollama
    • LlamaCpp
    • Hugging Face
    • Vertex AI
    • Mistral AI

  3. What is EmbedJs?

    main

    EmbedJs is an open-source framework designed to simplify the creation and deployment of personalized AI applications. It follows a "Conventional but Configurable" design principle, making it suitable for both software engineers and machine learning engineers.

    EmbedJs automates the core components of a Retrieval-Augmented Generation (RAG) pipeline:

    • Data Management: Segmenting unstructured data into manageable chunks.
    • Embedding Generation: Creating vector representations of data.
    • Vector Storage: Storing embeddings in a vector database for optimized retrieval.
    • Retrieval & Interaction: Providing APIs to extract contextual information, perform precise searches, or power interactive chat conversations based on your specific data.

  4. Configure EmbedJs components

    main

    An EmbedJs application is composed of several configurable components that define its capabilities. You can configure the following types of components to build your application:

    • LLM: Large Language Model providers for text generation and reasoning.
    • Embedding Model: Models used to convert text into vector embeddings.
    • Data Source: The origin of the raw data used by the application.
    • Vector Database: Specialized databases for storing and querying high-dimensional vectors.
    • Stores: Mechanisms for persisting or managing application state and data.

  5. Supported embedding model providers in EmbedJs

    main

    EmbedJs provides a unified interface for interacting with various embedding model providers. You can use any of the following providers to generate vector embeddings:

    • OpenAI: Standard OpenAI embedding models.
    • Azure OpenAI: OpenAI models hosted on Microsoft Azure.
    • Vertex AI: Google Cloud's Vertex AI embedding models.
    • Cohere: Cohere's proprietary embedding models.
    • Huggingface: Models hosted on the Huggingface platform.
    • Ollama: Local embedding models running via Ollama.
    • LlamaCpp: Local embedding models using the Llama.cpp runtime.
    • TwelveLabs (Marengo): Specialized embedding models from TwelveLabs.

  6. Common use cases for EmbedJs

    main

    EmbedJs is designed to facilitate various LLM-powered applications involving vector embeddings and similarity searches. Common implementation patterns include:

    • Chatbots: Building conversational agents that leverage context from vector stores.
    • Question Answering: Implementing systems that retrieve relevant documents to answer specific user queries.
    • Semantic Search: Creating search engines that find information based on meaning rather than just keyword matching.

  7. Use TwelveLabs for video understanding with embedjs-twelvelabs

    main

    The @llm-tools/embedjs-twelvelabs package extends embedJs to provide video understanding capabilities using TwelveLabs models. It provides two primary features:

    1. MarengoEmbeddings: A multimodal embedding model that produces 512-dimensional embeddings using the Marengo model. It can be used as a drop-in embedding model for your existing workflows.
    2. TwelveLabsVideoLoader: A data source that uses the Pegasus model to analyze video content and automatically load the resulting descriptions into your RAG (Retrieval-Augmented Generation) application.

  8. Supported data sources in EmbedJs

    main

    EmbedJs provides built-in support for loading and processing unstructured data from a wide variety of sources. This abstraction allows you to ingest data from files, web content, and specialized platforms without manually handling the underlying loading complexity.

    Supported data sources include:

    • Files: PDF, CSV, JSON, XML, Text, DOCX, PPT, Excel, Markdown / MDX, and Images.
    • Web & Directories: Web pages, Sitemaps, and local Directories.
    • Video & Multimedia: YouTube (Videos, Channels, and Search results) and TwelveLabs Video.
    • Knowledge Bases: Confluence.
    • Extensibility: A Custom data source option is available if your specific source is not listed.

    Each data source is designed to be integrated into your application through a user-friendly interface, enabling easy customization of how data is ingested and processed.

  9. What are EmbedJs stores and what can they persist?

    main

    EmbedJs uses an abstraction called stores to manage data persistence beyond simple vector embeddings. This abstraction allows you to swap the underlying storage layer without changing your application logic.

    Stores are used to persist:

    • Loaders and chunks: The actual content loaded and their associated metadata.
    • Loader caching: Caching specific to the loader used.
    • Conversation history: The history of interactions in a conversation.

    Supported built-in stores include:

    • LMDB
    • LibSQL
    • MongoDB
    • Redis

  11. How the EmbedJs RAG pipeline works

    main

    EmbedJs manages the lifecycle of data for RAG (Retrieval-Augmented Generation) applications through two main phases: Data Ingestion and Query Processing.

    1. Data Ingestion (Adding data to your pipeline)

    1. Automatic Data Handling: The framework recognizes the data type and loads it automatically.
    2. Efficient Data Processing: It creates embeddings for key parts of the loaded data.
    3. Flexible Data Storage: Processed data is stored in a user-selected vector database.

    2. Query Processing (Responding to users)

    1. Query Processing: The user's question is converted into embeddings.
    2. Document Retrieval: These embeddings are used to find the most relevant documents within the vector database.
    3. Answer Generation: The retrieved documents are passed to a Large Language Model (LLM) to generate a precise, context-aware answer.

  12. Quickstart with Paid models using OpenAI

    main

    To use paid models like GPT-4, configure your OPENAI_API_KEY environment variable. You can use SIMPLE_MODELS.OPENAI_GPT4_O to quickly set the model and OpenAiEmbeddings for the embedding layer within the RAGApplicationBuilder.

    import { RAGApplicationBuilder, SIMPLE_MODELS } from '@llm-tools/embedjs';
import { OpenAiEmbeddings } from '@llm-tools/embedjs-openai';
import { WebLoader } from '@llm-tools/embedjs-loader-web';
import { HNSWDb } from '@llm-tools/embedjs-hnswlib';

//Replace this with your OpenAI key
process.env.OPENAI_API_KEY = "sk-xxxx"

const ragApplication = await new RAGApplicationBuilder()
.setModel(SIMPLE_MODELS.OPENAI_GPT4_O)
.setEmbeddingModel(new OpenAiEmbeddings())
.setVectorDatabase(new HNSWDb())
.build();

await ragApplication.addLoader(new WebLoader({ urlOrContent: 'https://www.forbes.com/profile/elon-musk' }));
await ragApplication.addLoader(new WebLoader({ urlOrContent: 'https://en.wikipedia.org/wiki/Elon_Musk' }));

await ragApplication.query('What is the net worth of Elon Musk today?')
//Answer: The net worth of Elon Musk today is $258.7 billion.