VectorAdmin Documentation

repository·master·Indexed 25 days ago

https://github.com/mintplex-labs/vector-admin

A universal management suite and GUI for vector databases such as Pinecone and Chroma. VectorAdmin provides a full-stack web interface for developers to inspect, manage, and manipulate vectorized text data, including tools for uploading documents, managing embeddings, and performing semantic searches. The system consists of a NodeJS/Express backend, a React frontend, a Flask-based document processor, and InngestJS background workers.

Tokens
16.4K
Snippets
21
Records
123
Agent score
81%

What's inside VectorAdmin

  1. What is VectorAdmin?

    master
    VectorAdmin is a universal GUI and tool suite designed for managing vector databases (such as Pinecone or Chroma) at scale. It provides a full-stack application to view, update, and delete individual text chunks of embeddings, manage multiple concurrent vector databases, and upload/embed new documents directly. It is designed to give users total control over vectorized text that is otherwise difficult to inspect via standard APIs or tools like LangChain.
  2. VectorAdmin Technical Architecture

    master

    The VectorAdmin monorepo is composed of the following core components:

    • document-processor: A Flask application used to digest, parse, and embed documents.
    • frontend: A ViteJS + React application for content management and UI.
    • backend: A NodeJS + Express server that handles vector database interactions and management.
    • workers: An InngestJS instance that manages long-running background tasks.
    • docker: Orchestration files to run the entire architecture as a single instance.
  3. Connect to a local Chroma vector database

    master

    When connecting to a Chroma instance running on the same machine as your Docker containers, use http://host.docker.internal:[CHROMA_PORT] as the URL.

    Similar to Pinecone, a Sync Chroma Data button may appear on the organization homepage if existing data is detected in Chroma.

  4. Run VectorAdmin using a pre-built Docker image

    master

    The fastest way to run VectorAdmin is to pull the latest pre-built image from Docker Hub. You must provide a valid DATABASE_CONNECTION_STRING for a Postgres database (RDS, remote, or local) before starting, as the vdbms database is created on boot using this string.

    docker pull mintplexlabs/vectoradmin
    
    docker run -d -p 3001:3001 \
    --name vectoradmin \
    -e SERVER_PORT="3001" \
    -e JWT_SECRET="your-random-string-here" \
    -e INNGEST_EVENT_KEY="background_workers" \
    -e INNGEST_SIGNING_KEY="random-string-goes-here" \
    -e INNGEST_LANDING_PAGE="true" \
    -e DATABASE_CONNECTION_STRING="postgresql://vectoradmin:password@xxxxxxx:5432/vdbms" \
    mintplexlabs/vectoradmin
  5. Set up VectorAdmin in a Development Environment

    master

    To run VectorAdmin locally for development, follow these steps. Note: These instructions do not work on Windows.

    1. Initial Setup

    From the project root directory, run:

    yarn dev:setup

    Then, set up the document processor:

    cd document-processor && python3.9 -m virtual-env v-env && source v-env/bin/activate && pip install -r requirements.txt

    2. Run Services

    Open separate terminal windows from the project root and run the following commands:

    Database and Server:

    yarn prisma:setup
    yarn dev:server

    Frontend:

    yarn dev:frontend

    Background Workers:

    yarn dev:workers

    Document Processor:

    cd document-processor && flask run --host '0.0.0.0' --port 8888

    3. Initial Configuration

    On the first boot, visit the homepage. You will be automatically redirected to a setup flow to create your primary admin account, organization, and database connection.

  6. Build VectorAdmin Docker image from source

    master

    To build the image from the local repository, follow these steps:

    1. Clone the repository and navigate to the docker/ directory.
    2. Copy the example environment file: cp .env.example .env.
    3. Edit the .env file to update the following required variables:
      • JWT_SECRET: A random string for authentication.
      • DATABASE_CONNECTION_STRING: A valid Postgres connection string.
      • INNGEST_SIGNING_KEY: A random string.
    4. Run the build command.

    Note: If connecting to a Postgres instance running on your host machine from within the container, use host.docker.internal in the connection string.

    git clone git@github.com:Mintplex-Labs/vector-admin.git
    cd vector-admin
    cd docker/
    cp .env.example .env
    # Edit .env with JWT_SECRET, DATABASE_CONNECTION_STRING, and INNGEST_SIGNING_KEY
    docker-compose up -d --build vector-admin
  7. Connect to Pinecone vector database

    master

    To use Pinecone, navigate to your organization settings and provide your Pinecone configuration and keys.

    After connecting, if you see a Sync Pinecone Data button on the organization homepage, it means there is existing data in your Pinecone index that can be pulled into VectorAdmin. The sync time depends on the number of embedded documents.