semantic-chunking

Instead of using the built-in ONNX models, you can provide a custom embedding function using the embedCallback option. This is useful for using providers like OpenAI or Cohere. When embedCallback is provided, the built-in ONNX model is not initialized, saving memory and startup time.

Type Signature: (texts: string[]) => Promise<number[][]>

Note: If using embedCallback, you can use modelName and dtypeLabel to override the metadata reported in the results (which defaults to custom-embedding and custom).

// OpenAI Example
import { chunkit } from 'semantic-chunking';
import OpenAI from 'openai';

const openai = new OpenAI();

const embedCallback = async (texts) => {
  const response = await openai.embeddings.create({
    input: texts,
    model: 'text-embedding-3-small'
  });
  return response.data.map(d => d.embedding);
};

const documents = [{ document_name: 'doc1', document_text: '...' }];
const chunks = await chunkit(documents, { embedCallback });

Authentication is optional. You can enable it by setting the API_AUTH_TOKEN environment variable. When enabled, all protected endpoints require a Bearer token in the Authorization header.

Enabling Authentication:

• Local: export API_AUTH_TOKEN=your-secret-token-here
• Docker: Use the -e API_AUTH_TOKEN=your-secret-token-here flag in docker run.
• Docker Compose: Add API_AUTH_TOKEN: your-secret-token-here under the environment key.

Using Authentication: Include the token in your request header as Authorization: Bearer <your-token>.

Unprotected Endpoints: The following endpoints do not require authentication:

• GET / (landing page)
• GET /api/health
• GET /api/version

curl -X POST http://localhost:3001/api/chunkit \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer your-secret-token-here" \
  -d '{"documents": [...]}'

For Retrieval-Augmented Generation (RAG) applications, you can improve embedding quality by using the chunkPrefix option. This is particularly effective for models that support task-specific prefixes (like nomic-embed-text-v1.5).

1. Chunk Documents: Use a prefix like search_document when chunking your knowledge base.
2. Process Queries: Use a different prefix like search_query when processing the user's search query.

This ensures the embedding model treats the document chunks and the query with the appropriate semantic context, improving retrieval accuracy.

// Chunking documents for the vector DB
const myDocumentChunks = await cramit(documents, { 
    chunkPrefix: "search_document", 
    returnEmbedding: true 
});

// Processing the user query
const mySearchQueryChunk = await cramit(queryDocuments, { 
    chunkPrefix: "search_query", 
    returnEmbedding: true 
});

Starting from v2.6.0, you can bypass the built-in ONNX models by using the embedCallback parameter. This allows you to integrate external embedding services. Below are the available implementation patterns:

Hugging Face Inference API (Free)

Best for beginners. Requires a free API key from Hugging Face.

export HF_API_KEY=your_token_here
npm run example-embedCallback-huggingface

Local Ollama (Private/Offline)

Best for privacy. Requires Ollama installed locally and a model pulled (e.g., nomic-embed-text).

# Install Ollama and pull model
curl https://ollama.ai/install.sh | sh
ollama pull nomic-embed-text

# Run example
npm run example-embedCallback-ollama

Mock Embeddings (Testing Only)

Best for CI/CD or zero-dependency testing. It is deterministic but not semantically meaningful.

npm run example-embedCallback-mock

OpenAI (Paid)

Best for production-grade quality. Requires a paid OpenAI API key.

export OPENAI_API_KEY=your_key_here
npm run example-embedCallback-openai

The API is designed to be deployed using Docker or Docker Compose.

Quick Start with Docker Compose:

• API Only: docker-compose up -d (Available at http://localhost:3001)
• With Web UI: docker-compose --profile webui up -d (API at http://localhost:3001, Web UI at http://localhost:3000)

Manual Docker Run: To run the API server manually:

docker run -d \
  -p 3001:3001 \
  -v ./models:/app/models \
  -e API_AUTH_TOKEN=your-token \
  --name semantic-chunking-api \
  semantic-chunking

Important Configuration:

• Volume Mounts: It is highly recommended to mount a local directory to /app/models (e.g., -v ./models:/app/models) to persist downloaded ONNX models. This prevents re-downloading large models (23MB to 548MB) on every restart.
• Environment Variables:
  • PORT: Server port (default: 3001)
  • NODE_ENV: Environment mode (e.g., production)
  • API_AUTH_TOKEN: Optional Bearer token for authentication

docker build -t semantic-chunking .

docker run -d \
  -p 3001:3001 \
  -v ./models:/app/models \
  -e API_AUTH_TOKEN=your-token \
  --name semantic-chunking-api \
  semantic-chunking