docetl

The fastest way to start the DocWrangler playground is using Docker. This method sets up a persistent Docker volume, builds the DocETL image, and runs both the UI (at http://localhost:3000) and the API (at http://localhost:8000).

1. Configure Environment Files

Create a .env file in the root directory for the FastAPI backend:

# Required: API key for your preferred LLM provider (OpenAI, Anthropic, etc)
OPENAI_API_KEY=your_api_key_here 
BACKEND_ALLOW_ORIGINS=http://localhost:3000,http://127.0.0.1:3000
BACKEND_HOST=localhost
BACKEND_PORT=8000
BACKEND_RELOAD=True
FRONTEND_HOST=localhost
FRONTEND_PORT=3000

Create a .env.local file in the website directory for the frontend:

# Optional: Required for AI assistant chatbot and prompt engineering tools (OpenAI only)
OPENAI_API_KEY=sk-xxx
OPENAI_API_BASE=https://api.openai.com/v1
MODEL_NAME=gpt-4o-mini

NEXT_PUBLIC_BACKEND_HOST=localhost
NEXT_PUBLIC_BACKEND_PORT=8000

2. Launch and Cleanup

To start the playground:

make docker

To remove all Docker resources (including the persistent data volume):

make docker-clean

When groups are too large for full processing, use value_sampling to select a representative subset of the data.

Available Methods:

• random: Randomly select a subset of values.
• first_n: Select the first N values.
• cluster: Use K-means clustering to select representative samples.
• sem_sim: Select samples based on semantic similarity to a query.

For sem_sim (semantic similarity), you must provide embedding_model, embedding_keys (the fields to embed), and query_text.

frame = frame.reduce(
    name="sampled_reduce_sem_sim",
    reduce_key="product_id",
    prompt="""Summarize the reviews for product {{ inputs[0].product_id }}, focusing on comments about battery life and performance:
{% for item in inputs %}
Review {{ loop.index }}: {{ item.review }}
{% endfor %}",""",
    value_sampling={
        "enabled": True,
        "method": "sem_sim",
        "sample_size": 30,
        "embedding_model": "text-embedding-3-small",
        "embedding_keys": ["review"],
        "query_text": "Battery life and performance",
    },
    output={"schema": {"summary": "string"}},
)

The Python API is recommended for production code, notebooks, and scripting. You can define pipelines using operators like map and reduce.

Key features include:

• Setting a default_model.
• Configuring rate_limits for llm_call and llm_tokens.
• Using read_json to load data.
• Using .map() for per-item operations with a prompt and output schema.
• Using .reduce() to aggregate data based on a reduce_key.
• Running the pipeline with .show() (for a sample) or .collect() (for a full run).
• Accessing pipeline.total_cost to see the execution cost.

import docetl

docetl.default_model = "gpt-4o-mini"
docetl.rate_limits = {
    "llm_call": [{"count": 500, "per": 1, "unit": "minute"}],
    "llm_tokens": [{"count": 200_000, "per": 1, "unit": "minute"}],
}

# Classify support tickets, then summarize each category
pipeline = docetl.read_json("tickets.json")

pipeline = pipeline.map(
    prompt="Classify this support ticket: {{ input.text }}",
    output={"schema": {"category": "str", "priority": "str"}},
)

pipeline = pipeline.reduce(
    reduce_key="category",
    prompt="Summarize these tickets: {% for t in inputs %}{{ t.text }}{% endfor %}",
    output={"schema": {"summary": "str"}},
)

pipeline.schema()  # {'category': 'str', 'summary': 'str'}
pipeline.show()  # run on 5 docs and print results
rows = pipeline.collect()  # full run
print(f"Cost: ${pipeline.total_cost:.4f}")

To optimize your DocETL pipeline using MOAR, you must provide an evaluation function that reads pipeline output and returns numeric metrics. MOAR uses a specific metric from your returned dictionary (defined by metric_key) as the accuracy metric for optimization.

Function Signatures

1. Single-argument signature: Use this if you only need the pipeline output. The function receives results_path (the path to the JSON file containing pipeline output). Note: Pipeline output often includes original input data (e.g., a src attribute), so you may not need the original dataset file separately.

2. Two-argument signature: Use this if you need to compare output against the original dataset. The function receives dataset_path and results_path automatically.

Return Value

Your function must return a dictionary of numeric metrics. The key provided to metric_key during optimization must exist in this dictionary.