PDFStract

PDFStract is a unified platform for PDF extraction, conversion, text chunking, and embedding. It is designed as a production-grade data preparation layer for RAG (Retrieval-Augmented Generation) workflows.

Users can interact with the system through three primary entry points:

1. CLI: For command-line operations.
2. Web API: For service-based integration (FastAPI).
3. Python Library: For direct programmatic use in Python applications.

The architecture is built on a factory-based pattern that supports over 25 implementations across three core layers: Converters, Chunkers, and Embeddings.

The Web UI allows you to select specific tools and parameters for the extraction pipeline.

PDF Conversion Libraries

• Docling: Balanced performance and quality (default).
• Marker: Best for complex layouts and figures.
• PyMuPDF4LLM: Fastest for simple documents.
• Unstructured: Great for document structure.
• PaddleOCR: Best for scanned/image PDFs.

Text Chunking Methods

• Token: Simple token-based splitting (default).
• Semantic: AI-powered semantic chunking.
• Recursive: Smart recursive text splitting.
• Sentence: Sentence-boundary aware.
• Code: Code-aware chunking.

Chunking Settings

• Chunk Size: Target size in tokens (default: 512).
• Overlap: Overlap between chunks (default: 50).
• Min Size: Minimum chunk size threshold.
• Max Size: Maximum chunk size limit.

PDFStract follows a modular linear architecture designed to transform unstructured PDF data into structured, vector-ready content for RAG (Retrieval-Augmented Generation) applications:

PDF Input $\rightarrow$ Converter Library $\rightarrow$ Raw Text $\rightarrow$ Chunker $\rightarrow$ Structured Chunks $\rightarrow$ Your RAG Pipeline

Because each stage is modular, you can swap out specific components (like changing the extraction library or the chunking strategy) by simply changing a parameter, without altering the rest of your pipeline logic.

Selecting the right chunk size and overlap is critical for retrieval performance. Use these recommended settings as a starting point:

Choose a library based on your document type and performance needs:

A typical request follows these phases:

1. Entry: Triggered via CLI, Web endpoint, or Library method.
2. Validation: Verifies the PDF path and configuration parameters.
3. Conversion: The ConverterFactory loads a converter to read the PDF and extract text.
4. Chunking: The ChunkerFactory loads a chunker to segment the text into Chunk objects.
5. Embedding: The EmbeddingsFactory loads a wrapper to call an LLM or local model to generate vectors.
6. Support:
   • Queue Manager: Parallelizes work.
   • Results Manager: Saves results to ~/.pdfstract/results/<task_id>/.
   • Logger: Records activity via Loguru to ~/.pdfstract/logs/.
   • Database: Persists metadata.

Start a local development server to preview the documentation. This command opens a browser window and supports live reloading for most changes.

yarn start

You can launch the PDFStract web interface using the CLI. By default, it runs on http://localhost:8000. You can customize the host and port using flags.

CLI Commands

• Basic start: pdfstract web
• Custom port: pdfstract web --port <port>
• Custom host and port: pdfstract web --host <host> --port <port>

# Start PDFStract web interface
pdfstract web

# Or specify custom port
pdfstract web --port 3000

# With custom host
pdfstract web --host 0.0.0.0 --port 8080

You can run the PDFStract frontend locally using make.

1. Clone the repository.
2. Run make up.
3. Access the Web UI at http://localhost:3000 and the API at http://localhost:8000.

git clone https://github.com/aksarav/pdfstract.git
cd pdfstract
make up

You can run the complete RAG pipeline (Extract → Chunk → Embed) in a single line of code using convert_chunk_embed, or execute the steps individually for more control.

Complete Pipeline

from pdfstract import PDFStract

pdfstract = PDFStract()
result = pdfstract.convert_chunk_embed('document.pdf')

Step-by-Step Pipeline

from pdfstract import PDFStract

pdfstract = PDFStract()

# 1. Extract
text = pdfstract.convert('document.pdf', library='auto')

# 2. Chunk
chunks = pdfstract.chunk(text, chunker='semantic', chunk_size=512)

# 3. Embed
vectors = pdfstract.embed_texts([c['text'] for c in chunks['chunks']])

from pdfstract import PDFStract

pdfstract = PDFStract()

# Complete pipeline: Extract → Chunk → Embed
result = pdfstract.convert_chunk_embed('document.pdf')

For faster installation, you can use uv to install the full suite of converters:

uv pip install pdfstract[all]