officeparser

repository·master·Indexed 19 days ago

https://github.com/harshankur/officeparser

A strictly-typed Node.js and Browser library for parsing office files (.docx, .pptx, .xlsx, .odt, .odp, .ods, .pdf, .rtf, .csv, .md, .html, .epub) into an Abstract Syntax Tree (AST). It supports generating high-fidelity outputs in Markdown, HTML, CSV, RTF, PDF, EPUB, and RAG-focused chunks using three distinct chunking strategies: Document Structure, Fixed-Size, and Semantic. The library includes a CLI and features for image extraction and OCR.

Tokens: 24K
Snippets: 70
Records: 93
Agent score: 67%

What's inside officeparser

Understand the OfficeParserAST structure
master
The OfficeParserAST is a format-agnostic representation of a document. It allows you to work with content from various formats (DOCX, PDF, XLSX, etc.) using a unified schema.
Key components include:
- type: The source format (e.g., 'docx', 'pdf', 'xlsx').
- metadata: Document-level properties like author, title, and created.
- content: An array of OfficeContentNode objects representing the document structure (paragraphs, headings, tables, etc.). Each node contains its own text, children, formatting, and metadata.
- auxiliary: Out-of-band elements like headers, footers, or slide masters.
- attachments: Extracted images or charts (if extractAttachments: true is used).
- warnings: An array of OfficeIssue objects for non-fatal parsing issues.
- .to(format, config?): A method to convert the AST into other formats like 'html', 'md', 'text', 'csv', 'rtf', 'pdf', or 'chunks'.
https://github.com/harshankur/officeparser/blob/master/README.md
Understand Chart Nodes and data extraction
master
Charts are extracted as chart nodes. The chart's visual data is stored in an attachment object.
Attachment structure:
- chartData: Contains { title, dataSets, labels }.
```
Chart Node (type: 'chart')
├── metadata: { attachmentName: 'chart1.xml' }
└── → Attachment: { chartData: { title, dataSets, labels } }
```
https://github.com/harshankur/officeparser/blob/master/README.md
Markdown Dialect and Round-trip Fidelity
master
The MarkdownParser and MarkdownGenerator support an extended dialect to ensure high-fidelity round-tripping (e.g., .md → AST → .md is idempotent).
Supported extended features include:
- Task lists: - [x] Done maps to ListMetadata.isTask and .checked.
- Footnotes: Text[^1] maps to type: 'note'.
- Abbreviations: *[HTML]: Hypertext... maps to TextMetadata.abbreviationTitle.
- Citations: [@smith2024] maps to TextMetadata.citationKey.
- Wikilinks: [[Page]] maps to TextMetadata.wikilink.
- Attribute lists: ![alt](img.png){width=50%} maps to ImageMetadata.width.
- Frontmatter arrays: Parsed into real arrays in metadata.customProperties.
https://github.com/harshankur/officeparser/blob/master/README.md
Extract footnotes, endnotes, and slide notes
master
Notes are handled differently depending on the file type and node type:
- Slide Speaker Notes (PPTX): These live directly on the node where type is 'slide'. Access them via slide.notes.
- Footnotes and Endnotes (DOCX/RTF): These can be deeply nested within the content tree. You must traverse ast.content recursively and check for the notes property on each node.
Note: The putNotesAtLast flag is deprecated and has no effect. Notes are now always attached via node.notes.
```
// Slide speaker notes (PPTX) live on the slide node itself
const slide = ast.content.find(n => n.type === 'slide');
console.log(slide?.notes?.map(n => n.text));

// Footnotes and endnotes (DOCX/RTF) can be deeply nested, so we traverse recursively:
const printNotes = (nodes: OfficeContentNode[]) => {
    nodes.forEach(node => {
        if (node.notes) {
            node.notes.forEach(note => console.log(note.text));
        }
        if (node.children) {
            printNotes(node.children);
        }
    });
};

printNotes(ast.content);
```
https://github.com/harshankur/officeparser/blob/master/README.md

Understand Admonitions, Embeds, and Definition Lists

master

The AST supports specialized content types:

Admonitions: Used for notes, tips, warnings, etc. They round-trip between Markdown (> [!NOTE]) and HTML (<div class="admonition...">).
Embeds: Currently models YouTube videos via videoId and url metadata.
Definition Lists: A hierarchy of definitionTerm and definitionDescription nodes.

Admonition Node (type: 'admonition')
├── metadata: { admonitionType: 'note' | 'tip' | 'important' | 'warning' | 'caution', title?: string }
└── children: [ Paragraph | List | ... ] (block content)

Embed Node (type: 'embed')
└── metadata: { embedType: 'youtube', videoId: string, url?: string, width?: string, align?: string }

Definition List Node (type: 'definitionList')
└── children:
    ├── Definition Term (type: 'definitionTerm')
    └── Definition Description (type: 'definitionDescription')

Understand the List Node structure
master
Lists in the AST are represented by list nodes. Each node contains metadata to track logical grouping and nesting. Items sharing the same listId belong to the same logical list. Even if a paragraph interrupts a list, the itemIndex continues to increment for that listId, ensuring numbering remains correct during reconstruction.
Metadata fields:
- listId: Unique identifier for the logical list.
- listType: 'ordered' or 'unordered'.
- indentation: 0-based nesting level.
- itemIndex: Sequential position within the list level.
- paragraphIndentation: { left, hanging, right, firstLine }.
```
List Node
├── type: 'list'
├── metadata: {
│       listId: '1',
│       listType: 'ordered' | 'unordered',
│       indentation: 0,
│       itemIndex: 0,
│       paragraphIndentation: { left, hanging, right, firstLine }
│   }
└── children: [ Text content ]
```
https://github.com/harshankur/officeparser/blob/master/README.md
Handle Break Nodes (DOCX and ODF)
master
When includeBreakNodes: true is set, break elements (page, column, etc.) are emitted as nodes.
Key behaviors:
- Break nodes do not have a text property.
- ast.toText() and ast.to('text') automatically convert break nodes into the configured newline delimiter.
- DOCX: Breaks are typically inline children of a paragraph.
- ODF: Breaks are emitted as siblings around a paragraph (based on paragraph style).
```
Break Node (type: 'break')
└── metadata: {
        breakType: 'textWrapping' | 'page' | 'column' | 'lastRenderedPage' | 'carriageReturn',
        clear?: 'all' | 'left' | 'none' | 'right'
    }
```
https://github.com/harshankur/officeparser/blob/master/README.md
Understand the Table Node hierarchy
master
Tables follow a strict table → row → cell hierarchy. Cells can contain nested content such as paragraphs, lists, or even other tables.
Metadata fields for Cells:
- row: Zero-based grid position.
- col: Zero-based grid position.
- rowSpan?: Merged rows (primarily ODF).
- colSpan?: Merged columns (primarily ODF).
```
Table Node (type: 'table')
└── children: Row Nodes (type: 'row')
    └── children: Cell Nodes (type: 'cell')
        ├── metadata: { row, col, rowSpan?, colSpan? }
        └── children: [ Paragraph | List | Table | ... ]
```
https://github.com/harshankur/officeparser/blob/master/README.md
Extract and use Equations (LaTeX)
master
Equations from all supported formats (DOCX, PPTX, ODT, ODP, ODS, HTML, EPUB, Markdown) are normalized to LaTeX. This ensures consistent representation regardless of the source markup.
Important for indexing: Treat code nodes with math metadata as opaque LaTeX strings. Do not split them into individual words, as the structure (like fractions) is critical to the meaning.
Node structure:
- type: 'code'
- text: The LaTeX string (e.g., '\frac{1}{2}').
- metadata.math: 'inline' or 'block'.
```
Code Node (type: 'code')
├── text: '\frac{1}{2}'          // LaTeX, whatever the source markup was
└── metadata: { math: 'inline' | 'block' }
```
https://github.com/harshankur/officeparser/blob/master/README.md
Convert to EPUB with Images
master
When converting to or from EPUB, you must pass extractAttachments: true if the document contains images.
Without this flag, the parser will not extract the embedded image bytes, causing images to disappear in the resulting EPUB. The EpubGenerator packages images as real zip entries (e.g., OEBPS/images/...) rather than using data: URIs, which is necessary for compatibility with most EPUB readers.
CLI Example:
```
npx officeparser book.docx --extractAttachments --to=epub --output=book.epub
```
https://github.com/harshankur/officeparser/blob/master/README.md

Generate RAG chunks with native chunking strategies

master

Use OfficeConverter.convert with the 'chunks' target to generate document chunks optimized for Retrieval-Augmented Generation (RAG) pipelines. The library supports three distinct strategies:

Document Structure (Default): Splits at natural AST boundaries like headings, paragraphs, pages, slides, or sheets. This preserves the logical flow of the document.
Fixed-Size (Recursive): Splits by character count with a specified overlap, similar to LangChain's RecursiveCharacterTextSplitter.
Semantic: Uses sentence embeddings and cosine similarity to find topic boundaries. This requires providing an embeddingFunction.

Each chunk is returned as an OfficeChunk object containing the text and rich metadata (like sourceType, pageNumber, slideNumber, or closestHeading) to support filtered retrieval and citations.

// Example: Document Structure Strategy
const { value: chunks } = await OfficeConverter.convert('report.docx', 'chunks', {
    generatorConfig: {
        chunksConfig: {
            strategy: 'document-structure',
            splitBy: 'heading',    // 'paragraph' | 'heading' | 'page' | 'slide' | 'sheet'
            maxChunkSize: 1500,
            tableSplitStrategy: 'row', // repeats header row in every chunk
        }
    }
});

// Example: Fixed-Size Strategy
const { value: chunks } = await OfficeConverter.convert('report.docx', 'chunks', {
    generatorConfig: {
        chunksConfig: {
            strategy: 'fixed-size',
            chunkSize: 1000,
            chunkOverlap: 200,
        }
    }
});

// Example: Semantic Strategy
const { value: chunks } = await OfficeConverter.convert('report.docx', 'chunks', {
    generatorConfig: {
        chunksConfig: {
            strategy: 'semantic',
            embeddingFunction: async (text) => {
                // implementation using an embedding provider
                return embedding;
            },
            similarityThreshold: 0.8,
            maxChunkSize: 2000,
        }
    }
});

Choose the right API for your goal

master

Use this guide to select the appropriate method based on your requirements:

Goal	API to use
Extract text / AST from a file	`OfficeParser.parseOffice(file)`
Convert directly to another format	`OfficeConverter.convert(file, 'md')`
Parse first, then generate	`parseOffice()` → `OfficeGenerator.generate(ast, 'html')`
Convert on the AST itself (shorthand)	`ast.to('md')`
RAG pipeline chunking	`OfficeConverter.convert(file, 'chunks', {...})`