docx2python

repository·master·Indexed 42 days ago

https://github.com/shayhill/docx2python

A Python library for extracting text, images, and structural metadata (headings, tables, footnotes) from Microsoft Word (.docx) files into structured Python objects. Version 3.7.1 supports HTML formatting for font styles, image extraction via binary data or folder export, and provides a nested list structure to preserve document hierarchy.

Tokens
7.2K
Snippets
23
Records
34
Agent score
48%

What's inside docx2python

  1. Understand the nested list return format

    master

    The document structure is preserved as nested lists. Paragraphs are always at depth 4 (output.body[i][j][k][l]).

    Structure Hierarchy:

    • document (List)
      • table (List)
        • row (List)
          • cell (List)
            • paragraph (String, Run, or Par instance)

    If a document contains no tables, the body will appear as a single-cell table containing all content. Tables are always returned as $n imes m$ (rows $ imes$ columns) lists. If duplicate_merged_cells=True, merged cells are filled with content from adjacent cells; if False, they are filled with empty cells.

    [
      # document
      [
        # table
        [
          # row
          [
            # cell
            "Paragraph 1",
            "Paragraph 2"
          ]
        ]
      ]
    ]
  2. Understand the DOCX file structure for extension

    master

    A .docx file is a zipped collection of XML files and media. Understanding this structure is useful if you intend to extend docx2python or manually manipulate docx files.

    Key components include:

    • _rels/.rels: Maps to the locations of major files (like document.xml).
    • Content Files: Files containing the actual text (e.g., document.xml, header1.xml).
    • Content-file rels: Files (e.g., document.xml.rels) that define relId numbers used by content files to reference non-content files like images.
    • Display Files: Files (e.g., numbering.xml) that instruct content files on how to display text (such as auto-numbering). docx2python specifically reads numbering.xml.
    • word/media: The directory containing all images/pictures attached to the document.

    Note on modification: You can unzip a .docx file, modify the XML content, and zip it back up to create a valid document, provided the XML changes remain valid.

  3. Extract docx content using docx2python()

    master

    The docx2python function unzips a .docx file and extracts headers, footers, text, footnotes, endnotes, properties, comments, and images into a DocxContent object.

    It is recommended to use the function as a context manager (with ... as) to ensure the underlying zipfile is closed properly. Alternatively, you can call .close() explicitly.

    Basic Usage:

    from docx2python import docx2python
    
    # Using context management (recommended)
    with docx2python('path/to/file.docx') as docx_content:
        print(docx_content.text)
    
    # Manual close
    docx_content = docx2python('path/to/file.docx')
    print(docx_content.text)
    docx_content.close()
    from docx2python import docx2python
    
    # extract docx content
    with docx2python('path/to/file.docx') as docx_content:
        print(docx_content.text)
  4. Extract images from docx files

    master

    You can extract images in two ways: by providing an image_folder argument during initialization, or by calling .save_images() on the resulting DocxContent object.

    Option 1: Specify folder during initialization

    with docx2python('path/to/file.docx', 'path/to/image_directory') as docx_content:
        print(docx_content.text)

    Option 2: Use save_images()

    with docx2python('path/to/file.docx') as docx_content:
        docx_content.save_images('path/to/image/directory')

    Option 3: Manual extraction from the images dictionary If you need to handle the binary data yourself, access the .images attribute, which maps image names to their binary content:

    for name, image in result.images.items():
        with open(name, 'wb') as image_destination:
            image_destination.write(image)
  5. Extract content with HTML formatting

    master

    By setting html=True, docx2python converts basic font styles into HTML tags.

    Supported tags include:

    • <i> (italic), <b> (bold), <u> (underline), <s> (strike)
    • <sup> (superscript), <sub> (subscript)
    • <span style="font-variant: small-caps"> (small caps)
    • <span style="text-transform:uppercase"> (all caps)
    • <span style="background-color: yellow"> (highlighted)
    • <span style="font-size:32"> (font size)
    • <span style="color:#ff0000"> (colored text)

    Note: Hyperlinks are always exported as HTML (<a href="...">link text</a>) regardless of the html setting. Special characters like &, >, and < are encoded as &amp;, &gt;, and &lt; when html=True.

    with docx2python('path/to/file.docx', html=True) as docx_content:
        print(docx_content.text)
  6. Understand the document tree structure in docx2python

    master

    The docx2python package organizes document content into a nested list structure representing the hierarchy of a .docx file. The depth of the nesting corresponds to the document elements:

    [
        # document
        [
            # table (tbl)
            [
                # row (tr)
                [
                    # cell (tc)
                    [
                        # paragraph (p)
                        [
                            "" # text run (r)
                        ]
                    ]
                ]
            ]
        ]
    ]

    Text in table cells and text outside of a table is captured at depth=5. You can access the full tree of Par (paragraph) objects via the DepthCollector.tree property, or a simplified nested list of strings via DepthCollector.tree_text.

  7. Understand the DocxContent output structure

    master

    The DocxContent class provides the structured data extracted from a .docx file. It offers two primary ways to access text content:

    1. Run-level access (5-deep nested list): Properties ending in _runs (e.g., body_runs, header_runs) return a 5-deep nested list where the innermost level contains individual text runs. This is useful for preserving fine-grained formatting information.

      • Structure: [tables [table [rows [cells [paragraphs [runs]]]]]]
    2. Paragraph-level access (4-deep nested list): Default properties (e.g., body, header, document) return a 4-deep nested list where individual runs within a paragraph have been joined into a single string.

      • Structure: [tables [table [rows [cells [paragraphs [strings]]]]]]
    3. Flattened text: The .text property returns all paragraphs joined by \n\n as a single string.

    Summary of Data Formats

    Property TypeExampleDepthInner Element
    Runsbody_runs5str (individual run)
    Paragraphsbody4str (joined paragraph)
    Flattenedtext1str (full document)
    # Example of the difference in structure
    # body_runs (5-deep)
    # [[[[['run1 ', 'run2']]]]]
    
    # body (4-deep)
    # [[[['run1 run2']]]]]
  8. Initialize DocxReader to parse docx files

    master

    Use the DocxReader class to load a .docx file or a BytesIO object. This is the primary entry point for extracting content from a Word document.

    Arguments:

    • docx_filename: Path to the .docx file or a BytesIO object.
    • html (optional): If True, converts XML to HTML. Defaults to False.
    • duplicate_merged_cells (optional): If True, duplicates text in merged cells. Defaults to True.

    Note that DocxReader supports the context manager pattern (with statement) to ensure the underlying zip file is closed correctly.

    from docx2python import DocxReader
    
    with DocxReader('example.docx') as reader:
        # Use reader here
        pass
  9. Convert document tables to an HTML map

    master

    Using enum_at_depth from docx2python.iterators, you can transform the nested list structure of a document into an HTML representation. This is useful for visually searching through extracted data in a browser.

    Note: The input tables typically comes from docx_to_text_output.document or docx_to_text_output.body.

    from docx2python.iterators import enum_at_depth
    
    def html_map(tables) -> str:
        # prepend index tuple to each paragraph
        for (i, j, k, l), paragraph in enum_at_depth(tables, 4):
            tables[i][j][k][l] = " ".join([str((i, j, k, l)), paragraph])
    
        # wrap each paragraph in <pre> tags
        for (i, j, k), cell in enum_at_depth(tables, 3):
            tables[i][j][k] = "".join(["<pre>{x}</pre>".format(x) for x in cell])
    
        # wrap each cell in <td> tags
        for (i, j), row in enum_at_depth(tables, 2):
            tables[i][j] = "".join(["<td>{x}</td>".format(x) for x in row])
    
        # wrap each row in <tr> tags
        for (i,), table in enum_at_depth(tables, 1):
            tables[i] = "".join(["<tr>{x}</tr>".format(x) for x in table])
    
        # wrap each table in <table> tags
        tables = "".join(['<table border="1">{x}</table>'.format(x) for x in tables])
    
        return ["<html><body>"] + tables + ["</body></html>"]
    
    # Example usage:
    tables = [[[['a', 'b'], ['a', 'd']]]]
    print(html_map(tables))
  10. Access docx comments

    master

    You can retrieve comments from a Word document using the comments attribute of the DocxContent object. This is best used within a context manager to ensure proper resource handling.

    Each comment is returned as a tuple containing: (reference_text, author, date, comment_text).

    from docx2python import docx2python
    
    with docx2python('path/to/file.docx') as docx_content:
        print(docx_content.comments)
  11. Process document tables with docx2python.iterators

    master

    The docx2python.iterators module provides helper functions to navigate and manipulate the nested list structures (tables) returned by docx2python.

    • enum_cells(tables): Iterates over cells in a table structure, yielding a tuple of indices (i, j, k) and the cell content.
    • enum_at_depth(tables, depth): Iterates through the nested structure at a specific depth, yielding indices and the content at that level.
    from docx2python.iterators import enum_cells
    
    def remove_empty_paragraphs(tables):
        for (i, j, k), cell in enum_cells(tables):
            tables[i][j][k] = [x for x in cell if x]
    
    # Example usage:
    tables = [[[['a', 'b'], ['a', '', 'd', '']]]]
    remove_empty_paragraphs(tables)
    # Result: [[[['a', 'b'], ['a', 'd']]]]