pdfreader

repository·master·Indexed 20 days ago

https://github.com/adrienjoly/npm-pdfreader

A Node.js library for reading text and parsing tables from PDF files. It features automatic column detection for tabular data, a rule-based engine for complex data extraction, and specialized parsers including TableParser, ColumnsParser, and SequentialParser. The library supports password-protected PDFs and can parse files from both disk and memory buffers.

Tokens
4.6K
Snippets
17
Records
20
Agent score
72%

What's inside pdfreader

  1. How PdfReader works and its item types

    master

    The PdfReader class is the primary interface for parsing PDF files. You can instantiate it with an optional configuration object, such as { debug: true } for troubleshooting.

    Parsing is performed using either parseFileItems(filename, callback) or parseBuffer(buffer, callback). Both methods use a callback that is invoked for every item found during parsing.

    An item object can be one of the following:

    • null: Indicates the end of the file or that an error occurred.
    • {file: {path: string}}: File metadata, always the first item.
    • {page: integer, width: float, height: float}: Page metadata (page numbers start at 1). This acts as a boundary for coordinate-based text processing.
    • {text: string, x: float, y: float, w: float, ...}: A text item containing the string and its 2D AABB coordinates on the page.
    import { PdfReader } from "pdfreader";
    
    new PdfReader().parseFileItems("test/sample.pdf", (err, item) => {
      if (err) console.error("error:", err);
      else if (!item) console.warn("end of file");
      else if (item.text) console.log(item.text);
    });
  2. Fix 'Cannot read property \'userAgent\' of undefined' in Express

    master

    If you encounter the error TypeError: Cannot read property 'userAgent' of undefined when using pdfreader within an Express-based Node.js application, you must polyfill the navigator object before importing the module:

    global.navigator = {
      userAgent: "node",
    };
    
    window.navigator = {
      userAgent: "node",
    };
  3. Parse a PDF from a buffer

    master

    If you have PDF data in memory (e.g., from a database or an upload) rather than a file on disk, use the parseBuffer method.

    import fs from "fs";
    import { PdfReader } from "pdfreader";
    
    fs.readFile("test/sample.pdf", (err, pdfBuffer) => {
      // pdfBuffer contains the file content
      new PdfReader().parseBuffer(pdfBuffer, (err, item) => {
        if (err) console.error("error:", err);
        else if (!item) console.warn("end of buffer");
        else if (item.text) console.log(item.text);
      });
    });
  4. Parse a password-protected PDF file

    master

    To parse a PDF that requires a password, pass the password key in the PdfReader constructor options.

    new PdfReader({ password: "YOUR_PASSWORD" }).parseFileItems(
      "test/sample-with-password.pdf",
      function (err, item) {
        if (err) console.error(err);
        else if (!item) console.warn("end of file");
        else if (item.text) console.log(item.text);
      }
    );
  5. Perform rule-based data extraction

    master

    The Rule class allows you to define specific extraction strategies using "accumulators". You can create an item processor using Rule.makeItemProcessor() which evaluates items against defined rules (like regex matches or table parsing) and executes a callback when a rule matches.

    Common rule patterns include:

    • Rule.on(regexp).extractRegexpValues(): Extracts values based on a regular expression.
    • Rule.on(regexp).parseNextItemValue(): Parses the value of the next item after a match.
    • Rule.on(regexp).parseTable(columns): Parses a table with a specific number of columns.
    • Rule.on(regexp).accumulateAfterHeading(): Accumulates data following a specific heading.
    const processItem = Rule.makeItemProcessor([
      Rule.on(/^Hello "(.*)"$/)
        .extractRegexpValues()
        .then(displayValue),
      Rule.on(/^Value//)
        .parseNextItemValue()
        .then(displayValue),
      Rule.on(/^c1$/).parseTable(3).then(displayTable),
      Rule.on(/^Values//)
        .accumulateAfterHeading()
        .then(displayValue),
    ]);
    
    new PdfReader().parseFileItems("test/sample.pdf", (err, item) => {
      if (err) console.error(err);
      else processItem(item);
    });
  6. Detect cell collisions in a matrix

    master

    If you are using a matrix representation of a table where multiple PDF text items might occupy the same logical cell, detectCollisions(matrix) identifies these overlaps.

    It returns an array of collision objects. Each object contains:

    • row: The row index.
    • col: The column index.
    • items: An array of the actual PDF items that collided in that cell.
    import { detectCollisions } from './lib/parseTable.js';
    
    // matrix is a 2D array where matrix[row][col] is an array of items
    const collisions = detectCollisions(matrix);
    
    collisions.forEach(collision => {
      console.log(`Collision at Row ${collision.row}, Col ${collision.col}`);
      console.log('Items involved:', collision.items);
    });
  7. Retrieve structured data with getRows, getMatrix, and getCleanMatrix

    master

    Once items have been processed by TableParser, you can extract the data in several formats:

    • getRows(): Returns an array of rows. Each row is an object where keys are column identifiers and values are arrays of items belonging to that column at that specific y coordinate.
    • getMatrix(): Returns a 3-dimensional matrix: row -> column -> items_colliding_in_column -> item. This is useful for low-level manipulation of the actual item objects.
    • getCleanMatrix({ collisionSeparator }): Returns a 2-dimensional matrix: row -> column -> value. This is the most common format for data extraction. If multiple items occupy the same cell (collision), they are joined using the provided collisionSeparator (defaults to an empty string).
  8. Use TableParser to extract tabular data

    master

    The TableParser class is used to classify PDF items into rows and columns based on their coordinates (x, y). It is designed to be used incrementally as you iterate through PDF items.

    To use it, you typically:

    1. Instantiate TableParser.
    2. Call processItem(item, col) for each item, where col is the column index/identifier.
    3. Use processHeadingItem(item, col) if the item is a column header (this treats the item as being at y: 0).
    4. Retrieve the structured data using getRows(), getMatrix(), or getCleanMatrix().
    import { TableParser } from 'pdfreader';
    
    const parser = new TableParser();
    
    // As you iterate through PDF items:
    parser.processItem({ x: 10, y: 50, text: 'Data' }, 'column1');
    
    // To get a clean 2D array (row -> column -> text value):
    const matrix = parser.getCleanMatrix({ collisionSeparator: ' ' });
  9. Render table data as TSV strings

    master

    The lib/parseTable.js module provides several utility functions to transform parsed PDF table structures into human-readable or machine-readable string formats (like Tab-Separated Values).

    • renderTable(table): Converts a 2D array of rows into a TSV string. Each cell's text is truncated to the first 7 characters.
    • renderMatrix(matrix): Converts a matrix (where cells may contain multiple items) into a TSV string. If a cell contains multiple items, they are joined with a + separator and truncated to 7 characters.
    • renderRows(rows): Renders rows with their index and the x-coordinate/text of each item (e.g., 0:123:text).
    • renderItems(items): Renders a flat list of items as y_coordinate\t x_coordinate\t text.
    import { renderTable, renderMatrix, renderItems } from './lib/parseTable.js';
    
    // Example: rendering a simple table
    const table = [['Header1', 'Header2'], ['Val1', 'Val2']];
    console.log(renderTable(table));
    
    // Example: rendering items
    const items = [{x: 10, y: 20, text: 'Hello'}];
    console.log(renderItems(items));
  10. Format TableParser output for debugging

    master

    The TableParser provides built-in methods to render the parsed data as strings for quick inspection:

    • renderRows(): Returns a string representation of the rows, showing the row index and the x coordinate followed by the text for each cell.
    • renderMatrix(): Returns a tab-separated string representation of the 3D matrix, where colliding items in a cell are joined by a + character.
  11. Use parseTable as a Rule accumulator

    master

    The parseTable function is a factory that creates an accumulator function designed to be used within a pdfreader Rule. It manages the collection of items and organizes them into rows and a matrix once parsing is complete.

    Usage Pattern

    1. Call parseTable(nbRows, headerRow) to create the accumulator.
    2. The returned function should be used as the accumulate method for a Rule.
    3. When the rule's whenDone lifecycle hook is triggered, the accumulator processes the collected items.

    Parameters

    • nbRows (Number): The expected number of rows in the table. Used to calculate row clusters.
    • headerRow (Number): The index of the row containing column headers. This is used to determine column boundaries.

    Output Structure

    The rule.output object will be populated with:

    • items: The raw list of collected items.
    • rows: A 2D array of items classified into rows.
    • matrix: A 2D array of items classified into rows and columns.
    import { parseTable } from './lib/parseTable.js';
    
    // Inside a pdfreader Rule definition
    const myRule = new Rule();
    
    // Initialize the accumulator
    // nbRows: 5, headerRow: 0
    myRule.accumulate = parseTable(5, 0);
    
    myRule.whenDone(function() {
      const { rows, matrix } = this.output;
      console.log('Parsed rows:', rows);
      console.log('Parsed matrix:', matrix);
    });