DOCSEARCHbyAlgolia
Submit Docs

Gmail Processor

repository·main·Indexed 20 days ago

https://github.com/ahochsteger/gmail-processor

An open-source Google Apps Script library for automating Gmail workflows. It uses JSON-based rules to match emails and attachments to trigger actions such as storing files in Google Drive, performing OCR text extraction, decrypting password-protected PDFs, and logging data to Google Spreadsheets.

Tokens
41.7K
Snippets
105
Records
184
Agent score
69%

What's inside gmail-processor

  1. What is Gmail Processor?

    main
    Gmail Processor is an open-source Google Apps Script library designed to automate the processing of Gmail messages and attachments. It uses a JSON configuration to match specific threads, messages, or attachments against criteria and triggers corresponding actions, such as storing attachments in Google Drive, decrypting PDFs, performing OCR text extraction, or logging data to Google Spreadsheets.

  2. Key Features of Gmail Processor

    main

    Gmail Processor provides the following automation capabilities:

    • Extensive Automation: Match threads, messages, and attachments to trigger specific actions.
    • Google Drive Integration: Store attachments, message PDFs, or entire threads in Google Drive.
    • PDF Decryption: Decrypt and store password-protected PDF attachments.
    • OCR Text Extraction: Extract text from images (JPEG, PNG, GIF) and PDFs to organize or annotate attachments.
    • Google Spreadsheet Logging: Log information about processed items into a Google Spreadsheet.
    • Flexible Configuration: Define rules and actions using a JSON configuration.
    • Extensible Architecture: Easily add new actions and integrations.

  3. Pass data between actions using actionMeta

    main

    You can pass data from one action to a subsequent action by using the actionMeta object within a custom action.

    1. Extract and Store: In your custom action (e.g., custom.parseInvoice), extract the required data and assign it to a key in actionMeta.
    2. Access via Placeholder: In the subsequent action (e.g., attachment.store), access that data using the placeholder syntax {{message.<keyName>}}.

    This allows for dynamic workflows where the output of a parsing or logic step determines the parameters of a storage or notification step.

    // Example logic flow:
// 1. custom.parseInvoice sets actionMeta.invoiceNumber
// 2. attachment.store uses {{message.invoiceNumber}}

  4. Extract structured JSON from attachments using Gemini AI

    main

    The processor supports extracting structured JSON payloads from email text and attachments using the Gemini REST API. This is achieved via the attachment.extractWithAI trigger. When this action is triggered with an expected response schema, the processor retrieves the structured JSON and stores it in the actionMeta map for subsequent use in the processing pipeline.

    // Scenario: Extract Structured JSON
// WHEN attachment.extractWithAI is triggered with an expected response schema
// THEN it must retrieve the structured JSON and store it in the actionMeta map.

  5. How Gmail Processor processes and tracks emails

    main

    Gmail Processor uses a hierarchical JSON configuration to match threads, messages, and attachments. Once a match is found, it executes a list of actions (e.g., exporting a thread as a PDF, adding a label, or storing an attachment).

    To prevent re-processing the same items, the library supports three methods for tracking processed entities:

    1. Mark processed threads by attaching a label:

      • Best for: Simple cases with only one message per thread.
      • Pros: Keeps messages in an unread state.
      • Cons: Cannot process new messages added to a thread that has already been labeled.

    2. Mark processed messages as read:

      • Best for: Most use cases, especially threads with multiple messages.
      • Pros: Can process additional messages within the same thread even after the thread has been previously processed.
      • Cons: Marks messages as read, which may affect your inbox view.

    3. Custom:

      • Best for: Complex edge cases requiring maximum flexibility.
      • Pros: Most flexible; allows the user to define their own logic via actions.
      • Cons: Requires careful configuration to ensure matching rules and marking actions align, otherwise items may be processed repeatedly.

  6. Planned AI-powered data extraction and categorization

    main

    The roadmap includes first-party AI integration designed to transform the processor from a rule-based tool into an intelligent data pipeline.

    Key planned features include:

    • AI-Powered Data Extraction: New actions like message.extractWithAI or attachment.extractWithAI that send content to an LLM (e.g., via Gemini REST API) and inject structured results into the context via actionMeta.
    • AI-Based Categorization & Summarization: Using LLMs to classify, tag, or summarize threads/messages. This allows for auto-labeling threads by topic or routing messages based on intent.

    Users can currently achieve similar results by implementing custom actions that use Google Apps Script's UrlFetchApp to call LLM APIs directly.

  7. How actionMeta enables action chaining and data injection

    main

    The actionMeta mechanism is a core architectural pattern used to store results from one action so they can be consumed by subsequent actions in a pipeline. This allows for complex workflows, such as extracting data from an attachment and then using that extracted data to construct a filename or a storage path.

    Currently, this pattern is implemented in attachment.extractText. Future updates aim to formalize this as a generalized way to inject structured results into the processing context.

    // Conceptual example of the pattern used in attachment.extractText
// Extract data -> use in filename -> store file

  8. Note on Async execution in Google Apps Script (GAS)

    main

    While the library uses Promise-based handling for certain operations (like storeDecryptedPdf), users should be aware of Google Apps Script (GAS) constraints when considering asynchronous patterns:

    • Execution Limit: GAS has a 6-minute execution limit.
    • No Background Workers: GAS does not support true background workers.
    • Async Support: While GAS V8 supports async/await, full async migrations for the entire processing pipeline are limited by these environment constraints.

  9. Commit Message Format and Requirements

    main

    When using the Commit Message Skill, the generated output follows a specific structure to ensure compliance with project standards:

    1. Header: <type>(<scope>): <description> (constrained to 50-72 characters).
    2. Body: A detailed explanation of the changes (included if necessary).
    3. Footer: Used for breaking changes or issue references.
    4. Required Trailers: Every message must include these exact metadata lines:
      • AI-Assisted: true
      • AI-Tool: Antigravity

    The final message is returned in a fenced code block for easy copying.

    feat(attachment): add support for inline image extraction

The new extraction logic handles 'multipart/related' messages correctly.

AI-Assisted: true
AI-Tool: Antigravity

  10. Strict 100% Test Coverage Enforcement for SpreadsheetConfigProvider

    main
    The SpreadsheetConfigProvider implementation must maintain a strict 100% unit test coverage threshold. This requirement specifically applies to all parsing and conversion branches within the provider code. Any regression in code coverage that drops below 100% must result in a CI (Continuous Integration) failure.

  11. Limitations of bidirectional conversion between JSON and Spreadsheets

    main

    When using the bidirectional conversion feature (JSON-to-Sheet or Sheet-to-JSON), be aware of the following constraints:

    Nesting Hierarchy Loss

    Converting highly complex, deeply nested JSON configurations into a flat row model may cause issues. For example, if you have multiple different message filters under a single thread, or multiple attachment actions per attachment rule, the conversion may result in repeated rows with duplicate thread queries.

    Non-Standard Types

    Spreadsheet cells only support standard types: strings, numbers, booleans, and JSON strings.

    Warning: JavaScript function expressions or custom validation callbacks configured via the API code cannot be serialized into spreadsheet cells and will be lost during conversion.

  12. Use SpreadsheetConfigProvider to configure Gmail Processor via Google Sheets

    main

    The SpreadsheetConfigProvider allows you to define Gmail Processor rules and settings using a Google Sheet instead of writing JSON or YAML. This is designed for non-technical users to provide a low-barrier interface for configuration.

    Key Features

    • Tabular Configuration: Define rules in a flat tabular structure within a spreadsheet.
    • Automatic Parsing: The provider parses the flat spreadsheet rows into the hierarchical context structure required by the Gmail Processor.
    • Validation: The parsed spreadsheet data is integrated into the existing Zod validation pipeline via ConfigSchema.parse() to ensure configuration integrity.
    • Bidirectional Conversion: Supports converting between JSON and Spreadsheet formats (JSON-to-Sheet and Sheet-to-JSON).

    Usage

    To run the processor using a spreadsheet as the configuration source, use the runWithSpreadsheet entry point provided by the GmailProcessor class.

    // Example entry point usage
GmailProcessor.runWithSpreadsheet(spreadsheetId);