DOCSEARCHbyAlgolia
Submit Docs

Posthorn

repository·main·Indexed 42 days ago

https://github.com/craigmccaskill/posthorn

A unified outbound mail layer for self-hosted projects that acts as a gateway between applications (via HTTP forms, HTTP APIs, or SMTP) and transactional email providers such as Postmark, Resend, Mailgun, AWS SES, or outbound SMTP relays.

Tokens
99.9K
Snippets
233
Records
417
Agent score
51%

What's inside Posthorn

  1. Overview of Posthorn

    main

    Posthorn is a self-hosted, unified outbound mail gateway that sits between your applications and a transactional mail provider (such as Postmark, Resend, Mailgun, AWS SES, or outbound-SMTP). It eliminates the need to integrate multiple mail APIs into every individual app in your stack by providing a single, centralized integration layer.

    Ingress Shapes (How apps talk to Posthorn)

    • HTTP form ingress: For contact forms or webhooks. Accepts application/x-www-form-urlencoded and multipart/form-data POST requests.
    • JSON API ingress: For server-to-server communication (workers, cron jobs). Uses Authorization: Bearer authentication and supports idempotency keys.
    • SMTP ingress: For legacy apps that only support SMTP (e.g., Ghost, Gitea, Mastodon). Posthorn accepts SMTP on your local network (requiring STARTTLS and AUTH PLAIN/client-cert) and forwards it via an HTTP API transport.

    Transports (How Posthorn talks to providers)

    • Postmark, Resend, Mailgun: Bespoke HTTP clients.
    • AWS SES: Uses a bespoke SigV4 implementation.
    • Outbound SMTP: Relays via any STARTTLS-capable upstream (e.g., Postfix smarthost).

  2. What Posthorn is and its core functionality

    main

    Posthorn is a unified outbound mail layer designed for self-hosted projects. It acts as a single gateway between your self-hosted applications and your chosen transactional mail provider, solving the problem of managing multiple API keys and integration methods across a stack.

    Ingress Shapes (How you send mail to Posthorn)

    Posthorn accepts incoming mail via three methods:

    • HTTP Form: Supports application/x-www-form-urlencoded and multipart/form-data. Includes honeypot, Origin/Referer checks, rate limiting, and optional CSRF tokens.
    • HTTP API: Bearer-authenticated JSON mode. Supports idempotency keys, per-request to_override, and per-key rate limiting.
    • SMTP: A TCP listener supporting AUTH PLAIN or client-cert. STARTTLS is required by default. Includes sender/recipient allowlists/caps and size limits.

    Egress Transports (How Posthorn sends mail to the provider)

    Posthorn delivers mail via five bespoke transports (without SDK dependencies):

    • Postmark
    • Resend
    • Mailgun (US + EU)
    • AWS SES
    • Outbound-SMTP relay

    Key Features in v1.0

    • Templating: Uses Go text/template for subject and body with custom-field passthrough.
    • Validation: Performs email syntax checks and required field validation.
    • Failure Handling: Retries once on transient/5xx errors (1s delay) and once on 429 errors (5s delay). No retries on 4xx configuration errors. 10s hard timeout.
    • Observability: Provides /healthz and /metrics (Prometheus) endpoints. Logs are structured JSON with UUIDv4 submission IDs.

  3. Posthorn v1.0 MVP Feature Set

    main

    Posthorn v1.0 is a mail gateway designed to handle outbound email via HTTP form submissions or API calls. It acts as an integration layer between web services and email providers (specifically Postmark in v1.0).

    Core Capabilities

    • Ingress Modes: Supports HTTP form-encoded POST submissions on configured paths. It supports multiple independent form configurations (endpoints).
    • Egress (Transports): Uses the Postmark HTTP API transport (v1.0 only). The architecture includes a pluggable Transport interface for future providers.
    • Spam Protection: Includes a configurable honeypot field, Origin/Referer checks, a token bucket rate limiter per endpoint, and maximum request body size limits.
    • Validation: Validates required fields (returning HTTP 422 on failure) and ensures correct email formats.
    • Email Customization: Uses Go template rendering for subjects and bodies, using form fields as template data. Unconfigured fields are passed through in a structured block at the bottom of the email.
    • Response Handling: Supports JSON responses for programmatic use and redirects for standard HTML forms via the Accept header. Configurable redirect_success and redirect_error URLs are available.
    • Deployment: Distributed as a standalone Go binary or a multi-arch Docker image (ghcr.io/craigmccaskill/posthorn). It is designed to run behind any TLS-terminating reverse proxy (Caddy, nginx, Traefik, Cloudflare, etc.).

  4. Posthorn's Outbound Abuse Posture and Responsibilities

    main

    Posthorn acts as a thin relay and does not operate as an outbound MTA (Mail Transfer Agent).

    Outbound Management

    Because Posthorn relays through external providers (Postmark, Resend, Mailgun, SES), outbound IP reputation management is the provider's responsibility. Posthorn does not manage sender reputation, implement content-based spam classification, or coordinate quotas across multiple providers.

    Posthorn's Role in Outbound Protection

    Posthorn provides the following mechanisms to protect the relay and the provider quota:

    • Token-bucket rate limit (FR8): Bounds per-endpoint, per-IP submission volume to prevent a single compromised caller from draining provider quotas.
    • Max body size cap (FR7): Bounds individual message size to protect Posthorn process memory and defend against payload-volume attacks.
    • Suppression list (v2, #4): Refuses to send to known-bouncing addresses to protect sender reputation.
    • Structured logs (FR17, NFR7): Logs every outbound decision with submission_id, endpoint, transport, and payload for operator forensics.

    Deliverability and DNS

    Posthorn does not own, promise, or test email deliverability, DKIM/SPF/DMARC, or inbox placement.

    • DKIM/SPF/DMARC: The provider handles signing (DKIM); the operator is responsible for DNS configuration.
    • Testing: Posthorn's live-provider tier verifies only that a well-formed request is accepted by the provider's API. It does not assert successful delivery to an inbox.

  5. How the Posthorn migration system works

    main

    Posthorn uses a bespoke, forward-only migration system.

    • File Location: Migration files are located at core/storage/migrations/00N_name.sql.
    • Naming Convention: Files must use a strict sequential numeric prefix (e.g., 001_baseline.sql, 002_add_xyz.sql).
    • Execution: On startup, the storage layer checks the _migrations table for the maximum applied version and applies any new embedded files in numerical order within a single BEGIN TRANSACTION.
    • Rollbacks: There are no rollback files. The system is forward-only. If a migration fails, you must restore from a backup. Operators are encouraged to run sqlite3 backup via posthorn validate or external tools regularly.

  6. Postmark transport behavior and error handling

    main

    The Postmark transport communicates with POST https://api.postmarkapp.com/email using the X-Postmark-Server-Token: <api_key> header. The request body is JSON containing fields such as From, To, Subject, TextBody, and ReplyTo.

    Posthorn classifies Postmark responses into the following error classes to determine retry logic:

    Postmark responseError classRetry?
    200 OK or 202 Accepted(success)no
    429 Too Many RequestsErrRateLimitedyes, after 5s
    5xx server errorErrTransientyes, after 1s
    Network timeout / connection refusedErrTransientyes, after 1s
    4xx other than 429 (401, 422, etc.)ErrTerminalno

    Note: 422 errors related to unverified sender signatures are common during onboarding. These are treated as ErrTerminal and will surface the upstream error in the submission_failed log line.

  7. JSON body shape and template variable coercion

    main

    The JSON request body must be a flat object. The keys in the object are used as template variables in the endpoint's subject and body configurations using the {{.key}} syntax.

    Coercion Rules:

    • Strings: Pass through unchanged.
    • Numbers: Render without a decimal if integer-valued (e.g., 42 becomes 42).
    • Booleans: Render as true or false.
    • Arrays of primitives: Become multi-value fields (e.g., ["a", "b"] renders as a b).
    • null: Omitted from the template.

    Restrictions (Returns HTTP 400):

    • Nested objects (e.g., {"user": {"name": "Alice"}}) are not supported. Flatten data at the caller level (e.g., "user_name": "Alice").
    • Arrays containing objects or other arrays.
    • Top-level arrays or primitives (the root must be an object).
    {
  "name": "Alice",
  "count": 42,
  "confirmed": true,
  "tags": ["urgent", "support"]
}

  8. Understand SMTP transport behavior

    main

    The SMTP transport follows a standard protocol flow: EHLO → STARTTLS → AUTH PLAIN → MAIL FROM → RCPT TO → DATA → QUIT.

    Key behavioral characteristics:

    • Connection Reuse: None. One TCP connection is established per Send request.
    • Timeout: 30s per request.
    • Message Construction: Hand-constructed RFC 5322 messages (UTF-8) containing From, To, Subject, and Reply-To headers plus a plain-text body.
    • Subject Encoding: Uses RFC 2047 B-encoding for non-ASCII subjects.
    • Message IDs: The transport_message_id is empty because the underlying net/smtp library does not expose the server's queued as <id> response. Correlate messages using timestamps and recipients in your relay's logs.

  9. Security and Threat Model for HTTP Form Ingress

    main

    In version 1.0, Posthorn focuses on securing HTTP form-based ingress. The following defenses are implemented to mitigate common web-based threats:

    • Drive-by scraper bots: Mitigated via a Honeypot field.
    • Direct-POST bots (skipping the form page): Mitigated via Origin/Referer checks (fails closed).
    • Targeted abuse: Mitigated via Token bucket rate limiting (supports trusted_proxies for proxy-aware limiting).
    • Postmark quota burn: Mitigated via Rate limiting and a Max request body size cap.
    • Email header injection: Mitigated by using a Structured-data transport API and explicit injection-payload test coverage.
    • API key theft: Keys are never included in logs or error output.

  10. Understand Posthorn's retry policy and error classification

    main

    Posthorn uses a simple, bounded retry policy designed for synchronous requests. It retries at most once per request and enforces a hard 10-second overall timeout for the entire operation. Failures are classified into three categories that determine the action taken:

    Transport responseClassAction
    200 OK / 202 AcceptedsuccessReturn 200 to client
    5xx server errorErrTransientWait 1s, retry once
    Network timeout / connection refusedErrTransientWait 1s, retry once
    429 Too Many RequestsErrRateLimitedWait 5s, retry once
    4xx other than 429 (401, 403, 422, etc.)ErrTerminalNo retry → 502 to client
    Either retry exhausted (still failing after retry)(terminal)502 to client
    10 seconds elapsed in total(timeout)Cancel in-flight, return 502

  11. How automatic suppression works

    main

    Posthorn automatically adds addresses to a global suppression list when it receives hard_bounce or spam_complaint events. This protects your provider reputation by preventing future sends to dead addresses.

    Behavior:

    • Future Sends: Any attempt to send to a suppressed address (via HTTP or SMTP) will be skipped.
    • HTTP Response: If all recipients in a request are suppressed, Posthorn returns 200 {"status": "suppressed", "suppressed": {"addr": "reason"}}. If only some are suppressed, the suppressed map in the response identifies the skipped addresses.
    • SMTP Response: Clients receive a 250 status.
    • Soft Bounces: soft_bounce events (e.g., full mailbox) do not trigger suppression.

    Suppression rows are exempt from standard retention pruning to ensure they persist.

  12. Handling failures and retries

    main

    Posthorn's current error handling behavior is as follows:

    • Transient Failures: Posthorn retries a request exactly once after a 1-second delay if a transient failure occurs.
    • Persistent Failures: If the retry also fails (e.g., the provider like Postmark is down), Posthorn returns a 502 error to the client.
    • Data Recovery: Because there is currently no persistent queue, failed submissions are not automatically retried later. However, the full form payload is logged at the ERROR level, allowing you to recover the data manually from the logs.

    Note: A SQLite-backed persistent retry queue is planned for v2.