WildDuck IMAP/POP3 Server

repository·master·Indexed 24 days ago

https://github.com/zone-eu/wildduck

A scalable, no-SPOF IMAP/POP3 mail server built with Node.js and a distributed, sharded, and replicated MongoDB backend. It includes tools for IMAP command parsing and compilation via imapHandler, MD5-based password hashing compatible with PHP, and a comprehensive installer for a full mail stack including Haraka, ZoneMTA, and Rspamd.

Tokens
32K
Snippets
55
Records
181
Agent score
85%

What's inside WildDuck

  1. Overview of WildDuck Mail Server

    master

    WildDuck is a scalable, no-SPOF (Single Point of Failure) IMAP/POP3 mail server. It is designed with a product philosophy similar to Gmail.

    Key architectural feature: WildDuck uses a distributed database (sharded and replicated MongoDB) as its backend for storing all data, including emails.

  2. WildDuck Logo Assets

    master

    The WildDuck graphics repository contains several versions of the logo for different use cases:

    • Main Logo: The standard WildDuck logo (wildduck.svg).
    • Logo with Text: The logo paired with the 'WildDuck' text using the Roboto Font (wildduck-type.svg).
    • Monochrome/Icon: A simplified version suitable for icon sets, Safari pinned tabs, and other small-scale UI elements (wildduck-mono.svg).
  3. IMAP Protocol Support and Standards

    master

    WildDuck IMAP server implements the IMAP4rev1 suite with several key extensions. Notable supported features include:

    • IDLE (RFC2177): Notifies clients about new/deleted messages and flag updates.
    • CONDSTORE (RFC4551) and ENABLE (RFC5161): Supports most of the spec (metadata is ignored).
    • STARTTLS (RFC2595): Secure connection.
    • UIDPLUS (RFC4315): Enhanced UID handling.
    • UTF8=ACCEPT (RFC6855): Native support for Unicode email usernames.
    • QUOTA (RFC2087): Global account quota based on the sum of RFC822 sources of stored messages.
    • COMPRESS=DEFLATE (RFC4978): Traffic compression.
    • MOVE (RFC6851): Message moving.

    Performance Note: WildDuck may be slower than Dovecot when fetching single users because it recomposes messages from different parts rather than reading directly from a filesystem. However, it offers better parallelization via MongoDB sharding and faster synchronization speeds due to its write-ahead log.

  4. Understand the WildDuck architecture and services

    master

    WildDuck is a mail server built on Node.js, MongoDB, and Redis. It provides several decoupled services that can be enabled or disabled individually, allowing you to distribute specific roles across different hosts (e.g., running IMAP on one host and LMTP on another).

    Core services include:

    • IMAP server: For mail retrieval.
    • POP3 server: For mail retrieval.
    • LMTP server: For pushing messages into the mail store.
    • HTTP API server: For administrative tasks like creating new users.
  5. Leverage built-in email features: Address labels and dot handling

    master

    WildDuck supports several automated email addressing behaviors:

    • Address Labels: You can use the format username+label@example.com, which is automatically delivered to username@example.com.
    • Dot Handling: Dots in usernames and addresses are treated as informational only. For example, username@example.com is treated identically to user.name@example.com.
  6. Access messages via IMAP and HTTP API

    master

    WildDuck provides two primary ways to interact with and access email messages:

    1. IMAP: Traditional mail access protocol.
    2. HTTP API: A modern interface that serves parsed data. Unlike IMAP, which requires fetching full RFC822 messages and parsing them manually, the HTTP API provides structured data (such as HTML, plaintext content, and attachments) directly. This makes it ideal for building webmail interfaces.

    For detailed API specifications, refer to the HTTP API documentation.

  7. How WildDuck manages session state and updates

    master

    WildDuck is designed to be distributed by keeping minimal session state (primarily a list of known UIDs and the latest MODSEQ value).

    Mailbox Loading and Updates:

    • Mailbox Opening: When a mailbox is opened, the entire message list is loaded as an array of UID values.
    • Journaling: Information about updates (new/deleted messages, flag changes) is stored in a journal log.
    • Synchronization: An update beacon is propagated via Redis pub/sub whenever a change occurs.
    • Applying Updates: If a session detects changes (e.g., via a NOOP call), it loads the journaled log from the database and applies updates to the UID array one by one until the state matches the latest database state. If a change cannot be notified (e.g., during a FETCH call), the user continues to see the previous state until the next opportunity for synchronization.
  8. What the CONDSTORE extension provides in WildDuck

    master
    WildDuck implements the CONDSTORE extension (RFC 4551). This extension allows IMAP clients to perform conditional STORE operations and track modification sequences (modseq). This is useful for synchronizing mailbox states efficiently by only acting on messages that have changed since a specific sequence number.
  9. How WildDuck auditing works

    master

    WildDuck provides built-in auditing capabilities that allow administrators to enable message auditing for specific email accounts.

    Key characteristics of the auditing process:

    • Message Isolation: Audited messages are copied rather than linked. If a message is deleted from the original email account, the copy remains available for the audit.
    • Time-Bound: Audits are restricted to a specific time frame, limiting the scope of messages exposed to the auditor.
    • Automatic Expiration: Once the audit period expires, all copied messages are automatically deleted and auditor access is revoked.
    • Secure Access: Administrators can generate designated access credentials encrypted with PGP keys. Administrators cannot access email content directly; instead, they provide the encrypted credentials to the auditor, who then decrypts them into a CSV file.
    • Auditor Capabilities: Once decrypted, auditors can view message listings and metadata, search emails by address, date, or subject, and download individual emails or ZIP bundles containing multiple emails.
  10. Hash format and PHP compatibility

    master

    The hashes generated by this library are compatible with PHP's crypt() function.

    Hash Structure: $1$X9U0NCH4$1.cDTvOaCzP41UQ699rOU0

    • $1$: Identifies the hash as being based on MD5.
    • X9U0NCH4: The Salt.
    • 1.cDTvOaCzP41UQ699rOU0: The Hashed string.

    Comparison Example: Both the JavaScript implementation and PHP's crypt() will produce the same output for the same input:

    JavaScript:

    var CryptMD5 = require('./cryptmd5.js');
    console.log(CryptMD5.cryptMD5('focus123', 'erXgIjX7'));
    // Returns: $1$erXgIjX7$fi/gmab/rku/qc6.ivndo0

    PHP:

    echo crypt('focus123', '$1$erXgIjX7');
    // Returns: $1$erXgIjX7$fi/gmab/rku/qc6.ivndo0
  11. How IMAP command parsing and compilation work together

    master

    The IMAP handler follows a two-step lifecycle for processing commands:

    1. Parsing: A raw IMAP string (received from a client) is passed to imapHandler.parser(). This decomposes the string into a structured commandObject that identifies tags, commands, and attributes (like SEQUENCE, ATOM, or LITERAL).

    2. Compilation: To respond to a client, you construct a commandObject (either by modifying a parsed one or creating a new one) and pass it to imapHandler.compileStream(). This returns a Stream rather than a string. This design allows the server to pipe large data (like email attachments or bodies) directly to the network socket, minimizing memory footprint.

    Note for Server Developers: Because the compilation returns a stream, it is currently not possible to pause the output stream to wait for a '+' tagged server response for literal values. This makes the handler optimized for server-to-client communication rather than client-to-server.