Chatmail Core

repository·main·Indexed 21 days ago

https://github.com/chatmail/core

A low-level library for secure, scalable email-based messaging implementing network and encryption protocols including SMTP, IMAP, rPGP, and P2P networking. It provides a C interface (deltachat_ffi), a JSON-RPC 2.0 API via deltachat-rpc-server, and client libraries for Python, TypeScript/JavaScript, and Go.

Tokens
84.9K
Snippets
283
Records
400
Agent score
72%

What's inside chatmail-core

  1. Overview of Delta Chat CI and maintenance scripts

    main

    The project provides several scripts to maintain code quality and automate testing/documentation:

    Rust Quality

    • clippy.sh: Runs cargo clippy for all Rust code.
    • deny.sh: Runs cargo deny for all Rust code.

    Python Testing

    • run-python-test.sh: Runs CFFI Python tests (recreates environment from scratch).
    • run-rpc-test.sh: Runs JSON-RPC Python tests.
    • remote_tests_python.sh: Rsyncs to a build machine and runs JSON-RPC Python tests remotely.

    Documentation and Builds

    • run-doxygen.sh: Generates C-docs for upload to https://c.delta.chat/.
    • run_all.sh: Builds Python wheels.
    • android-rpc-server.sh: Compiles deltachat-rpc-server binaries using the Android NDK.
    • codespell.sh: Spellchecks the source code using the codespell tool.
    • remote_tests_rust.sh: Rsyncs to the build machine and runs Rust tests remotely.
  2. Overview of CFFI Python Bindings

    main
    The CFFI Python bindings provide access to the chatmail core library. This core library implements standard email protocols including IMAP, SMTP, MIME, and OpenPGP. It exposes a low-level API for managing Chats, Contacts, and Messages, making it suitable for building user interfaces or automated bots.
  3. Use the deltachat Python High Level API

    main

    The Delta Chat Python High Level API is organized around four primary classes that manage the lifecycle of your account, contacts, chats, and messages.

    • deltachat.Account: The main entry point for the library. You use this class to initialize your session and it serves as the factory for creating other objects.
    • deltachat.Contact: Represents a person or entity you can interact with.
    • deltachat.Chat: Represents a conversation thread between users.
    • deltachat.Message: Represents an individual message within a chat.
  4. Choose between JSON-RPC and CFFI Python bindings

    main

    Delta Chat provides two distinct sets of Python bindings for the Rust Core. Choosing the right one depends on your project requirements:

    • JSON-RPC Bindings (Recommended): Use these for all new projects. They are the primary focus for core developers and are used in the Delta Chat Desktop app. New APIs are most likely to be released here first.
    • CFFI Bindings: These are the legacy bindings used by many existing bot projects and core library tests. While they will be maintained through 2024, they are moving into a maintenance-only mode, meaning new features may not be added.
  5. Understand the deltachat Python binding layers

    main

    The deltachat package provides two distinct layers of abstraction for interacting with the Delta Chat core (written in Rust):

    1. Low-level CFFI bindings: These map directly to the C interface of the Delta Chat core.
    2. High-level Python bindings: These are built on top of the CFFI bindings to provide a more idiomatic Python experience.
  6. Use deterministic collections (BTreeMap/BTreeSet)

    main

    Prefer BTreeMap over HashMap and BTreeSet over HashSet.

    Using B-Tree based collections ensures that iterating over the structures returns items in a deterministic order. This is critical for preventing non-deterministic bugs, flaky tests, and inconsistent behavior across different devices when processing messages.

  7. Access the DeltaChat JSON-RPC API

    main

    The deltachat-jsonrpc crate provides a JSON-RPC 2.0 interface to DeltaChat, which can be accessed in two ways:

    1. Via CLI (stdio): Use the deltachat-rpc-server executable to expose the JSON-RPC API through standard input/output.
    2. Via C FFI: Call the JSON-RPC API directly through the C interface. The exposed functions are:
      • dc_jsonrpc_init
      • dc_jsonrpc_request
      • dc_jsonrpc_next_response
      • dc_jsonrpc_unref

    Detailed documentation for these functions is available in the deltachat-ffi/deltachat.h header file.

  8. Resolve group membership inconsistencies via GMM replay

    main

    To heal inconsistent group memberships, the system can replay Group Membership Modification (GMM) messages. Delta Chat uses explicit GMM messages (as detailed in the Spec) which are typically encrypted to the group members.

    When an inconsistency is detected, the following mechanism is used to synchronize members:

    1. group-membership-tracking: All valid GMM messages must be persisted in their full raw MIME-format in the database (e.g., in the msgs table, potentially using an extended mime_header column).
    2. consistency_checking: If an incoming GMM contains a member list that contradicts the local view, the device broadcasts a Group-Member-Correction message to all members. This message contains a multipart list of the relevant GMMs.
    3. correcting_memberships: Upon receiving a Group-Member-Correction message, the device passes the contained GMMs directly to the incoming mail pipeline.

    Note: To avoid infinite recursion, GMMs received via a Group-Member-Correction message must bypass the consistency_checking step and be processed as if they were standard incoming messages. If a device has already seen the Message-ID of a replayed GMM, it should ignore it.

  9. How webxdc works

    main

    webxdc is a decentralized web app ecosystem that uses existing chat groups as a communication layer instead of traditional web servers. This allows for serverless, secure, and offline-first web applications.

    Technical Workflow

    1. Distribution: An app (provided as an archive or bundled HTML) is sent as an attachment to a chat group.
    2. Execution: When a chat member taps the attachment, the app opens in a sandboxed system web view.
    3. Sandboxing: The sandbox prevents the app from making network requests, reloading code, or fetching external resources.
    4. Communication: Since the app cannot access the internet directly, it communicates by sending "app-state" update messages to other app users within the same chat group.
  10. Encryption standards for Chatmail

    main

    Chatmail messages SHOULD be encrypted using the Autocrypt standard. The setting prefer-encrypt=mutual MAY be used by default.

    Metadata SHOULD be protected using the Header Protection standard (RFC 9788) following this specific Header Confidentiality Policy:

    • from: Returns the RFC 5322 addr-spec part.
    • to: Returns "hidden-recipients": ;.
    • date: Returns a UTC timestamp of a random date within the last 7 days.
    • subject: Returns [...].
    • message-id or chat-is-post-message: Returns the original value.
    • All other headers: Returns null.
  11. How plugin hooks work in Delta Chat Python bindings

    main

    The Delta Chat Python bindings use pluggy to manage plugin registration and hook execution. Plugins are categorized into two types based on their scope:

    1. Global plugins: These are active across all accounts. They are useful for lifecycle events that happen at the application level, specifically implementing hooks at account-creation and account-shutdown times.
    2. Account plugins: These are scoped to a single Account instance and are only active during that specific account's lifetime.
  12. How the Delta Chat JSON-RPC architecture works

    main
    The JSON-RPC implementation follows a client-server model. The deltachat-rpc-client Python package acts as a client that connects to a standalone deltachat-rpc-server binary. The server manages the Delta Chat logic, while the Python client provides a high-level interface for developers to interact with the server via JSON-RPC calls.