vdirsyncer

repository·main·Indexed 23 days ago

https://github.com/pimutils/vdirsyncer

A command-line utility for synchronizing calendars and addressbooks between servers and local filesystems, or between two servers directly. It supports various storage types including CalDAV, CardDAV, filesystem, singlefile, and read-only HTTP, with specialized support for Google Calendar and Contacts via OAuth.

Tokens
18.3K
Snippets
36
Records
111
Agent score
82%

What's inside vdirsyncer

  1. Overview of vdirsyncer

    main

    vdirsyncer is a command-line tool designed to synchronize calendars and addressbooks between various servers and the local filesystem.

    Common use cases include:

    • Client-Server Sync: Synchronizing a remote server with a local folder. This allows you to use other local programs to modify events and contacts, which vdirsyncer then synchronizes back to the server.
    • Server-to-Server Sync: Synchronizing calendars or addressbooks directly between two different servers.

    It is intended to serve as a calendar and contact equivalent to OfflineIMAP for emails.

  2. Understand the vdirsyncer testsuite structure

    main

    The testsuite is organized into three main directories within tests/ to facilitate separate coverage reporting:

    • system: Contains system- and integration tests. Use this directory for tests that utilize temporary files.
    • unit: Contains tests where each testcase targets a single class or function.
    • storage: Runs a generic storage testsuite against all supported storage backends.
  3. When to use vdirsyncer instead of file synchronization

    main

    While vdirs (virtual directories) are composed of files and can technically be synchronized using file-sync tools like Syncthing, Dropbox, unison, or git, vdirsyncer is preferred for specific use cases:

    1. Conflict Resolution: File-sync services like Dropbox are often agnostic to file contents. If a file changes on both sides, they may create duplicate copies rather than merging data. In the context of tools like khal, this can lead to errors such as duplicate events with the same UID.
    2. Mobile Support: File-syncing setups (like git or Dropbox) are difficult to implement on smartphones. vdirsyncer synchronizes via CardDAV/CalDAV protocols, which are natively supported by mobile applications such as DAVx⁵ or other smartphone-bundled apps.
  4. Understand the vdirsyncer configuration format

    main

    vdirsyncer uses an ini-like format where all values are interpreted as JSON. If a value is not valid JSON, it is treated as a string.

    Supported JSON types include:

    • Strings: x = "foo" or x = foo (shorthand)
    • Integers: x = 42
    • Lists of strings: x = ["a", "b", "c"]
    • Booleans: x = true or x = false
    • Null: x = null (equivalent to Python's None)
  5. Fetch credentials using the `.fetch` configuration key

    main

    To avoid storing passwords or usernames in cleartext within your configuration file, you can use the .fetch suffix on any parameter in a [storage ...] section. This allows you to dynamically retrieve values using different strategies like executing commands or prompting the user.

    [storage foo]
    type = "caldav"
    url = ...
    username.fetch = ["command", "~/get-username.sh"]
    password.fetch = ["command", "~/get-password.sh"]
  6. Understand collections and how to pair them

    main

    A Collection is a collective term for addressbooks or calendars.

    Collection Identifiers

    Each collection has a unique identifier used in the collections parameter of a [pair] section:

    • Filesystem storage: The name of the directory representing the collection.
    • DAV storages: The last segment of the URL.
    • Note: This identifier is distinct from the displayname (the human-friendly name), which can change.

    Special Collection Names

    • "from a" or "from b": Placeholders that include all collections found on side A or B during discovery.
    • null: Tells vdirsyncer that the storage itself is the collection, rather than a container of collections. This is useful if your DAV server doesn't support listing collections or if you want to point directly to a specific collection URL.

    Manual Pairing (Server-to-Server Sync)

    When synchronizing two remote servers (e.g., NextCloud to iCloud), their collection names (UUIDs) likely won't match. You can manually pair them by providing a list of pairs in the collections parameter. Each sub-list in the array represents a mapping: ["local_name", "remote_name_a", "remote_name_b"].

    Example of manual pairing:

    [pair doublecloud]
    a = "my_nextcloud"
    b = "my_icloud"
    # Maps 'mytest' to the specific collections on both servers
    collections = [["mytest", "test", "3b4c9995-5c67-4021-9fa0-be4633623e1c"]]

    You can then sync only this specific mapping using: vdirsyncer sync doublecloud/mytest.

    [pair doublecloud]
    a = "my_nextcloud"
    b = "my_icloud"
    collections = [["mytest", "test", "3b4c9995-5c67-4021-9fa0-be4633623e1c"]]
  7. Understand the Vdir Storage Format

    main

    The Vdir storage format is a filesystem-based standard for storing calendars and contacts. vdirsyncer uses the filesystem storage type to synchronize to these directories. In this model, a 'vdir' is a directory representing a single calendar or address book.

    Structure

    • Root Folder: Contains multiple subfolders called collections (also known as addressbooks or calendars).
    • Items: Files contained within collections. Each file must represent exactly one event, task, or contact.

    Item File Types

    • vCard: Must use the .vcf file extension.
    • iCalendar: Must use the .ics file extension.

    Requirements

    • Items should contain a UID property. If multiple UID properties exist, their values must be identical.
    • Filenames should ideally relate to the UID of the content, though they do not have to match exactly.
  8. Use custom root CAs for certificate validation

    main

    To point vdirsyncer to a specific custom set of root CA certificates, use the verify parameter in your storage configuration. This is useful if your Python/aiohttp installation has broken default CA bundles or if you are using a private CA.

    [storage foo]
    type = "caldav"
    ...
    verify = "/path/to/cert.pem"
  9. Install vdirsyncer via OS/distro packages

    main

    vdirsyncer is available through various community-contributed package managers. Using a distribution's package manager is the recommended method.

    Supported platforms and packages:

    • Arch Linux: Available in extra.
    • Ubuntu and Debian: Available via packagecloud.io/pimutils/vdirsyncer (official repositories may be outdated).
    • macOS: Available via homebrew.
    • GNU Guix, NetBSD, OpenBSD, and Slackware (via SlackBuilds.org) also have packages.

    Important: Only the latest version of vdirsyncer is supported. Do not use older versions from 'stable' release channels (like Debian or Fedora stable) as they are not supported.

  10. Configure vdirsyncer to use DavMail for Exchange/Outlook

    main

    DavMail acts as a proxy that allows Card- and CalDAV clients (like vdirsyncer) to interact with Microsoft Outlook or Exchange servers.

    Warning: Success with DavMail varies depending on your specific Exchange server configuration. There is a risk of encountering errors or data loss. Always ensure you are using the latest version of DavMail.

    To use DavMail, configure a storage section in your vdirsyncer configuration with the caldav type, pointing to your local DavMail instance (typically http://localhost:1080).

    [storage outlook]
    type = "caldav"
    url = "http://localhost:1080/users/user@example.com/calendar/"
    username = "user@example.com"
    password = "..."
  11. Manual installation requirements

    main

    If a distribution package is unavailable, you can install vdirsyncer manually using Python's package manager.

    Prerequisites:

    • Python: Version 3.9 to 3.13 and pip must be installed.
    • System Libraries: libxml, libxslt, and zlib must be present.
    • Operating System: Linux or macOS. Windows is not supported.

    On Ubuntu/Debian, you can install the necessary system dependencies with:

    sudo apt-get install libxml2 libxslt1.1 zlib1g python3
  12. Configure the vdirsyncer timer interval

    main

    You can customize the synchronization frequency by overriding the vdirsyncer.timer unit. Use systemctl --user edit vdirsyncer.timer to open an editor and add override settings.

    Commonly used keys:

    • OnBootSec=: Defines how long after system boot the first run occurs.
    • OnUnitActiveSec=: Defines the interval between subsequent runs.
    systemctl --user edit vdirsyncer.timer

    Inside the editor, add overrides like this:

    [Timer]
    OnBootSec=5m
    OnUnitActiveSec=15m