certificate-transparency-go

Migrillian is a tool designed to transfer data from existing Certificate Transparency (CT) logs into Trillian PREORDERED_LOG trees.

Note: This tool is currently in development and is not ready for production use.

Common use cases include:

• One-off data migrations: Moving data from legacy CT implementations to a new Trillian-based solution.
• Continuous migration (Log Mirroring): Keeping a local copy of a log up-to-date with a remote log continuously.

Trillian provides a general gRPC API for Merkle tree operations, but it requires a personality to perform application-specific operations. For Certificate Transparency, the ctfe/ directory contains the CTFE personality.

Key features of the CTFE personality include:

• Providing HTTP/JSON API entrypoints as described in RFC 6962.
• Validating that Log submissions are valid X.509 certificates with a signature chain reaching an acceptable root.

The CTFE is generally stateless, multi-tenant (supporting parallel log instances), and is controlled via a configuration file.

A CT Log is a cryptographic entity that signs data using a private key. This key is required by all distributed CTFE instances and must be kept highly secure.

Security Requirements:

• No Re-use: The CT Log key must not be re-used for distinct Logs.
• No TLS Re-use: The CT Log key should not be re-used for HTTPS/TLS termination.
• Public Key: The corresponding public key is required to register the Log with browsers for trust.

To prevent unbounded growth of Log instances, it is recommended to use temporal sharding. This involves setting up a collection of separate Log instances (each with its own private key) that only accept certificates with a NotAfter date within a specific range (e.g., a calendar year).

In Trillian-based Logs, you can implement this by setting the [not_after_start, not_after_limit) range in the CTFE configuration files.

The Trillian CT Personality acts as an adapter layer that enables a Trillian Log to function as a Certificate Transparency (CT) Log.

Key architectural components:

• trillian/ctfe: The core logic. It listens for HTTP requests following the RFC 6962 CT API and translates them into gRPC requests for the underlying Trillian Log.
• Trillian Dependencies: The personality relies on the github.com/google/trillian repository for gRPC API definitions, cryptographic key management (github.com/google/trillian/crypto/...), and monitoring/statistics via Prometheus (github.com/google/trillian/monitoring/...).

While regular backups are recommended for persistent data, restoring a CT Log backup is dangerous.

The Risk: If any data has been accepted and a signed promise-to-include has been issued since the backup was taken, performing a restore will effectively fork the underlying Merkle tree. This breaks the tree's append-only property and results in log disqualification.

The Trillian log signer requires a single active instance to ensure a unique sequencing of entries. To achieve resilience without sacrificing the single-signer constraint, an election process is used via an etcd cluster.

To run multiple instances of the trillian_log_signer with election enabled:

1. Set up an etcd cluster.
2. Run the signer with the --etcd_servers flag pointing to the cluster (comma-separated list of host:port).
3. Ensure the --force_master option is removed.

Example:

# Run signer with etcd for election
trillian_log_signer --etcd_servers etcd1:2379,etcd2:2379,etcd3:2379

Trillian uses an internal storage interface to record logged certificates. This deployment guide uses the MySQL implementation. You must first configure a MySQL database according to the Trillian core schema.

Cross-check: Verify the setup by connecting to your MySQL instance and checking for the required tables. A successful setup should include the following tables:

• LeafData
• SequencedLeafData
• Subtree
• TreeControl
• TreeHead
• Trees
• Unsequenced

% mysql --host=127.0.0.1 --port=3306 --user=root --database=test
MariaDB [test]> show tables;
+-------------------+
| Tables_in_test    |
+-------------------+
| LeafData          |
| SequencedLeafData |
| Subtree           |
| TreeControl       |
| TreeHead          |
| Trees             |
| Unsequenced       |
+-------------------+

Deploying a CT Log requires configuring multiple components. There are two primary deployment paths documented in the repository:

1. Manual Deployment: Use docs/ManualDeployment.md to set up components and processes on individual machines. This is recommended for operators of trusted CT Logs to ensure full understanding of the environment.
2. Containerized Deployment: Use docs/ContainerDeployment.md for sample container scripts designed to make deployment more automatic and easier.

Reliable operation requires connecting monitoring to an alerting system. You should monitor both standard operational metrics and CT-specific metrics.

Standard Operational Metrics:

• Error rates (categorized by read/write paths and 4xx/5xx status codes).
• Request latency distribution.
• Task health, CPU, and memory usage.

CT-Specific Metrics:

• Merkle tree head age: The age of the most recent Merkle tree head.
• Unmerged backlog: The size of the current backlog of unmerged submissions.
• Primary signer count: The number of primary signer instances per log instance (should normally be 1; can transiently be 0, but must never be > 1).