Tessera

repository·main·Indexed 42 days ago

https://github.com/transparency-dev/tessera

A Go library for building high-performance, tile-based transparency logs (tlogs). Designed as a scalable, low-TCO successor to Trillian v1, it is optimized for cloud and on-premises infrastructure using the tlog-tiles API. It includes tools for conformance testing on AWS and GCP, log integrity verification via fsck, and support for POSIX-compliant filesystem logs.

Tokens
55.1K
Snippets
154
Records
246
Agent score
54%

What's inside Tessera

  1. Overview of Hammer load testing tool

    main

    Hammer is a load testing tool designed for tlog-mirror servers. It generates write traffic according to the tlog-mirror specification to test both the correctness and performance of a mirror under load.

    To use Hammer, the target mirror server must support POST requests at the following paths:

    • /add-checkpoint
    • /add-entries

    By default, Hammer runs with an interactive text-based UI (TUI) in the terminal, allowing you to monitor status, logs, and adjust read/write traffic levels in real-time. For headless environments like CI pipelines, you should disable the UI using the --show_ui=false flag.

  2. Overview of Tessera's architecture and purpose

    main

    Tessera is a library designed for managing Merkle trees and building transparency applications. Unlike Trillian v1, which is a collection of microservices, Tessera is a library that provides data storage and management tools. It is intended to be used as a foundation to build other transparency-related applications rather than being run as a standalone production binary.

    The architecture is divided into several layers:

    1. Application Layer: Uses PersonalityKit (Tessera's application support library) to manage t-log instances, fetch data, or build/verify proofs.
    2. Tessera Log API: The core interface for interacting with logs.
    3. Storage Layer: Composed of Tessera StorageKit (for authors of storage implementations) and infrastructure libraries. This layer manages the persistence of t-log state.

    Tessera is opinionated: it only supports building t-logs that conform to the tlog-tiles or static-ct-api specifications and enforces the use of tlog-checkpoint for state commitments.

  3. Overview of Tessera

    main

    Tessera is a Go library designed for building tile-based transparency logs (tlogs) following the tlog-tiles API. It is intended as the successor to Trillian v1, offering a library-based architecture (rather than microservices) to reduce operational complexity and Total Cost of Ownership (TCO).

    Key characteristics include:

    • Infrastructure Support: Drivers available for AWS, GCP, and POSIX filesystems.
    • Performance: High read-throughput and read-availability, with fast sequencing and integration.
    • Extensibility: Allows building arbitrary log personalities, including support for Static CT API compliance.
    • Optional Features: Supports 'best-effort' de-duplication and synchronous integration.

    Note: To achieve maximum read scaling, logs should use the tlog-tiles API. While you can build a custom personality to adapt other APIs to Tessera, you may lose the scaling benefits.

  4. Overview of the POSIX MTC Mirror

    main

    The POSIX MTC Mirror is a server implementation backed by Tessera's POSIX storage. It is designed to comply with the tlog-mirror specification.

    A tlog-mirror serves two roles:

    1. A regular tlog-witness server (which can optionally sign consistent log checkpoints).
    2. An "extended" tlog-witness that signs a consistent checkpoint only after it has successfully stored a copy of the log content associated with that checkpoint.
    WARNING

    This binary and its internal packages are under active development, are considered experimental, and are not production-ready. They are outside the SemVer policy.

  5. Overview of Tessera on AWS storage architecture

    main

    The AWS storage implementation for Tessera uses a hybrid approach to balance write coordination and read scalability:

    • Transactional Storage: Uses Amazon Aurora (MySQL) to coordinate writes. New entries are held in transactional storage to be batched and assigned sequence numbers. This allows the Add API to return quickly with durably assigned sequence numbers.
    • Long-term/Read Storage: Uses Amazon S3 for low-cost, low-complexity serving of read traffic. An asynchronous process derives entry bundles and Merkle tree structures from sequenced batches and writes them to S3.

    Because entries are sequenced in chunks before being stored in the tree, all tree derivations are idempotent.

  6. Understand the CloudBuild pipeline lifecycle

    main

    The CloudBuild pipeline is automatically triggered by commits to the main branch of the repository. Its primary responsibilities are:

    1. Building the cmd/gcp/conformance Docker image from the main branch.
    2. Provisioning a fresh conformance testing environment.
    3. Executing conformance tests against the newly built Docker image.
    4. Decommissioning (turning down) the conformance testing environment.
  7. What is a Tessera Personality?

    main

    Tessera is a Go library designed to efficiently serve logs via the tlog-tiles API.

    Because Tessera is a generic library, the code you write to integrate it into your specific ecosystem is referred to as a personality. When building a personality, your first decision is choosing a storage driver. The native drivers available are:

    • GCP: Recommended for production in Google Cloud.
    • AWS: Recommended for production in AWS.
    • POSIX: Simplest for local development or environments where you already serve static files.

    Cloud implementations (GCP/AWS) are generally easier to scale and operate for production use cases.

  8. Serve log data via the tlog-tiles API

    main

    Tessera makes logs readable via the tlog-tiles API. How you serve this data depends on your storage driver:

    • AWS/GCP: Data is written to object storage. The operator must ensure the object storage is publicly readable and set up a URL pointing to it.
    • POSIX: The operator must serve the files (which are written according to the API spec) using an HTTP File Server.
  9. How integration works in Tessera

    main

    Integration is a background process that begins once a Tessera lifecycle object is created. It takes entries that have been sequenced and merges them into the log.

    An entry is considered integrated when:

    • It becomes available via the read API at its assigned index.
    • It has Merkle tree hashes that commit to its inclusion in the tree.
  10. Understand Conformance Personalities

    main

    Conformance Personalities are server implementations built on top of Tessera that serve two purposes: providing simple deployment examples for different backends and acting as conformance/performance harnesses for Tessera using hammer.

    Available implementations include:

    • Local POSIX-compliant filesystem
    • GCP
    • AWS

    Each personality exposes a POST endpoint at /add. When you send a request body to this endpoint, the contents are appended to the log, and the server returns the decimal index assigned to that newly sequenced entry.

  11. How Tessera operates on Google Cloud Platform

    main

    Tessera on GCP uses a hybrid storage architecture to balance high-performance writes with low-cost, scalable reads:

    1. Transactional Storage (Cloud Spanner): Used for coordinating the cluster and holding new entries temporarily. This allows the Add API to return quickly with durably assigned sequence numbers.
    2. Long-term Storage (Google Cloud Storage - GCS): An asynchronous process derives entry bundles and Merkle tree structures from sequenced batches and writes them to GCS. This provides low-cost and low-complexity serving of read traffic.

    Because entries are sequenced in chunks before being stored in GCS, all tree derivations are idempotent.

  12. How Tessera storage implementations work

    main

    Tessera manages t-log state through concrete storage implementations. While the library provides implementations for GCP and POSIX filesystem, third parties can construct their own.

    All storage implementations must follow these guidelines:

    • High-level APIs: Implementations must expose APIs that support the various lifecycle modes (e.g., Appender, LogReader).
    • Native Abstractions: APIs are designed to be high-level enough to allow implementations to act natively to their target infrastructure (e.g., avoiding forced transactional models that might hinder optimization).
    • Spec Compliance: If the storage infrastructure can be directly exposed to clients (like S3 or GCS via HTTP), internal Merkle tree node data and leaves MUST be stored according to the tlog-tiles or static-ct-api specifications.
    • Option Support: Implementations SHOULD support the "With Options" patterns defined by Tessera, such as WithBatching and WithCheckpointSigner.