tebako Documentation

repository·main·Indexed 19 days ago

https://github.com/tamatebako/tebako

A packaging and loading ecosystem that bundles applications and their runtime environments into single, zero-dependency executables. It utilizes a virtual filesystem (TFS), a bootstrap loader, and a preload interposition shim (libtfs-preload) to provide isolated, portable, and secure execution across platforms. Features include 'lean' and 'fat' packaging modes, a 'Jail' mechanism for host access restriction, and specialized deployment flows for Ruby applications using Gemfiles.

Tokens
111.7K
Snippets
295
Records
526
Agent score
65%

What's inside tebako

  1. What is a toolkit slice and when to use one

    main

    A toolkit slice is a mechanism for packaging optional, specialized, or large capabilities (such as cryptography or UI toolkits) as downloadable images.

    Instead of bloating the core tebako bootstrap with heavy dependencies (like full OpenPGP libraries), capabilities are moved into slices. This allows the core to remain small (e.g., under 3 MB) while providing machinery that other system components can call upon when needed.

    Key characteristics:

    • It is not an interpreter or an end-user application; it is machinery for other components.
    • It is downloaded and verified on demand.
    • If a required capability is missing, the system reports the specific missing capability by name rather than failing with a generic error.
  2. What is a data slice and how is it used

    main

    A data slice is a versioned artifact that packages content (such as fonts, schemas, templates, dictionaries, model weights, or datasets) rather than executable code. While it shares the same physical form as other slices—an image plus a manifest—it does not provide commands. Instead, applications mount it as read-only content.

    Key Characteristics

    • Shared Content: One copy on disk can be shared by multiple applications, preventing duplication.
    • Independent Lifecycle: Data slices have their own version lines and can be upgraded without touching the consuming applications.
    • Platform-Free: Most data slices are platform-independent, meaning a single image can serve multiple CPU architectures or operating systems.
    • Security: Data slices are verified using the same checksums and signatures as code slices.
  3. What is the tebako packaging and loading ecosystem?

    main

    Tebako is a generic packaging and loading ecosystem designed to be runtime-agnostic, platform-agnostic, and payload-agnostic. It allows developers to stitch multiple payload images (application code, data, fonts, runtimes, etc.) into a single executable (a "fat" binary) or a small executable that resolves parts on demand (a "lean" binary).

    Core capabilities include:

    • Stitch: Combining $N$ payload images and a bootstrap into one executable.
    • Load: Recursively mounting payloads into a single Virtual File System (VFS) and transferring control to a declared entrypoint.
    • Share: Using a machine-wide content cache so runtimes and payloads are downloaded only once.
    • Shim: Automatically managing shims for every executable provided by an installed payload.
    • Jail: Enforcing declarative host-filesystem policies per run via the VFS layer.
    • Trust: Providing opt-in authentication, signatures, and encryption for packages.
  4. The lifecycle of a data slice

    main

    The lifecycle of a data slice follows five distinct stages:

    1. Press: The content tree is packaged into an image with a manifest containing its name, version, and content hash.
    2. Publish: The slice is uploaded to a registry.
    3. Resolve: When an application starts, the resolver checks the local store for a compatible version; if not found, it is downloaded.
    4. Mount: The slice is mounted read-only at the specific path declared by the application.
    5. Prune: The local store tracks dependencies; cleanup processes (garbage collection) remove only the data slices that are no longer required by any application.
  5. Understand the tebako content store structure

    main

    Downloaded artifacts are kept in a content store organized by type. Versions of the same artifact sit side-by-side and are never overwritten. Installations are atomic (using a lock, temporary directory, and rename) to prevent half-written artifacts.

    Artifacts include two marker files: a verified checksum and an origin file.

    Store Layout:

    store/
      runtimes/<language>/<version>/<tebako-abi>/<platform>/
      toolkits/<name>/<version>/<platform>/
      payloads/<name>/<version>/<platform or universal>/
      data/<name>/<version>/
  6. How language bindings for tpkg work

    main

    The tpkg contract is manipulated directly within each language rather than by shelling out to foreign binaries. This ensures high fidelity and performance. The contract is implemented via:

    • crates/tpkg for Rust
    • tpkg.h for C (via cbindgen)
    • tpkg.rb for Ruby (formalized from the original gem's trailer reading logic)
  7. Configure runtime versioning and compatibility

    main

    Applications should declare a version range rather than an exact runtime version to allow for seamless updates.

    Compatibility Rules:

    • Pure-language applications (e.g., pure Ruby): Can typically accept a wide range (e.g., >= 3.3, < 5.0).
    • Applications with native extensions: Must lock to the specific ABI line they were built against.
    • Contract Versioning: The loader and runtime share a contract version. If a runtime changes its launch requirements, the bootstrap will detect this and return a named compatibility error rather than attempting a mis-launch.

    Upgrading a runtime within a compatible range almost never requires an application rebuild.

  8. Distinguish between Schemas and Declarations

    main

    In the tebako contract model, it is critical to distinguish between the grammar and the instance:

    1. Schemas: The versioned grammars themselves. These live in a single registry: docs/spec/schemas/<name>.yaml within the product repository. Every schema must contain schema: <name> and schema_version: <int>.
    2. Declarations: Specific YAML instances that conform to a schema. Examples include in-image manifests, layout.yaml, release-manifest.yaml, tpkg-registry.yaml, the package L2 block, and config.yaml.

    Validation Workflow:

    • Producers validate their declarations against their owned schema during CI.
    • Consumers validate declarations before use and must refuse invalid input with a named error.
  9. Understand the tebako key model and sharing

    main

    tebako uses a hierarchical key model based on OpenPGP (via rnp-rs) to manage access control without a central authority.

    Key Types

    • DEKs (Data Keys): These encrypt the actual content blocks. They are persisted as OpenPGP session keys using native wrap/unwrap (PKESK packets).
    • Session Keys (Grant Envelopes): Sharing access involves wrapping a DEK for a specific recipient (using ephemeral ECDH). A wrapped DEK combined with a manifest digest acts as a capability: possessing it grants access to that specific subtree.

    Access Control Patterns

    • Hierarchical Derivation: Subtree keys are derived from a root DEK using HKDF with the file path as the info parameter. A key for /a/b grants access to /a/b/** but cannot access /a/c or /a (one-way derivation).
    • Selective Disclosure: A single artifact can serve multiple audiences. The manifest maps specific recipients to specific directories or layers, allowing each recipient to open only their assigned slice.
    • Crypto-shredding: Access can be revoked prospectively by re-issuing the manifest without the recipient's envelope, or by rotating the DEK for future versions. Destroying a DEK renders the content unrecoverable everywhere.
  10. Understand the Contract Graph and Component Lifecycle

    main

    The tebako ecosystem operates on a strict 'Contract Graph' where every interface between components is governed by a directed edge (owner → consumer) and a declarative YAML contract.

    Core Principles:

    • Fail-Closed Verification: Consumers must verify contracts before execution. A mismatch results in a named error identifying both parties.
    • Single Ownership: Every interface has exactly one owner, one versioned YAML schema, and one verification point.
    • Era-based Security: Anything undeclared is considered 'pre-era (era 1)' and is refused by name; it is never assumed or silently served.

    **Key Lifecycle Stages (Contracts):

    • C1 (Source Tarball → Runtime Factory): Uses tarball-layout.yaml.
    • C2 (Runtime Factory → Loader): Produces a release-manifest.yaml.
    • C3 (Runtime Exe ↔ Env Image): Governed by layout.yaml.
    • C4 (Exe ↔ Static Libs): Governed by c_api.h and filesystem TU.
    • C5 (Handoff): Uses argv/env with contract_version (per Spec 17).
            ┌────────────────────────────────────────────────────────────────┐
            │ tamatebako/ruby — SOURCE FACTORY                               │
            │  out: patched src tarball (tebako-mount-root, SHA256SUMS)      │
            └───────────────┬────────────────────────────────────────────────┘
                            │ C1 tarball-layout.yaml
                            ▼
    ┌──────────────┐  C19   ┌───────────────────────────────────────────────┐
    │ vcpkg        │───────▶│ tebako-runtime-ruby — RUNTIME FACTORY         │
    │ baseline     │ tags   │  builds: runtime exe + env image              │
    └──────────────┘        │  in:  link unit (tebako-rs artifact)           │
                            │  out: runtime release (exe, image, manifest)   │
                            └──────┬───────────────┬───────────────┬────────┘
                                   │ C2 release-   │ C3 exe↔image  │ C4 link-time
                                   │ manifest.yaml │ (layout.yaml) │ (c_api.h, fs TU)
                                   ▼               ▼               ▼
                            ┌──────────────┐  ┌─────────────┐  ┌───────────────┐
                            │ registries / │  │ runtime exe │  │ static libs   │
                            │ release idx  │  │ (driver in) │  │ (driver, tfs) │
                            └──────┬───────┘  └──────┬───────────────┘
                                   │ C12             │ C5 handoff (spec 17, 
                                   │ registry.yaml   │ argv/env — contract_version)
                                   ▼                 │
    ┌─────────────┐  C13 store    ┌─────────────────▼───────────────────────┐
    │ store       │◀──────────────│ LOADER FAMILY                           │
    │ (~/.tebako) │               │ bootstrap · shim · cli · tfs-cli       │
    └──────┬------┘               └───┬────────┬─────────┬────────┬─────────┘
           │ C13                      │ C6     │ C7      │ C8     │ C9
           ▼                          ▼        ▼         ▼        ▼
      everything local           package   payload    registry  runtime
                               trailer   manifests          release manifest
  11. How Tebako shims and dispatch work

    main

    Tebako uses a shim-based model to manage executable registration and versioning. Instead of calling binaries directly, you call a tebako-shim (the dispatcher) located in ~/.tebako/shims/.

    When a command is invoked, the dispatcher performs the following resolution steps:

    1. Payload Version Resolution (First match wins):
      • TEBAKO_<TOOL>_VERSION environment variable.
      • The nearest .tebako-tools.yaml file (walking up from the current working directory).
      • User default version (configured via tebako use <tool>@<version>).
      • The default version specified in the registry.
    2. Runtime Resolution:
      • The dispatcher looks at the entrypoint's runtime_requirement.
      • It selects the newest compatible cached runtime.
      • If no compatible runtime is cached, it downloads the newest compatible version.
    3. Execution:
      • The dispatcher mounts the payload, the selected runtime(s), and any declared dependency mounts.
      • It applies the jail view (if applicable) and executes the entrypoint.

    Note: Swapping runtimes does not require changing the payload, as payloads are immutable and runtime selection happens at dispatch time.

    # Example of how a tool might be invoked via the shim
    # The shim resolves the version and runtime automatically
    tebako-tool --version
  12. Understand the Tebako vocabulary and core abstractions

    main

    To use the Tebako ecosystem, you must understand its fundamental building blocks:

    • payload: A single TFS image (.tfs) containing an in-image manifest.
    • runtime: A specific kind of payload that provides an interpreter.
    • package / tpkg: The complete three-part executable consisting of a bootstrap, slots, and a trailer.
    • bootstrap: The Rust loader (tebako-bootstrap) which serves as the process entry point.
    • TFS: The userland virtual filesystem layer.
    • shim: A command registered on the system PATH that dispatches execution into a payload.