AKD (Auditable Key Directory)

repository·main·Indexed 43 days ago

https://github.com/facebook/akd

An implementation of an auditable key directory (verifiable registry/authenticated dictionary) designed for key transparency in end-to-end encrypted messaging. Based on the SEEMless and Parakeet protocols, it provides a stateless API for managing directory entries and includes a specialized WhatsApp Key Transparency Auditor for verifying audit proofs across v1 and v2 logs. Version 0.13.0 requires Rust 1.51 or higher.

Tokens
27.1K
Snippets
100
Records
136
Agent score
56%

What's inside AKD

  1. Overview of akd package organization

    main

    The repository is organized into several key packages:

    SubfolderDescription
    akdMain implementation for service providers managing the directory.
    akd_coreMinimal library of core AKD operations.
    examplesUsage examples and utilities (e.g., verifying audit proofs from WhatsApp's deployment).
    xtaskInternal package used for the code coverage pipeline.
  2. Use the WhatsApp Key Transparency Auditor

    main

    The WhatsApp Key Transparency Auditor is a tool for verifying WhatsApp key transparency audit proofs. It fetches published proofs from WhatsApp's CloudFront storage and uses the akd library to cryptographically verify the append-only proofs for each epoch.

    By default, the auditor targets the v2 log (the current, actively published log). To audit the legacy v1 log, you must provide the --log v1 flag.

    # Audit the latest epoch (v2 log)
    cargo run -p examples --release -- whatsapp-kt-auditor -l
    
    # Audit a specific epoch (v2 log)
    cargo run -p examples --release -- whatsapp-kt-auditor -e 722610
    
    # Audit the latest epoch in the legacy log (v1)
    cargo run -p examples --release -- whatsapp-kt-auditor --log v1 -l
  3. Build WASM bindings for AKD client operations

    main

    To create a lightweight WASM library for client-side proof verification, target the akd_core crate instead of the full akd crate. Because akd_core has fewer dependencies, the resulting WASM binary will be significantly smaller.

    Example bindings for a client verifying server-generated proofs can be found in the wasm_client/ sub-directory.

  4. Understand the akd stateless API model

    main

    The akd library provides a stateless API for an auditable key directory (also known as a verifiable registry or authenticated dictionary).

    Because the API is stateless, consumers of the library are responsible for providing their own storage solution for the directory entries.

    Key components include:

    • akd: The main implementation intended for service providers managing an underlying directory.
    • akd_core: A minimal library containing the core operations of the AKD protocol.
  5. Add integration tests

    main

    Integration tests in AKD are located in the integration_tests folder and run end-to-end tests against real storage implementations.

    • Structure: They use the #[tokio::test] attribute and follow a common structure defined by directory_test_suite in integration_tests/src/test_util.rs. This utility requires a database, a number of users, and a VRF signing function.
    • Adding a new data-layer: To add a new storage implementation to the integration suite, add it as a dev-dependency in the akd_integration_tests crate, create a new <storage>_tests.rs file, and reference it in integration_tests/src/lib.rs.
  6. Use the Proof-of-Concept (POC) CLI for manual testing

    main

    The poc application is a command-line REPL used to interact directly with an AKD instance in various configurations. It is useful for manual verification of the system.

    Common Flags and Subcommands:

    • -d, --debug: Activate debugging mode.
    • --memory: Use the memory-based database implementation.
    • -l, --log_level <LEVEL>: Adjust console log level (Default: Info. Options: Error, Warn, Info, Debug, Trace).
    • -m, --multirow_size <SIZE>: MySQL multi-row insert size (Default: 100).
    • bench-db-insert: Benchmark database insertion.
    • bench-lookup: Benchmark lookup API.
    • bench-publish: Benchmark publish API.
    • drop: Drop existing database tables.
    • flush: Flush data from database tables.
    # Example: Run the POC app with help
    cargo run -- --help
  7. Run tests locally

    main

    You can run tests using standard Cargo commands from the root of the repository.

    • Run all tests: cargo test (runs all tests across all crates with default features).
    • Run tests for a specific crate: cargo test --package akd (useful to isolate tests, especially since some integration tests require Docker).
    • Run tests with specific features: To test the akd crate without the VRF (Verifiable Random Function) implementation, use:
      cargo test --package akd --no-default-features --features public_tests
    # Run all tests
    cargo test
    
    # Run tests for the akd package only
    cargo test --package akd
    
    # Run akd tests without default features (e.g., disabling vrf)
    cargo test --package akd --no-default-features --features public_tests
  8. Audit the latest or specific epoch in WhatsApp KT logs

    main

    You can run the auditor using several modes depending on whether you want to verify the most recent state or a specific point in time.

    Audit the latest epoch

    Use the -l flag to audit the most recent epoch available in the log.

    Audit a specific epoch

    Use the -e <epoch_number> flag to audit a specific epoch.

    Interactive mode

    Use the -i flag to load all available epochs and select which one to audit via an interactive prompt.

    # Audit latest
    cargo run -p examples --release -- whatsapp-kt-auditor -l
    
    # Audit specific epoch
    cargo run -p examples --release -- whatsapp-kt-auditor -e 722610
    
    # Interactive mode
    cargo run -p examples --release -- whatsapp-kt-auditor -i
  9. Test a new `Storage` trait implementation

    main

    If you are implementing a new storage layer for AKD, you can leverage the existing suite of standard storage tests by enabling the public_tests feature in the akd crate.

    1. Setup Dev-Dependency: Add akd to your [dev-dependencies] with the public_tests and serde features enabled.
    2. Run Test Suite: Initialize your storage implementation and pass it to akd::storage::tests::run_test_cases_for_storage_impl. This function will panic if an error occurs, providing a stack trace.

    Configuration (Cargo.toml):

    [dev-dependencies]
    akd = { path = "../akd", version = "^0.5.0", features = ["public_tests", "serde"] }
    #[tokio::test]
    async fn my_new_storage_tests() {
        // setup
        let storage = ...;
    
        // Run the test cases (will panic if error occurs so you get a stack trace)
        akd::storage::tests::run_test_cases_for_storage_impl(&storage).await;
    
        // teardown / cleanup (if necessary)
    }
  10. Write unit tests in AKD

    main

    AKD uses standard Rust testing practices. When writing unit tests, follow these guidelines:

    1. Async Tests: If your test requires calling asynchronous code, use the #[tokio::test] attribute instead of the standard #[test].
    2. Organization: Use the #[cfg(test)] attribute on test or tests sub-modules to ensure test code is only included during test compilation.
    3. Logging: To enable log messages when tests fail, use a global startup constructor with the ctor crate. This ensures your crate benefits from centralized log management.

    Example of async vs sync tests:

    #[test]
    fn my_test() {
      panic!("boom");
    }
    
    #[tokio::test]
    async fn my_async_test() {
      panic!("async boom!");
    }
    #[cfg(test)]
    #[ctor::ctor]
    fn test_start() {
        init_logger(Level::Info);
    }