DOCSEARCHbyAlgolia
Submit Docs

denokv

repository·main·Indexed 20 days ago

https://github.com/denoland/denokv

A self-hostable backend for Deno KV that provides a scalable, network-accessible key-value database using SQLite. It implements the KV Connect protocol, allowing multiple Deno processes to share the same database. The project includes the @deno/kv client library for Node.js (18+), a Docker image for deployment, and the denokv_proto crate for implementing Deno KV-compatible database engines.

Tokens
12.9K
Snippets
44
Records
61
Agent score
71%

What's inside denokv

  1. Understand the denokv_proto crate

    main

    The denokv_proto crate is the foundational layer for the Deno KV protocol. It serves three primary purposes:

    1. Protobuf Definitions: It contains the protobuf definitions required for the KV Connect protocol.
    2. Protocol Specification: It includes the KV Connect specification.
    3. Database Abstraction: It defines a Database trait. Developers can implement this trait to create their own Deno KV-compatible database engines.

  3. What is the KV Connect protocol?

    main

    The KV Connect protocol is the communication standard used by the Deno CLI and third-party clients to interact with remote Deno KV backends.

    When to use it:

    • Use this protocol when accessing a remote database via a URL, for example: await Deno.openKv("http://<remote>").
    • Note: This protocol is not used when the Deno CLI accesses a local KV database backed by SQLite.
    // Example of triggering the KV Connect protocol
const kv = await Deno.openKv("http://<remote-backend-url>");

  4. Supported Deno KV features in denokv

    main

    When connecting via KV Connect (to denokv or Deno Deploy), most Deno KV operations are supported. However, queues are not supported over the network protocol.

    FeatureLocal databaseKV Connect (denokv / Deno Deploy)
    get / getMany / set / delete / list✅✅
    atomic() (checks, sum / min / max)✅✅
    Key expiration (expireIn)✅✅
    watch✅✅
    enqueue / listenQueue✅❌

    Use local databases (e.g., Deno.openKv("./db.sqlite")) if you require queue functionality.

  5. Understand KV Connect protocol versioning and compatibility

    main

    The KV Connect protocol uses version negotiation during the metadata exchange phase.

    Compatibility Rules

    • Version 2 is backwards compatible with Version 1: A client supporting Version 2 can connect to a Version 1 server.
    • Version 1 is NOT forwards compatible: A client supporting only Version 1 cannot connect to a Version 2 server.

    Protocol Versions

    Version 1

    • The initial version of the protocol.
    • Used as the default version if the client does not specify a list of supported versions during metadata exchange.

    Version 2

    Adds support for multiple versions and databases, and consistency metadata:

    • x-denokv-version header: Added to all Data Path Protocol requests to indicate the client's protocol version.
    • x-denokv-database-id header: Added to all Data Path Protocol requests to indicate the UUID of the database being accessed.
    • read_is_strongly_consistent field: Added to the response body of Snapshot Read Requests to indicate if the server has a strongly consistent view. Clients can use this to decide whether to retry a request.

    Version 3

    Adds advanced observation and status features:

    • status field: Added to the response body of Snapshot Read Requests. This replaces the read_disabled boolean field used by clients.
    • Watch operation: Adds the "Watch" data path operation to the protocol.

  6. Connection requirements for KV Connect backends

    main

    To implement or connect to a KV Connect backend, the following technical requirements apply:

    • Transport: Uses HTTP, JSON, and ProtoBuf.
    • HTTP Versions: Backends MUST support both HTTP/1.1 and HTTP/2.
    • Security: It is RECOMMENDED that backends accessible over the public internet support HTTPS only.
    • Model: It is a stateless, request-response protocol. Clients and servers MAY send multiple requests/responses over the same underlying HTTP connection.
    • Ordering: The protocol is stateless; clients and servers MUST NOT assume that requests and responses are received in the same order they were sent.
    • Client Optimization: While the protocol is stateless, clients SHOULD maintain a cache of metadata and authentication information to reduce network round trips.

  7. Compare supported KV features across backends

    main

    Not all features are available on all backends. Specifically, Queue operations (enqueue/listenQueue) are not supported on remote (KV Connect) backends.

    | Feature | sqlite | in-memory | remote (KV Connect) |
| :--- | :--- | :--- | :--- |
| `get` / `getMany` / `set` / `delete` / `list` | ✅ | ✅ | ✅ |
| `atomic()` (checks, `sum` / `min` / `max`) | ✅ | ✅ | ✅ |
| Key expiration (`expireIn`) | ✅ | ✅ | ✅ |
| `watch` | ✅ | ✅ | ✅ |
| `enqueue` / `listenQueue` | ✅ | ✅ | ❌ |

  8. Implement a Deno KV compatible database using the Database trait

    main
    To build a database that is compatible with Deno KV, you must implement the Database trait provided by the denokv_proto crate. This allows your custom database implementation to communicate using the standard KV Connect protocol.

  9. Understand the KV Connect Protocol structure

    main

    The KV Connect protocol is divided into two distinct phases used to interact with a KV database:

    1. Metadata Exchange Protocol: A JSON-based phase performed at startup. It is used for authentication and to discover database metadata, such as the protocol version, database UUID, available HTTP endpoints, and authentication tokens.
    2. Data Path Protocol: A Protobuf-over-HTTP phase used for performing actual CRUD (Create, Read, Update, Delete) operations on the database.

    Clients must successfully complete the Metadata Exchange before attempting any Data Path operations.

  11. Deploy denokv to Fly.io

    main

    You can deploy denokv to Fly.io using a fly.toml configuration. This setup requires a persistent volume for the SQLite database and uses Fly secrets for the access token.

    Steps:

    1. Create a volume: flyctl volumes create denokv_data.
    2. Set the access token: flyctl secrets set DENO_KV_ACCESS_TOKEN=<random-token>.
    3. Deploy: flyctl deploy.

    Note on Scaling: By default, the configuration below allows the database to scale to 0 instances. The first request after inactivity will be slow due to startup time. To prevent this, set min_machines_running = 1 and auto_stop_machines = false in fly.toml.

    # fly.toml
app = "<your-app-name>"
primary_region = "<region>"

[build]
  image = "ghcr.io/denoland/denokv:latest"

[http_service]
  internal_port = 4512
  force_https = true
  auto_stop_machines = true
  auto_start_machines = true
  min_machines_running = 0

[env]
  DENO_KV_SQLITE_PATH="/data/denokv.sqlite3"

[mounts]
  destination = "/data"
  source = "denokv_data"

  12. Handle V8 serialization in Bun

    main

    Bun uses JavaScriptCore serialization, which is incompatible with the V8 serialization used by Deno and Node. To avoid data corruption, openKv will throw by default on Bun.

    To use local databases in Bun, you must explicitly provide Bun's serializers. Note that data written this way will be unreadable by Node or Deno.

    import { openKv } from "@deno/kv";
import { serialize as encodeV8, deserialize as decodeV8 } from "v8"; // actually JavaScriptCore format on Bun!

const kv = await openKv("kv.db", { encodeV8, decodeV8 });