SurrealDB JavaScript SDK

repository·main·Indexed 43 days ago

https://github.com/surrealdb/surrealdb.js

JavaScript SDK for connecting to remote and embedded SurrealDB instances across Node.js, Bun, Deno, and Web Browsers. Includes the @surrealdb/memory package, a typed REST client for the Agent Memory API supporting memory operations, chat streaming, and entity management. Supports SurrealDB versions >= 2.1.0 and < 4.0.0.

Tokens
52.2K
Snippets
183
Records
228
Agent score
55%

What's inside SurrealDB

  1. What is SQON (SurrealQL Object Notation)?

    main

    SQON is a family of data representation formats used by SurrealDB to encode its rich value system. It provides three main capabilities:

    1. Value classes: Specialized classes for types that have no direct JavaScript equivalent, such as RecordId, Table, DateTime, and Decimal.
    2. Codecs: Serializers and deserializers (CborCodec and JsonCodec) that allow converting between JavaScript value instances and wire formats without losing type information.
    3. Utilities: Helper functions for comparing (equals), converting (jsonify), and escaping (escapeIdent, toSurqlString) SurrealQL values.

    Note: The full surrealdb SDK re-exports everything from @surrealdb/sqon.

  2. Retries and Idempotency

    main

    The SDK provides automatic retries for certain operations:

    • Idempotent GET requests: Automatically retry on 5xx and connection failures with an exponential backoff (250ms, 500ms, 1000ms) up to a default of 3 retries.
    • remember and rememberMany: These write operations are safe to retry because they include an Idempotency-Key derived from the request, valid for a 30-second window.
    • Other mutating methods: These are not retried automatically.
  3. Handle pagination in `@surrealdb/memory`

    main

    Most list surfaces in the SDK use cursor-based pagination.

    • list: Returns a single page of rows and a page object containing hasMore, nextCursor, and totalSize. To walk the full collection, follow nextCursor until it is absent.
    • listAll: An unbounded read that walks the entire collection for you. Use this for trees or filter sources, but avoid it for large datasets intended for a single screen.
    • count: Returns the total number of items without fetching them (where supported).

    Important Notes:

    • Termination: Always terminate on the absence of a cursor, not on a short page. A page might be shorter than the limit due to visibility filtering.
    • Ranked/Aggregate Heads: Methods like entities.search, entities.top, and client.lookup sections are ranked heads and do not use cursors. If they are truncated, follow the specific collection endpoint they name instead of re-requesting with a larger limit.
    • Exclusive Options: scopes.list and client.audit only support limit and cursor. entities.neighbours does not support count alongside minFacts (this will result in a 400 error).
    const first = await client.entities.list({ limit: 50 });
    first.entities; // the rows
    first.page; // { hasMore, nextCursor?, totalSize? }
    
    // Walk by hand: follow `nextCursor` until it is absent.
    let cursor: string | undefined;
    do {
      const page = await client.entities.list({ limit: 50, cursor });
      handle(page.entities);
      cursor = page.page.nextCursor ?? undefined;
    } while (cursor);
    
    await client.scopes.listAll(); // ScopeNodeJson[], cursors followed to exhaustion
    await client.documents.count(); // number, without fetching the documents
  4. Configure storage backends in @surrealdb/wasm

    main

    The WebAssembly engine supports two primary storage backends:

    • mem://: In-memory storage. Data is lost when the browser tab is closed.
    • indxdb://<name>: IndexedDB persistence. Data persists across sessions.
    // In-memory (data is lost when the tab closes)
    await db.connect("mem://");
    
    // IndexedDB persistence
    await db.connect("indxdb://myapp");
  5. Understand SQON JSON (SQON-J) encoding

    main

    SQON JSON (MIME: application/vnd.surrealdb.sqon+json) is a JSON-compatible serialization format for SurrealDB values. It is designed for portability and readability in environments where only JSON is available, such as HTTP APIs, browser clients, and logging infrastructure.

    It achieves full type fidelity by using $ prefixed keys to describe custom SurrealDB types, similar to the EJSON specification. While less compact than the binary SQON-B format, it is highly portable and LLM-friendly.

  6. How scopes and lenses work in @surrealdb/memory

    main

    In @surrealdb/memory, scopes and lenses are used to control the visibility and access regions for operations.

    • Write/Session operations (write, session) accept a scopes option.
    • Read operations (recall, context, documents.query, documents.keywords.search) accept a lens option.

    Wire Format: ScopeSets

    The underlying wire format is a ScopeSets object, which is a Disjunctive Normal Form (DNF) selector represented as a string[][]:

    1. The outer array represents an OR of clauses.
    2. Each inner array represents an AND of key/value slash-paths.

    Example Logic: `[[

    // OR logic: (team/a) OR (team/b AND clearance/secret)
    const scope = [["team/a"], ["team/b", "clearance/secret"]];
    
    // Ergonomic shorthand supported by normaliseScope:
    
    // Single-path clause
    client.remember("...", { scopes: "team/eng" }); 
    
    // OR of single-path clauses
    client.remember("...", { scopes: ["team/eng", "org/acme"] }); 
    
    // AND of paths
    client.remember("...", { scopes: [["team/eng", "org/acme"]] });
  7. Use lenses to narrow read regions

    main

    A read lens narrows the region from which a query reads. You can use the /* wildcard to select an entire subtree.

    Important Security Note: A lens can only narrow the region. The key's existing grants still apply on top of the lens. If a lens attempts to access a region outside of the currently granted region, it will return no rows rather than throwing a 403 error.

    // Using a subtree wildcard
    client.recall("...", { lens: "team/eng/*" });
    
    // Using an OR of lenses
    client.documents.query({
      query: "...", 
      lens: ["team/eng/*", "org/acme"]
    });
    
    // Using a lens in keyword search
    client.documents.keywords.search({
      query: "...", 
      lens: "team/eng/*"
    });
  8. Understand the SQON data representation formats

    main

    SQON (SurrealQL Object Notation) is a family of data representation formats used by SurrealDB to encode its rich data value system. There are three semantically equivalent representations optimized for different environments:

    • SQON (Textual): Uses SurrealQL syntax. Best for direct database interaction, queries, and SurrealQL expressions.
    • SQON Binary (CBOR): Uses CBOR with custom tags. Best for efficient wire transport, compact storage, and binary-safe environments.
    • SQON JSON (Textual): Standard JSON. Best for HTTP APIs, browser environments, and human-readable interchange.

    Any value expressible in one format can be round-tripped through any other without loss of type information.

  9. How `client.lookup` works and how to handle its results

    main

    The client.lookup(query) method answers what the context knows about a subject in a single round trip. It returns stored rows without generation or summarization.

    Results should be branched based on resolution.kind:

    • entity: A single subject is found. Access via answer.resolution.subject and answer.facts.items.
    • ambiguous: Multiple candidates exist. Each carries a distinguisher. Access via answer.resolution.candidates.
    • topic: No single subject, but a cluster of related entities/passages. Access via answer.entities.items and answer.passages.items.
    • empty: Nothing found. answer.resolution.nearest provides suggestions for similar stored items.

    If a section is truncated, you can walk the full collection using the specific endpoint provided (e.g., client.facts.attributes({ entity }) for facts).

    const answer = await client.lookup("Atlas");
    
    switch (answer.resolution.kind) {
      case "entity":
        render(answer.resolution.subject, answer.facts.items);
        break;
      case "ambiguous":
        // Each candidate carries a `distinguisher` so they can be told apart.
        offerChoice(answer.resolution.candidates);
        break;
      case "topic":
        // No single subject: the answer is the cluster in `entities` / `passages`.
        renderCluster(answer.entities.items, answer.passages.items);
        break;
      case "empty":
        // `nearest` separates "not stored" from "stored under another name".
        suggest(answer.resolution.nearest);
        break;
    }
  10. Use @surrealdb/node as an embedded engine

    main

    The @surrealdb/node package is a native engine plugin that embeds SurrealDB directly into your server-side JavaScript runtime (Node.js, Bun, or Deno) using a NAPI addon. This allows you to run a database in-process without a separate server.

    To use it, register the engine using createNodeEngines() when constructing your Surreal client, then connect to an embedded endpoint (e.g., mem://).

    import { createNodeEngines } from "@surrealdb/node";
    import { Surreal } from "surrealdb";
    
    const db = new Surreal({
        engines: createNodeEngines(),
    });
    
    await db.connect("mem://");
    
    await db.use({ namespace: "test", database: "test" });
    
    await db.query("CREATE person SET name = 'Tobie'");
  11. Install @surrealdb/memory

    main

    Add the @surrealdb/memory SDK to your project using your preferred package manager. This is a lightweight, typed REST client for the Agent Memory API that uses the platform's native fetch and has no runtime dependencies.

    # using npm
    npm i @surrealdb/memory
    
    # or using pnpm
    pnpm i @surrealdb/memory
    
    # or using yarn
    yarn add @surrealdb/memory
    
    # or using bun
    bun add @surrealdb/memory
  12. Run SurrealDB in a Web Worker

    main

    To keep your UI responsive, you can offload database operations to a Web Worker using createWasmWorkerEngines. This requires importing a worker agent from @surrealdb/wasm/worker?worker.

    import { createWasmWorkerEngines } from "@surrealdb/wasm";
    import WorkerAgent from "@surrealdb/wasm/worker?worker";
    import { Surreal, createRemoteEngines } from "surrealdb";
    
    const db = new Surreal({
        engines: {
            ...createRemoteEngines(),
            ...createWasmWorkerEngines({
                createWorker: () => new WorkerAgent(),
            }),
        },
    });
    
    await db.connect("mem://");