Superdiff

repository·main·Indexed 22 days ago

https://github.com/donedeal0/superdiff

A high-performance, zero-dependency JavaScript/TypeScript library for deep structural diffing of objects, arrays, text, and coordinates. It features move detection, support for large datasets via streaming and file inputs, and specialized functions including getObjectDiff, getListDiff, streamListDiff, getTextDiff, and getGeoDiff.

Tokens
8.6K
Snippets
34
Records
42
Agent score
71%

What's inside @donedeal0/superdiff

  1. Overview of Superdiff

    main

    Superdiff is a high-performance, zero-dependency library designed to provide rich and readable diffs for various data types. It is optimized for speed and scalability, even with deeply nested data or massive datasets.

    Key Capabilities:

    • Data Types Supported: Arrays (Lists), Objects, Text, and Coordinates (Geo).
    • Large Dataset Handling: Supports streaming and file inputs to process huge datasets efficiently without exhausting memory.
    • Advanced Diffing: Includes move detection (detecting when items in a list have changed position) and output refinement.
    • Performance: Scales linearly and consistently outperforms or matches many specialized diff libraries in benchmarks.
  2. Stream large list diffs with streamListDiff

    main

    For high performance and large datasets, use streamListDiff to process object lists as a stream. This prevents memory exhaustion by processing data in chunks.

    Environment Setup

    • Server: Import from @donedeal0/superdiff/server. Supports Node.js Readable streams, FilePath (strings), or arrays.
    • Browser: Import from @donedeal0/superdiff/client. Supports ReadableStream, File objects, or arrays. Requires ESM support.

    Usage streamListDiff returns a stream listener that emits three events:

    • data: Emits a chunk (an array of StreamListDiff objects).
    • finish: Emitted when processing is complete.
    • error: Emitted if an error occurs.

    Options

    • referenceKey: (Required) A key common to all objects (e.g., id).
    • chunksSize: (number, default 0) Number of object diffs per chunk. 0 means 1 diff per chunk.
    • useWorker: (boolean, default true) Runs the diff in a worker for performance. Recommended for lists > 100,000 items.
    • considerMoveAsUpdate: (boolean, default false) Treats moved as updated.
    // Browser example
    import { streamListDiff } from "@donedeal0/superdiff/client";
    
    const diff = streamListDiff(
      [{ id: 1, name: "Item 1" }, { id: 2, name: "Item 2" }],
      [{ id: 2, name: "Item 2" }, { id: 3, name: "Item 3" }],
      "id",
      { chunksSize: 2 }
    );
    
    diff.on("data", (chunk) => {
      console.log("Received chunk:", chunk);
    });
    
    diff.on("finish", () => console.log("Done"));
  3. Configure getObjectDiff options

    main

    The getObjectDiff function accepts an optional options object:

    • ignoreArrayOrder: (boolean, default false) If true, arrays with the same values in different orders are considered equal.
    • showOnly: Filters the output to specific statuses.
      • statuses: An array of statuses to include: "added" | "deleted" | "updated" | "equal".
      • granularity:
        • "basic" (default): Returns only the top-level keys matching the status.
        • "deep": Returns nested keys that match the status, filtering the tree accordingly.
  4. Configure getTextDiff options

    main

    The getTextDiff function accepts an optional options object:

    • separation: "character" | "word" | "sentence" (default "word").
    • accuracy:
      • "normal" (default): Fast, simple tokenization.
      • "high": Slower, exact tokenization (handles Unicode, emoji, CJK, and locale-aware segmentation).
    • detectMoves: (boolean, default false) If true, token swaps are marked as updated instead of deleted/added pairs.
    • ignoreCase: (boolean, default false) If true, case is ignored.
    • ignorePunctuation: (boolean, default false) If true, punctuation is ignored.
    • locale: (string | Intl.Locale) Enables locale-aware segmentation in high accuracy mode.
  5. Configure getGeoDiff options

    main

    The getGeoDiff function accepts an optional options object:

    • unit: The distance unit: "centimeter" | "foot" | "inch" | "kilometer" | "meter" | "mile" | "mile-scandinavian" | "millimeter" | "yard" (default "kilometer").
    • accuracy:
      • "normal" (default): Fast, uses Haversine formula (spherical Earth).
      • "high": Precise, uses Vincenty formulae (ellipsoidal Earth).
    • maxDecimals: (number, default 2) Maximum decimals for the distance.
    • locale: (string | Intl.Locale, default "en-US") Enables locale-aware distance labels.
  6. Configure getListDiff options

    main

    The getListDiff function accepts an optional options object:

    • showOnly: An array of statuses to return: "added" | "deleted" | "moved" | "updated" | "equal".
    • referenceKey: (string) If provided, an object is considered updated rather than added or deleted if this key remains stable (e.g., an id). Only affects objects.
    • ignoreArrayOrder: (boolean, default false) If true, arrays with the same values in different orders are considered equal.
    • considerMoveAsUpdate: (boolean, default false) If true, a moved status is treated as updated.
  7. Understand the StreamListDiff data format

    main

    The StreamListDiff<T> type represents the payload emitted during a list diff stream. Each emitted object contains the current and previous state of an item and its position in the list.

    Fields:

    • value: The current value of the item (T | null).
    • previousValue: The previous value of the item (T | null).
    • index: The current index in the list (number | null).
    • previousIndex: The previous index in the list (number | null).
    • status: The type of change, matching the ListStatus type (e.g., 'added', 'removed', 'updated', 'moved').
    type StreamListDiff<T extends Record<string, unknown>> = {
      value: T | null;
      previousValue: T | null;
      index: number | null;
      previousIndex: number | null;
      status: `${ListStatus}`;
    };
  8. Available Superdiff API functions

    main

    Superdiff exports five primary functions to handle different diffing requirements:

    • getObjectDiff: For comparing objects.
    • getListDiff: For comparing arrays/lists (includes move detection).
    • streamListDiff: For comparing extremely large lists using streams.
    • getTextDiff: For comparing text/strings.
    • getGeoDiff: For comparing coordinates/geospatial data.
  9. Compare coordinates with getGeoDiff

    main

    Use getGeoDiff to calculate the difference between two geographical coordinates. Coordinates must follow GeoJSON order: [longitude, latitude].

    It returns the distance, the direction of movement, and a localized label.

    import { getGeoDiff } from "@donedeal0/superdiff";
    
    // [longitude, latitude]
    const diff = getGeoDiff([2.3522, 48.8566], [-0.1278, 51.5074]);
    // Output includes distance, direction, and label
  10. Compare text with getTextDiff

    main

    Use getTextDiff to compare two strings at a character, word, or sentence level. It uses the Longest Common Subsequence (LCS) algorithm.

    Modes

    • Default (No Move Detection): Optimized for UI. Token moves are ignored so insertions don't break equality. Updates are represented as a deleted followed by an added token.
    • Move Detection (detectMoves: true): Semantically precise. Direct token swaps are marked as updated.
    import { getTextDiff } from "@donedeal0/superdiff";
    
    // Default word-based diff
    const diff = getTextDiff("The brown fox", "The orange cat", { 
      separation: "word", 
      detectMoves: false 
    });
  11. Compare two arrays with getListDiff

    main

    Use getListDiff to compare two arrays. It supports primitive values, objects, and duplicate values, returning the status of each entry (e.g., added, deleted, moved, updated, equal).

    import { getListDiff } from "@donedeal0/superdiff";
    
    const diff = getListDiff(
      ["mbappe", "mendes", "verratti", "ruiz"],
      ["mbappe", "messi", "ruiz"]
    );
  12. Compare two objects with getObjectDiff

    main

    Use getObjectDiff to compare two objects and receive a structured diff of their values, including deeply nested properties. It supports any value type and provides a recursive diff for nested keys.

    import { getObjectDiff } from "@donedeal0/superdiff";
    
    const diff = getObjectDiff(
      { id: 54, user: { name: "joe", member: true } },
      { id: 54, user: { name: "joe", member: false } }
    );