similar

repository·main·Indexed 23 days ago

https://github.com/mitsuhiko/similar

A dependency-free Rust diff library (v3.1.1) that implements various diffing algorithms including Myers, Patience, Histogram, and classic LCS table. It provides high-level interfaces for comparing text, bytes, or arbitrary comparable sequences at line, word, character, or grapheme levels. The library includes the TextDiff API for line-based diffing and the DiffHook trait for customizing diffing behavior. It supports no_std + alloc environments with optional hashbrown integration.

Tokens
9.2K
Snippets
20
Records
52
Agent score
79%

What's inside similar

  1. Overview of similar diffing capabilities

    main

    similar is a dependency-free Rust crate that implements multiple diffing algorithms and granular diffing levels. It supports:

    Algorithms

    • Myers' diff
    • Patience diff
    • Hunt-style diff
    • Histogram diff
    • Classic LCS table diff

    Diffing Granularity

    • Line level
    • Word level
    • Character level
    • Grapheme level

    Data Types

    • Text diffing
    • Byte diffing
    • Diffing on arbitrary comparable sequences
    • Unified diff generation
  2. Explore diff test cases and scenarios

    main

    The repository provides several specific diffing scenarios to test different library behaviors:

    CaseScenarioPurpose
    case01simple_editStraightforward line edits and small value changes.
    case02patience_reorderSection reordering; tests how the patience diff algorithm handles moved blocks.
    case03repeated_linesRepeated near-identical lines; tests behavior with ambiguous anchors.
    case04code_refactorRealistic code-like refactor (extracting helpers and behavior tweaks via dedup).
    case05whitespace_punctuationCase, style, punctuation, and version/number changes.
    case06insertions_edgesInsertions at the very start and end of files (edge hunks).
  3. Validate DiffOp ranges for TextDiff::iter_changes

    main

    In version 3.0, TextDiff::iter_changes will panic if a DiffOp range is out of bounds. While iterating directly over diff.ops() is safe, you must manually validate any DiffOp values that are deserialized, transformed, or manually constructed.

    Use the following pattern to validate that an operation is within the bounds of the TextDiff:

    use similar::{DiffOp, TextDiff};
    
    fn op_in_bounds(diff: &TextDiff<'_, '_, str>, op: &DiffOp) -> bool {
        let (_, old, new) = op.as_tag_tuple();
        old.end <= diff.old_len() && new.end <= diff.new_len()
    }
  4. Understand the diff input sample naming convention

    main

    The examples/diffs/ directory uses a consistent naming scheme designed for lexicographic glob expansion. This allows you to easily pair 'before' (old) and 'after' (new) files using patterns like caseNN.*.txt.

    • Old files (before): caseNN.01.before_<scenario>.txt
    • New files (after): caseNN.02.after_<scenario>.txt
  5. Migrate from Similar 2.7 to 3.0

    main

    When upgrading from version 2.7.x to 3.0, ensure your Rust toolchain is updated to 1.85 or newer. Key breaking changes include:

    • std is now an explicit default feature. For no_std environments, you must disable default features.
    • TextDiff slice accessors (old_slices() and new_slices()) have been replaced with new accessor APIs.
    • TextDiff::iter_changes is stricter and will panic if a DiffOp range is out of bounds.
    • TextDiff type annotations have a different lifetime shape.
    • get_diff_ratio has been renamed to diff_ratio.
    • Several constructors are now const and can be used in const contexts.
  6. Configure similar for no_std + alloc environments

    main

    By default, similar enables std. If you are working in a no_std environment with an allocator, you must disable default features.

    Depending on your requirements, you can choose between two backends for internal collections:

    1. Default (BTreeMap): Use default-features = false to use alloc::collections::BTreeMap.
    2. Hashbrown (HashMap): Use default-features = false, features = ["hashbrown"] to use hashbrown::HashMap.
    [dependencies]
    similar = { version = "3", default-features = false }
  7. Use the Replace hook to combine deletions and insertions into replacements

    main

    The Replace<D> struct is a DiffHook wrapper that optimizes diff results. It intercepts delete and insert events and, when they occur consecutively, combines them into a single replace event. This is useful for obtaining blocks of maximal length and ensuring a consistent order of operations. While the core text processing in similar may resolve these back to deletes and inserts, using Replace is recommended for consistent diff output.

    To use it, wrap an existing DiffHook (such as Capture) using Replace::new(d).

  8. How sequence and text diffing work in similar

    main

    The library provides two distinct layers of abstraction depending on your needs:

    1. Sequence Diffing (Low Level)

    Used for any indexable collection. Functions like capture_diff and capture_diff_slices return DiffOp objects. These operations represent ranges of differences by index in the source sequence. This is efficient for large sequences but requires manual index management.

    2. Text Diffing (High Level)

    Specifically designed for text and line-based operations. The TextDiff type wraps the underlying algorithms to provide a more ergonomic API. Instead of working with raw indices, you can work with Change objects that represent lines or characters.

    Ops vs Changes

    Because sequences often match in large chunks, the library distinguishes between:

    • Diff Operations (DiffOp): Ranges of differences by index. Use DiffOp::iter_changes to expand these into individual item-by-item changes.
    • Changes: Individual items (like a single line of text) that have been modified. TextDiff provides iter_changes to facilitate this.
  9. Use DiffInput to handle borrowed and owned text inputs

    main

    The DiffInput<'a, T> enum is used by diffing APIs to accept both borrowed and owned text values efficiently. This prevents unnecessary allocations when the caller already owns the data or when working with slices.

    • DiffInput::Borrowed(&'a T): Holds a reference to a DiffableStr.
    • DiffInput::Owned(<T as ToOwned>::Owned): Holds an owned version of the DiffableStr.

    You can retrieve the underlying DiffableStr reference using .as_diffable_str().

    pub enum DiffInput<'a, T: DiffableStr + ?Sized> {
        Borrowed(&'a T),
        Owned(<T as ToOwned>::Owned),
    }
    
    impl<T: DiffableStr + ?Sized> DiffInput<'_, T> {
        pub fn as_diffable_str(&self) -> &T {
            match self {
                DiffInput::Borrowed(value) => value,
                DiffInput::Owned(value) => value.borrow(),
            }
        }
    }
  10. How Myers' diff algorithm works with deadlines and heuristics

    main

    The Myers' diff implementation in this library uses several layers of optimization to balance accuracy and performance:

    1. Heuristics & Shortcuts: The core recursion applies local shortcuts like prefix/suffix trimming, front-anchor peeling, and small-side exact fallbacks.
    2. Deadlines: An optional Instant can be provided. The algorithm checks deadline_exceeded at various intervals (e.g., during prefix/suffix matching or within the small-side exact fallback) to bail early if the time limit is reached.
    3. Small-Side Exact Fallback: If one side of the diff is relatively small and the other is large, the algorithm may switch to an exact LCS (Longest Common Subsequence) dynamic programming approach to find the optimal edit script quickly.
    4. Divide and Conquer: For larger inputs, it uses a divide-and-conquer strategy by finding a 'middle snake' in the edit graph to split the problem into smaller sub-problems.
  11. Select tokenization modes with InlineChangeMode

    main

    The InlineChangeMode enum determines how text is broken down into tokens during the second-level refinement process.

    • Auto: Uses the crate default. If the unicode feature is enabled, it uses unicode words; otherwise, it uses whitespace-based words.
    • Words: Tokenizes by whitespace runs and non-whitespace runs.
    • Chars: Tokenizes by individual characters.
    • UnicodeWords: Tokenizes by unicode words (requires unicode feature).
    • Graphemes: Tokenizes by unicode grapheme clusters (requires unicode feature).
  12. Handle missing newlines in text diffs

    main

    When performing line diffs, the library accounts for the fact that files may or may not end in a newline character.

    • If a diff was created via TextDiff::from_lines, you can check TextDiff::newline_terminated() to see if the system is tracking missing newlines.
    • Individual Change objects provide a missing_newline() method which returns true if the change is missing a trailing newline. This allows callers to render special markers like \ No newline at end of file in unified diffs.