Pretext: Fast and Accurate Text Measurement & Layout

repository·main·Indexed 13 days ago

https://github.com/chenglou/pretext

A pure JavaScript/TypeScript library for multiline text measurement and layout that avoids expensive browser layout reflows. It provides APIs for calculating paragraph height, manual line layout for Canvas/SVG/WebGL, and rich-text inline flow helpers. Version 0.0.8 requires Intl.Segmenter and Canvas 2D support.

Tokens
12K
Snippets
40
Records
60
Agent score
99%

What's inside Pretext

  1. Overview of Pretext Corpora

    main

    The corpora package provides starter long-form stress corpora designed for browser-layout experiments. These files serve as stable canaries for testing how different languages, punctuation systems, and complex text runs (like RTL/LTR mixing, emoji ZWJ, and various Unicode joiners) affect layout.

    Key components include:

    • Main Corpus Set: Wired into /corpus.
    • Long-form Rows: Wired into /benchmark.
    • Status Pages: Act as the source of truth for current results.
    • Metadata: Machine-readable metadata is stored in sources.json, and corpus status is in dashboard.json.
  2. Configure `pre-wrap` white-space mode behavior

    main

    When using the { whiteSpace: 'pre-wrap' } mode, Pretext deliberately mimics standard browser textarea-like behavior:

    • Spaces: Ordinary spaces are supported.
    • Hard Breaks: \n characters create hard breaks.
    • Tabs: Tabs use default browser-style tab stops (advancing to the next stop from the current line start).
    • Preserved Spaces: Spaces at the end of a line are preserved and hang at the line end.
    • Empty Lines: Consecutive hard breaks result in empty lines; however, a trailing final hard break does not invent an extra empty line.

    This mode is designed to match the wrapping behavior users expect from standard text input elements.

  3. Classify canary mismatches using the Pretext Taxonomy

    main

    When a corpus or probe disagrees with the engine, use the following taxonomy to determine the class of problem. This helps decide whether to clean data, adjust segmentation, or change the shaping model.

    Taxonomy Categories

    CategoryDescriptionTypical Response
    corpus-dirtyThe source text is untrustworthy (e.g., wrapped lines, scaffolding, typos, scraping artifacts).Clean the corpus or reject it entirely.
    normalizationThe text model is incorrect before line fitting (e.g., whitespace collapse, NBSP/ZWSP mishandling).Fix preprocessing or break-kind modeling; do not patch layout().
    boundary-discoveryCandidate break opportunities are wrong or too coarse (e.g., Intl.Segmenter output is plausible but merged units are not).Adjust preprocessing boundaries; keep rules semantic and narrow.
    glue-policyRaw boundaries are correct, but units are attached incorrectly (e.g., punctuation should stay with the previous word).Change glue/attachment rules, not measurement.
    edge-fitThe browser keeps/drops a short phrase at the line edge by a tiny amount (usually < 1px).Inspect browser-specific tolerance; avoid broad heuristics.
    shaping-contextLine breaks change shaping/glyph metrics such that segment sums are no longer exact.Accept an approximate envelope or move toward a richer shaping-aware model.
    font-mismatchThe engine and browser measure different fonts or font-resolution behaviors.Verify the exact font stack; avoid script heuristics until the font story is clean.
    diagnostic-sensitivityThe mismatch is in the probe, extractor, or environment (e.g., span vs range disagreement).Re-run with explicit extractor/method/environment; do not change the engine yet.
  4. Understand the Pretext measurement design

    main

    Pretext uses a two-phase measurement design to avoid expensive DOM layout thrashing (repeatedly calling getBoundingClientRect() or similar). By separating text analysis from layout arithmetic, it remains fast enough for resize-driven UI work.

    The Two Phases

    1. prepare(text, font): This phase segments the text, measures individual segments using Canvas measureText() (which avoids DOM layout), and caches the widths.
    2. layout(prepared, maxWidth, lineHeight): This phase performs pure arithmetic using the cached widths to determine line breaks and positions. It does not perform any DOM reads.

    This design ensures that even if components recalculate layout frequently, the browser does not have to relay out the entire document repeatedly.

  5. Understand Pretext's text rendering limitations and requirements

    main

    Pretext is not a full font rendering engine and targets a specific subset of CSS text properties. When using Pretext for layout calculations, be aware of the following constraints:

    Supported CSS Properties

    • White-space: white-space: normal and pre-wrap.
    • Word-break: word-break: normal, keep-all, and { wordBreak: 'keep-all' } (the latter behaves as expected for CJK/Hangul and mixed text while maintaining overflow-wrap: break-word fallback).
    • Overflow-wrap: overflow-wrap: break-word. Note that very narrow widths may still break inside words, but only at grapheme boundaries.
    • Line-break: line-break: auto.
    • Letter-spacing: Handled via a numeric pixel value passed to prepare() or prepareWithSegments().
    • Tabs: Follows default browser-style tab-size: 8.

    Unsupported/Unsafe Features

    • System Fonts: Using system-ui is unsafe for layout() accuracy on macOS due to platform-specific bugs. Use a specific named font instead.
    • Advanced CSS Text Features: font-optical-sizing, font-feature-settings, and standalone font-variation-settings are not modeled. Variable-font axes only work if the active axis is reflected in the canvas font string (e.g., via weight).

    Runtime Requirements

    • Intl.Segmenter: The runtime must support Intl.Segmenter. Browsers or runtimes without this are currently unsupported.
    • Canvas 2D: Requires Canvas 2D text measurement capabilities.
  6. Handle Apple Color Emoji width discrepancies in Chrome and Firefox

    main

    On macOS, Chrome and Firefox report Apple Color Emoji widths via measureText() that differ from the actual DOM text width at small font sizes. This can cause premature emoji line breaks.

    How Pretext handles this: The prepare() method includes a capability-detection mechanism that identifies this canvas/DOM difference for a specific font. It calculates a correction factor per emoji grapheme and caches it. This correction is applied during preparation and does not affect the core layout() logic.

  7. Use corpus tooling for diagnostics

    main

    Corpus tooling allows for deep analysis of how text is handled across different widths and fonts:

    • bun run corpus-check: Diagnoses a corpus at one or a few widths.
    • bun run corpus-check:safari: Safari-specific corpus check.
    • bun run corpus-sweep: Maintains a Chrome step=10 corpus width sweep.
    • bun run corpus-sweep:safari: Maintains a Safari step=10 corpus width sweep.
    • bun run corpus-font-matrix: Runs the same corpus under alternate fonts.
    • bun run corpus-font-matrix:safari: Safari-specific font matrix check.
    • bun run corpus-taxonomy: Groups corpus mismatches by likely cause.
    • bun run corpus-status: Rebuilds corpora/dashboard.json.
    • bun run corpus-status:refresh: Refreshes Chrome and Safari step=10 sweeps, then rebuilds the corpus dashboard.
    bun run corpus-sweep
    bun run corpus-status:refresh
  8. Run Pretext demos locally

    main

    To run the project's demos locally, clone the repository and use the Bun runtime.

    1. Install dependencies: bun install
    2. Start the dev server: bun start (or bun run start:windows on Windows)
    3. Open /demos/index in your browser.
    # For macOS/Linux
    bun install
    bun start
    
    # For Windows
    bun install
    bun run start:windows
  9. Run day-to-day development tasks

    main

    Use the following commands for local development, typechecking, and testing:

    • bun start: Starts a stable local page server at http://localhost:3000.
    • bun run start:windows: A Windows-friendly fallback for the local server that avoids automatic port cleanup.
    • bun run check: Performs typechecking, linting, and a dead-code scan using knip.
    • bun test: Runs the small, durable invariant test suite.
    bun start
    bun run start:windows
    bun run check
    bun test
  10. Perform manual paragraph line layout

    main

    For advanced use cases like Canvas, SVG, or WebGL rendering, use the prepareWithSegments and manual layout APIs. This allows you to control exactly how and where lines are drawn.

    Key APIs:

    • prepareWithSegments(text, font, options): Returns a PreparedTextWithSegments handle.
    • layoutWithLines(prepared, maxWidth, lineHeight): Returns an array of LayoutLine objects containing text and dimensions.
    • measureLineStats(prepared, maxWidth): Returns { lineCount, maxLineWidth } without allocating strings, useful for 'shrink-wrap' container calculations.
    • walkLineRanges(prepared, maxWidth, callback): Iterates through lines by calling a callback with LayoutLineRange (width and cursors) for each line.
    • layoutNextLineRange(prepared, cursor, maxWidth): An iterator-like API for variable-width layouts (e.g., flowing text around an image). Returns a LayoutLineRange or null when exhausted.
    • materializeLineRange(prepared, range): Converts a LayoutLineRange into a full LayoutLine containing the actual text string.
    import { prepareWithSegments, layoutWithLines } from '@chenglou/pretext'
    
    const prepared = prepareWithSegments('AGI 春天到了. بدأت الرحلة 🚀', '18px "Helvetica Neue"')
    const { lines } = layoutWithLines(prepared, 320, 26) // 320px max width, 26px line height
    for (let i = 0; i < lines.length; i++) ctx.fillText(lines[i].text, 0, i * 26)
  11. Steering rules for new mismatches

    main

    Follow these steps when a new mismatch is identified to determine the correct classification:

    1. Rule out corpus-dirty and diagnostic-sensitivity first.
    2. If widths are obviously tiny-edge cases (sub-pixel), classify as edge-fit.
    3. If a semantic merge or split fixes multiple widths cleanly, classify as boundary-discovery or glue-policy.
    4. If repeated clean corpora still miss after good preprocessing, escalate to shaping-context.
    5. If only one font family or fallback stack misses, classify as font-mismatch.