DOCSEARCHbyAlgolia
Submit Docs

rustybuzz

repository·main·Indexed 20 days ago

https://github.com/harfbuzz/rustybuzz

A pure Rust port of the HarfBuzz shaping algorithm (matching v10.1.0) designed for integration without C++ dependencies. It provides shaping functionality using the ttf-parser crate for TrueType parsing, though it does not include subsetting, Unicode routines, or system library glue. Note: This project is currently unmaintained and archived; users are encouraged to migrate to HarfRust.

Tokens
6.9K
Snippets
27
Records
37
Agent score
71%

What's inside rustybuzz

  1. Understand rustybuzz limitations and behavior

    main

    While rustybuzz matches HarfBuzz v10.1.0 and passes most shaping tests, it is not a 1:1 faithful port and has several key differences:

    Core Differences

    • Shaping Only: Unlike HarfBuzz, rustybuzz only provides shaping. It does not include subsetting, Unicode routines, or glue for system libraries (like FreeType, CoreText, or DirectWrite).
    • Font Parsing: TrueType parsing is handled by the ttf-parser crate rather than the HarfBuzz internal parser.
    • No Font Size Property: Shaping always uses UnitsPerEm. You must scale the resulting glyph positions and advances manually to match your desired font size.
    • Error Handling: Malformed fonts will cause an error in rustybuzz, whereas HarfBuzz typically uses a fallback/dummy shaper.

    Unsupported Features

    • Subsetting
    • mort tables (deprecated by Apple)
    • Arabic fallback shaper
    • avar2 and other parts of the boring-expansion-spec

  2. Determine which harfbuzz changes to ignore

    main

    Because rustybuzz is a shaper and not a full collection like harfbuzz, many changes in the upstream repository do not apply.

    Ignore these changes:

    • Subsetting: Any files/functions named subset or commits prefixed with [subset].
    • Font Parsing: Changes to font parsing logic (unless new tests fail, in which case consult the project discussions).
    • Unicode Tables: rustybuzz uses different routines/crates for Unicode tables.
    • Custom C++ Containers: rustybuzz uses Rust's std instead of harfbuzz's custom containers.
    • Build Systems: Any changes to the harfbuzz build systems.

    Focus on these changes:

    • Any change in an hb-ot-* file likely affects rustybuzz.

  3. Set up a working environment for backporting

    main

    To backport changes from harfbuzz to rustybuzz, it is recommended to use Linux or macOS due to harfbuzz's reliance on Unix tools.

    Required dependencies:

    • git
    • Python
    • Rust
    • clang/gcc
    • meson
    • ragel (potentially, must be built from source if *.rl files change)

  4. Generate Rust source files using Python scripts

    main

    The repository includes Python scripts to automate the generation of Rust source files for specific Harfbuzz components. Use these scripts to regenerate the universal table or vowel constraints files if the underlying data has changed.

    # Generate the OT shaper use table
python3 gen-universal-table.py > ../src/hb/ot_shaper_use_table.rs

# Generate vowel constraints and format the output
python3 ./gen-vowel-constraints.py > ../src/complex/vowel_constraints.rs
rustfmt ../src/complex/vowel_constraints.rs

  5. Build Ragel with Rust support

    main

    To generate Rust code from Ragel scripts, you must use the latest development branch of Ragel, as the stable version does not support native Rust generation. Ragel requires colm as a dependency.

    On macOS, ensure you have the necessary build tools installed via Homebrew:

    brew install automake autoconf libtool

    Follow these steps to build the dependencies:

    1. Build and install colm to a custom prefix.
    2. Build and install ragel using the --with-colm flag pointing to your colm installation.

    Note: Building on Windows is not recommended due to the use of autotools.

    # build `colm` first
git clone https://github.com/adrian-thurston/colm
cd colm
./autogen.sh
./configure --prefix=/path/to/colm/install # prefer a custom path to /usr/local
make
make install

cd ..
git clone https://github.com/adrian-thurston/ragel
./autogen.sh
# --with-colm takes the same path we used above
./configure --prefix=/path/to/ragel/install --with-colm=/path/to/colm/install
make
make install

  6. Install rustybuzz via Cargo

    main

    rustybuzz is a pure Rust port of the HarfBuzz shaping algorithm. Because it is written in Rust and does not rely on C++ or external system libraries, you can add it to your project without a C++ compiler or complex configuration.

    rustybuzz = "*"

  7. Post-generation tasks for Ragel-generated Rust code

    main

    After generating Rust code from Ragel scripts, you must perform the following manual steps to ensure code quality and correctness:

    1. Format the code: Ragel produces non-standard Rust formatting. You must run cargo fmt on the generated files.
    2. Add warnings: Ragel does not preserve comments. Manually add a header to each generated file to prevent accidental edits:
      // This file is autogenerated. Do not edit it!
//
// See docs/ragel.md for details.
    3. Handle the Universal state machine: If generating universal_machine.rs from universal_machine.rl, you must manually modify the output. Replace instances where Rust complains about variables set to 0 (e.g., ts = 0;) by replacing 0 with p0.

  9. Run rustybuzz benchmarks

    main

    To run the benchmarks for the rustybuzz project, use cargo bench with the nightly toolchain. You may need to unset the HARFBUZZ_SYS_NO_PKG_CONFIG environment variable to ensure proper configuration during the benchmark run.

    HARFBUZZ_SYS_NO_PKG_CONFIG="" cargo +nightly bench

  10. Update shaping tests using gen-shaping-tests.py

    main

    To update the auto-generated Rust shaping tests, you must first build the specific harfbuzz commit you are porting using meson. Do not use your system's installed harfbuzz version.

    1. Build the specific harfbuzz commit:
    git clone https://github.com/harfbuzz/harfbuzz
cd harfbuzz
git checkout <HASH> # Replace <HASH> with the target commit hash
meson builddir --reconfigure
ninja -Cbuilddir
    1. Generate the tests: Run the generation script from the rustybuzz/scripts directory, passing the path to the harfbuzz clone directory (not the src subdirectory):
    cd rustybuzz/scripts
./gen-shaping-tests.py /path/to/harfbuzz
    1. Verify: Run cargo test to ensure the new tests pass.
    # Build harfbuzz at the specific commit
git clone https://github.com/harfbuzz/harfbuzz
cd harfbuzz
git checkout HASH
meson builddir --reconfigure
ninja -Cbuilddir

# Generate rustybuzz tests
cd rustybuzz/scripts
./gen-shaping-tests.py /path/to/harfbuzz

  11. Generate Rust state machines using ragel-rust

    main

    Once Ragel is built, use the ragel-rust binary to convert .rl scripts into Rust code.

    Use the following command structure: [path/to/ragel/bin/ragel-rust] -e -F1 [input_file.rl]

    • -e: Enables specific generation modes.
    • -F1: Specifies the output format.

    Note: Ragel creates temporary *.ri files during this process which can be safely deleted after generation.

    /path/to/ragel/install/bin/ragel-rust -e -F1 src/hb/ot_shape_complex_indic_machine.rl # or any other .rl file

  12. Backport a harfbuzz commit to rustybuzz

    main

    Follow this general algorithm to port changes:

    1. Identify the commit: Find the latest harfbuzz commit that has not yet been ported (check rustybuzz commit messages for the last ported hash).
    2. Analyze: Determine if the change applies to rustybuzz (see "Analyzing Changes" for guidance on what to skip).
    3. Port: Apply the logic to the rustybuzz codebase.
    4. Update Tests: Port harfbuzz tests if necessary (see "Updating Tests" for details).
    5. Verify: Run cargo test to ensure all tests pass.
    6. Format: Run cargo fmt.
    7. Commit: Commit the changes with a link to the original harfbuzz commit.