DOCSEARCHbyAlgolia
Submit Docs

regex

repository·master·Indexed 26 days ago

https://github.com/rust-lang/regex

A high-performance regular expression library for Rust (version 1.13.1) that guarantees linear time complexity O(m * n). It provides the `regex::Regex` API for UTF-8 strings, `regex::bytes::Regex` for arbitrary byte slices, and `RegexSet` for simultaneous multi-pattern scanning. The repository also includes `regex-automata` for fine-grained engine control, `regex-cli` for debugging and testing, and `regex-capi` for C language interoperability via the `rure` API.

Tokens
29.2K
Snippets
96
Records
156
Agent score
86%

What's inside regex

  1. Overview of regex-automata

    master
    The regex-automata crate exposes the various regex engines used by the regex crate. It provides an "expert" level API designed for fine-grained control over regex engines. The engines focus on finite automata implementations and guarantee worst-case O(m * n) time complexity for all searches, where m is the length of the regex and n is the length of the haystack.

  2. Use regex-lite for string searching

    master
    The regex-lite crate provides a lightweight regex engine with a syntax nearly identical to the regex crate. It is designed for users who need a smaller binary size and faster compilation times by opting out of complex optimizations and robust Unicode support. All searches have a worst-case time complexity of O(m * n), where m is the regex size and n is the string size.

  3. Understand the difference between Ast and Hir in regex-syntax

    master

    The regex-syntax crate exports two primary types for representing regular expressions:

    • Ast: A faithful abstract syntax of a regular expression. It can be converted back to concrete syntax while mostly preserving its original form.
    • Hir: A high-level intermediate representation (HIR). It simplifies the syntactic structure to facilitate analysis and compilation. While it can be converted back to concrete syntax, the output will likely not resemble the original input syntax.

  4. Compile a regex::Regex from a regex_syntax::hir::Hir

    master

    There is no direct way to compile a regex::Regex from a regex_syntax::hir::Hir. To achieve this, you must:

    1. Convert the Hir to a string using its std::fmt::Display implementation.
    2. Compile that string using Regex::new from the regex crate.

  7. Generate and serialize a DFA

    master

    The generate serialize command automates the process of compiling a regex into a DFA, writing the binary serialization to files, and generating Rust code to deserialize it lazily.

    Command Structure: regex-cli generate serialize <type> <variable_name> <output_dir> <pattern>

    Arguments:

    • <type>:
      • sparse: Sacrifices search performance for smaller DFA size.
      • dense: Faster searches but larger DFAs.
      • dfa: Generates a single DFA (finds end of match).
      • regex: Generates two DFAs (one for end of match, one for start) to support full regex functionality.
    • <variable_name>: The name of the variable used in the generated Rust source.
    • <output_dir>: The directory where files will be written.
    • <pattern>: The regex pattern(s) to build the DFA for.

    Common Flags:

    • --minimize: Applies a DFA minimization algorithm to shrink the size.
    • --shrink: Uses heuristics to make the NFA smaller, speeding up determinization.
    • --start-kind anchored: Builds a DFA that only supports anchored searches (matches must start at the search offset).
    • --rustfmt: Runs rustfmt on the generated Rust code.
    • --safe: Uses only safe Rust code for deserialization.

    Workflow Example:

    1. Add regex-automata to your Cargo.toml with features std and dfa-search.
    2. Run the regex-cli command to generate files.
    3. Include the generated module in your Rust project and use the constant to perform searches.
    regex-cli generate serialize sparse dfa \
  --minimize \
  --shrink \
  --start-kind anchored \
  --rustfmt \
  --safe \
  SIMPLE_WORD_FWD \
  ./src/ \
  "\w"

  9. Run compilation tests for the regex crate

    master

    To measure from-scratch compilation time and relative binary size increases for the regex or regex-automata crates, use the regex-cli compile-test command.

    Prerequisites:

    1. You must have regex-cli installed.
    2. You must run the command from the root of a checkout of this repository.

    Steps:

    1. Create a temporary directory for the output.
    2. Run regex-cli compile-test pointing to the repository root and your output directory.
    3. Pipe the output to a .csv file.

    Note: Relative binary size is calculated by comparing a binary using the regex crate against a baseline 'hello world' program.

    $ mkdir /tmp/regex-compile-test
$ regex-cli compile-test ./ /tmp/regex-compile-test | tee record/compile-test/2023-04-19_1.7.3.csv

  10. Optimize regex performance

    master

    To ensure high performance when using the regex crate, follow these best practices:

    1. Avoid re-compiling in loops: Compiling a regex is expensive. Use std::sync::LazyLock or the regex! macro to compile once and reuse.
    2. Ask only for what you need: Use is_match if you only need a boolean result. Avoid captures if you don't need group offsets, as computing them is more expensive.
    3. Use literals: Including literals in your pattern (e.g., @ in \w+@\w+) helps the engine accelerate the search.
    4. Minimize Unicode impact: If you only need ASCII, use ASCII-only character classes like [0-9] or (?-u:\w) to reduce memory usage and increase speed.
    5. Handle thread contention: While Regex is thread-safe, sharing a single instance across many threads can cause contention. Consider cloning the Regex for each thread; clones share the same compiled state but provide optimized access to mutable search space.

  11. Print debug output for regex-automata types

    master

    Use the debug command to inspect the internal representation of principle types in the regex-automata crate (e.g., NFA). This is useful for debugging regex objects.

    Common flags:

    • --no-table: Omits extra metadata about NFA size and build time to make output more concise.
    • -b or --no-utf8-syntax: Allows patterns that can match invalid UTF-8. This is required if your pattern (like (?-u:.)) can match non-UTF-8 bytes, as the default setting forbids such patterns.
    $ regex-cli debug thompson '.' --no-table

  12. Avoid re-compiling regex in loops using LazyLock

    master

    Compiling a Regex is expensive. To avoid re-compiling the same pattern inside a loop or a frequently called function, use std::sync::LazyLock (or the once_cell crate if on an older Rust version) to compile the regex once and reuse it.

    use std::sync::LazyLock;

use regex::Regex;

fn some_helper_function(haystack: &str) -> bool {
    static RE: LazyLock<Regex> = LazyLock::new(|| Regex::new(r"...").unwrap());
    RE.is_match(haystack)
}

fn main() {
    assert!(some_helper_function("abc"));
    assert!(!some_helper_function("ac"));
}