regexp2

repository·master·Indexed 22 days ago

https://github.com/dlclark/regexp2

A feature-rich regular expression engine for Go that provides backtracking and compatibility with Perl5 and .NET. Unlike the built-in RE2-based regexp package, regexp2 supports lookarounds and backreferences. It includes compatibility modes for RE2 and ECMAScript, a compat package for standard library method signatures, and advanced tuning options for backtracking stack size and memory caching.

Tokens
8.8K
Snippets
31
Records
36
Agent score
76%

What's inside regexp2

  1. Handle Catastrophic Backtracking and Timeouts

    master

    Because regexp2 supports features like lookarounds and backreferences, it is susceptible to catastrophic backtracking.

    1. Stack Limits: By default, the backtracking stack is limited to 100,000 slots. If exceeded, it returns regexp2.ErrBacktrackingStackLimit. You can increase this via OptionMaxBacktrackingStackSize(n) or disable it with a negative value.
    2. Match Timeouts: You can set Regexp.MatchTimeout to bound the duration of a match.

    Note on Performance: Timeout checking uses a background worker that updates a clock approximately every 100ms. This incurs a constant background CPU load (~0.15%) as long as any live matches have a timeout set, even if the matches finish quickly.

    // Increase stack size
    re := regexp2.MustCompile(pattern, regexp2.OptionMaxBacktrackingStackSize(200000))
    
    // Disable stack limit
    re := regexp2.MustCompile(pattern, regexp2.OptionMaxBacktrackingStackSize(-1))
    
    // Check for error
    if errors.Is(err, regexp2.ErrBacktrackingStackLimit) {
        // handle limit exceeded
    }
  2. Use Unicode character classes

    master

    The engine supports Unicode character classes using \p{...} (positive) and \P{...} (negated). It supports Go Unicode categories, aliases, scripts, and properties. It also supports property selection syntax like \p{property=value} (e.g., \p{grapheme_cluster_break=regional_indicator}).

    letter := regexp2.MustCompile(`\p{L}+`)
    katakana := regexp2.MustCompile(`\p{Katakana}+`)
    notEmoji := regexp2.MustCompile(`\P{Emoji}+`)
  3. Enable ECMAScript compatibility mode

    master

    You can configure regexp2 to match the ECMAScript specification as closely as possible by enabling the ECMAScript flag.

    Important Notes:

    • This mode prioritizes the ECMAScript specification behavior over matching the C# RegexOptions.ECMAScript interpretation.
    • To support the \u{CodePoint} syntax, you must provide both the ECMAScript and Unicode flags.
  4. Use the regexp compatibility adapter

    master

    The github.com/dlclark/regexp2/v2/compat package allows you to use regexp2 while maintaining the method signatures of the standard library regexp package.

    • compat.MustCompile: Compiles a pattern using the regexp2 engine but returns a type compatible with standard regexp signatures.
    • compat.Wrap: Wraps an existing *regexp2.Regexp into a compatible interface.
    • compat.Matcher: An interface implemented by both *regexp.Regexp and the compat adapter, allowing functions to accept either engine.

    Warning: Because standard library signatures do not return errors, the adapter will panic if a regexp2 error occurs (like a match timeout or stack limit error).

    import (
    	"github.com/dlclark/regexp2/v2"
    	"github.com/dlclark/regexp2/v2/compat"
    )
    
    // Compile using the adapter
    re := compat.MustCompile(`Your pattern`, regexp2.RE2)
    if re.MatchString(`Something to match`) {
    	// do something
    }
    
    // Or wrap an existing regexp2 instance
    base := regexp2.MustCompile(`Your pattern`)
    reWrapped := compat.Wrap(base)
    
    // Use the common interface
    func findWords(re compat.Matcher, input string) []string {
    	return re.FindAllString(input, -1)
    }
  5. Install regexp2 via go get

    master

    Install the regexp2 library using the standard Go toolchain. Note that version 2 requires the /v2 suffix in the module path and requires Go 1.25 or later.

    go get github.com/dlclark/regexp2/v2@latest
  6. Basic usage of regexp2

    master

    Usage is similar to the standard Go regexp package. Use regexp2.Compile to create a regex state machine, or regexp2.MustCompile if you want the application to panic on an invalid pattern. The resulting *regexp2.Regexp is safe for concurrent use across goroutines.

    re := regexp2.MustCompile(`Your pattern`)
    if isMatch, _ := re.MatchString(`Something to match`); isMatch {
        //do something
    }
  7. Enable RE2 Compatibility Mode

    master

    By default, regexp2 matches the .NET engine. You can enable RE2 compatibility mode by passing the regexp2.RE2 option to Compile or MustCompile. This changes parsing behavior to support:

    • Named ASCII character classes (e.g., [[:foo:]])
    • Python-style capture groups (e.g., (?P<name>re))
    • Python-style named backreferences (e.g., (?P=name))
    • $ matching only the end of the string
    • \d, \s, and \w matching RE2-specific character sets
    • Relaxed character escape sequences (e.g., \_ matches literal _)
    re := regexp2.MustCompile(`Your RE2-compatible pattern`, regexp2.RE2)
    if isMatch, _ := re.MatchString(`Something to match`); isMatch {
        // do something
    }
  8. Understand OptimizationOptions and default limits

    master

    The OptimizationOptions struct controls runtime limits and caching behavior. By default, regexp2 uses DefaultOptimizationOptions to ensure that Compile is safe for mixed-cardinality inputs by providing sensible bounds on memory and stack usage.

    Key behaviors for tuning:

    • Stack Size: MaxBacktrackingStackSize prevents stack overflow. Use a negative value to allow unbounded growth.
    • Caching: For replacement data, setting a value of 0 disables persistent retention, while -1 allows unbounded growth. For pooled buffers, 0 disables pooling and -1 allows all built-in size classes.
    // Default values used if no options are provided:
    // MaxBacktrackingStackSize: 100000
    // MaxCachedRuneBufferLength: 256 KB
    // MaxCachedReplaceBufferLength: 256 KB
    // MaxCachedReplacerDataEntries: 16
    // MaxCachedReplacerDataBytes: 4 KB
    // DisableCharClassASCIIBitmap: false
  9. Understand the hierarchy of Match, Group, and Capture

    master

    The regexp2 matching results follow a hierarchical structure:

    1. Match: The top-level result. It contains the entire match and a collection of Group objects.
    2. Group: A named or numbered collection of captures.
      • A Group embeds a Capture representing its last capture for easy access.
      • A Group contains a slice of Captures ([]Capture), which is useful for patterns that allow repeated captures in a single group.
    3. Capture: The most granular level, representing the specific start and end positions (in runes) of a matched substring.
  10. Use special replacement tokens in patterns

    master

    When performing replacements using standard pattern strings (rather than a MatchEvaluator), regexp2 supports several special tokens to reference parts of the match or the surrounding text. These are represented by negative integer constants in the engine:

    TokenConstantDescription
    replaceLeftPortion-1The portion of the text to the left of the match
    replaceRightPortion-2The portion of the text to the right of the match
    replaceLastGroup-3The last captured group in the match
    replaceWholeString-4The entire input string

    Note: These constants are used internally by the syntax.ReplacerData rules. When using standard replacement strings, you typically use group references (like $1, $2) which the engine maps to these logic paths.

  11. Configure MatchTimeout to prevent catastrophic backtracking

    master

    The Regexp struct includes a MatchTimeout field of type time.Duration. A match will time out if it takes approximately more than this duration. This is a safety mechanism against catastrophic backtracking.

    By default, MatchTimeout is set to DefaultMatchTimeout (the maximum possible duration), which effectively suppresses timeout checking. You can set a specific duration on your Regexp instance to enforce limits.

    re, _ := regexp2.Compile("complex-pattern")
    re.MatchTimeout = 100 * time.Millisecond
  12. Prevent Goroutine Leaks in Tests when using MatchTimeout

    master

    If you use MatchTimeout, regexp2 starts a background goroutine for the timeout clock. This can cause failures in testing tools like uber-go/goleak. To fix this, you must call regexp2.StopTimeoutClock() during test teardown.

    You can also speed up tests by reducing the clock cycle rate using regexp2.SetTimeoutCheckPeriod. This must be called in an init() function because it is not thread-safe.

    func TestSomething(t *testing.T) {
        defer goleak.VerifyNone(t)
        defer regexp2.StopTimeoutClock()
    
        // ... test
    }
    
    // Or in TestMain
    func TestMain(m *testing.M) {
        // ... setup
        m.Run()
        regexp2.StopTimeoutClock()
        goleak.VerifyNone(t)
    }
    
    // Speed up testing in an init function
    func init() {
        regexp2.SetTimeoutCheckPeriod(time.Millisecond)
    }