DOCSEARCHbyAlgolia
Submit Docs

Oniguruma Regular Expressions Library

repository·master·Indexed 25 days ago

https://github.com/kkos/oniguruma

A modern and flexible regular expressions library supporting various character encodings and features from multiple regex implementations. It provides a highly configurable C API via the OnigSyntaxType struct, allowing users to toggle basic and advanced regex constructs—such as named capture groups, possessive quantifiers, conditional patterns, and Unicode grapheme cluster matching—through specific configuration flags (op, op2, and syn).

Tokens
13.9K
Snippets
16
Records
43
Agent score
79%

What's inside Oniguruma

  1. Configure Oniguruma syntax using OnigSyntaxType

    master

    Oniguruma's syntax and behavior are configured by populating an OnigSyntaxType struct. This struct allows you to define specific syntax operators and behaviors, similar to how built-in syntaxes are defined.

    Configuration is organized into three 32-bit unsigned int groups:

    1. op: Roughly corresponds to "basic regex" configuration.
    2. op2: Roughly corresponds to "advanced regex" configuration.
    3. behavior: Describes how the engine handles broken input, bad input, or undefined corner cases.

    Additionally, the struct contains:

    • options: Defines the default OnigOptionType used if no options are provided to onig_new().
    • meta_char_table: Used when ONIG_SYN_OP_VARIABLE_META_CHARACTERS is enabled, allowing metacharacters (like * or ?) to be replaced with custom alternates (e.g., using % instead of .* for SQL-style syntax).
    typedef struct {
  unsigned int   op;
  unsigned int   op2;
  unsigned int   behavior;
  OnigOptionType options;   /* default option */
  OnigMetaCharTableType meta_char_table;
} OnigSyntaxType;

  2. Configure variable metacharacters with ONIG_SYN_OP_VARIABLE_META_CHARACTERS

    master

    Enables the use of onig_set_meta_char() to redefine the standard six special regex characters. When this flag is set, Oniguruma uses the custom characters provided via onig_set_meta_char(). If this flag is clear, the default regex characters are used and any data set by onig_set_meta_char() is ignored.

    Supported standard metacharacters include:

    • ONIG_META_CHAR_ESCAPE: \
    • ONIG_META_CHAR_ANYCHAR: .
    • ONIG_META_CHAR_ANYTIME: *
    • ONIG_META_CHAR_ZERO_OR_ONE_TIME: ?
    • ONIG_META_CHAR_ONE_OR_MORE_TIME: +
    • ONIG_META_CHAR_ANYCHAR_ANYTIME: Equivalent to .* (useful for SQL % or shell * wildcards).

  3. Handle independent repeat operators

    master

    When operators like ?, *, +, or {n,m} are not directly attached to an operand (e.g., ^* or (*)), their behavior is controlled by two flags:

    1. ONIG_SYN_CONTEXT_INDEP_REPEAT_OPS:

      • If set: The operators are treated as literals.
      • If clear: The behavior is determined by ONIG_SYN_CONTEXT_INVALID_REPEAT_OPS.

    2. ONIG_SYN_CONTEXT_INVALID_REPEAT_OPS:

      • (Only relevant if ONIG_SYN_CONTEXT_INDEP_REPEAT_OPS is set)
      • If set: Independent operators produce an error message.
      • If clear: Independent operators are silently discarded.

  4. Configure Oniguruma Syntax Flags (syn)

    master
    Oniguruma uses syntax flags (syn) to handle regex syntax corner cases and constructs that may be valid in some engines but errors in others. These flags allow you to fine-tune how the engine interprets specific operators, character classes, and grouping constructs to ensure compatibility with other regex engines (like Python, Ruby, or Perl) or to enforce stricter validation.

  5. Configure Group Two Flags (op2) for advanced regex syntax

    master
    Oniguruma provides Group Two Flags (op2) to enable support for advanced and lesser-known regex syntax constructs. These flags allow you to opt-in to specific features like named captures, possessive quantifiers, conditional patterns, and Unicode grapheme cluster matching. Enabling these flags is necessary if your patterns use these specific syntaxes.

  6. How to use the Oniguruma API in C

    master

    To use the Oniguruma API, include oniguruma.h in your program.

    Handling Name Collisions

    If you need to avoid collisions with existing definitions, you can define the following macros before including the header:

    • To disable UChar type definition (which is unsigned char): define ONIG_ESCAPE_UCHAR_COLLISION.
    • To disable regex_t type definition: define ONIG_ESCAPE_REGEX_T_COLLISION.

    Compiling and Linking (Unix/Cygwin)

    When compiling with cc (assuming a prefix of /usr/local), use the -L flag to specify the library path and -lonig to link the library:

    cc sample.c -L/usr/local/lib -lonig
    cc sample.c -L/usr/local/lib -lonig

  7. Install Oniguruma via Linux distribution packages

    master

    You can install Oniguruma using your Linux distribution's package manager:

    • Fedora: dnf install oniguruma-devel
    • RHEL/CentOS: yum install oniguruma
    • Debian/Ubuntu: apt install libonig5
    • Arch: pacman -S oniguruma
    • openSUSE: zypper install oniguruma
    apt install libonig5

  8. Install Oniguruma on Windows (Visual Studio or vcpkg)

    master

    Using Visual Studio

    Run the following commands in your terminal:

    • To build the library: .\make_win.bat (produces onig_s.lib for static linking and onig.dll for dynamic linking).
    • To run test programs: .\make_win.bat all-test.

    Note: If you want to use the static link library (onig_s.lib) in Win32, add the option -DONIG_EXTERN=extern to your C compiler.

    Using vcpkg

    1. git clone https://github.com/Microsoft/vcpkg.git
    2. cd vcpkg
    3. ./bootstrap-vcpkg.bat
    4. ./vcpkg integrate install
    5. ./vcpkg install oniguruma
    .\make_win.bat

  9. Supported Regular Expression types and encodings

    master

    Oniguruma is a superset of many regular expression implementations. It supports the following types:

    • POSIX
    • Grep
    • GNU Regex
    • Perl
    • Java
    • Ruby
    • Emacs

    Character encoding can be specified per regular expression object. Supported encodings include:

    • ASCII, UTF-8, UTF-16BE, UTF-16LE, UTF-32BE, UTF-32LE
    • EUC-JP, EUC-TW, EUC-KR, EUC-CN
    • Shift_JIS, Big5, GB18030, KOI8-R, CP1251
    • Various ISO-8859 series (1 through 16)
    • Others: GB18030, CP1251

  10. Supported character encodings in Oniguruma

    master

    Oniguruma allows specifying a character encoding for each regular expression object. Supported encodings include:

    • Standard: ASCII, UTF-8, UTF-16 (BE/LE), UTF-32 (BE/LE)
    • East Asian: EUC-JP, EUC-TW, EUC-KR, EUC-CN, Shift_JIS, Big5, GB18030
    • Cyrillic/Other: KOI8-R, CP1251
    • ISO-8859 series: ISO-8859-1 through ISO-8859-16

  12. Install Oniguruma on Linux distributions

    master

    You can install Oniguruma using your Linux distribution's package manager:

    • Fedora: dnf install oniguruma-devel
    • RHEL/CentOS: yum install oniguruma
    • Debian/Ubuntu: apt install libonig5
    • Arch: pacman -S oniguruma
    • openSUSE: zypper install oniguruma
    apt install libonig5