DOCSEARCHbyAlgolia
Submit Docs

wcmatch

repository·main·Indexed 41 days ago

https://github.com/facelessuser/wcmatch

A Python library providing enhanced file matching and globbing capabilities. It serves as an advanced replacement for fnmatch, glob, and pathlib, bringing Bash-like features to Python including brace expansion, extended globbing, Zsh-style numerical ranges, and symlink recursion via ***. It includes utilities for filename matching (wcmatch.fnmatch), path-centric matching (wcmatch.globmatch), and a pathlib variant.

Tokens
20.5K
Snippets
60
Records
105
Agent score
50%

What's inside wcmatch

  1. Overview of Wildcard Match features

    main

    Wildcard Match is an enhanced replacement for Python's built-in fnmatch, glob, and pathlib libraries. It translates simple, path-centric glob patterns into regular expressions to provide file matching and globbing that closely follows Bash-like behavior.

    Key features include:

    • Enhanced Globbing: Supports ** for recursive globbing and Zsh-style *** for recursive globbing that follows symlinks.
    • Bash-like Syntax: Supports backslash escaping (\), brace expansion (e.g., a{b,{c,d}} expands to ab ac ad), and extended match patterns (@(...), +(...), *(...), ?(...), and !(...)).
    • POSIX Support: Supports POSIX-style character classes (e.g., [[:alnum:]]) using the C locale.
    • Path Utilities: Includes globmatch for path-centric matching and a pathlib variant that uses Wildcard Match's engine instead of Python's default.
    • User Expansion: Automatically expands ~ or ~username to the appropriate user path.
    • File Crawling: Provides a dedicated file search utility called wcmatch built on top of fnmatch and globmatch.

  2. Enable directory traversal with glob.GLOBSTAR and glob.GLOBSTARLONG

    main

    Control how ** patterns match directories.

    • glob.GLOBSTAR (or glob.G): Enables ** to match zero or more directories.
    • glob.GLOBSTARLONG (or glob.GL): Enables *** to act like ** but specifically allows traversing symlinks. Enabling this automatically enables GLOBSTAR.

    Note: When using GLOBSTARLONG, the FOLLOW flag is ignored for **; you must use an explicit *** to traverse symlinks, unless MATCHBASE is also enabled, in which case the implicit leading ** acts as ***.

  3. How glob patterns and flags work together

    main

    Wildcard Match uses a system of flags to enable advanced pattern features. These features add specific 'magic' symbols to the default set of ?*[]\.

    FeatureSymbols Added
    EXTGLOB()
    BRACE{}
    NEGATE!
    MINUSNEGATE-
    SPLIT|
    GLOBTILDE~

    When using glob.is_magic(pattern, flags=...), the function checks if any of the symbols associated with the enabled flags are present in the pattern.

  4. Expand patterns with glob.BRACE

    main

    glob.BRACE (or glob.B) enables Bash-style brace expansion (e.g., a{b,{c,d}} expands to ab ac ad).

    • Numerical Ranges: Supports {0..100} and increments like {0..6..2}.
    • Character Ranges: Supports {a..z}.
    • Uniqueness: By default, glob and iglob return unique results. Use glob.NOUNIQUE (or glob.Q) to mimic Bash behavior and return all results, including duplicates.

    Warning: Massive expansion risk. Patterns like {1..100} generate 100 separate patterns, which can cause the filesystem to be crawled 100 times.

    >>> from wcmatch import glob
>>> glob.glob('{*,README}.md', flags=glob.BRACE)
['LICENSE.md', 'README.md']
>>> glob.glob('{*,README}.md', flags=glob.BRACE | glob.NOUNIQUE)
['LICENSE.md', 'README.md', 'README.md']

  5. Manage pattern expansion limits

    main
    The WcMatch class allows expanding a single pattern into multiple patterns using the pipe | operator or brace expansion {}. To prevent excessive resource usage, there is a default limit of 1000 patterns. You can adjust this using the limit keyword argument. Setting limit=0 removes the limit entirely.

  6. Handle dot files with glob.DOTGLOB and glob.NODOTDIR

    main

    Control how files or directories starting with a dot . are matched.

    • glob.DOTGLOB (or glob.D / glob.DOTMATCH): Allows meta-characters (like *) to match dots. By default, dots are only matched if explicitly provided in the pattern.
    • glob.NODOTDIR (or glob.Z): Prevents "magic" patterns like .* from matching the special directories . and ... To match these specifically, you must use literal . or .. patterns.
    >>> from wcmatch import glob
>>> glob.globfilter(['.', '..'], '.*', flags=glob.NODOTDIR)
[]

  7. Enable extended pattern matching with fnmatch.EXTMATCH

    main

    The fnmatch.EXTMATCH (or fnmatch.E) flag enables extended globbing, allowing special pattern groups like +(...), *(...), and ?(...).

    Important Considerations:

    • Negation Conflict: If EXTMATCH and NEGATE are used together, a pattern starting with !( is treated as an extended glob pattern rather than a negation pattern. To negate a literal ( in an extended pattern, you must escape it: !\(.
    • Security Warning: Extended glob groups are translated into regular expression groups (e.g., +(...) becomes (...)+). This makes them susceptible to Regular Expression Denial of Service (ReDoS) attacks. Avoid using these in performance-critical or untrusted production environments if the risk is high.

  8. How wcmatch.pathlib differs from standard pathlib

    main

    While wcmatch.pathlib mimics the standard Python pathlib API, it introduces several key differences:

    • Enhanced Globbing: The glob(), rglob(), and match() methods are driven by wcmatch.glob, allowing for advanced features via the flags keyword.
    • Pattern Support: glob(), rglob(), and match() accept either a single string pattern or a list of patterns.
    • Explicit Flags: Unlike standard pathlib, GLOBSTAR and DOTGLOB are not enabled by default; you must pass them via the flags argument.
    • Full Matching: PurePath classes include globmatch() (and the alias full_match()) for performing a "full" match, similar to Python 3.13's full_match().
    • Duplicate Filtering: When multiple patterns are provided (or expanded via BRACE or SPLIT), glob() and rglob() automatically filter out duplicate results unless the NOUNIQUE flag is used.
    • Literal Dot Matching: wcmatch.pathlib allows explicit use of . as a literal directory in matching, whereas standard pathlib often ignores it.

  9. Security considerations for Wildcard Match

    main

    Because Wildcard Match translates glob patterns into regular expressions using the Python regular expression engine, it is susceptible to ReDoS (Regular Expression Denial of Service).

    Patterns containing nested quantifiers or overlapping alternations can cause catastrophic backtracking. When using this library in production systems, be cautious about the patterns provided to ensure they do not introduce performance vulnerabilities.

  10. How WcMatch handles hidden files

    main

    In wcmatch, dots (.) are not treated specially by default.

    • Without HIDDEN flag: All hidden files (system and dot files) are excluded from the crawling process. This means *, ?, and [.] will not match dot files because they are never seen by the crawler.
    • With HIDDEN flag: The crawler includes hidden files, allowing *, ?, and [.] to match dot files.
    # Matches standard files
wcmatch.WcMatch('.', '*.yml').match()

# Matches hidden files like .travis.yml
wcmatch.WcMatch('.', '*.yml', flags=wcmatch.HIDDEN).match()

  11. Use negation and exclusion patterns with fnmatch.NEGATE

    main

    The fnmatch.NEGATE (or fnmatch.N) flag allows you to treat patterns starting with ! as exclusion patterns.

    Key behaviors:

    • Pairing Requirement: Exclusion patterns cannot stand alone. They must be paired with an inclusion pattern, either by using the fnmatch.SPLIT flag (e.g., inclusion|!exclusion) or by providing multiple patterns in a list.
    • Dot Files: Enabling NEGATE automatically enables fnmatch.DOTMATCH for all exclusion patterns (this cannot be disabled for excludes).
    • Conflict Resolution: If you pass exclusion patterns via a matching function's exclude parameter, the NEGATE flag is ignored in favor of those parameters. Do not use both exclude and NEGATE simultaneously.

    To force exclusion patterns to act as the primary filter (assuming all files match unless excluded), use the fnmatch.NEGATEALL (or fnmatch.A) flag.

  12. Use exclusion patterns with glob.NEGATE and glob.NEGATEALL

    main

    Manage exclusion patterns (patterns starting with !) to exclude specific files from a match.

    • glob.NEGATE (or glob.N): Treats patterns starting with ! as exclusions. Exclusion patterns must be paired with an inclusion pattern (e.g., using SPLIT or a list of patterns). Enabling NEGATE also automatically enables DOTGLOB for exclusion patterns.
    • glob.NEGATEALL (or glob.A): Forces exclusion patterns to act as if an implicit ** (globstar) inclusion pattern was provided. This allows you to match everything except the specified patterns (e.g., !*.md).
    • glob.MINUSNEGATE (or glob.M): When used with NEGATE, allows exclusion patterns to start with - instead of !, which is useful when using EXTGLOB features that already use !.
    >>> from wcmatch import glob
>>> glob.glob(['..', '!.*'], flags=glob.NEGATE)
[]
>>> glob.glob(['..', '!.*'], flags=glob.NEGATE | glob.NODOTDIR)
['..']