doublestar

doublestar provides path pattern matching and globbing with support for the ** (globstar) pattern. This allows for recursive matching of files and directories.

Pattern Behavior

• ** matches any number of path components (files and directories) recursively.
• Constraint: The ** must appear as a path component by itself. For example, /path** is invalid and behaves like /path*. To match everything under a path, use /path/** or /path/*/**.
• Directory Matching: /path/** matches all files and directories under the path, whereas /path/**/ will only match directories.

Examples

Given the structure:

grandparent
`-- parent
    |-- child1
    `-- child2

• **/child* matches child1 and child2.
• grandparent/**/child? matches child1 and child2.
• **/parent/* matches child1 and child2.
• ** matches everything recursively.

When using doublestar, be aware of how specific pattern features impact performance:

• Alternations ({alts}): Using alternatives is conceptually similar to running multiple separate patterns. Performance may degrade significantly if you use deeply nested alternations, as this increases the number of patterns that must be evaluated during Match() or Glob() operations.
• Doublestars (**):
  • In matching operations (Match()), the performance of ** is comparable to a single *.
  • In globbing operations (Glob(), GlobWalk()), ** can trigger a large number of filesystem reads because it forces a recursive traversal of the directory tree.

Doublestar supports the following special terms:

Character Classes:

• [abc123]: matches any character in the set.
• [a-z0-9]: matches any character in the range.
• [^class] or [!class]: matches any character NOT in the class.

Notes:

• Escape special characters with a backslash \.
• A ** should ideally be surrounded by separators (e.g., /**/). A mid-pattern ** (e.g., path/to/**.txt) behaves like bash's globstar and returns the same as path/to/*.txt.

If you have already called ValidatePattern and do not need to check if the pattern is well-formed, you can use the Unvalidated variants of the matching functions for a small performance improvement. This is most effective when the pattern matching reaches the end of the name before the end of the pattern (e.g., Match("a/b/c", "a")).

Available functions:

• MatchUnvalidated(pattern, name string) bool
• PathMatchUnvalidated(pattern, name string) bool

v3 introduced the use of ! to negate character classes (in addition to ^).

If your existing patterns use an exclamation mark as the first character in a character class (e.g., [!...]), you must update the pattern by escaping the exclamation mark or moving it to a different position within the class to avoid unintended negation behavior.

The change from v1 to v2 involved updating the OS interface.

• The return type of the Open method on the OS interface changed from *os.File to a new interface called doublestar.File.
• The doublestar.File interface only defines io.Closer and Readdir, making it compatible with libraries like go-billy or afero.

If you were using the OS interface, update the return type of your Open implementation. Since *os.File already implements doublestar.File, this is a straightforward change.

Install the doublestar package using go get. Note that v4 is a rewrite optimized for performance and requires Go v1.16+ because it utilizes the io/fs package.

go get github.com/bmatcuk/doublestar/v4

To use the library in your project, import the v4 module path.

import "github.com/bmatcuk/doublestar/v4"

To prevent GlobWalk from entering a specific directory during traversal, return fs.SkipDir from your GlobWalkFunc callback.

doublestar provides a package-level variable SkipDir which is an alias for fs.SkipDir to make this explicit.

• If the matched path is a directory: GlobWalk will not recurse into it.
• If the matched path is not a directory: GlobWalk will skip the rest of the current parent directory.

err := doublestar.GlobWalk(fsys, "**", func(path string, d fs.DirEntry) error {
    if d.IsDir() && someCondition(path) {
        return doublestar.SkipDir // Do not recurse into this directory
    }
    return nil
})

Since globs are not regular expressions, use GlobWalk with a custom filter function to perform complex matching. Use the glob pattern to perform an initial broad pass, then apply your custom logic in the callback.

var matches []string
err := doublestar.GlobWalk(fsys, pattern, func(p string, d fs.DirEntry) error {
  if (customFilter(p, d)) {
    matches = append(matches, p)
  } else if (d.isDir()) {
    return doublestar.SkipDir
  }
  return nil
})
return matches, err

Use GlobWalk(fsys fs.FS, pattern string, fn GlobWalkFunc, opts ...GlobOption) to call a callback function fn for every file matching the pattern.

Key behaviors:

• It is more memory-efficient than Glob() because it avoids allocating a slice of all matches.
• It provides access to fs.DirEntry objects for each match.
• You can quit early by returning a non-nil error from the callback. Returning SkipDir (from io/fs) will skip the current directory.
• Like Glob(), it assumes / as the path separator.

type GlobWalkFunc func(path string, d fs.DirEntry) error

func GlobWalk(fsys fs.FS, pattern string, fn GlobWalkFunc, opts ...GlobOption) error