fdir Documentation

repository·master·Indexed 23 days ago

https://github.com/thecodrr/fdir

A high-performance directory crawler and globbing alternative for NodeJS designed to handle massive directory trees, capable of crawling 1 million files in under 1 second. It features a fluent Builder API for configuring filters, depth limits, symlink handling, and path formats. fdir supports synchronous, promise-based, and callback-based execution, and provides specialized options for grouping files or retrieving only file and directory counts.

Tokens
5.5K
Snippets
18
Records
47
Agent score
83%

What's inside fdir

  1. Use the Builder API to configure fdir

    master
    The primary way to interact with fdir is through the Builder class located in src/builder/index.js. The Builder provides a fluent API to define flags and filters. These configurations are eventually compiled into an options object that is passed to the core engine to control the walking process.
  2. How fdir works internally

    master

    At its core, fdir is a directory walker that takes a rootDirectory path and various flags/filters to recursively output all file paths.

    To maximize performance, fdir uses a conditional function building strategy. Instead of using heavy branching logic during the walk, it builds tiny, specialized internal functions based on the provided flags. These functions are designed to be inlined by the JavaScript engine, which reduces branching overhead and memory allocations during the filesystem traversal.

  3. How fdir works: The Builder Pattern

    master

    fdir uses a builder pattern to configure a crawler instance fluently. Instead of passing a large configuration object to a single function, you chain configuration methods on a new fdir instance, call .crawl(path) to prepare the crawler, and finally call one of the execution methods (.withPromise(), .withCallback(), or .sync()).

    Example workflow:

    new fdir()
      .withBasePath()
      .crawl("path/to/dir")
      .sync();
  4. Quickstart: Crawl directories with fdir

    master

    To use fdir, instantiate the fdir builder, configure your desired options (like withFullPaths()), and specify the target directory with crawl(). You can then retrieve the files either synchronously using .sync() or asynchronously using .withPromise().

    import { fdir } from "fdir";
    
    // create the builder
    const api = new fdir().withFullPaths().crawl("path/to/dir");
    
    // get all files in a directory synchronously
    const files = api.sync();
    
    // or asynchronously
    api.withPromise().then((files) => {
      // do something with the result here.
    });
  5. Understand fdir output formats

    master

    Depending on your Options, fdir returns one of three primary output types:

    1. PathsOutput: An array of strings (string[]) representing the paths found. This is the default behavior.
    2. OnlyCountsOutput (Counts): An object containing the total number of files and directories found.
      • files: number
      • directories: number
      • dirs: number (Deprecated, use directories instead)
    3. GroupOutput (Group[]): An array of objects grouping files by their parent directory.
      • directory: string (The path to the directory)
      • files: string[] (Array of file paths within that directory)
      • dir: string (Deprecated, use directory instead)
  6. Use crawlWithOptions for non-chained configuration

    master

    If you prefer not to use method chaining, you can pass an Options object directly to crawlWithOptions.

    Supported Options:

    type Options = {
      includeBasePath?: boolean;
      includeDirs?: boolean;
      normalizePath?: boolean;
      maxDepth?: number;
      maxFiles?: number;
      resolvePaths?: boolean;
      suppressErrors?: boolean;
      group?: boolean;
      onlyCounts?: boolean;
      filters: FilterFn[];
      resolveSymlinks?: boolean;
      useRealPaths?: boolean;
      excludeFiles?: boolean;
      excludeSymlinks?: boolean;
      exclude?: ExcludeFn;
      relativePaths?: boolean;
      pathSeparator: PathSeparator;
      signal?: AbortSignal;
      globFunction?: Function;
    };

    Example:

    new fdir()
      .crawlWithOptions("path/to/dir", {
        includeBasePath: true,
      })
      .sync();
  7. Use glob patterns to match files

    master

    Apply glob filters to include only matching files. fdir uses picomatch internally, but you must install it manually if you want to use custom glob functions.

    • glob(...string[]): Applies a glob filter to all files.
    • globWithOptions(string[], Object): Applies a glob filter with specific matcher options.
    • withGlobFunction(Function): Uses a provided function (like picomatch) to perform the matching.

    Example:

    // Using built-in glob
    const crawler = new fdir().glob("./**/*.js", "./**/*.md");
    
    // Using a custom glob function
    import picomatch from 'picomatch';
    const crawler = new fdir().withGlobFunction(picomatch);
    const crawler = new fdir().glob("./**/*.js", "./**/*.md");
  8. Get file and directory counts with onlyCounts

    master

    To optimize performance when you only need the totals, use onlyCounts(). This changes the output from an array of paths to an object containing files and dirs counts.

    Example:

    const { files, dirs } = new fdir().onlyCounts().sync();