include_dir Rust Crate

repository·master·Indexed 18 days ago

https://github.com/michael-f-bryan/include_dir

A Rust crate that allows developers to embed entire directory trees directly into compiled binaries at compile time. It extends the functionality of include_str! and include_bytes! via the include_dir! macro, providing a Dir object to retrieve files, navigate the embedded filesystem, and optionally search using glob patterns or access file metadata.

Tokens
3.5K
Snippets
12
Records
15
Agent score
63%

What's inside include_dir

  1. Embed a directory tree using include_dir!

    master

    The include_dir! macro allows you to embed an entire directory tree into your binary at compile time. It functions similarly to the standard include_str! and include_bytes! macros. You provide a file path to the macro and assign the resulting Dir to a static variable.

    Once embedded, you can use the Dir object to retrieve files by their full path, inspect their contents, or search for them using glob patterns (if the glob feature is enabled).

    use include_dir::{include_dir, Dir};
    
    static PROJECT_DIR: Dir = include_dir!("$CARGO_MANIFEST_DIR");
    
    // Retrieve a file by its full path
    let lib_rs = PROJECT_DIR.get_file("src/lib.rs").unwrap();
    
    // Inspect file contents as UTF-8
    let body = lib_rs.contents_utf8().unwrap();
  2. Considerations for large directories

    master

    Because include_dir! embeds files as Rust byte strings, including large directories or many files can significantly impact build performance:

    • RAM Usage: Large inclusions can cause the compiler to consume high amounts of RAM.
    • Parsing Time: The compiler may spend significant time parsing the expanded code.
    • Binary Size: The resulting binary size will increase by approximately the size of the included data.

    Example observation: Including a target/ directory (64MB) increased compile time from 1.5s to 5s and RAM usage from 200MB to 730MB.

  3. Embed an entire directory into your binary with include_dir!

    master

    The include_dir! macro allows you to embed an entire directory tree into your Rust binary at compile time, extending the functionality of include_str! and include_bytes!.

    Best Practice: Use Environment Variables

    Because rustc does not guarantee the current working directory when running procedural macros, you should avoid using relative paths. Instead, use environment variable interpolation to ensure paths are absolute and reliable. Common variables provided by Cargo include $CARGO_MANIFEST_DIR and $OUT_DIR.

    Example of including a folder relative to your crate root: include_dir!("$CARGO_MANIFEST_DIR/assets")

    use include_dir::{include_dir, Dir};
    
    // Embed the crate's source directory
    static PROJECT_DIR: Dir<'_> = include_dir!("$CARGO_MANIFEST_DIR");
    
    // Retrieve a file by its path within the directory
    let lib_rs = PROJECT_DIR.get_file("src/lib.rs").unwrap();
    
    // Access contents as UTF-8
    let body = lib_rs.contents_utf8().unwrap();
  4. Search for files using glob patterns

    master

    If you enable the glob feature, you can use the .find() method on a Dir instance to search for files and directories using glob patterns. The .find() method returns an iterator over the matching entries.

    #[cfg(feature = "glob")]
    {
        let glob = "**/*.rs";
        for entry in PROJECT_DIR.find(glob).unwrap() {
            println!("Found {}", entry.path().display());
        }
    }
  5. Retrieve files and sub-directories from a Dir

    master

    You can iterate over the contents of a Dir using the following methods:

    • files(): Returns an iterator over all items in the directory that are File entries.
    • dirs(): Returns an iterator over all items in the directory that are Dir entries.
    • entries(): Returns an iterator over all DirEntry items (both files and directories).
    // Iterate over all files in the directory
    for file in embedded_dir.files() {
        println!("File: {:?}", file.path());
    }
    
    // Iterate over all sub-directories
    for sub_dir in embedded_dir.dirs() {
        println!("Sub-dir: {:?}", sub_dir.path());
    }
  6. Use the Dir struct to access embedded files and directories

    master

    The Dir struct is the primary interface for interacting with embedded files and directories included via the include_dir!() macro. It provides methods to navigate the embedded filesystem, retrieve specific files or sub-directories, and extract the entire embedded structure to the host filesystem.

    Key Capabilities:

    • Navigation: Access the relative path of the directory using .path() and iterate over its contents using .entries(), .files(), or .dirs().
    • Lookup: Use .get_entry(path), .get_file(path), or .get_dir(path) to find specific items. These methods perform a recursive search.
    • Verification: Use .contains(path) to check if a specific path exists within the embedded directory.
    • Extraction: Use .extract(base_path) to write all embedded files and directories to the actual filesystem starting at base_path.
    // Example of navigating and extracting a Dir
    let embedded_dir = include_dir!("my_assets");
    
    // Look up a specific file
    if let Some(file) = embedded_dir.get_file("images/logo.png") {
        println!("Found file with size: {}", file.contents().len());
    }
    
    // Extract everything to a local folder named "output"
    embedded_dir.extract("output").expect("Failed to extract files");
  7. Use the include_dir! macro to embed directories

    master

    The include_dir! macro allows you to embed the entire contents of a directory into your Rust binary at compile time. This is useful for bundling assets, configuration files, or static data directly within your executable.

    Usage

    Pass a single string literal representing the path to the directory you wish to include. The macro supports environment variable expansion using the $VAR syntax.

    Key Features

    • Recursive Embedding: Automatically traverses subdirectories and embeds them as include_dir::Dir entries.
    • Environment Variables: You can use $VARIABLE_NAME within the path string to resolve paths dynamically based on environment variables.
    • Metadata Support: If the metadata feature is enabled, the macro will also embed file metadata (accessed, created, and modified timestamps).
    • Path Normalization: Paths are normalized to use forward slashes (/) regardless of the host operating system (Windows or Linux).
    // Example usage of the include_dir! macro
    let my_dir = include_dir!("assets");
    
    // You can also use environment variables
    let dynamic_dir = include_dir!("$ASSET_PATH/data");
  8. Use the DirEntry enum to navigate the embedded tree

    master

    The DirEntry<'a> enum represents an item within the embedded directory tree, acting as a wrapper for either a directory (Dir) or a file (File). It provides a unified interface to access common properties like the path, regardless of whether the underlying item is a file or a directory.

    Key methods:

    • path(): Returns a reference to the Path of the entry.
    • as_dir(): Returns Some(&Dir) if the entry is a directory, otherwise None.
    • as_file(): Returns Some(&File) if the entry is a file, otherwise None.
    • children(): Returns a slice of DirEntry items if the entry is a directory; returns an empty slice if it is a file.
    use include_dir::DirEntry;
    
    // Assuming `entry` is a DirEntry<'a> obtained from the tree
    let path = entry.path();
    
    if let Some(dir) = entry.as_dir() {
        // Handle directory logic
        let children = entry.children();
    }
    
    if let Some(file) = entry.as_file() {
        // Handle file logic
    }
  9. Use the Metadata struct to inspect embedded file timestamps

    master

    The Metadata struct provides access to the timestamps of embedded files. It stores time offsets as std::time::Duration relative to the SystemTime::UNIX_EPOCH. You can retrieve these as standard SystemTime objects using the accessed(), created(), and modified() methods.

    use include_dir::Metadata;
    use std::time::Duration;
    
    // Creating metadata manually (e.g., for testing or custom embedding)
    let meta = Metadata::new(
        Duration::from_secs(1600000000), // accessed
        Duration::from_secs(1600000000), // created
        Duration::from_secs(1600000000), // modified
    );
    
    // Accessing the SystemTime values
    let accessed_time = meta.accessed();
    let created_time = meta.created();
    let modified_time = meta.modified();
  10. Extract embedded files to the filesystem with extract()

    master

    The extract<S: AsRef<Path>>(&self, base_path: S) method writes the entire contents of the embedded Dir to the host filesystem at the specified base_path.

    Behavioral Notes:

    • Directory Creation: It automatically creates parent directories for the target path if they do not exist.
    • Conflict Handling: The method will fail if any files intended to be written already exist at the target location.
    • Error State: In the event of an error during extraction, a partially extracted directory structure may remain on the filesystem.
    // Extracts the embedded directory to the current working directory's "extracted_assets" folder
    dir.extract("extracted_assets")?;