@yao-pkg/pkg

repository·main·Indexed 21 days ago

https://github.com/yao-pkg/pkg

A tool that packages Node.js projects into single, self-contained executables for Linux, macOS, and Windows, allowing them to run on devices without Node.js installed. It supports two packaging strategies: Traditional Mode (V8 bytecode and patched binaries) and Enhanced SEA Mode (official Node.js Single Executable Application APIs for native ESM support). The tool includes a Virtual Filesystem (VFS) accessible via the /snapshot/ prefix and provides debugging tools to inspect the VFS tree and optimize binary size.

Tokens
45.8K
Snippets
132
Records
210
Agent score
74%

What's inside @yao-pkg/pkg

  1. What is pkg?

    main
    pkg is a command-line tool that packages Node.js projects into single, self-contained executables for Linux, macOS, and Windows. These binaries run on target machines without requiring a Node.js installation, do not require node_modules to be present, and behave like native CLI tools. They can also include project assets inside the executable to increase portability.
  2. Compare @yao-pkg/pkg, Bun, and Deno for JS executables

    main

    When choosing a tool to turn JavaScript into a standalone executable, consider these primary trade-offs:

    • @yao-pkg/pkg (SEA): Uses stock Node.js (V8). It is the most compatible with the npm ecosystem and native addons because it uses the exact same ABI as the Node.js you tested against. It supports compression (Brotli, GZip, Zstd) and provides same-day security updates as Node.js.
    • @yao-pkg/pkg (Standard): The only option that can strip source code to V8 bytecode for better intellectual property protection.
    • Bun (bun build --compile): Fastest build times and smallest binaries. It uses JavaScriptCore instead of V8. While it has high npm compatibility, it can struggle with node-pre-gyp packages that require manual intervention to pin .node files.
    • Deno (deno compile): Uses V8 and provides excellent cross-compilation. It auto-detects node_modules and embeds assets via --include, but requires --self-extracting for native addons and has partial Node.js built-in support.
  3. Understand the Snapshot filesystem

    main

    When you package an application with pkg, all project files are collected into a bundle called a snapshot. At runtime, the application accesses these files through a virtual snapshot filesystem.

    Files within the snapshot are prefixed with /snapshot/ (on Unix-like systems) or C:\snapshot\ (on Windows). For example, if your source file is at /project/app.js, its runtime path inside the binary will be /snapshot/project/app.js.

  4. How Enhanced SEA Mode works

    main

    Enhanced SEA (Single Executable Application) mode is a specialized build pipeline used when a project has a package.json and targets Node.js >= 22. Unlike traditional modes, it preserves native ESM files (skipping ESM $\to$ CJS transformation) and avoids V8 bytecode compilation.

    Instead of individual files, it concatenates all project files into a single __pkg_archive__ binary blob and generates a __pkg_manifest__.json containing file offsets and metadata. At runtime, a custom bootstrap (sea-bootstrap.bundle.js) mounts this archive as a Virtual File System (VFS) at the /snapshot path, allowing the application to read its own bundled assets using standard fs calls.

    /* 
    Enhanced Mode Detection Requirements:
    - Presence of `package.json`
    - Target Node.js version >= 22
    */
  5. Understand ESM support in pkg

    main

    Starting from version 6.13.0, pkg provides improved support for ECMAScript Modules (ESM). By default, most ESM features are automatically transformed into CommonJS during the packaging process to ensure compatibility with the generated executable.

    Supported ESM features

    • import and export statements: Automatically transformed to require() and module.exports.
    • Top-level await: Wrapped in an async IIFE to function within a CommonJS context.
    • Top-level for await...of: Wrapped in an async IIFE to function within a CommonJS context.
    • import.meta.url: Polyfilled to provide the current module's file URL.
    • import.meta.dirname: Polyfilled to provide the directory path (requires Node.js 20.11+).
    • import.meta.filename: Polyfilled to provide the file path (requires Node.js 20.11+).
  6. Standard streams, signals, and exit codes in packaged apps

    main

    The packaged binary behaves like a standard Node.js process regarding system interactions:

    • Standard Streams: stdin, stdout, and stderr support standard piping, redirection, TTY detection, and terminal colors.
    • Signals: Signal handlers (e.g., SIGINT, SIGTERM) and unhandled rejection behaviors function as they do in Node.js.
    • Exit Codes: Calling process.exit() works normally, and the binary propagates the final exit code back to the shell.
  7. How native addons are handled in packaged apps

    main

    Both Traditional and Enhanced SEA modes use a shared patching mechanism (patchDlopen) to handle native .node addons. Since native modules cannot be loaded directly from a virtual filesystem, they are extracted to a local cache.

    The Extraction Workflow:

    1. A .node file is requested via process.dlopen.
    2. The system checks if the file is inside the snapshot/VFS.
    3. If yes, the content is read via the VFS and SHA256 hashed.
    4. The file is cached in ~/.cache/pkg/<hash>/.
    5. If the addon is part of a node_modules package, the entire package folder is copied to ensure dependencies are met.
    6. If it is a standalone file, only the single file is copied.
    7. The original dlopen is called using the path to the extracted file in the cache.
  8. Protect source code using pkg Standard mode

    main

    If intellectual property protection is a hard requirement, use pkg in Standard mode. This is the only tool in the comparison that can strip source code from the binary. It compiles JavaScript into V8 bytecode and stores it as a STORE_BLOB with sourceless: true, meaning a byte dump of the executable will contain only bytecode and no recoverable source code.

    Warning: For all other tools (pkg SEA, Bun, Deno), the source code is embedded in the binary and is not considered secret.

  9. Access packaged files via the /snapshot/ virtual filesystem

    main

    Regardless of the mode used, pkg exposes all files included in the package under a /snapshot/ prefix. This is handled by a Virtual Filesystem (VFS) layer that intercepts standard Node.js fs calls.

    To read a file that was included in your package, use the absolute path starting with /snapshot/.

    Example: If your project has a file at app/config.json, you access it in your code as: fs.readFileSync('/snapshot/app/config.json').

    Note on Native Addons: Native addons (.node files) cannot be loaded directly from memory. Both modes will automatically extract these files to ~/.cache/pkg/<sha256>/ on their first load and then execute them using the real process.dlopen.

    // Accessing a packaged file at runtime
    const fs = require('fs');
    const config = fs.readFileSync('/snapshot/app/config.json');
  10. Choose a compression algorithm

    main

    Select an algorithm based on your priority (binary size vs. speed) and the target Node.js runtime:

    AlgorithmCompression ratioDecompression speedUse when
    BrotliHighestSlowestBinary size is the only priority
    ZstdHighVery fastBalanced default for small binary and fast cold start
    GZipLowerFastOlder Node.js runtimes without Zstd support

    Important Note on Zstd: Zstd requires node:zlib's zstdCompress / zstdDecompress support, which was added in Node.js 22.15.0. Both your build host and the packaged Node runtime must support it. If you are targeting older Node 22.x releases, use Brotli instead.

  11. Understand the SEA Binary Format

    main

    An SEA executable is a standard Node.js binary with a NODE_SEA_FUSE activated and an injected NODE_SEA_BLOB resource. The blob contains:

    • main: The sea-bootstrap.js entry point.
    • Asset: __pkg_manifest__: A JSON file containing directory structures, stats, symlinks, and the offsets map.
    • Asset: __pkg_archive__: A single large binary blob containing all application files.

    Files are accessed via the offsets map, which provides [byteOffset, byteLength] for zero-copy extraction using Buffer.subarray().

  12. Worker Thread Support in SEA

    main

    Standard Node.js worker threads do not automatically inherit VFS hooks. The SEA bootstrap solves this by monkey-patching the workerThreads.Worker constructor.

    When a worker is spawned with a path inside /snapshot:

    1. The bootstrap reads the worker source from the VFS.
    2. It prepends the worker VFS bootstrap (sea-vfs-setup.js).
    3. It wraps the code in Module._compile to ensure correct require() resolution (setting __filename, __dirname, and module.paths).
    4. It spawns the worker using { eval: true } so the worker runs entirely in-memory.