DOCSEARCHbyAlgolia
Submit Docs

Atlas for Nim

repository·master·Indexed 41 days ago

https://github.com/nim-lang/atlas

A package cloner for Nim that manages project dependencies in an isolated `deps/` directory. Atlas provides environment isolation, workspace support, and a built-in parallel test runner, serving as an alternative or augmentation to Nimble. It includes tools for managing virtual environments, linking local projects, and executing tasks via `atlas-run`.

Tokens
5.8K
Snippets
25
Records
30
Agent score
100%

What's inside Atlas

  1. How Projects and Dependencies work in Atlas

    master

    Atlas distinguishes between Projects and Dependencies primarily by their location:

    1. Projects: A directory containing an atlas.config file. Projects can share dependencies with each other using the link command.
    2. Dependencies: Packages located inside a project's deps/ directory.

    Key behaviors:

    • Priority: For dependency resolution, a project always has higher priority than a dependency.
    • Interchangeability: You can move a dependency out of the deps/ directory to turn it into a project, or move a project into the deps/ directory to turn it into a dependency.
    • Linking: Projects can share dependencies via nimble-link files created by the link command.

  2. Use Feature Statements in Nimble files

    master

    Features in .nimble files allow for optional requirements (e.g., for testing). Atlas lazily clones these dependencies until they are explicitly requested via a feature flag or a requirement in another package.

    To enable a feature when running Atlas commands, use the --feature=foobar flag. Alternatively, use --keepFeatures or -k to reuse feature defines from your current nim.cfg.

    Atlas automatically enables the dev and patch features for the root package. You can declare development-only dependencies using dev:, and patch features using a standard feature "patch": block.

    require "normallib"

feature "testing":
  require "mytestlib"

  3. How the Atlas Solver works

    master

    The Atlas solver uses a SAT (Satisfiability) solver to resolve package versions. When lazy resolution is enabled, the process follows these steps:

    1. Initial Graph Expansion: A dependency graph is built. Child dependencies are marked as LazyDeferred (using a placeholder version *@- with a #head payload) instead of being fully loaded.
    2. SAT Construction: A formula is created where each package version and release feature is represented by a SAT variable. Dependency implications for LazyDeferred nodes are skipped in this pass.
    3. First Solve Pass:
      • If SAT succeeds: Atlas identifies any required dependencies that are still LazyDeferred, promotes them to DoLoad, and triggers a rerun.
      • If SAT fails (UNSAT): Atlas attempts a conflict-recovery pass by pulling in referenced lazy dependencies (prioritizing special #... versions and features enabled via CLI/context) to try and find a valid solution.
    4. Rerun Loop: If a rerun is triggered, Atlas resets variable IDs, clears feature mappings, re-expands the graph with the newly loaded packages, and solves again. This repeats until no LazyDeferred dependencies remain.
    5. Activation: Once a stable solution is found, Atlas checks for duplicate module names, checks out the selected commits, runs build hooks, and updates nim.cfg.

  4. How version constraints and root pins work

    master

    Atlas manages versioning using git commits, tags, and Nimble manifests.

    Selection Logic

    • Atlas selects one release per required package that satisfies all requirements.
    • Only the selected release's dependencies are activated.
    • Feature dependencies are only applied if the feature is enabled.
    • Version selection is deterministic, picking the minimum required version.

    Root Pins

    If your root project (your application) specifies a specific commit pin (e.g., siwin#6e37ac8), this pin overrides any transitive dependency requests for that package (e.g., a dependency requesting siwin#head). This allows you to lock a known-good commit even if the repository's tip moves.

    Conflict Resolution

    • If requirements conflict, Atlas reports the incompatible requirements and the packages/features that requested them.
    • If resolution fails, Atlas leaves the existing nim.cfg untouched and skips dependency activation.
    • Use --verbosity:debug for detailed release-level information during conflicts.

  5. Set up a Workspace Style environment

    master

    You can configure multiple projects to share a single deps/ folder by using a workspace-style setup. This is done by setting the deps field in an atlas.config file located in each project's root.

    To initialize a workspace folder ws/:

    mkdir ws/ && cd ws/
git clone https://github.com/nim-lang/choosenim
cd choosenim/
atlas --deps=../ --confdir=. init
atlas install

    In this setup, ws/ acts as the central repository for all dependencies used by projects within that workspace.

    atlas --deps=../ --confdir=. init

  6. Set up Virtual Nim environments with atlas env

    master

    Atlas can manage multiple Nim versions within a single project using the env command.

    Usage:

    • atlas env <version>: Installs a specific version (e.g., 1.6.12 or devel).
    • --binary: Requires downloading a prebuilt binary. Fails if no binary is available.
    • --source: Always builds Nim from source.
    • --git-ref <ref>: Builds a custom Nim checkout from a branch, tag, or SHA (implies source build).
    • --git-url <url>: Specifies a custom remote for the Nim repository.
    • --github-path: (CI only) Appends the Nim bin directory to the GitHub Actions GITHUB_PATH.

    Activation: After installation, activate the environment by sourcing the generated script:

    • UNIX: source deps/nim-<version>/activate.sh
    • Windows: deps\nim-<version>\activate.bat
    atlas env 2.2.10 --binary --github-path
atlas env my-nim --git-url=https://github.com/me/Nim.git --git-ref=fix/windows

  7. Use Atlas in GitHub Actions CI

    master

    Atlas is optimized for CI. Since dependencies are installed into the local deps/ directory, you can simply cache that directory to speed up builds.

    Recommended Workflow:

    1. Install Atlas via nimble.
    2. Cache the deps/ directory using the hash of your .nimble file as the key.
    3. Run atlas install with required features.
    - name: Install Latest Atlas
  run: |
    nimble install 'https://github.com/nim-lang/atlas@#head'

- name: Cache packages
  uses: actions/cache@v3
  with:
    path: deps/
    key: ${{ runner.os }}-${{ hashFiles('foo.nimble') }}

- name: Install Deps
  run: |
    atlas install --feature=test --feature=other

  8. Enable and use Atlas plugins

    master

    Atlas can integrate foreign projects (non-Nim projects) as dependencies using plugins. A plugin is a NimScript (*.nims) that can execute external tools.

    Setup Steps:

    1. In your atlas.config file, add: plugins="_plugins".
    2. Create a directory named _plugins in your project root.
    3. Place your .nims plugin files inside that directory.

    Security Note: To prevent any plugin from running arbitrary code, use the --noexec switch when running Atlas commands.

    # atlas.config
plugins = "_plugins"

  9. Basic Usage Tutorial

    master

    To manage dependencies in a Nim project using Atlas:

    1. Initialize Atlas in your project directory:
      atlas install
    2. Add a dependency (e.g., malebolgia):
      atlas use malebolgia
    3. Use the dependency in your code:
      import malebolgia
    4. Compile normally using the Nim compiler. Atlas manages the configuration via a nim.cfg file created in your project root.
    atlas install
atlas use malebolgia
nim c myproject.nim

  10. Run project tests with atlas-run tests

    master

    Use atlas-run tests to run project tests matching tests/t*.nim in parallel.

    Key Options:

    • --backend:<ic|cpp|js>: Selects the Nim compiler command. JavaScript tests are run via node.
    • --compiler-output: Shows successful compiler output (hidden by default).
    • --only-errors: Prints only failed test chunks.
    • --stream: Prints output as it arrives (use with --jobs:1).
    • --stats: Prints compile and run time for each test.
    • --compile-only: Compiles tests without running them.
    • --skip <pattern>: Removes matching tests from the set.

    Selectors:

    • foo: Matches tests/tfoo*.nim.
    • foo.nim: Matches tests/tfoo.nim.
    • examples/*.nim: Direct path/glob selection.

    Compiler Flags: Pass flags via the NIMFLAGS environment variable or after a -- separator. Arguments after -- take precedence.

    atlas-run tests --backend:js --only-errors
atlas-run tests t --skip integration --skip slow
NIMFLAGS="-d:release" atlas-run tests
atlas-run tests tatlasrun -- -d:release --mm:orc

  11. Create reproducible builds with pin and rep

    master

    Atlas provides mechanisms to ensure reproducible builds by locking dependencies to specific commit hashes using a lockfile (defaulting to atlas.lock).

    • atlas pin [atlas.lock]: Pins repositories to their current commit hashes.
      • Running this in the project root pins the entire project.
      • Running this within a specific project pins that project's dependencies, but not the project itself.
    • atlas rep [atlas.lock]: Replays or repeats the project using the pinned commit hashes. This command also executes any defined 'build' instructions unless the --noexec switch is used. replay and reproduce are aliases for rep.

    Note: For true reproducibility, you must also pin your C++ compiler, operating system version, and third-party system dependencies (like libc).

    atlas pin
atlas rep
# Or using aliases
atlas replay
atlas reproduce
# To prevent executing build instructions
atlas rep --noexec

  12. Use forge aliases to add dependencies

    master

    Instead of providing full URLs, you can use a shorthand forge alias syntax: <alias>:<user>/<repo>. This works in both the atlas use command and in Nimble require statements.

    Supported Aliases:

    • gh or github: https://github.com/
    • gl: https://gitlab.com/
    • srht or sourcehut: https://git.sr.ht/~
    • cb, cberg, or codeberg: https://codeberg.org/

    Examples:

    # Using CLI
atlas use gh:zedeus/nitter
atlas use cb:someuser/somepkg

# Using Nimble require
require "gh:user/repo"
require "gl:user/repo >= 1.0"
    atlas use gh:zedeus/nitter