DOCSEARCHbyAlgolia
Submit Docs

Scalar

repository·main·Indexed 23 days ago

https://github.com/microsoft/scalar

A tool that optimizes Git for large-scale repositories by automating advanced features such as partial clone, sparse-checkout, background prefetch, and commit-graph. Originally a standalone .NET tool, Scalar is now included as a component of the microsoft/git fork.

Tokens
2.5K
Snippets
7
Records
15
Agent score
81%

What's inside Scalar

  1. What is Scalar?

    main

    Scalar is a tool designed to help Git scale to extremely large repositories. It automates the configuration of several advanced Git features to improve performance and reduce resource usage.

    Key features enabled by Scalar include:

    • Partial clone: Reduces initial repository download time by not fetching all objects immediately.
    • Background prefetch: Periodically downloads object data from remotes to speed up foreground git fetch calls.
    • Sparse-checkout: Limits the size of your working directory.
    • File system monitor: Tracks modified files to avoid scanning the entire worktree.
    • Commit-graph: Accelerates commit walks and reachability calculations (e.g., git log).
    • Multi-pack-index: Enables fast object lookups across multiple pack-files.
    • Incremental repack: Repacks Git data into fewer pack-files without disrupting concurrent commands.

  2. Run functional tests on Mac

    main

    To run functional tests on macOS, use the provided shell scripts.

    Execution

    1. Build Scalar: Scripts/Mac/BuildScalarForMac.sh
    2. Run the tests: Scripts/Mac/RunFunctionalTests.sh

    Debugging with VS for Mac

    If you need to attach the VS for Mac debugger:

    1. Build your latest changes and open Scalar.sln in VS for Mac.
    2. Go to Run -> Run With -> Custom Configuration...
    3. Select Start external program and specify the published functional test binary (e.g., /Users/<USERNAME>/Repos/Scalar/Publish/Scalar.FunctionalTests).
    4. Specify any desired arguments (see Running Specific Tests).
    5. Select Run Action -> Debug - .Net Core Debugger and click Debug.
    Scripts/Mac/BuildScalarForMac.sh
Scripts/Mac/RunFunctionalTests.sh

  4. Install the latest version of Scalar via microsoft/git

    main

    Scalar has transitioned from a standalone .NET tool into a thin shell around core Git features, now included in the microsoft/git fork. To use the latest version of Scalar, you should install microsoft/git instead of the standalone Scalar package.

    To verify your installation, run scalar version. The output should be identical to the output of git version.

    scalar version

  5. Run functional tests on Windows

    main

    Functional tests in Scalar are built on NUnit 3 and require administrative privileges because they create and delete a test service.

    Prerequisites

    1. Build Scalar using one of these methods:
      • Open Scalar.sln in Visual Studio and build everything.
      • Run Scripts\BuildScalarForWindows.bat from the command line.
    2. Run the Scalar installer located in BuildOutput\Scalar.Installer.Windows\bin\x64\<Debug or Release> to ensure correct pre/post-command hooks are found.

    Execution

    Run the tests with elevation (as Administrator) using one of these methods:

    • Command Line: Run Scripts\RunFunctionalTests.bat from CMD as Administrator.
    • Visual Studio: Run the Scalar.FunctionalTests project from inside Visual Studio launched as Administrator.

    To run the entire suite across all configurations, use the --full-suite option.

    Scripts\RunFunctionalTests.bat

  6. How to write functional tests in Scalar

    main

    Functional tests in Scalar should clone a repository and use existing tools/filesystem APIs to interact with a virtual repo. To ensure a clean environment, tests should derive from one of two base classes.

    Important: Place your new test class in the same namespace as the base class. NUnit treats namespaces as test suites, and Scalar uses this to manage enlistments.

    Base Classes

    1. TestsWithEnlistmentPerFixture: Creates an enlistment before any tests in the fixture run and deletes it after all tests in the fixture are complete. Use this when you have a sequence of tests that need to manipulate the same repository.
    2. TestsWithEnlistmentPerTestCase: Creates a new enlistment for every single test case. This is the most reliable method for isolation but is the most expensive in terms of performance.

    Customizing Settings

    Test parameters (paths and URLs) are defined in Scalar.FunctionalTests\Settings.cs.

  7. Trace child processes recursively

    main

    When diagnosing complex builds (like xcodebuild) that spawn multiple sub-processes (compilers, linkers, etc.), use the -f flag to trace the root command and all processes launched by it recursively.

    ./Scripts/Mac/Tracing/dtruss -d -e -f xcodebuild
    ./Scripts/Mac/Tracing/dtruss -d -e -f xcodebuild

  8. Collect a syscall trace for a failing command on macOS

    main

    If a build command or specific process is misbehaving, you can capture a trace of its system calls using the custom dtruss script provided in the repository. This is useful for diagnosing errors by tracking syscalls, return values, and arguments.

    To capture a trace of a failing command and save it to a file while still seeing the output in your terminal, use the following pattern:

    ++path-to-vfs4g-source++/Scripts/Mac/Tracing/dtruss -d -e -F -f ++command-to-run++ 2> >(tee ++trace-filename.txt++ >&2)

    Note: The script uses sudo internally to run dtrace as root, but the command you are tracing will run as your current user. You will be prompted for your admin password.

    ++path-to-vfs4g-source++/Scripts/Mac/Tracing/dtruss -d -e -F -f ++command-to-run++ 2> >(tee ++trace-filename.txt++ >&2)

  9. Migrate from the old Scalar version to microsoft/git

    main

    If you have previously installed the standalone .NET version of Scalar, follow these steps to migrate to the new version included in microsoft/git:

    Windows

    Install microsoft/git. The installer will automatically remove the .NET version and update any registered enlistments to work with the new version.

    macOS

    1. Uninstall the old version using Homebrew:
      brew uninstall --cask scalar
# OR, depending on your version:
brew uninstall --cask scalar-azrepos
    2. Install the new version:
      brew install --cask microsoft-git
    3. Note: You must re-run scalar register on your enlistments to ensure they are registered for future upgrades.

    Linux

    1. Remove the .NET version manually:
      sudo rm -rf /usr/local/lib/scalar/
    2. Install the new version (which will overwrite the scalar binary in /usr/local/bin).
    3. Note: You must re-run scalar register on your enlistments to ensure they are registered for future upgrades.

  10. Trace processes by name, PID, or command line

    main

    The custom dtruss script allows you to target specific processes using several methods:

    Trace by Process Name

    Use the -n <processname> flag. Note that macOS process names are limited to 16 characters (MAXCOMLEN).

    sudo ./Scripts/Mac/Tracing/dtruss -d -e -n clang

    Trace by PID

    Use the -p <PID> flag to attach to a specific running process ID.

    sudo dtruss -p 1000 2> >(tee clang-trace.txt >&2)

    Trace a Command Directly

    To trace a command from the moment it starts, pass the command as an argument to dtruss. This is the most convenient method for new processes.

    ./Scripts/Mac/Tracing/dtruss ls
    sudo ./Scripts/Mac/Tracing/dtruss -d -e -n clang

  11. Update the remote test branch

    main

    Functional tests default to cloning main and checking out a branch named FunctionalTests/YYYYMMDD (based on the creation date). This prevents remote changes from breaking tests.

    To update the functional tests to use a new branch:

    1. Create a new branch following the pattern FunctionalTests/YYYYMMDD.
    2. Update the Commitish setting in Scalar.FunctionalTests\Settings.cs with the new branch name.
    3. Verify your scenarios locally.
    4. Push the new FunctionalTests/YYYYMMDD branch and then push your changes.