Source Link

repository·main·Indexed 23 days ago

https://github.com/dotnet/sourcelink

A system that embeds source control metadata into binaries and symbols, allowing developers to debug NuGet dependencies by automatically downloading the correct source code from repositories such as GitHub, Azure Repos, GitLab, and Bitbucket. It provides provider-specific packages, MSBuild properties for configuration (e.g., EmbedAllSources, PublishRepositoryUrl), and support for standardized CI environment variables.

Tokens
7.9K
Snippets
16
Records
36
Agent score
80%

What's inside dotnet-sourcelink

  1. What is Source Link and how does it work?

    main

    Source Link is a system that embeds source control metadata into symbols (PDBs), binaries, and NuGet packages. This allows developers debugging a binary (like a NuGet dependency) to automatically download and display the exact commit-specific source code from the original repository. This enables features like breakpoints and stepping into source code for arbitrary dependencies.

    To provide a good debugging experience, you must:

    1. Include source control information in PDBs.
    2. Include source control information in the NuGet package manifest.
    3. Distribute PDBs along with your binaries.
  2. Locate a Git repository from a path

    main

    When searching for a repository starting from an initial path /a/b, the search follows a specific hierarchy of paths, looking for either a valid git directory or a .git file.

    Search Order:

    1. /a/b/.git
    2. /a/b/
    3. /a/.git
    4. /a
    5. /.git
    6. /

    Key Rules:

    • Only paths on the same device as the initial path are considered.
    • On Windows, \ separators are normalized to / before searching.
    • A .git file must start with the gitdir: prefix followed by a valid git directory path. Relative paths in a .git file are relative to the directory containing the file.
    • Linked Working Trees: A repository is a linked working tree if its git directory is different from its common directory and a gitdir file is present in the git directory containing the absolute path to the working directory.
    • Bare Repositories: A repository is considered bare if core.bare is set to true and it is not a linked working tree.
  3. Standardized Environment Variables for CI Services

    main

    The .NET Core SDK uses a standardized set of environment variables to embed source control information (Commit ID, Repository URL, and SCM name) into built assets like assemblies and NuGet packages. This creates a strong link between binaries and their source code for debugging and auditing.

    To support this, CI services should provide the following environment variables, all prepended with STANDARD_CI_:

    • STANDARD_CI_SOURCE_REVISION_ID: The commit hash or ID (e.g., 2ba93796dcf132de447886d4d634414ee8cb069d).
    • STANDARD_CI_REPOSITORY_URL: The URL for the repository (e.g., https://github.com/dotnet/corefx).
    • STANDARD_CI_REPOSITORY_TYPE: The Source Control Manager name (e.g., git, tfvc, svn, mercurial).
  4. Understand Git configuration file parsing and includes

    main

    Git configuration files follow a specific format and support recursive includes.

    Parsing Rules

    • Files may start with an optional BOM.
    • Section and subsection names can be separated by multiple whitespace characters.
    • No whitespace is allowed between a terminating subsection quote " and the closing bracket ].
    • Variables defined outside of a section are allowed (they appear in git config --list but cannot be queried via git config --get).
    • If a fully qualified variable name already exists, it is treated as multi-valued.

    Includes and Conditional Includes

    • Recursion Limit: Recursive includes stop at a depth of 10.
    • Path Resolution:
      • If the path starts with ./ or .\ (Windows), it is joined with the directory containing the current config file.
      • If the path starts with ~/ or ~\ (Windows), the tilde is expanded to the home directory.
      • If the path is not absolute, it is joined with the current directory.
      • If the path ends with a directory separator, ** is appended to the path.
    • Pattern Matching: Path patterns are matched against the repository's git directory using fnmatch. Patterns must use / for directory separators.
  5. Reduce security scan overhead using Multiple Outputs in 1ES

    main

    When using 1ES pipeline templates (templates-official), every publish artifact execution triggers additional security scans. To minimize this overhead, you should use the outputParentDirectory feature of 1ES templates.

    Implementation Steps:

    1. Gather all build outputs into the $(Build.ArtifactStagingDirectory) using a task like CopyFiles@2.
    2. Configure the templateContext.outputs parameter in your template call to point to these locations.

    Note: Multiple outputs are ONLY applicable to 1ES PT publishing (using templates-official).

    # azure-pipelines.yml
    extends:
      template: azure-pipelines/MicroBuild.1ES.Official.yml@MicroBuildTemplate
      parameters:
        stages:
        - stage: build
          jobs:
          - template: /eng/common/templates-official/jobs/jobs.yml@self
            parameters:
              # 1ES makes use of outputs to reduce security task injection overhead
              templateContext:
                outputs:
                - output: pipelineArtifact
                  displayName: 'Publish logs from source'
                  continueOnError: true
                  condition: always()
                  targetPath: $(Build.ArtifactStagingDirectory)/artifacts/log
                  artifactName: Logs
              jobs:
              - job: Windows
                steps:
                - script: echo "friendly neighborhood" > artifacts/marvel/spiderman.txt
              # copy build outputs to artifact staging directory for publishing
              - task: CopyFiles@2
                  displayName: Gather build output
                  inputs:
                    SourceFolder: '$(System.DefaultWorkingDirectory)/artifacts/marvel'
                    Contents: '**'
                    TargetFolder: '$(Build.ArtifactStagingDirectory)/artifacts/marvel'
  6. Compare PDB distribution methods for managed code

    main

    There are three primary ways to distribute PDBs (debug symbols) for managed code. Choose the method based on your requirements for package size, user opt-in, and feed compatibility.

    |                                         | snupkg                      | Include in main package | Embed in assembly |
    |-----------------------------------------|-----------------------------|--------------------------|-------------------|
    | No user opt-in required to load symbols | ✅                          | ❌                       | ✅                 |
    | No increase in size of main package     | ✅                          | ❌                       | ❌                 |
    | No increase in size of assemblies       | ✅                          | ✅                       | ❌                 |
    | Supported by all package feeds          | ❌ (Supported on NuGet.org) | ✅                       | ✅                 |
  7. Minimal git metadata requirements for Source Link

    main

    Source Link can operate on repositories that lack a full .git directory, provided certain minimal metadata is present. If you are building from a directory without full git metadata, ensure the following files exist:

    • .git/HEAD: A text file containing a commit SHA.
    • .git/config: A configuration file specifying the origin remote URL.

    Submodule Requirements: If the repository uses submodules, you must also provide:

    • .gitmodules in the repository root listing relative paths for all submodules.
    • The origin remote in .git/config must contain URLs for all initialized submodules.
    • For each submodule, a .git/modules/<submodule-name>/HEAD file containing a commit SHA.
    • For each submodule, a <submodule-path>/.git file containing: gitdir: ../.git/modules/<submodule-name>.
  8. Understand .gitignore pattern matching

    main

    .gitignore files determine which files are ignored by Git. They follow a specific order of precedence and matching logic.

    Precedence (Order of evaluation)

    1. Default patterns: ., .., .git
    2. The inner-most .gitignore file (in the directory being checked)
    3. The outer-most .gitignore file (search stops at the working directory)
    4. The info/exclude file under the common directory
    5. The file specified by core.excludesFile in the configuration

    Pattern Types

    • Negative Pattern: Starts with !. Removes the ! to match the file instead of ignoring it.
    • Directory Pattern: Ends with /.
    • Full Path Pattern: Contains a / after the trailing / is removed. These match against the path relative to the directory containing the .gitignore file.
    • Standard Pattern: Matches only against the last component of the path (file or directory name).

    Matching Rules

    • Case Sensitivity: Matching is case-insensitive if core.ignorecase is set to true.
    • Separators: Input paths must use / directory separators. Even on Windows, \ is not considered a directory separator for matching.
    • Wildcards: In Full Path Patterns, the wildcards ?, *, and [] do not match the / character.
    • Directory Matching: If a pattern is a directory pattern, it will only match if the input path is a directory path.
  9. Prerequisites for C++ Source Link support

    main

    To debug native binaries using Source Link information embedded in PDBs, you must use Visual Studio 2017 Update 9 or newer.

    The VC++ linker supports the /SOURCELINK switch starting from Visual Studio 2017 Update 8. However, PDBs produced by that version are incompatible with case-sensitive source control systems like git. To ensure compatibility with git, use Visual Studio 2019 or newer.

  10. Distribute symbols via .snupkg symbol packages

    main

    If publishing to NuGet.org, it is recommended to build and publish a .snupkg symbol package. This allows the debugger to download symbols from the NuGet.org symbol server on demand.

    Limitations of .snupkg:

    • Does not support Windows PDBs (generated by VC++ or managed projects with DebugType set to full).
    • Requires the library to be built with Visual Studio 2017 Update 9 or newer.
    • Consumers require Visual Studio 2017 Update 9 or newer.
    • Not supported by Azure DevOps Artifacts.
  11. Configure Source Link for GitLab

    main

    For projects hosted on GitLab, add the Microsoft.SourceLink.GitLab package.

    By default, version 8.0.0+ assumes GitLab 12.0+. If you are using an older version, you must specify the host and version using the SourceLinkGitLabHost item group. You can also use the CI_SERVER_VERSION environment variable.

    <ItemGroup>
      <PackageReference Include="Microsoft.SourceLink.GitLab" Version="8.0.0" PrivateAssets="All"/>
    </ItemGroup>
    
    <!-- For GitLab versions older than 12.0 -->
    <ItemGroup>
      <SourceLinkGitLabHost Include="gitlab.yourdomain.com" Version="11.0"/>
    </ItemGroup>
  12. Configure Source Link for GitHub

    main

    For projects hosted on github.com or GitHub Enterprise, add the Microsoft.SourceLink.GitHub package.

    <ItemGroup>
      <PackageReference Include="Microsoft.SourceLink.GitHub" Version="8.0.0" PrivateAssets="All"/>
    </ItemGroup>