Repo

repository·main·Indexed 19 days ago

https://github.com/gerritcodereview/git-repo

A Python-based tool built on top of Git to manage multiple Git repositories and automate development workflows, commonly used in large-scale projects like Android. It provides a wrapper to handle collections of repositories via manifests, custom fetch commands, and integrated Gerrit review interactions.

Tokens
28.9K
Snippets
95
Records
135
Agent score
65%

What's inside git-repo

  1. Overview of Repo

    main
    Repo is a tool built on top of Git designed to manage multiple Git repositories simultaneously. It automates parts of the development workflow and handles uploads to revision control systems. It is not a replacement for Git, but a wrapper to make working with large collections of Git repositories easier. The repo command is a standalone executable Python script.
  2. Understand the .repo/ directory layout

    main

    The .repo/ directory contains the internal state of your repo client checkout. Key components include:

    • config: Per-repo client checkout settings in git-config format.
    • .repo_config.json: A JSON cache of the config file for faster processing.
    • repo/: A git checkout of the repo project itself, used for self-updating.
    • .repo_fetchtimes.json: Records fetch times used by repo sync.
    • .repo_localsyncstate.json: Used by repo sync to detect partial tree syncs.
    • manifests/: A git checkout of the manifest project.
    • manifests.git/: A bare checkout of the manifest project.
    • manifest.xml: The active manifest file used by repo (often a symlink to a file in manifests/).
    • local_manifests/: Directory for user-authored manifest fragments used to tweak the manifest.
    • projects/: Bare checkouts of every project synced by the manifest.
    • project-objects/: Shared git objects to allow multiple checkouts of the same remote repo to save space.
    • modules/ & subproject-objects/: Similar to projects/ and project-objects/ but for git submodules.
    • worktrees/: Used when git worktree is enabled.
  3. How repo Smart Syncing works

    main

    Smart Syncing allows repo sync to fetch specific, known source states from a remote server instead of just tracking the latest revisions of all projects in a manifest.

    Instead of blindly fetching the latest code, the client sends a request to a manifest server via an XML-RPC connection. The server responds with a specific manifest (typically specifying exact commits for every project) that matches the user's requested state (e.g., a specific release or build). This ensures a reproducible local build environment.

  4. Understand the `~/ .repoconfig/` directory layout

    main

    Repo maintains user-specific configuration and state in a .repoconfig/ directory. By default, this is located in the user's home directory, but the location can be customized by setting the REPO_CONFIG_DIR environment variable.

    Directory Structure

    • .repoconfig/config: Per-user settings using the standard [git-config] file format.
    • .repoconfig/keyring-version: A cache file used to check if the gnupg subdirectory contains the same keys as the repo launcher, preventing slow, constant GPG executions.
    • .repoconfig/gnupg/: An isolated GnuPG internal state directory used when repo runs gpg, ensuring it does not interfere with the user's standard ~/.gnupg/ directory.

    JSON Cache Files

    Repo maintains JSON caches of configuration files to improve processing speed:

    • .repoconfig/.repo_config.json: A JSON cache of the .repoconfig/config file.
    • .repo_.gitconfig.json: A JSON cache of the .gitconfig file.
  5. Understand the repo Manifest XML structure

    main

    A repo manifest file (typically default.xml) is an XML document that defines the collection of Git repositories to be managed. It follows a specific DTD that allows for defining remotes, default settings, submanifests, and individual projects.

    Key behaviors:

    • Extensibility: All unknown elements are silently ignored for compatibility. To use custom elements, use the x-* namespace to avoid collisions with future repo updates.
    • Hierarchy: Elements like project can be nested to represent Git submodules, inheriting attributes from their parents unless overridden.
    • Layering: Local manifests can use extend-project and remove-project to modify an existing manifest without replacing it entirely.
    <!DOCTYPE manifest [
      <!ELEMENT manifest (notice?, remote*, default?, manifest-server?, submanifest*?, remove-project*, project*, extend-project*, repo-hooks?, superproject?, contactinfo?, include*)>
      <!-- ... other element definitions ... -->
    ]>
    <manifest>
      <!-- Manifest content goes here -->
    </manifest>
  6. Implement the fetchcmd contract and invariants

    main

    If you are implementing a custom fetch command, your command must adhere to the following contract to ensure repo sync works correctly:

    Postconditions (Required on exit 0)

    After your command exits with status 0, repo requires:

    1. The commit must exist in the object store (verified via git cat-file -e REPO_TREV).
    2. The mapped local tracking ref (e.g., refs/remotes/REPO_REMOTE/<branch> or the tag ref) must point to REPO_TREV.
    3. FETCH_HEAD must point to REPO_TREV.
    4. The commit graph from REPO_TREV must be reachable enough to compute merge bases with local branches.

    Invariants and Constraints

    • Idempotency: Fetching the same REPO_TREV twice should be a no-op.
    • Scope of Changes: Only modify FETCH_HEAD and refs/remotes/*. Do not touch HEAD or local branches. This preserves repo sync --network-only semantics.
    • Worktree Safety: You must preserve the dirty worktree state.
    • Exclusions: The command is not executed for MetaProjects (the internal repo repository at .repo/repo or the manifests repository at .repo/manifests).

    Error Handling

    • A non-zero exit status will abort the project's sync and surface the command's stderr to the user.
    • If repo detects a mismatch in tracking refs or target reachability after your command exits with 0, it will treat it as a failure.
  7. Understand hook runtime behavior and constraints

    main

    When developing hooks, keep the following runtime characteristics in mind:

    • Execution Directory: Hooks run from the top level of the repo client (the workspace root), not the subdirectory of the project being operated on. Hooks often use os.chdir to move into a specific project directory.
    • Failure Mechanism: Hook return values are ignored. To fail a step, you must call sys.exit() with a non-zero exit code.
    • Stdout/Stderr: Output is not filtered. Avoid excessive verbosity; use long/verbose output only when a hook fails.
    • Git State: Repo does not modify the git checkout state for the hook. The hook may run in a 'dirty' repo. If a hook needs to operate on specific commits, it must manually discover and extract them.
    • Python Path: sys.path is modified so that the top of the repohooks directory comes first, allowing easy imports of local modules within the hooks project.
    • Approvals: For the first execution, users are prompted for approval. For https:// manifests, this happens once. For http:// manifests, users are prompted whenever the hooks project is updated.
  8. How repo hooks interact with Python versions

    main

    Projects using [repo hooks] run on independent schedules. Because it is not possible to detect which Python version the hooks were written or tested against, repo always imports and executes them using the active Python version.

    If the active Python version is too new for the hooks, the responsibility for updating the hooks lies with the hooks maintainer.

  9. Understand the repo Manifest structure

    main

    A repo manifest defines the structure of a repo client, specifying which directories are visible and their corresponding Git source locations.

    A manifest is implemented as a bare Git repository containing a single default.xml file at its top level. Because manifests are stored in Git repositories, they are version-controlled; when a user runs repo sync, the client automatically fetches updates to the manifest from the remote repository.

  10. How repo hooks work

    main

    Repo allows you to hook specific runtime stages (like pre-upload or post-sync) with custom Python modules.

    Core Workflow:

    1. Storage: All hooks reside in a dedicated Git project.
    2. Registration: The manifest file (used during repo init) specifies which project contains the hooks and which stages they are enabled for.
    3. Execution: When a triggered stage occurs, Repo dynamically loads the corresponding Python module from the registered project and calls its main function.

    These hooks are ideal for running linters, checking code formatting, or executing unit tests before allowing operations like uploading commits to Gerrit.

  11. How the repo launcher handles old Python versions

    main

    The repo launcher is an independent script designed to support older Python versions without restricting the main codebase.

    If the launcher detects that the current Python version is too old to run the main codebase, it attempts to re-execute itself using a newer Python interpreter via standard pythonX.Y interpreter names.

    If your default Python interpreters are too old to run the launcher even when newer versions are installed, you can:

    1. Modify the repo launcher's shebang to match your environment.
    2. Download and use an older version of the repo launcher (note: old launchers are not guaranteed to work with current versions of repo, and bug reports using old launchers will not be accepted).
  12. Use Git worktrees to avoid symlink issues on Windows

    main

    Repo 2.4+ supports Git worktrees, which allows Repo to create client checkouts that do not require symlinks. This is useful on Windows because it removes the need for Administrator access to sync code. To use this feature, opt in by passing the --worktree flag during the repo init command.

    Note: This is an experimental feature. Maintain backups and report any bugs.

    repo init --worktree