Copybara Documentation

repository·master·Indexed 26 days ago

Copybara is a stateless tool for transforming and moving code between repositories, commonly used to synchronize confidential and public repositories. It supports Git natively and provides experimental support for Mercurial. Workflows are defined using a Starlark configuration language to specify origins, destinations, authoring, and transformations such as core.replace and core.move. The tool can be built via Bazel, run using Docker, or integrated into Bazel workspaces.

Tokens: 42.5K
Snippets: 75
Records: 323
Agent score: 86%

What's inside Copybara

Overview of the Starlark interface framework
master
The net.starlark.java.annot package provides a framework of annotations and interfaces designed to enable Starlark to access Java-implemented data types and methods. This is used to bridge Java logic with Starlark scripts within the Copybara ecosystem.
https://github.com/google/copybara/blob/master/third_party/bazel/main/java/net/starlark/java/annot/README.md
Overview of Copybara
master
Copybara is a tool for transforming and moving code between repositories. It is designed to maintain synchronization between an authoritative repository (the source of truth) and one or more non-authoritative repositories (e.g., a public repository synced from a confidential one).
Key features:
- Statelessness: State is stored in the destination repository as a label in the commit message, allowing multiple users or services to achieve consistent results.
- Git Support: Currently supports Git repositories natively; Mercurial support is experimental.
- Extensible: Architecture allows for custom origins and destinations.
https://github.com/google/copybara/blob/master/README.md
Use the built-in set type in Starlark
master
A set is a mutable collection of unique, hashable values. Sets provide constant-time operations for insertion, removal, and membership testing. They are implemented using a hash table.
Key Characteristics:
- Construction: Use set() for an empty set or set([iterable]) to create a set from an existing collection. There is no literal syntax.
- Membership: Use in and not in operators to check for presence.
- Uniqueness: Duplicate elements are not stored.
- Ordering: Sets are iterable; the order of iteration follows the order in which elements were first added.
- Boolean Context: An empty set is False; a non-empty set is True.
- Comparison: Sets can be compared for equality (==) and inequality (!=). Order of elements does not matter for equality. However, sets do not support ordered comparisons like <, <=, >, or >=.
```
s = set(["a", "b", "c"])
"a" in s  # True
"z" in s  # False

s = set(["z", "y", "z", "y"])
len(s)       # 2
s.add("x")
len(s)       # 3
for e in s:
    print e  # prints "z", "y", "x"
```
https://github.com/google/copybara/blob/master/docs/reference.md
Use the gerrit_api_obj for Gerrit migrations
master
The gerrit_api_obj provides a Gerrit API endpoint implementation designed for feedback migrations and after-migration hooks. It allows for programmatic interaction with Gerrit changes, including abandoning, submitting, and reviewing changes, as well as managing votes and retrieving change information.
https://github.com/google/copybara/blob/master/docs/reference.md
Understand transformations and transformation_status
master
A transformation is a single operation that modifies the source checked out from the origin before it is written to the destination. Transformations can also be used for validations or checks.
When a transformation is executed, it returns a transformation_status object which indicates the result:
- is_success: Boolean indicating if the status is SUCCESS.
- is_noop: Boolean indicating if the status is NO-OP (no changes were made).
https://github.com/google/copybara/blob/master/docs/reference.md
Build and run Copybara using Docker
master
Docker usage is experimental. You can build the image and run it against your local directory.
Build the image: docker build --rm -t copybara .
Run the container: Mount your current directory to /usr/src/app inside the container to allow Copybara to access your code and config files.
Environment Variables: You can configure Copybara behavior via environment variables:
- COPYBARA_SUBCOMMAND: The command to run (defaults to migrate).
- COPYBARA_CONFIG: Path to the config file (defaults to copy.bara.sky).
- COPYBARA_WORKFLOW: The workflow name to run (defaults to default).
- COPYBARA_SOURCEREF: The source reference.
- COPYBARA_OPTIONS: Additional options for Copybara.
```
# Build
docker build --rm -t copybara .

# Run with specific environment variables
docker run \
    -e COPYBARA_SUBCOMMAND='validate' \
    -e COPYBARA_CONFIG='other.config.sky' \
    -v "$(pwd)":/usr/src/app \
    -it copybara
```
https://github.com/google/copybara/blob/master/README.md
Share Git credentials with Copybara Docker container
master
To allow the Copybara Docker container to use your local Git configuration and SSH credentials, mount the relevant files/sockets as follows:
```
docker run \
    -v ~/.gitconfig:/root/.gitconfig:ro \
    -v ~/.ssh:/root/.ssh \
    -v ${SSH_AUTH_SOCK}:${SSH_AUTH_SOCK} -e SSH_AUTH_SOCK \
    -v "$(pwd)":/usr/src/app \
    -it copybara
```
https://github.com/google/copybara/blob/master/README.md
Build Copybara from source
master
To build Copybara from the repository HEAD, you need JDK 11 and Bazel installed.
1. Clone the repository: git clone https://github.com/google/copybara.git
2. Build the standard binary: bazel build //java/com/google/copybara
3. Build an executable uberjar: bazel build //java/com/google/copybara:copybara_deploy.jar
4. Run tests: bazel test //...
```
git clone https://github.com/google/copybara.git
bazel build //java/com/google/copybara
bazel build //java/com/google/copybara:copybara_deploy.jar
bazel test //...
```
https://github.com/google/copybara/blob/master/README.md

Use pre-built Copybara in a Bazel workspace

master

To use a weekly snapshot release of Copybara within your Bazel project:

Requirement: Copybara requires Java Runtime 21 or greater. Add run --java_runtime_version=remotejdk_21 to your .bazelrc.
Download the JAR: In your WORKSPACE or MODULE.bazel, use http_jar to fetch the release artifact from the GitHub releases page.
Define the binary: In a BUILD file, declare a java_binary pointing to com.google.copybara.Main.
Run: Use bazel run to execute commands like migrate.

# In WORKSPACE or MODULE.bazel
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_jar")

http_jar(
    name = "com_github_google_copybara",
    # Get sha256 from https://github.com/google/copybara/releases/download/[version]/copybara_deploy.jar.sha256
    sha256 = "",
    urls = ["https://github.com/google/copybara/releases/download/[version]/copybara_deploy.jar"],
)

# In a BUILD file
load("@rules_java//java:java_binary.bzl", "java_binary")

java_binary(
   name = "copybara",
   main_class = "com.google.copybara.Main",
   runtime_deps = ["@com_github_google_copybara//jar"],
)

Customize change identity in workflows
master
By default, Copybara hashes several fields to create a unique identifier for each change, allowing reuse of destination changes. You can customize this using change_identity to ensure the same identity is used across multiple workflows.
At least ${copybara_config_path} must be present in the string. The current user is added to the hash automatically.
Available variables:
- ${copybara_config_path}: Main config file path.
- ${copybara_workflow_name}: The name of the workflow being run.
- ${copybara_reference}: The requested reference (e.g., Gerrit change number, GitHub PR number, or resolved revision).
- ${label:label_name}: A specific label present for the current change.
https://github.com/google/copybara/blob/master/docs/reference.md
Manage destination file protection and pruning
master
To control which files are affected by a migration in the destination, use the following parameters:
- destination_files: Use a glob to specify which files are part of the migration. Files outside this glob are preserved even if they are missing from the origin. Example: glob(['**'], exclude = ['**/BUILD']) protects all BUILD files.
- smart_prune: (Only for CHANGE_REQUEST mode) When set to true, Copybara performs a best-effort attempt to restore non-affected snippets/files that were previously scrubbed.
- migrate_noop_changes: By default, Copybara only migrates changes affecting origin_files or config files. Set this to true to include all changes, though this may increase empty change errors.
https://github.com/google/copybara/blob/master/docs/reference.md
Configure merge import and post-merge transformations
master
If you need to perpetuate destination-only changes in non-source-of-truth repositories, use merge_import.
- merge_import: A migration mode that uses a diffing tool (default is diff3) to merge files. It takes (1) origin file, (2) baseline file, and (3) destination file as inputs.
- after_merge_transformations: A sequence of transformations to run after merge_import but before writing to the destination. This is useful for generating files (like BUILD files) that depend on the results of the merge.
https://github.com/google/copybara/blob/master/docs/reference.md