git-filter-repo

Choosing between git-filter-repo and git rebase depends on whether you want to manipulate the state of files or the changes (diffs) between commits.

To use git-filter-repo as a Python library or to use the demonstration scripts in contrib/filter-repo-demos/, you must ensure a file named git_filter_repo.py is available in your $PYTHONPATH.

This file should be a copy of or a symlink to the main git-filter-repo script. You can place it in your Python site-packages directory.

In Git, a commit hash is a cryptographic hash of its entire content, including the commit message, author, committer, the top-level tree hash, and the hashes of its parent(s).

If git-filter-repo modifies any part of a commit (such as removing a file or changing a path), the commit's tree hash changes. Because the tree hash changes, the commit hash changes. Furthermore, because the parent hash is part of the next commit's content, any change to a commit causes a chain reaction that changes the hashes of all subsequent commits in the history.

Beyond its CLI capabilities, git-filter-repo functions as a library that allows you to write custom history rewriting tools. When building custom tools, your scripts will automatically inherit core functionality such as:

• Rewriting hashes in commit messages
• Pruning commits that become empty
• Handling filenames with special characters or non-standard encodings
• Handling of replace refs

To prevent accidental data loss, git-filter-repo will detect if you are working in a fresh clone. If the repository is not a fresh clone, the tool will bail out to encourage a safer workflow (where you can simply delete the clone if an error occurs).

To bypass this safety check and run the tool on an existing repository, use the --force flag.

git filter-repo --force [options]

When writing multi-line strings inside a git filter-repo callback, Python's indentation can cause unwanted leading spaces to be included in the data. To prevent this, use textwrap.dedent() within your callback string.

git filter-repo --blob-callback '
  import textwrap
  blob.data = bytes(textwrap.dedent("""
    This is the new
    file content.
    """), "utf-8")
'

The Git project recommends git-filter-repo over git filter-branch.

vs filter-branch

• Performance: filter-branch is significantly slower for non-trivial repositories.
• Safety: filter-branch has known issues that can lead to silent corruption or messy rewrites.
• Complexity: filter-branch is difficult to use for non-trivial rewrites.

vs BFG Repo Cleaner

• Capability: BFG is limited to specific types of rewrites, whereas git-filter-repo is more versatile.
• Architecture: BFG's architecture is less amenable to handling diverse rewrite types and has known shortcomings/bugs.

You can learn how to use git-filter-repo through several resources:

Manuals

• User Manual: Available via HTML preview.
• Built-in Help: Use the -h flag in the CLI.

Cheat Sheets & Examples

• Converting from filter-branch: Use the filter-branch conversion cheat sheet.
• Converting from BFG Repo Cleaner: Use the BFG conversion cheat sheet.
• User-filed issues: See examples from user-filed issues.
• FAQ: Refer to the Frequently Answered Questions.

If git fsck --full identifies corrupt commit objects, you can fix them by creating a replacement object using git replace and then making the change permanent with git filter-repo --proceed.

1. Export the corrupt commit to a file: git cat-file -p <commit_hash> > tmp.
2. Edit the file to fix the corruption (e.g., fixing malformed author/committer lines).
3. Create a new commit object from the fixed file: git hash-object -t commit -w tmp.
4. Apply the replacement: git replace -f <corrupt_hash> <new_hash>.
5. Make it permanent: git filter-repo --proceed.

$ git cat-file -p 166f57b3fbe31257100361ecaf735f305b533b21 >tmp
# ... edit tmp ...
$ git replace -f 166f57b3fbe31257100361ecaf735f305b533b21 $(git hash-object -t commit -w tmp)
$ rm tmp
$ git filter-repo --proceed