From: Grant Moyer <dev@grantmoyer.com>
To: git@vger.kernel.org
Cc: Patrick Steinhardt <ps@pks.im>,
Junio C Hamano <gitster@pobox.com>,
Grant Moyer <dev@grantmoyer.com>,
Michele Locati <michele@locati.it>
Subject: [PATCH v3] filter-branch: fix commit map init from state branch
Date: Fri, 2 Oct 2026 21:01:27 -0400 [thread overview]
Message-ID: <20261003010128.256757-1-dev@grantmoyer.com> (raw)
In-Reply-To: <20261001012347.3998801-1-dev@grantmoyer.com>
The "--state-branch" option asks git-filter-branch(1) to write a
mapping from old to new objects into a branch to enable incremental
processing of large histories. This object mapping is stored as
a simple blob at "$state_branch:filter.map" with one object pair
per line in the format "$from_commit:$to_commit". Before processing
commits in a subsequent run, the state branch is used to populate an
object mapping directory, where each file is named "map/$from_commit"
and has contents "$to_commit".
In f6d855091e (filter-branch: stop depending on Perl, 2025-04-16),
we refactored git-filter-branch(1) to no longer require Perl,
but accidentally started to interpret object pairs in reverse (as
"$to_commit:$from_commit") when populating the map directory from
the state branch. This can cause all kinds of bad behavior from the
state-branch being effectively ignored to previously filtered objects
accidentally being mapped back to unfiltered objects. One especially
evident case occurs when "git filter-branch --prune-empty ..." maps
some commits to nothing, then on subsequent runs outputs many errors
while trying to create files with empty names, like:
> /usr/lib/git-core/git-filter-branch: line 305: ../map/: Is a directory
This regression went unnoticed because, since the introduction of
the only test for "--state-branch" in 709cfe848a (filter-branch: skip
commits present on --state-branch, 2018-06-26), the test accidentally
passes even if commits are not skipped. The test checks that after
populating a state branch with git-filter-branch(1), then running
it again with that state branch, the resulting filtered commits for
the first run and second run match. However since the filter used is
deterministic, the commits always match, even if the commits in the
state branch are re-filtered.
Fix the population of the object mapping dir by interpreting
object pairs as "$from_commit:$to_commit". Also fix the existing
"--state-branch" test by directly exiting with a non-zero code if
any commits from the state branch aren't skipped. Finally, add a new
"--state-branch" test which directly checks that a commit from the
state branch is used when incrementally filtering a repo.
Tested-by: Michele Locati <michele@locati.it>
Co-authored-by: Michele Locati <michele@locati.it>
Signed-off-by: Michele Locati <michele@locati.it>
Signed-off-by: Grant Moyer <dev@grantmoyer.com>
---
git-filter-branch.sh | 4 +++-
t/t7003-filter-branch.sh | 25 ++++++++++++++++++++++++-
2 files changed, 27 insertions(+), 2 deletions(-)
diff --git a/git-filter-branch.sh b/git-filter-branch.sh
index 24fa317aaa..9aa07be6e1 100755
--- a/git-filter-branch.sh
+++ b/git-filter-branch.sh
@@ -302,7 +302,9 @@ then
do
case "$line" in
*:*)
- echo "${line%:*}" >../map/"${line#*:}";;
+ from_commit=${line%:*}
+ to_commit=${line#*:}
+ echo "$to_commit" >../map/"$from_commit";;
*)
die "Unable to load state from $state_branch:filter.map";;
esac
diff --git a/t/t7003-filter-branch.sh b/t/t7003-filter-branch.sh
index 86011e7b1f..801cc83e5e 100755
--- a/t/t7003-filter-branch.sh
+++ b/t/t7003-filter-branch.sh
@@ -121,10 +121,33 @@ W=$(git rev-parse HEAD)
test_expect_success 'using --state-branch to skip already rewritten commits' '
test_when_finished git reset --hard $V &&
git reset --hard $V &&
- git filter-branch --state-branch state -f --tree-filter "touch file || :" HEAD &&
+ git filter-branch --state-branch state -f --tree-filter "exit 1" HEAD &&
test_cmp_rev $W HEAD
'
+test_expect_success '--state-branch incremental rewrite uses the rewritten parents' '
+ test_when_finished "rm -fr incremental" &&
+ git init incremental &&
+ (
+ cd incremental &&
+ mkdir sub &&
+ test_commit first sub/file &&
+ test_commit outside root-file &&
+ git filter-branch --state-branch refs/state \
+ --prune-empty --subdirectory-filter sub -- HEAD &&
+ rewritten_first=$(git rev-parse HEAD) &&
+ git reset --hard outside &&
+ test_commit second sub/file &&
+ git filter-branch -f --state-branch refs/state \
+ --prune-empty --subdirectory-filter sub -- outside..HEAD &&
+ test_cmp_rev $rewritten_first HEAD^ &&
+ git show refs/state:filter.map >map &&
+ echo "$(git rev-parse second):$(git rev-parse HEAD)" >expect &&
+ grep "^$(git rev-parse second):" map >actual &&
+ test_cmp expect actual
+ )
+'
+
git tag oldD HEAD~4
test_expect_success 'rewrite one branch, keeping a side branch' '
git branch modD oldD &&
--
2.55.0
next prev parent reply other threads:[~2026-10-03 1:09 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-08-01 3:31 [PATCH] fiter-branch: fix commit map init from state branch Grant Moyer
2026-09-29 14:31 ` Michele Locati
2026-09-30 14:40 ` Patrick Steinhardt
2026-09-30 15:48 ` Grant Moyer
2026-09-30 16:08 ` Patrick Steinhardt
2026-09-30 20:38 ` Michele Locati
2026-10-01 1:23 ` [PATCH v2] filter-branch: " Grant Moyer
2026-10-01 5:36 ` Patrick Steinhardt
2026-10-01 6:33 ` Michele Locati
2026-10-01 16:10 ` Junio C Hamano
2026-10-03 1:01 ` Grant Moyer [this message]
2026-10-05 6:11 ` [PATCH v3] " Patrick Steinhardt
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261003010128.256757-1-dev@grantmoyer.com \
--to=dev@grantmoyer.com \
--cc=git@vger.kernel.org \
--cc=gitster@pobox.com \
--cc=michele@locati.it \
--cc=ps@pks.im \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox