* Re: [PATCH] submodule-config: fix reading submodule.fetchJobs
From: Pablo @ 2026-05-03 11:07 UTC (permalink / raw)
To: Saagar Jha via GitGitGadget; +Cc: git, Saagar Jha
In-Reply-To: <pull.2287.git.git.1777803242592.gitgitgadget@gmail.com>
El dom, 3 may 2026 a las 12:14, Saagar Jha via GitGitGadget
(<gitgitgadget@gmail.com>) escribió:
>
> From: Saagar Jha <saagar@saagarjha.com>
>
> Signed-off-by: Saagar Jha <saagar@saagarjha.com>
> ---
> submodule-config: fix reading submodule.fetchJobs
>
> Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-2287%2Fsaagarjha%2Fmaint-v1
> Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-2287/saagarjha/maint-v1
> Pull-Request: https://github.com/git/git/pull/2287
>
> submodule-config.c | 2 +-
> t/t7406-submodule-update.sh | 10 ++++++++++
> 2 files changed, 11 insertions(+), 1 deletion(-)
>
> diff --git a/submodule-config.c b/submodule-config.c
> index 1f19fe2077..57b190678e 100644
> --- a/submodule-config.c
> +++ b/submodule-config.c
> @@ -1037,5 +1037,5 @@ static int gitmodules_update_clone_config(const char *var, const char *value,
>
> void update_clone_config_from_gitmodules(int *max_jobs)
> {
> - config_from_gitmodules(gitmodules_update_clone_config, the_repository, &max_jobs);
> + config_from_gitmodules(gitmodules_update_clone_config, the_repository, max_jobs);
> }
> diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh
> index 3adab12091..234a021fb3 100755
> --- a/t/t7406-submodule-update.sh
> +++ b/t/t7406-submodule-update.sh
> @@ -1055,6 +1055,16 @@ test_expect_success 'submodule update can be run in parallel' '
> )
> '
>
> +test_expect_success 'submodule update honors fetch jobs config from .gitmodules' '
> + test_when_finished "rm -rf super3" &&
> + git clone cloned super3 &&
> + (cd super3 &&
> + git config -f .gitmodules submodule.fetchJobs 67 &&
> + GIT_TRACE="$(pwd)/trace.out" git submodule update --init &&
> + grep "67 tasks" trace.out
> + )
> +'
> +
> test_expect_success 'git clone passes the parallel jobs config on to submodules' '
> test_when_finished "rm -rf super4" &&
> GIT_TRACE=$(pwd)/trace.out git clone --recurse-submodules --jobs 7 . super4 &&
>
> base-commit: 67ad42147a7acc2af6074753ebd03d904476118f
> --
> gitgitgadget
>
Hi Saagar!
A few hints before this can be reviewable:
- There is no commit body which, not only is a must, it makes it very
hard reviewing because I cannot know what you wanted to do and I have
to imagine it.
- CI tests seem to be failing, but the file that reports fail doesn't
seem to be related with your changes.
About the code, I can understand the 'why': the function above
'fetch_config_from_gitmodules' builds an struct inside and sends the
address &config, 'update_clone_config_from_gitmodules' calls the same
function with &max_jobs but the error would be that max_jobs is
already a pointer so there would be no need to pass by reference
max_jobs.
But this kind of explanation is what should be on the commit body.
Maybe not this technical as the code can be easily seen, but a high
level explanation about what this is.
Looking forward for a v2,
--
Pablo
^ permalink raw reply
* [PATCH] submodule-config: fix reading submodule.fetchJobs
From: Saagar Jha via GitGitGadget @ 2026-05-03 10:14 UTC (permalink / raw)
To: git; +Cc: Saagar Jha, Saagar Jha
From: Saagar Jha <saagar@saagarjha.com>
Signed-off-by: Saagar Jha <saagar@saagarjha.com>
---
submodule-config: fix reading submodule.fetchJobs
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-git-2287%2Fsaagarjha%2Fmaint-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-git-2287/saagarjha/maint-v1
Pull-Request: https://github.com/git/git/pull/2287
submodule-config.c | 2 +-
t/t7406-submodule-update.sh | 10 ++++++++++
2 files changed, 11 insertions(+), 1 deletion(-)
diff --git a/submodule-config.c b/submodule-config.c
index 1f19fe2077..57b190678e 100644
--- a/submodule-config.c
+++ b/submodule-config.c
@@ -1037,5 +1037,5 @@ static int gitmodules_update_clone_config(const char *var, const char *value,
void update_clone_config_from_gitmodules(int *max_jobs)
{
- config_from_gitmodules(gitmodules_update_clone_config, the_repository, &max_jobs);
+ config_from_gitmodules(gitmodules_update_clone_config, the_repository, max_jobs);
}
diff --git a/t/t7406-submodule-update.sh b/t/t7406-submodule-update.sh
index 3adab12091..234a021fb3 100755
--- a/t/t7406-submodule-update.sh
+++ b/t/t7406-submodule-update.sh
@@ -1055,6 +1055,16 @@ test_expect_success 'submodule update can be run in parallel' '
)
'
+test_expect_success 'submodule update honors fetch jobs config from .gitmodules' '
+ test_when_finished "rm -rf super3" &&
+ git clone cloned super3 &&
+ (cd super3 &&
+ git config -f .gitmodules submodule.fetchJobs 67 &&
+ GIT_TRACE="$(pwd)/trace.out" git submodule update --init &&
+ grep "67 tasks" trace.out
+ )
+'
+
test_expect_success 'git clone passes the parallel jobs config on to submodules' '
test_when_finished "rm -rf super4" &&
GIT_TRACE=$(pwd)/trace.out git clone --recurse-submodules --jobs 7 . super4 &&
base-commit: 67ad42147a7acc2af6074753ebd03d904476118f
--
gitgitgadget
^ permalink raw reply related
* Re: [PATCH v3 1/1] git-gui: handle missing worktree and separated gitdir
From: Johannes Sixt @ 2026-05-03 8:53 UTC (permalink / raw)
To: Mark Levedahl; +Cc: git, Shroom Moo
In-Reply-To: <93e1c61f-e58b-4a0c-8ece-7a8d945fa900@gmail.com>
Am 02.05.26 um 23:51 schrieb Mark Levedahl:
> Two basic problems I see here are beyond the question of if (and when) git-gui should try
> to locate a worktree:
>
> - git gui blame in a gitdir was apparently broken by the git repo commit 2d92ab32fd
> ("rev-parse: make --show-toplevel without a worktree an error", 2019-11-19). Prior to that
> commit, git gui would stay in the startup directory enabling only features that cannot
> modify the repository, and gitk could bring this view up in a gitdir. This doesn't work
> right now.
True, `blame` used to work in bare repositories, but is broken now.
> - git-gui's logic includes a conceptual error embodied in proc is_bare: is_bare uses $(git
> rev-parse --is-bare-repository) but what we need is $(git rev-parse --is-inside-git-dir),
> and these are not synonyms.
> It does not matter whether a worktree exists that points at the gitdir, and as discussed
> before, main worktrees can easily exist that we cannot locate from the gitdir. At best,
> is_bare is a guess.
> So, is_bare should be replaced by is_inside_gitdir, and we should also have
> is_inside_worktree. These are mutually exclusive, though both can be false.
I would not call the use of --is-bar-repository instead of
--is-inside-git-dir an error, just a choice that has been made. In
particular, when the startup directory is named '.git' and is not marked
as bare, then its parent directory can very reasonably be taken as its
worktree. (That's how things worked before --show-toplevel was used.) If
the check is for --is-inside-git-dir, this treatment would be ruled out
early.
> My current idea of an improved startup flow enables features only at the end:
>
> 1) If not in a gitdir or worktree,
> 1a) if git gui's subcommand is not 'gui', abort with an error (citool, browser, or
> blame invocations carry information specific to a gitdir/worktree).
> 1b) otherwise, invoke repository_chooser, which either aborts, or changes directory
> to a worktree.
>
> -- we are now in a gitdir or a worktree, and this may or may not be the startup directory.
>
> 2) Look at the combination of git gui subcommand and directory type (worktree / gitdir) to
> decide to continue.
> 2a) blame / browser are ok in either directory type.
> 2b) citool requires a specific worktree, which should have been the initial startup
> directory. Abort if not.
> 2b) gui requires a worktree. Abort if not (my recommendation), or offer to find (or
> automatically find) a worktree.
>
> 3) Change directory to the top level of of the directory_type (git rev-parse knows
> toplevel of a worktree, different code is needed for a gitdir).
>
> 4) Enable features based upon subcommand and directory type.
>
> There are 12 combinations of initial directory type (gitdir, worktree, neither) and
> subcommand (gui, blame, browser, citool) to consider, with a lot of duplicated code
> amongst the 12 cases. So, obviously, steps 1 and 2 can be convolved in many ways that are
> different than what I wrote above.
So true.
But perhaps there is a simpler solution: Let's present an error if
--show-toplevel fails except in the case where the startup directory is
named '.git' (and is a valid Git repository) and is not bare (then the
worktree is the parent). I insist in this exception, because this
use-case was considered important in the past (87cd09f43e56 "git-gui:
work from the .git dir", 2010-01-23).
-- Hannes
^ permalink raw reply
* Re: [PATCH v3 0/8] builtin: implement, document and test url-parse
From: Matheus Afonso Martins Moreira @ 2026-05-03 4:29 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git, Torsten Bögershausen, Ghanshyam Thakkar
In-Reply-To: <xmqqik95taoh.fsf@gitster.g>
Junio C Hamano <gitster@pobox.com> writes:
> Nitpick. With "git url-parse", these scripts has to do what they
> traditionally have always done, i.e., shell out to the command, no?
It's a good point. I should have worded it better:
Tools wanting to reason about them have historically had to
reimplement git's parsing logic externally. With git url-parse,
scripts can delegate URL parsing to git's own parser: validate
a URL, extract a component (scheme, user, host, port, path,
password), or both.
What I meant to say is that git's URL parsing was never exposed
as a standalone operation, leading external tools to reimplement
the logic themselves.
For example:
npm/git-url-parse millions of weekly downloads
crates/git-url-parse half a million downloads
With this builtin, scripts can rely on git's own parser
instead of a third party reimplementation.
References:
https://www.npmjs.com/package/git-url-parse
https://crates.io/crates/git-url-parse
^ permalink raw reply
* Re: [PATCH v3 0/8] builtin: implement, document and test url-parse
From: Junio C Hamano @ 2026-05-03 3:49 UTC (permalink / raw)
To: Matheus Moreira via GitGitGadget
Cc: git, Torsten Bögershausen, Ghanshyam Thakkar,
Matheus Moreira
In-Reply-To: <pull.1715.v3.git.git.1777699722.gitgitgadget@gmail.com>
"Matheus Moreira via GitGitGadget" <gitgitgadget@gmail.com> writes:
> ... Tools wanting to reason
> about them have historically had to reimplement git's parsing or shell out
> indirectly. With git url-parse, scripts can ask git directly: validate a
> URL, extract a component (scheme, user, host, port, path, password), or
> both.
Nitpick. With "git url-parse", these scripts has to do what they
traditionally have always done, i.e., shell out to the command, no?
^ permalink raw reply
* What's cooking in git.git (May 2026, #01)
From: Junio C Hamano @ 2026-05-03 3:47 UTC (permalink / raw)
To: git
Here are the topics that have been cooking in my tree. Commits
prefixed with '+' are in 'next' (being in 'next' is a sign that a
topic is stable enough to be used and is a candidate to be in a
future release). Commits prefixed with '-' are only in 'seen', and
aren't considered "accepted" at all and may be annotated with a URL
to a message that raises issues but they are by no means exhaustive.
A topic without enough support may be discarded after a long period
of no activity (of course they can be resubmitted when new interests
arise).
Git 2.54 has been released, and the tip of 'next' has acquired a
handful more topics, but no update to 'master' yet to kick off the
new cycle has happened (yet). I am still mostly offline for another
week, but just had a chance to pick up updates to a handful of
topics.
Copies of the source code to Git live in many repositories, and the
following is a list of the ones I push into or their mirrors. Some
repositories have only a subset of branches.
With maint, master, next, seen, todo:
git://git.kernel.org/pub/scm/git/git.git/
git://repo.or.cz/alt-git.git/
https://kernel.googlesource.com/pub/scm/git/git/
https://github.com/git/git/
https://gitlab.com/git-scm/git/
With all the integration branches and topics broken out:
https://github.com/gitster/git/
Even though the preformatted documentation in HTML and man format
are not sources, they are published in these repositories for
convenience (replace "htmldocs" with "manpages" for the manual
pages):
git://git.kernel.org/pub/scm/git/git-htmldocs.git/
https://github.com/gitster/git-htmldocs.git/
Release tarballs are available at:
https://www.kernel.org/pub/software/scm/git/
--------------------------------------------------
[New Topics]
* jh/alias-i18n-fixes (2026-04-24) 1 commit
- alias: restore support for simple dotted aliases
Further update to the i18n alias support to avoid regressions.
source: <20260424161707.1514255-1-jonatan@jontes.page>
* kn/refs-generic-helpers (2026-04-27) 9 commits
- refs: use peeled tag values in reference backends
- refs: add peeled object ID to the `ref_update` struct
- refs: move object parsing to the generic layer
- update-ref: handle rejections while adding updates
- update-ref: move `print_rejected_refs()` up
- refs: return `ref_transaction_error` from `ref_transaction_update()`
- refs: extract out reflog config to generic layer
- refs: introduce `ref_store_init_options`
- refs: remove unused typedef 'ref_transaction_commit_fn'
Refactor service routines in the ref subsystem backends.
Will merge to 'next'?
source: <20260427-refs-move-to-generic-layer-v3-0-e4638dfb7897@gmail.com>
* ob/more-repo-config-values (2026-04-23) 8 commits
- env: move "warn_on_object_refname_ambiguity" into `struct repo_config_values`
- env: move "sparse_expect_files_outside_of_patterns" into `repo_config_values`
- env: move "core_sparse_checkout_cone" into `struct repo_config_values`
- environment: move "precomposed_unicode" into `struct repo_config_values`
- environment: move "pack_compression_level" into `struct repo_config_values`
- environment: move `zlib_compression_level` into `struct repo_config_values`
- environment: move "check_stat" into `struct repo_config_values`
- environment: move "trust_ctime" into `struct repo_config_values`
Expecting a reroll.
cf. <CAD=f0L8-_3sDGGkCzF4WA0xmUtaY_qiz__3zq5AemLgwTsqvsg@mail.gmail.com>
source: <20260423165432.143598-1-belkid98@gmail.com>
* ps/history-fixup (2026-04-26) 3 commits
- builtin/history: introduce "fixup" subcommand
- builtin/history: generalize function to commit trees
- replay: allow callers to control what happens with empty commits
"git history" learned "fixup" command.
Comments?
source: <20260427-b4-pks-history-fixup-v3-0-cb908f06264b@pks.im>
* rs/grep-column-only-match-fix (2026-04-24) 1 commit
- grep: fix --column --only-match for 2nd and later matches
"git grep" update.
Will merge to 'next'?
source: <9bd69678-f04b-41d2-ad74-a386820d34c8@web.de>
* sb/unpack-index-pack-buffer-resize (2026-04-28) 1 commit
- index-pack, unpack-objects: increase input buffer from 4 KiB to 128 KiB
Use a larger buffer size in the code paths to ingest pack stream.
Will merge to 'next'?
source: <pull.2282.v4.git.git.1777387660841.gitgitgadget@gmail.com>
* bc/sign-commit-with-custom-encoding (2026-04-27) 2 commits
- commit: sign commit after mutating buffer
- commit: name UTF-8 function appropriately
Signing commit with custom encoding was passing the data to be
signed at a wrong stage in the pipeline, which has been corrected.
Will merge to 'next'?
source: <20260427221834.1824543-1-sandals@crustytoothpaste.net>
* cc/promisor-auto-config-url-more (2026-04-27) 9 commits
- doc: promisor: improve acceptFromServer entry
- promisor-remote: auto-configure unknown remotes
- promisor-remote: trust known remotes matching acceptFromServerUrl
- promisor-remote: introduce promisor.acceptFromServerUrl
- promisor-remote: add 'local_name' to 'struct promisor_info'
- urlmatch: add url_normalize_pattern() helper
- urlmatch: change 'allow_globs' arg to bool
- t5710: simplify 'mkdir X' followed by 'git -C X init'
- Merge branch 'cc/promisor-auto-config-url' into cc/promisor-auto-config-url-more
(this branch uses cc/promisor-auto-config-url.)
source: <20260427124108.3524129-1-christian.couder@gmail.com>
* js/maintenance-fix-deadlock-on-win10 (2026-04-28) 2 commits
- maintenance(geometric): do release the `.idx` files before repacking
- mingw: optionally use legacy (non-POSIX) delete semantics
To help Windows 10 installations, avoid removing files whose
contents are still mmap()'ed.
Will merge to 'next'?
source: <pull.2103.git.1777380768.gitgitgadget@gmail.com>
* js/objects-larger-than-4gb-on-windows (2026-04-28) 6 commits
- t5608: add regression test for >4GB object clone
- test-tool: add a helper to synthesize large packfiles
- delta, packfile: use size_t for delta header sizes
- odb, packfile: use size_t for streaming object sizes
- git-zlib: handle data streams larger than 4GB
- index-pack, unpack-objects: use size_t for object size
Update code paths that assumed "unsigned long" was long enough for
"size_t".
Expecting a reroll to mark EXPENSIVE tests as such.
source: <pull.2102.git.1777393580.gitgitgadget@gmail.com>
* js/t5564-socks-use-short-path (2026-04-29) 1 commit
- t5564: use a short path for the SOCKS proxy socket
Avoid hitting the pathname limit for socks proxy socket during the
test..
Will merge to 'next'.
source: <pull.2100.v2.git.1777450974159.gitgitgadget@gmail.com>
* kh/doc-log-decorate-list (2026-04-27) 2 commits
- doc: log: use the same delimiter in description list
- doc: log: fix --decorate description list
Doc update.
Comments?
source: <CV_doc_log_--decorate_list.626@msgid.xyz>
* za/t2000-modernise-more (2026-04-29) 1 commit
- t2000: consolidate second scenario into a single test block
Test update.
Comments?
source: <20260429103607.406339-1-zakariyahali100@gmail.com>
* hn/checkout-track-fetch (2026-04-28) 1 commit
- checkout: extend --track with a "fetch" mode to refresh start-point
"git checkout --track=..." learned to optionally fetch the branch
from the remote the new branch will work with.
Comments?
source: <pull.2281.v5.git.git.1777367012441.gitgitgadget@gmail.com>
* mf/revision-max-count-oldest (2026-04-30) 1 commit
- revision.c: implement --max-count-oldest
"git rev-list" (and "git log" family of commands) learned a new "--max-count-oldest"
that picks oldest N commits in the range instead of the usual newest.
Comments?
source: <2f71a00b035e25b971641b77a6fa7626f1e2459c.1777578676.git.mroik@delayed.space>
* mm/line-log-cleanup (2026-04-27) 3 commits
- line-log: allow non-patch diff formats with -L
- line-log: integrate -L output with the standard log-tree pipeline
- revision: move -L setup before output_format-to-diff derivation
Code clean-up.
Comments?
source: <pull.2094.git.1777349126.gitgitgadget@gmail.com>
* pw/rename-to-get-current-worktree (2026-05-01) 1 commit
- worktree: rename get_worktree_from_repository()
Code clean-up.
Will merge to 'next'.
source: <bd48396137f8d1352d11b3bd2dca2848f24a347d.1777648798.git.phillip.wood@dunelm.org.uk>
* ds/path-walk-filters (2026-05-02) 7 commits
- pack-objects: support sparse:oid filter with path-walk
- path-walk: add pl_sparse_trees to control tree pruning
- path-walk: support blob size limit filter
- backfill: die on incompatible filter options
- path-walk: support blobless filter
- t/perf: add pack-objects filter and path-walk benchmark
- pack-objects: pass --objects with --path-walk
The "git pack-objects --path-walk" traversal has been integrated
with several object filters, including blobless and sparse filters.
Comments?
source: <pull.2101.git.1777731354.gitgitgadget@gmail.com>
--------------------------------------------------
[Cooking]
* en/ort-cached-rename-with-trivial-resolution (2026-04-20) 1 commit
- merge-ort: handle cached rename & trivial resolution interaction better
"ort" merge backend improvements.
Will merge to 'next'.
source: <pull.2095.git.1776724214171.gitgitgadget@gmail.com>
* en/ort-harden-against-corrupt-trees (2026-04-20) 5 commits
- cache-tree: fix verify_cache() to catch non-adjacent D/F conflicts
- merge-ort: abort merge when trees have duplicate entries
- merge-ort: free diff pairs queue in clear_or_reinit_internal_opts()
- merge-ort: drop unnecessary show_all_errors from collect_merge_info()
- merge-ort: propagate callback errors from traverse_trees_wrapper()
"ort" merge backend handles merging corrupt trees better by
aborting when it should.
Needs review.
source: <pull.2096.git.1776731171.gitgitgadget@gmail.com>
* jk/revert-aa-reap-transport-child-processes (2026-04-22) 1 commit
- Revert "transport-helper, connect: use clean_on_exit to reap children on abnormal exit"
Revert a recent change that introduced a regression to help mksh users.
Will merge to 'next'.
source: <20260422230020.GA1839627@coredump.intra.peff.net>
* js/ci-github-actions-update (2026-04-30) 6 commits
- l10n: bump mshick/add-pr-comment from v2 to v3
- ci: bump git-for-windows/setup-git-for-windows-sdk from v1 to v2
- ci: bump actions/checkout from v5 to v6
- ci: bump actions/github-script from v8 to v9
- ci: bump actions/{upload,download}-artifact to v7 and v8
- ci: bump microsoft/setup-msbuild from v2 to v3
Update various GitHub Actions versions.
Will merge to 'next'.
source: <pull.2097.v3.git.1777534500.gitgitgadget@gmail.com>
* mf/format-patch-cover-letter-format-docfix (2026-04-22) 1 commit
- Fix docs for format.commitListFormat
Docfix.
Will merge to 'next'.
source: <576d29f15e016889e02c253713656cd8cbf1f04c.1776894255.git.mroik@delayed.space>
* sg/t6112-unwanted-tilde-expansion-fix (2026-04-21) 1 commit
- t6112: avoid tilde expansion
Test fix.
Will merge to 'next'?
source: <20260421192132.51172-1-szeder.dev@gmail.com>
* pw/status-rebase-todo (2026-05-01) 2 commits
- status: improve rebase todo list parsing
- sequencer: factor out parsing of todo commands
The display of the rebase todo list in "git status" has been
improved to correctly abbreviate object IDs for more commands and
avoid misinterpreting refs as object IDs.
Needs review.
source: <cover.1777648598.git.phillip.wood@dunelm.org.uk>
* sb/userdiff-lisp-family (2026-04-14) 2 commits
(merged to 'next' on 2026-04-20 at 5897c04899)
+ userdiff: extend Scheme support to cover other Lisp dialects
+ userdiff: tighten word-diff test case of the scheme driver
The userdiff driver for the Scheme language has been extended to
cover other Lisp dialects.
Will merge to 'master'.
source: <pull.2000.v4.git.1776220063.gitgitgadget@gmail.com>
* ss/t7004-unhide-git-failures (2026-04-20) 3 commits
- t7004: avoid subshells to capture git exit codes
- t7004: dynamically grab expected state in tests
- t7004: drop hardcoded tag count for state verification
Test clean-up.
Will merge to 'next'.
cf. <aecNc-BNwaqFlg5c@pks.im>
source: <20260421053334.5414-1-r.siddharth.shrimali@gmail.com>
* tb/pseudo-merge-bugfixes (2026-04-21) 9 commits
- pack-bitmap: prevent pattern leak on pseudo-merge re-assignment
- Documentation: fix broken `sampleRate` in gitpacking(7)
- pack-bitmap: reject pseudo-merge "sampleRate" of 0
- pack-bitmap: parse commits in `find_pseudo_merge_group_for_ref()`
- pack-bitmap: fix pseudo-merge lookup for shared commits
- pack-bitmap: fix inverted binary search in `pseudo_merge_at()`
- pack-bitmap-write: sort pseudo-merge commit lookup table in pack order
- t5333: demonstrate various pseudo-merge bugs
- t/helper: add 'test-tool bitmap write' subcommand
Fixes many bugs in pseudo-merge code.
Expecting (hopefully minor and final) reroll.
cf. <CABPp-BGkfavqezk2SV3+K6iF8MLm8j_=ijHiPDLmv_U_o_Ykgg@mail.gmail.com>
source: <cover.1776801694.git.me@ttaylorr.com>
* ds/fetch-negotiation-options (2026-04-22) 7 commits
- send-pack: pass negotiation config in push
- remote: add remote.*.negotiationInclude config
- fetch: add --negotiation-include option for negotiation
- remote: add remote.*.negotiationRestrict config
- transport: rename negotiation_tips
- fetch: add --negotiation-restrict option
- t5516: fix test order flakiness
The negotiation tip options in "git fetch" have been reworked to
allow requiring certain refs to be sent as "have" lines, and to
restrict negotiation to a specific set of refs.
Needs review.
source: <pull.2085.v3.git.1776871546.gitgitgadget@gmail.com>
* en/backfill-fixes-and-edges (2026-04-15) 3 commits
- backfill: default to grabbing edge blobs too
- backfill: document acceptance of revision-range in more standard manner
- backfill: reject rev-list arguments that do not make sense
The 'git backfill' command now rejects revision-limiting options that
are incompatible with its operation, uses standard documentation for
revision ranges, and includes blobs from boundary commits by default
to improve performance of subsequent operations.
Will merge to 'next'.
cf. <6e95b82a-19e3-460e-86f7-f899c2df261d@gmail.com>
source: <pull.2088.git.1776297482.gitgitgadget@gmail.com>
* mc/http-emptyauth-negotiate-fix (2026-04-30) 4 commits
- doc: clarify http.emptyAuth values
(merged to 'next' on 2026-04-20 at 6539524ca2)
+ t5563: add tests for http.emptyAuth with Negotiate
+ http: attempt Negotiate auth in http.emptyAuth=auto mode
+ http: extract http_reauth_prepare() from retry paths
The 'http.emptyAuth=auto' configuration now correctly attempts
Negotiate authentication before falling back to manual credentials.
This allows seamless Kerberos ticket-based authentication without
requiring users to explicitly set 'http.emptyAuth=true'.
Will merge to 'next' and then to 'master'.
source: <e0f236767f81ea60f90749d1bc00ab78081efd0e.1777546472.git.gitgitgadget@gmail.com>
source: <pull.2087.git.1776331259.gitgitgadget@gmail.com>
* en/batch-prefetch (2026-04-17) 3 commits
- grep: prefetch necessary blobs
- builtin/log: prefetch necessary blobs for `git cherry`
- patch-ids.h: add missing trailing parenthesis in documentation comment
In a lazy clone, "git cherry" and "git grep" often fetch necessary
blob objects one by one from promisor remotes. It has been corrected
to collect necessary object names and fetch them in bulk to gain
reasonable performance.
Needs review.
source: <pull.2089.v2.git.1776472347.gitgitgadget@gmail.com>
* en/diffstat-utf8-truncation-fix (2026-04-20) 1 commit
- diff: fix out-of-bounds reads and NULL deref in diffstat UTF-8 truncation
The computation to shorten the filenames shown in diffstat measured
width of individual UTF-8 characters to add up, but forgot to take
into account error cases (e.g., an invalid UTF-8 sequence, or a
control character).
Will merge to 'next'?
source: <pull.2093.v3.git.1776699778177.gitgitgadget@gmail.com>
* sp/refs-reduce-the-repository (2026-04-04) 3 commits
(merged to 'next' on 2026-04-09 at bb1d626802)
+ refs/reftable-backend: drop uses of the_repository
+ refs: remove the_hash_algo global state
+ refs: add struct repository parameter in get_files_ref_lock_timeout_ms()
Code clean-up to use the right instance of a repository instance in
calls inside refs subsystem.
Will merge to 'master'.
source: <20260404135914.61195-1-shreyanshpaliwalcmsmn@gmail.com>
* ps/odb-in-memory (2026-04-10) 18 commits
- t/unit-tests: add tests for the in-memory object source
- odb: generic in-memory source
- odb/source-inmemory: stub out remaining functions
- odb/source-inmemory: implement `freshen_object()` callback
- odb/source-inmemory: implement `count_objects()` callback
- odb/source-inmemory: implement `find_abbrev_len()` callback
- odb/source-inmemory: implement `for_each_object()` callback
- odb/source-inmemory: convert to use oidtree
- oidtree: add ability to store data
- cbtree: allow using arbitrary wrapper structures for nodes
- odb/source-inmemory: implement `write_object_stream()` callback
- odb/source-inmemory: implement `write_object()` callback
- odb/source-inmemory: implement `read_object_stream()` callback
- odb/source-inmemory: implement `read_object_info()` callback
- odb: fix unnecessary call to `find_cached_object()`
- odb/source-inmemory: implement `free()` callback
- odb: introduce "in-memory" source
- Merge branch 'jt/odb-transaction-write' into ps/odb-in-memory
(this branch uses jt/odb-transaction-write.)
Add a new odb "in-memory" source that is meant to only hold
tentative objects (like the virtual blob object that represents the
working tree file used by "git blame").
Will merge to 'next'?
source: <20260410-b4-pks-odb-source-inmemory-v3-0-22fd0fad58fe@pks.im>
* jc/doc-timestamps-in-stat (2026-04-10) 1 commit
(merged to 'next' on 2026-04-20 at 0680260012)
+ CodingGuidelines: st_mtimespec vs st_mtim vs st_mtime
Doc update.
Will merge to 'master'.
source: <xmqqzf3aofdj.fsf_-_@gitster.g>
* ps/test-set-e-clean (2026-04-21) 12 commits
(merged to 'next' on 2026-04-23 at 4f69b47b94)
+ t: detect errors outside of test cases
+ t9902: fix use of `read` with `set -e`
+ t6002: fix use of `expr` with `set -e`
+ t1301: don't fail in case setfacl(1) doesn't exist or fails
+ t0008: silence error in subshell when using `grep -v`
+ t: prepare `test_when_finished ()`/`test_atexit()` for `set -e`
+ t: prepare execution of potentially failing commands for `set -e`
+ t: prepare conditional test execution for `set -e`
+ t: prepare `git config --unset` calls for `set -e`
+ t: prepare `stop_git_daemon ()` for `set -e`
+ t: prepare `test_must_fail ()` for `set -e`
+ t: prepare `test_match_signal ()` calls for `set -e`
The test suite harness and many individual test scripts have been
updated to work correctly when 'set -e' is in effect, which helps
detect misspelled test commands.
Will merge to 'master'.
source: <20260421-b4-pks-tests-with-set-e-v6-0-26330e3061ab@pks.im>
* js/adjust-tests-to-explicitly-access-bare-repo (2026-04-26) 8 commits
- safe.bareRepository: default to "explicit" with WITH_BREAKING_CHANGES
- status tests: filter `.gitconfig` from status output
- ls-files tests: filter `.gitconfig` from `--others` output
- t5601: restore `.gitconfig` after includeIf test
- t1305: use `--git-dir=.` for bare repo in include cycle test
- t1300: remove global config settings injected by test-lib.sh
- t7900: do not let `$HOME/.gitconfig` interfere with XDG tests
- test-lib: allow bare repository access when breaking changes are enabled
Some tests assume that bare repository accesses are by default
allowed; rewrite some of them to avoid the assumption, rewrite
others to explicitly set safe.bareRepository to allow them.
Will merge to 'next'?
source: <pull.2098.v2.git.1777214316.gitgitgadget@gmail.com>
* cl/conditional-config-on-worktree-path (2026-04-03) 2 commits
- config: add "worktree" and "worktree/i" includeIf conditions
- config: refactor include_by_gitdir() into include_by_path()
The [includeIf "condition"] conditional inclusion facility for
configuration files has learned to use the location of worktree
in its condition.
Comments?
source: <20260403-includeif-worktree-v3-0-109ce5782b03@black-desk.cn>
* bc/rust-by-default (2026-04-09) 4 commits
(merged to 'next' on 2026-04-23 at fb9310bfae)
+ Enable Rust by default
+ Linux: link against libdl
+ ci: install cargo on Alpine
+ docs: update version with default Rust support
Rust support is enabled by default (but still allows opting out) in
some future version of Git.
Will merge to 'master'.
source: <20260409224434.1861422-1-sandals@crustytoothpaste.net>
* ps/shift-root-in-graph (2026-04-27) 1 commit
- graph: add indentation for commits preceded by a parentless commit
In a history with more than one root commit, "git log --graph
--oneline" stuffed an unrelated commit immediately below a root
commit, which has been corrected by making the spot below a root
unavailable.
Will merge to 'next'?
source: <20260427102838.44867-2-pabloosabaterr@gmail.com>
* jt/config-lock-timeout (2026-04-03) 1 commit
- config: retry acquiring config.lock for 100ms
The code path to update the configuration file has been taught to
use a short timeout to retry.
Waiting for a review response.
cf. <adYvSZeN0ZVqwRhi@pks.im>
source: <20260403100135.3901610-1-joerg@thalheim.io>
* dl/cache-tree-fully-valid-fix (2026-04-06) 1 commit
(merged to 'next' on 2026-04-13 at 68c82a9f37)
+ cache-tree: fix inverted object existence check in cache_tree_fully_valid
The check that implements the logic to see if an in-core cache-tree
is fully ready to write out a tree object was broken, which has
been corrected.
Will merge to 'master'.
source: <20260406192711.68870-1-davidlin@stripe.com>
* ja/doc-difftool-synopsis-style (2026-04-04) 4 commits
(merged to 'next' on 2026-04-13 at 0e6c98f313)
+ doc: convert git-describe manual page to synopsis style
+ doc: convert git-shortlog manual page to synopsis style
+ doc: convert git-range-diff manual page to synopsis style
+ doc: convert git-difftool manual page to synopsis style
Doc mark-up updates.
Will merge to 'master'.
source: <pull.2077.git.1775322767.gitgitgadget@gmail.com>
* lp/repack-propagate-promisor-debugging-info (2026-04-18) 6 commits
- repack-promisor: add missing headers
- t7703: test for promisor file content after geometric repack
- t7700: test for promisor file content after repack
- repack-promisor: preserve content of promisor files after repack
- repack-promisor add helper to fill promisor file after repack
- pack-write: add explanation to promisor file content
When fetching objects into a lazily cloned repository, .promisor
files are created with information meant to help debugging. "git
repack" has been taught to carry this information forward to
packfiles that are newly created.
Comments?
source: <cover.1776384902.git.lorenzo.pegorari2002@gmail.com>
* th/promisor-quiet-per-repo (2026-04-06) 1 commit
- promisor-remote: fix promisor.quiet to use the correct repository
The "promisor.quiet" configuration variable was not used from
relevant submodules when commands like "grep --recurse-submodules"
triggered a lazy fetch, which has been corrected.
Comments?
source: <20260406183041.783800-1-vikingtc4@gmail.com>
* jt/odb-transaction-write (2026-04-02) 7 commits
- odb/transaction: make `write_object_stream()` pluggable
- object-file: generalize packfile writes to use odb_write_stream
- object-file: avoid fd seekback by checking object size upfront
- object-file: remove flags from transaction packfile writes
- odb: update `struct odb_write_stream` read() callback
- odb/transaction: use pluggable `begin_transaction()`
- odb: split `struct odb_transaction` into separate header
(this branch is used by ps/odb-in-memory.)
ODB transaction interface is being reworked to explicitly handle
object writes.
Will merge to 'next'?
source: <20260402213220.2651523-1-jltobler@gmail.com>
* sa/cat-file-batch-mailmap-switch (2026-04-15) 1 commit
- cat-file: add mailmap subcommand to --batch-command
"git cat-file --batch" learns an in-line command "mailmap"
that lets the user toggle use of mailmap.
Will merge to 'next'?
source: <20260416033250.4327-2-siddharthasthana31@gmail.com>
* tb/incremental-midx-part-3.3 (2026-04-29) 16 commits
- repack: allow `--write-midx=incremental` without `--geometric`
- repack: introduce `--write-midx=incremental`
- repack: implement incremental MIDX repacking
- packfile: ensure `close_pack_revindex()` frees in-memory revindex
- builtin/repack.c: convert `--write-midx` to an `OPT_CALLBACK`
- repack-geometry: prepare for incremental MIDX repacking
- repack-midx: extract `repack_fill_midx_stdin_packs()`
- repack-midx: factor out `repack_prepare_midx_command()`
- midx: expose `midx_layer_contains_pack()`
- repack: track the ODB source via existing_packs
- midx: support custom `--base` for incremental MIDX writes
- midx: introduce `--no-write-chain-file` for incremental MIDX writes
- midx: use `strvec` for `keep_hashes`
- midx: build `keep_hashes` array in order
- midx: use `strset` for retained MIDX files
- midx-write: handle noop writes when converting incremental chains
The repacking code has been refactored and compaction of MIDX layers
have been implemented, and incremental strategy that does not require
all-into-one repacking has been introduced.
Will merge to 'next'?
source: <cover.1777507303.git.me@ttaylorr.com>
* jd/unpack-trees-wo-the-repository (2026-03-31) 2 commits
- unpack-trees: use repository from index instead of global
- unpack-trees: use repository from index instead of global
A handful of inappropriate uses of the_repository have been
rewritten to use the right repository structure instance in the
unpack-trees.c codepath.
Comments?
source: <pull.2258.v2.git.git.1774971267.gitgitgadget@gmail.com>
* ps/setup-wo-the-repository (2026-04-20) 18 commits
- setup: stop using `the_repository` in `init_db()`
- setup: stop using `the_repository` in `create_reference_database()`
- setup: stop using `the_repository` in `initialize_repository_version()`
- setup: stop using `the_repository` in `check_repository_format()`
- setup: stop using `the_repository` in `upgrade_repository_format()`
- setup: stop using `the_repository` in `setup_git_directory()`
- setup: stop using `the_repository` in `setup_git_directory_gently()`
- setup: stop using `the_repository` in `setup_git_env()`
- setup: stop using `the_repository` in `set_git_work_tree()`
- setup: stop using `the_repository` in `setup_work_tree()`
- setup: stop using `the_repository` in `enter_repo()`
- setup: stop using `the_repository` in `verify_non_filename()`
- setup: stop using `the_repository` in `verify_filename()`
- setup: stop using `the_repository` in `path_inside_repo()`
- setup: stop using `the_repository` in `prefix_path()`
- setup: stop using `the_repository` in `is_inside_git_dir()`
- setup: stop using `the_repository` in `is_inside_worktree()`
- setup: replace use of `the_repository` in static functions
Many uses of the_repository has been updated to use a more
appropriate struct repository instance in setup.c codepath.
Needs review.
source: <20260420-pks-setup-wo-the-repository-v1-0-f4a81c4988e8@pks.im>
* kh/doc-trailers (2026-04-13) 9 commits
- doc: interpret-trailers: document comment line treatment
- doc: interpret-trailers: commit to “trailer block” term
- doc: interpret-trailers: add key format example
- doc: interpret-trailers: explain key format
- doc: interpret-trailers: explain the format after the intro
- doc: interpret-trailers: not just for commit messages
- doc: interpret-trailers: use “metadata” in Name as well
- doc: interpret-trailers: replace “lines” with “metadata”
- doc: interpret-trailers: stop fixating on RFC 822
Documentation updates.
Needs review.
source: <V2_CV_doc_int-tr_key_format.613@msgid.xyz>
* ps/graph-lane-limit (2026-03-27) 3 commits
- graph: add truncation mark to capped lanes
- graph: add --graph-lane-limit option
- graph: limit the graph width to a hard-coded max
The graph output from commands like "git log --graph" can now be
limited to a specified number of lanes, preventing overly wide output
in repositories with many branches.
Needs review.
cf. <bdff0a5d-b738-4053-9b72-08eba88156de@kdbg.org>
source: <20260328001113.1275291-1-pabloosabaterr@gmail.com>
* cc/promisor-auto-config-url (2026-04-07) 10 commits
(merged to 'next' on 2026-04-13 at 289fcba081)
+ t5710: use proper file:// URIs for absolute paths
+ promisor-remote: remove the 'accepted' strvec
+ promisor-remote: keep accepted promisor_info structs alive
+ promisor-remote: refactor accept_from_server()
+ promisor-remote: refactor has_control_char()
+ promisor-remote: refactor should_accept_remote() control flow
+ promisor-remote: reject empty name or URL in advertised remote
+ promisor-remote: clarify that a remote is ignored
+ promisor-remote: pass config entry to all_fields_match() directly
+ promisor-remote: try accepted remotes before others in get_direct()
(this branch is used by cc/promisor-auto-config-url-more.)
Promisor remote handling has been refactored and fixed in
preparation for auto-configuration of advertised remotes.
Will merge to 'master'.
source: <20260407115243.358642-1-christian.couder@gmail.com>
* jr/bisect-custom-terms-in-output (2026-04-17) 2 commits
- rev-parse: use selected alternate terms to look up refs
- bisect: use selected alternate terms in status output
"git bisect" now uses the selected terms (e.g., old/new) more
consistently in its output.
Needs review.
source: <20260417-bisect-terms-v3-0-d659fa547261@schlaraffenlan.de>
* ua/push-remote-group (2026-04-27) 3 commits
- SQUASH??? -Werror=sign-compare workaround
- push: support pushing to a remote group
- remote: move remote group resolution to remote.c
"git push" learned to take a "remote group" name to push to, which
causes pushes to multiple places, just like "git fetch" would do.
source: <20260427140530.856125-1-usmanakinyemi202@gmail.com>
* hn/git-checkout-m-with-stash (2026-04-28) 5 commits
- checkout -m: autostash when switching branches
- checkout: rollback lock on early returns in merge_working_tree
- sequencer: teach autostash apply to take optional conflict marker labels
- sequencer: allow create_autostash to run silently
- stash: add --label-ours, --label-theirs, --label-base for apply
"git checkout -m another-branch" was invented to deal with local
changes to paths that are different between the current and the new
branch, but it gave only one chance to resolve conflicts. The command
was taught to create a stash to save the local changes.
Will merge to 'next'?
source: <pull.2234.v16.git.git.1777401552.gitgitgadget@gmail.com>
* kh/name-rev-custom-format (2026-04-30) 6 commits
. SQUASH???
. format-rev: introduce builtin for on-demand pretty formatting
. name-rev: make dedicated --annotate-stdin --name-only test
. name-rev: factor code for sharing with a new command
. name-rev: run clang-format before factoring code
. name-rev: wrap both blocks in braces
A new builtin "git format-rev" is introduced for pretty formatting
one revision expression per line or commit object names found in
running text.
Breaks CI.
cf. <8cdfbf18-6137-47c0-8134-4cd9f767fa4f@app.fastmail.com>
source: <V3_CV_format-rev.66a@msgid.xyz>
* js/parseopt-subcommand-autocorrection (2026-04-27) 11 commits
- SQUASH???
- doc: document autocorrect API
- parseopt: add tests for subcommand autocorrection
- parseopt: enable subcommand autocorrection for git-remote and git-notes
- parseopt: autocorrect mistyped subcommands
- autocorrect: provide config resolution API
- autocorrect: rename AUTOCORRECT_SHOW to AUTOCORRECT_HINT
- autocorrect: use mode and delay instead of magic numbers
- help: move tty check for autocorrection to autocorrect.c
- help: make autocorrect handling reusable
- parseopt: extract subcommand handling from parse_options_step()
The parse-options library learned to auto-correct misspelled
subcommand names.
Expecting a reroll.
source: <SY0P300MB0801677A2A1E0FD38D06A841CE2A2@SY0P300MB0801.AUSP300.PROD.OUTLOOK.COM>
* ab/clone-default-object-filter (2026-03-14) 1 commit
- clone: add clone.<url>.defaultObjectFilter config
"git clone" learns to pay attention to "clone.<url>.defaultObjectFilter"
configuration and behave as if the "--filter=<filter-spec>" option
was given on the command line.
Expecting review responses.
cf. <abe1l8ONmFIhzaxi@pks.im>
source: <pull.2058.v6.git.1773553022381.gitgitgadget@gmail.com>
* jc/neuter-sideband-fixup (2026-03-05) 6 commits
(merged to 'next' on 2026-03-13 at 5a4098b0cd)
+ sideband: drop 'default' configuration
+ sideband: offer to configure sanitizing on a per-URL basis
+ sideband: add options to allow more control sequences to be passed through
+ sideband: do allow ANSI color sequences by default
+ sideband: introduce an "escape hatch" to allow control characters
+ sideband: mask control characters
(this branch is used by jc/neuter-sideband-post-3.0.)
Try to resurrect and reboot a stalled "avoid sending risky escape
sequences taken from sideband to the terminal" topic by Dscho. The
plan is to keep it in 'next' long enough to see if anybody screams
with the "everything dropped except for ANSI color escape sequences"
default.
Will keep in 'next' a bit longer than usual.
source: <20260305233452.3727126-1-gitster@pobox.com>
* jc/neuter-sideband-post-3.0 (2026-03-05) 2 commits
- sideband: delay sanitizing by default to Git v3.0
- Merge branch 'jc/neuter-sideband-fixup' into jc/neuter-sideband-post-3.0
(this branch uses jc/neuter-sideband-fixup.)
The final step, split from earlier attempt by Dscho, to loosen the
sideband restriction for now and tighten later at Git v3.0 boundary.
On hold, until jc/neuter-sideband-fixup cooks long enough in 'next'.
(this branch uses jc/neuter-sideband-fixup.)
source: <20260305233452.3727126-8-gitster@pobox.com>
* cs/subtree-split-recursion (2026-03-05) 3 commits
- contrib/subtree: reduce recursion during split
- contrib/subtree: functionalize split traversal
- contrib/subtree: reduce function side-effects
When processing large history graphs on Debian or Ubuntu, "git
subtree" can die with a "recursion depth reached" error.
Comments?
source: <20260305-cs-subtree-split-recursion-v2-0-7266be870ba9@howdoi.land>
* pt/promisor-lazy-fetch-no-recurse (2026-03-11) 1 commit
- promisor-remote: prevent lazy-fetch recursion in child fetch
The mechanism to avoid recursive lazy-fetch from promisor remotes
was not propagated properly to child "git fetch" processes, which
has been corrected.
Will discard.
cf. <xmqqik9s6qvd.fsf@gitster.g>
source: <pull.2224.v3.git.git.1773238778894.gitgitgadget@gmail.com>
* pt/fsmonitor-linux (2026-04-15) 13 commits
- fsmonitor: convert shown khash to strset in do_handle_client
- fsmonitor: add tests for Linux
- fsmonitor: add timeout to daemon stop command
- fsmonitor: close inherited file descriptors and detach in daemon
- run-command: add close_fd_above_stderr option
- fsmonitor: implement filesystem change listener for Linux
- fsmonitor: rename fsm-settings-darwin.c to fsm-settings-unix.c
- fsmonitor: rename fsm-ipc-darwin.c to fsm-ipc-unix.c
- fsmonitor: use pthread_cond_timedwait for cookie wait
- compat/win32: add pthread_cond_timedwait
- fsmonitor: fix hashmap memory leak in fsmonitor_run_daemon
- fsmonitor: fix khash memory leak in do_handle_client
- t9210, t9211: disable GIT_TEST_SPLIT_INDEX for scalar clone tests
The fsmonitor daemon has been implemented for Linux.
Will merge to 'next'?
source: <pull.2147.v15.git.git.1776259657.gitgitgadget@gmail.com>
* ar/parallel-hooks (2026-04-10) 13 commits
(merged to 'next' on 2026-04-13 at 0a6acf0d17)
+ t1800: test SIGPIPE with parallel hooks
+ hook: allow hook.jobs=-1 to use all available CPU cores
+ hook: add hook.<event>.enabled switch
+ hook: move is_known_hook() to hook.c for wider use
+ hook: warn when hook.<friendly-name>.jobs is set
+ hook: add per-event jobs config
+ hook: add -j/--jobs option to git hook run
+ hook: mark non-parallelizable hooks
+ hook: allow pre-push parallel execution
+ hook: allow parallel hook execution
+ hook: parse the hook.jobs config
+ config: add a repo_config_get_uint() helper
+ repository: fix repo_init() memleak due to missing _clear()
Will merge to 'master'.
source: <20260410090608.75283-1-adrian.ratiu@collabora.com>
* pw/xdiff-shrink-memory-consumption (2026-04-02) 4 commits
. xdiff: reduce the size of array
. xprepare: simplify error handling
. xdiff: cleanup xdl_clean_mmatch()
. xdiff: reduce size of action arrays
Shrink wasted memory in Myers diff that does not account for common
prefix and suffix removal.
Needs to be rebased on updated en/xdiff-cleanup-3.
source: <cover.1775141855.git.phillip.wood@dunelm.org.uk>
* en/xdiff-cleanup-3 (2026-04-29) 6 commits
- xdiff/xdl_cleanup_records: make execution of action easier to follow
- xdiff/xdl_cleanup_records: make setting action easier to follow
- xdiff/xdl_cleanup_records: make limits more clear
- xdiff/xdl_cleanup_records: use unambiguous types
- xdiff: use unambiguous types in xdl_bogo_sqrt()
- xdiff/xdl_cleanup_records: delete local recs pointer
Preparation of the xdiff/ codebase to work with Rust.
Will merge to 'next'?
source: <pull.2156.v6.git.git.1777500495.gitgitgadget@gmail.com>
^ permalink raw reply
* Re: [PATCH v3 1/1] git-gui: handle missing worktree and separated gitdir
From: Mark Levedahl @ 2026-05-02 21:51 UTC (permalink / raw)
To: Johannes Sixt, Shroom Moo; +Cc: git
In-Reply-To: <a1a7237c-ffed-4a7a-ae58-55769aaa4453@gmail.com>
On 5/1/26 12:42 PM, Mark Levedahl wrote:
> On 5/1/26 9:13 AM, Johannes Sixt wrote:
>> Am 30.04.26 um 18:18 schrieb Mark Levedahl:
>>> On 4/30/26 6:02 AM, Shroom Moo wrote:
>>> We have quite a bit of code that attempts to make Git GUI work from the
>> .git directory and also in bare repositories.
>>
>> 87cd09f43e56 ("git-gui: work from the .git dir", 2010-01-23) made the
>> first step. The original code just used the $_gitdir as the working
>> directory. However, at that time we did not have alternate worktrees,
>> and the old code, when used today, does not work in a `git
>> worktree`-created worktree. Later, the `git rev-parse --show-toplevel`
>> call came with 38ec8d3e2652 ("git-gui: correct assignment of work-tree",
>> 2010-10-20). However, it also changes the fall-back code slightly, so
>> that running Git GUI from the .git directory would not work the same way
>> as before and takes the .git directory as the work tree (because in the
>> .git directory --show-cdup is not "..", but empty).
>>
>> I think we need to restructure the existing flow a bit and not just fix
>> a single spot in the code. I suggest this order of operation:
>>
>> 1. Handle the bare repository case. If not enabled, fail. Otherwise, we
>> can work with an empty $_gitworktree.
>>
>> 2. Collect --show-toplevel into $_gitworktree.
>>
>> 2a. If this failed: If --is-inside-git-dir is true, and the last
>> $_gitdir directory component is exactly ".git", take the parent
>> repository as $_gitworktree. Otherwise, fail.
>>
>> 3. Handle all the other edge cases, if any, with the so determined
>> $_gitworktree. (I didn't think through, yet, what needs to be done.)
>>
>> -- Hannes
>>
> I found one horrid edge case:
>
> Start git-gui in a gitdir not embedded in a worktree, with core.bare=false as there are
> one or more gitfile and/or symlinked worktrees elsewhere.
> - current git-gui aborts with an uncaught error. Good.
> - git-gui with the wrapped --show-toplevel call finds no worktree to switch to, so runs in
> the gitdir allowing commits of the gitdir items.
> - I just added and committed the *file* refs/heads/master to branch master in such a gitdir.
>
> git-gui's normal gui must be started ONLY if rev-parse --is-inside-work-tree is true. (The
> blame view invoked by gitk in theory could be allowed to run in a bare repository
> read-only mode.).
>
> For read/write mode:
> if --is-inside-git-dir == true at startup, we must abort, or find a valid worktree and
> switch to that.
>
> My personal preference is for git-gui to abort:
> I started git-gui where it cannot run.
> My error. Let me learn and fix that.
>
> Alternatively, ask me what to do:
> e.g., prepare a dialog after looking at git-worktree list, and the parent dir IFF this
> dir is named .git, telling me of my mistake and offering me one or more worktrees to
> switch to.
>
> But please, don't just switch to another directory without asking. This is just
> encouraging me to make careless errors.
>
> Mark
>
I dug a bit more into the startup logic, and I think I better understand rework that is
needed.
Two basic problems I see here are beyond the question of if (and when) git-gui should try
to locate a worktree:
- git gui blame in a gitdir was apparently broken by the git repo commit 2d92ab32fd
("rev-parse: make --show-toplevel without a worktree an error", 2019-11-19). Prior to that
commit, git gui would stay in the startup directory enabling only features that cannot
modify the repository, and gitk could bring this view up in a gitdir. This doesn't work
right now.
- git-gui's logic includes a conceptual error embodied in proc is_bare: is_bare uses $(git
rev-parse --is-bare-repository) but what we need is $(git rev-parse --is-inside-git-dir),
and these are not synonyms.
It does not matter whether a worktree exists that points at the gitdir, and as discussed
before, main worktrees can easily exist that we cannot locate from the gitdir. At best,
is_bare is a guess.
So, is_bare should be replaced by is_inside_gitdir, and we should also have
is_inside_worktree. These are mutually exclusive, though both can be false.
My current idea of an improved startup flow enables features only at the end:
1) If not in a gitdir or worktree,
1a) if git gui's subcommand is not 'gui', abort with an error (citool, browser, or
blame invocations carry information specific to a gitdir/worktree).
1b) otherwise, invoke repository_chooser, which either aborts, or changes directory
to a worktree.
-- we are now in a gitdir or a worktree, and this may or may not be the startup directory.
2) Look at the combination of git gui subcommand and directory type (worktree / gitdir) to
decide to continue.
2a) blame / browser are ok in either directory type.
2b) citool requires a specific worktree, which should have been the initial startup
directory. Abort if not.
2b) gui requires a worktree. Abort if not (my recommendation), or offer to find (or
automatically find) a worktree.
3) Change directory to the top level of of the directory_type (git rev-parse knows
toplevel of a worktree, different code is needed for a gitdir).
4) Enable features based upon subcommand and directory type.
There are 12 combinations of initial directory type (gitdir, worktree, neither) and
subcommand (gui, blame, browser, citool) to consider, with a lot of duplicated code
amongst the 12 cases. So, obviously, steps 1 and 2 can be convolved in many ways that are
different than what I wrote above.
Mark
^ permalink raw reply
* Re: [Bug] fetch --deepen truncates history in v2.54.0
From: René Scharfe @ 2026-05-02 20:26 UTC (permalink / raw)
To: Owen Stephens, git; +Cc: Samo Pogačnik
In-Reply-To: <a5fd970d-fd78-41bc-98f8-a6a87a7f39cc@web.de>
On 5/2/26 11:22 AM, René Scharfe wrote:
> On 4/29/26 1:27 PM, Owen Stephens wrote:
>>> What did you do before the bug happened? (Steps to reproduce your issue)
>>
>> Repeatedy called `git fetch --deepen 2` inside a shallow repo that was a
>> file:// clone of another repo. Once all commits had been fetched, a subsequent
>> `fetch --deepen` appears to "reset" the repo back to being shallow with a depth
>> of 2. A reproduction script is included below. This issue appears to have been
>> introduced in v2.54.0.
>>
>>> What did you expect to happen? (Expected behavior)
>>
>> I expected `git fetch --deepen` in a non-shallow repo with no upstream commits
>> to be a no-op.
>>
>>> What happened instead? (Actual behavior)
>>
>> `git log` history is truncated to two commits, and repo is considered shallow
>> by `git rev-parse --is-shallow-repository`.
>>
>>> What's different between what you expected and what actually happened?
>>
>> The previously-present commits in `git log` are missing, and the repo is again
>> considered shallow.
>>
>>> Anything else you want to add:
>>
>> Commit 3ef68ff seems relevant.
>
> Indeed, bisect identifies 3ef68ff40e (shallow: handling fetch relative-deepen,
> 2026-02-15) and reverting it fixes the issue. Copying its author.
Here's a simple fix, but it feels like cheating. A proper one should
live in shallow.c, no?
diff --git a/builtin/fetch.c b/builtin/fetch.c
index a22c319467..310099b96d 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -2664,7 +2664,8 @@ int cmd_fetch(int argc,
die(_("negative depth in --deepen is not supported"));
if (depth)
die(_("options '%s' and '%s' cannot be used together"), "--deepen", "--depth");
- depth = xstrfmt("%d", deepen_relative);
+ if (is_repository_shallow(the_repository))
+ depth = xstrfmt("%d", deepen_relative);
}
if (unshallow) {
if (depth)
^ permalink raw reply related
* [ANNOUNCE] Git Rev News edition 134
From: Christian Couder @ 2026-05-02 18:30 UTC (permalink / raw)
To: git
Cc: Junio C Hamano, Jakub Narebski, Markus Jansen, Kaartic Sivaraam,
Štěpán Němec, Taylor Blau,
Johannes Schindelin, Michael Montalbo, Kristoffer Haugsbakk,
Matthew Hughes, Paulo Gomes, Toon Claes, Patrick Steinhardt, lwn,
Meet Soni
Hi everyone,
The 134th edition of Git Rev News is now published:
https://git.github.io/rev_news/2026/04/30/edition-134/
Thanks a lot to Meet Soni, Toon Claes and Paulo Gomes who helped this month!
Enjoy,
Christian, Jakub, Markus and Kaartic.
PS: An issue for the next edition is already opened and contributions
are welcome:
https://github.com/git/git.github.io/issues/844
^ permalink raw reply
* [PATCH 7/7] pack-objects: support sparse:oid filter with path-walk
From: Derrick Stolee via GitGitGadget @ 2026-05-02 14:15 UTC (permalink / raw)
To: git
Cc: christian.couder, gitster, johannes.schindelin, johncai86,
karthik.188, kristofferhaugsbakk, me, newren, peff, ps,
Derrick Stolee, Derrick Stolee
In-Reply-To: <pull.2101.git.1777731354.gitgitgadget@gmail.com>
From: Derrick Stolee <stolee@gmail.com>
The --filter=sparse:<oid> option to 'git pack-objects' allows focusing
an object set to a sparse-checkout definition. This reduces the set of
matching blobs while retaining all reachable trees. No server currently
supports fetching with this filter because it is expensive to compute
and reachability bitmaps do not help without a significant effort to
extend the bitmap feature to store bitmaps for each supported sparse-
checkout definition.
Without focusing on serving fetches and clones with these filters, there
are still benefits that could be realized by making this faster. With
the sparse index, it's more realistic now than ever to be able to
operate a local clone that was bootstrapped by a packfile created with
a sparse filter, because the missing trees are not needed to move a
sparse-checkout from one commit to another or to view the history of any
path in scope. Such clones could perhaps be bootstrapped by partial
bundles.
Previously, constructing these sparse packs has been incredibly
computationally inefficient. The revision walk that explores which
objects are in scope spends a lot of time checking each object to see if
it matches the sparse-checkout patterns, causing quadratic behavior
(number of objects times number of sparse-checkout patterns). This
improves somewhat when using cone-mode sparse-checkout patterns that can
use hashtables and prefix matches to determine containment. However, the
check per object is still too expensive for most cases.
This is where the path-walk feature comes in. We can proceed as normal
by placing objects in bins by path and _then_ check a group of objects
all at once. Since sparse:<oid> only restricts blobs, the path-walk must
include all reachable trees while using the cone-mode patterns to skip
blobs at paths outside the sparse scope. This establishes a baseline for
a potential future "treesparse:<oid>" filter that would also restrict
trees, but introducing such a new filter is deferred to a later change.
The implementation here is focused around loading the sparse-checkout
patterns from the provided object ID and checking that the patterns are
indeed cone-mode patterns. We can then load the correct pattern list
into the path walk context and use the logic that already exists from
bff45557675 (backfill: add --sparse option, 2025-02-03), though that
feature loads sparse-checkout patterns from the worktree's local
settings and also restricts tree objects. We use a combination of errors
and warnings to signal problems during this load. The difference is that
errors are likely fatal for the non-path-walk version while the warnings
are probably just implementation details for the path-walk version and
the 'git pack-objects' command can fall back to the revision walk
version.
Now that the SEEN flag is deferred until after pattern checks (from the
previous commit), handle the case where a tree with a shared OID appears
at both an out-of-cone and in-cone path. When trees are not being pruned
(pl_sparse_trees == 0), the path-walk re-walks the tree at the in-cone
path so that in-cone blobs within it are discovered. The new tests in
t5317 and t6601 demonstrate this behavior and would fail without these
changes.
The performance test p5315 shows the impact of this change when using
sparse filters:
Test HEAD~1 HEAD
----------------------------------------------------------------------
5315.10: repack (sparse:oid) 77.98 77.47 -0.7%
5315.11: repack size (sparse:oid) 187.5M 187.4M -0.0%
5315.12: repack (sparse:oid, --path-walk) 77.91 31.41 -59.7%
5315.13: repack size (sparse:oid, --path-walk) 187.5M 161.1M -14.1%
These performance tests were run on the Git repository. The --path-walk
feature shows meaningful space savings (14% smaller for sparse packs)
and dramatic time savings (60% faster) by leveraging the path-walk's
ability to skip blobs outside the sparse scope.
Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
builtin/pack-objects.c | 2 +
path-walk.c | 81 ++++++++++++++-
t/t5317-pack-objects-filter-objects.sh | 125 +++++++++++++++++++++++
t/t6601-path-walk.sh | 131 +++++++++++++++++++++++++
4 files changed, 335 insertions(+), 4 deletions(-)
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index bc9fb5b457..ba00d8148a 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -4777,6 +4777,8 @@ static void get_object_list_path_walk(struct rev_info *revs)
result = walk_objects_by_path(&info);
trace2_region_leave("pack-objects", "path-walk", revs->repo);
+ path_walk_info_clear(&info);
+
if (result)
die(_("failed to pack objects via path-walk"));
}
diff --git a/path-walk.c b/path-walk.c
index c25392b901..700617ee2f 100644
--- a/path-walk.c
+++ b/path-walk.c
@@ -10,6 +10,7 @@
#include "hex.h"
#include "list-objects.h"
#include "list-objects-filter-options.h"
+#include "object-name.h"
#include "odb.h"
#include "object.h"
#include "oid-array.h"
@@ -180,10 +181,6 @@ static int add_tree_entries(struct path_walk_context *ctx,
return -1;
}
- /* Skip this object if already seen. */
- if (o->flags & SEEN)
- continue;
-
strbuf_setlen(&path, base_len);
strbuf_add(&path, entry.path, entry.pathlen);
@@ -194,6 +191,40 @@ static int add_tree_entries(struct path_walk_context *ctx,
if (type == OBJ_TREE)
strbuf_addch(&path, '/');
+ if (o->flags & SEEN) {
+ /*
+ * A tree with a shared OID may appear at multiple
+ * paths. Even though we already added this tree to
+ * the output at some other path, we still need to
+ * walk into it at this in-cone path to discover
+ * blobs that were not found at the earlier
+ * out-of-cone path.
+ *
+ * Only do this for paths not yet in our map, to
+ * avoid duplicate entries when the same tree OID
+ * appears at the same path across multiple commits.
+ */
+ if (type == OBJ_TREE && ctx->info->pl &&
+ ctx->info->pl->use_cone_patterns &&
+ !ctx->info->pl_sparse_trees &&
+ !strmap_contains(&ctx->paths_to_lists, path.buf)) {
+ int dtype;
+ enum pattern_match_result m;
+ m = path_matches_pattern_list(path.buf, path.len,
+ path.buf + base_len,
+ &dtype,
+ ctx->info->pl,
+ ctx->repo->index);
+ if (m != NOT_MATCHED) {
+ add_path_to_list(ctx, path.buf, type,
+ &entry.oid,
+ !(o->flags & UNINTERESTING));
+ push_to_stack(ctx, path.buf);
+ }
+ }
+ continue;
+ }
+
if (ctx->info->pl) {
int dtype;
enum pattern_match_result match;
@@ -533,6 +564,48 @@ static int prepare_filters(struct path_walk_info *info,
}
return 1;
+ case LOFC_SPARSE_OID:
+ if (info) {
+ struct object_id sparse_oid;
+ struct repository *repo = info->revs->repo;
+
+ if (info->pl) {
+ warning(_("sparse filter cannot be combined with existing sparse patterns"));
+ return 0;
+ }
+
+ if (repo_get_oid_with_flags(repo,
+ options->sparse_oid_name,
+ &sparse_oid,
+ GET_OID_BLOB)) {
+ error(_("unable to access sparse blob in '%s'"),
+ options->sparse_oid_name);
+ return 0;
+ }
+
+ CALLOC_ARRAY(info->pl, 1);
+ info->pl->use_cone_patterns = 1;
+
+ if (add_patterns_from_blob_to_list(&sparse_oid, "", 0,
+ info->pl) < 0) {
+ clear_pattern_list(info->pl);
+ FREE_AND_NULL(info->pl);
+ error(_("unable to parse sparse filter data in '%s'"),
+ oid_to_hex(&sparse_oid));
+ return 0;
+ }
+
+ if (!info->pl->use_cone_patterns) {
+ clear_pattern_list(info->pl);
+ FREE_AND_NULL(info->pl);
+ warning(_("sparse filter is not cone-mode compatible"));
+ return 0;
+ }
+
+ list_objects_filter_release(options);
+ }
+ return 1;
+
default:
error(_("object filter '%s' not supported by the path-walk API"),
list_objects_filter_spec(options));
diff --git a/t/t5317-pack-objects-filter-objects.sh b/t/t5317-pack-objects-filter-objects.sh
index 501d715b9a..dddb79ba62 100755
--- a/t/t5317-pack-objects-filter-objects.sh
+++ b/t/t5317-pack-objects-filter-objects.sh
@@ -478,4 +478,129 @@ test_expect_success 'verify pack-objects w/ --missing=allow-any' '
EOF
'
+# Test that --path-walk produces the same object set as standard traversal
+# when using sparse:oid filters with cone-mode patterns.
+#
+# The sparse:oid filter restricts only blobs, not trees. Both standard
+# and path-walk should produce identical sets of blobs, commits, and trees.
+
+test_expect_success 'setup pw_sparse for path-walk comparison' '
+ git init pw_sparse &&
+ mkdir -p pw_sparse/inc/sub pw_sparse/exc/sub &&
+
+ for n in 1 2
+ do
+ echo "inc $n" >pw_sparse/inc/file$n &&
+ echo "inc sub $n" >pw_sparse/inc/sub/file$n &&
+ echo "exc $n" >pw_sparse/exc/file$n &&
+ echo "exc sub $n" >pw_sparse/exc/sub/file$n &&
+ echo "root $n" >pw_sparse/root$n || return 1
+ done &&
+
+ git -C pw_sparse add . &&
+ git -C pw_sparse commit -m "first" &&
+
+ echo "inc 1 modified" >pw_sparse/inc/file1 &&
+ echo "exc 1 modified" >pw_sparse/exc/file1 &&
+ echo "root 1 modified" >pw_sparse/root1 &&
+ git -C pw_sparse add . &&
+ git -C pw_sparse commit -m "second" &&
+
+ # Cone-mode sparse pattern: include root + inc/
+ printf "/*\n!/*/\n/inc/\n" |
+ git -C pw_sparse hash-object -w --stdin >sparse_oid
+'
+
+test_expect_success 'sparse:oid with --path-walk produces same blobs' '
+ oid=$(cat sparse_oid) &&
+
+ git -C pw_sparse pack-objects --revs --stdout \
+ --filter=sparse:oid=$oid >standard.pack <<-EOF &&
+ HEAD
+ EOF
+ git -C pw_sparse index-pack ../standard.pack &&
+ git -C pw_sparse verify-pack -v ../standard.pack >standard_verify &&
+
+ git -C pw_sparse pack-objects --revs --stdout \
+ --path-walk --filter=sparse:oid=$oid >pathwalk.pack <<-EOF &&
+ HEAD
+ EOF
+ git -C pw_sparse index-pack ../pathwalk.pack &&
+ git -C pw_sparse verify-pack -v ../pathwalk.pack >pathwalk_verify &&
+
+ # Blobs must match exactly
+ grep -E "^[0-9a-f]{40} blob" standard_verify |
+ awk "{print \$1}" | sort >standard_blobs &&
+ grep -E "^[0-9a-f]{40} blob" pathwalk_verify |
+ awk "{print \$1}" | sort >pathwalk_blobs &&
+ test_cmp standard_blobs pathwalk_blobs &&
+
+ # Commits must match exactly
+ grep -E "^[0-9a-f]{40} commit" standard_verify |
+ awk "{print \$1}" | sort >standard_commits &&
+ grep -E "^[0-9a-f]{40} commit" pathwalk_verify |
+ awk "{print \$1}" | sort >pathwalk_commits &&
+ test_cmp standard_commits pathwalk_commits
+'
+
+test_expect_success 'sparse:oid with --path-walk includes all trees' '
+ # The sparse:oid filter restricts only blobs, not trees.
+ # Both standard and path-walk should include the same trees.
+ grep -E "^[0-9a-f]{40} tree" standard_verify |
+ awk "{print \$1}" | sort >standard_trees &&
+ grep -E "^[0-9a-f]{40} tree" pathwalk_verify |
+ awk "{print \$1}" | sort >pathwalk_trees &&
+
+ test_cmp standard_trees pathwalk_trees
+'
+
+# Test the edge case where the same tree/blob OID appears at both an
+# in-cone and out-of-cone path. When sibling directories have identical
+# contents, they share a tree OID. The path-walk defers marking objects
+# SEEN until after checking sparse patterns, so an object at an out-of-cone
+# path can still be discovered at an in-cone path.
+
+test_expect_success 'setup pw_shared for shared OID across cone boundary' '
+ git init pw_shared &&
+ mkdir pw_shared/aaa pw_shared/zzz &&
+ echo "shared content" >pw_shared/aaa/file &&
+ echo "shared content" >pw_shared/zzz/file &&
+ echo "root file" >pw_shared/rootfile &&
+ git -C pw_shared add . &&
+ git -C pw_shared commit -m "aaa and zzz share tree OID" &&
+
+ # Verify they share a tree OID
+ aaa_tree=$(git -C pw_shared rev-parse HEAD:aaa) &&
+ zzz_tree=$(git -C pw_shared rev-parse HEAD:zzz) &&
+ test "$aaa_tree" = "$zzz_tree" &&
+
+ # Cone pattern: include root + zzz/ (not aaa/)
+ printf "/*\n!/*/\n/zzz/\n" |
+ git -C pw_shared hash-object -w --stdin >shared_sparse_oid
+'
+
+test_expect_success 'shared tree OID: --path-walk blobs match standard' '
+ oid=$(cat shared_sparse_oid) &&
+
+ git -C pw_shared pack-objects --revs --stdout \
+ --filter=sparse:oid=$oid >shared_std.pack <<-EOF &&
+ HEAD
+ EOF
+ git -C pw_shared index-pack ../shared_std.pack &&
+ git -C pw_shared verify-pack -v ../shared_std.pack >shared_std_verify &&
+
+ git -C pw_shared pack-objects --revs --stdout \
+ --path-walk --filter=sparse:oid=$oid >shared_pw.pack <<-EOF &&
+ HEAD
+ EOF
+ git -C pw_shared index-pack ../shared_pw.pack &&
+ git -C pw_shared verify-pack -v ../shared_pw.pack >shared_pw_verify &&
+
+ grep -E "^[0-9a-f]{40} blob" shared_std_verify |
+ awk "{print \$1}" | sort >shared_std_blobs &&
+ grep -E "^[0-9a-f]{40} blob" shared_pw_verify |
+ awk "{print \$1}" | sort >shared_pw_blobs &&
+ test_cmp shared_std_blobs shared_pw_blobs
+'
+
test_done
diff --git a/t/t6601-path-walk.sh b/t/t6601-path-walk.sh
index 1126afaea1..520269dfc6 100755
--- a/t/t6601-path-walk.sh
+++ b/t/t6601-path-walk.sh
@@ -590,4 +590,135 @@ test_expect_success 'all, blob:limit=3 filter' '
test_cmp_sorted expect out
'
+test_expect_success 'setup sparse filter blob' '
+ # Cone-mode patterns: include root, exclude all dirs, include left/
+ cat >patterns <<-\EOF &&
+ /*
+ !/*/
+ /left/
+ EOF
+ sparse_oid=$(git hash-object -w -t blob patterns)
+'
+
+test_expect_success 'all, sparse:oid filter' '
+ test-tool path-walk --filter=sparse:oid=$sparse_oid -- --all >out &&
+
+ cat >expect <<-EOF &&
+ 0:commit::$(git rev-parse topic)
+ 0:commit::$(git rev-parse base)
+ 0:commit::$(git rev-parse base~1)
+ 0:commit::$(git rev-parse base~2)
+ 1:tag:/tags:$(git rev-parse refs/tags/first)
+ 1:tag:/tags:$(git rev-parse refs/tags/second.1)
+ 1:tag:/tags:$(git rev-parse refs/tags/second.2)
+ 1:tag:/tags:$(git rev-parse refs/tags/third)
+ 1:tag:/tags:$(git rev-parse refs/tags/fourth)
+ 1:tag:/tags:$(git rev-parse refs/tags/tree-tag)
+ 1:tag:/tags:$(git rev-parse refs/tags/blob-tag)
+ 2:blob:/tagged-blobs:$(git rev-parse refs/tags/blob-tag^{})
+ 2:blob:/tagged-blobs:$(git rev-parse refs/tags/blob-tag2^{})
+ 3:tree::$(git rev-parse topic^{tree})
+ 3:tree::$(git rev-parse base^{tree})
+ 3:tree::$(git rev-parse base~1^{tree})
+ 3:tree::$(git rev-parse base~2^{tree})
+ 3:tree::$(git rev-parse refs/tags/tree-tag^{})
+ 3:tree::$(git rev-parse refs/tags/tree-tag2^{})
+ 4:blob:a:$(git rev-parse base~2:a)
+ 5:blob:file2:$(git rev-parse refs/tags/tree-tag2^{}:file2)
+ 6:tree:a/:$(git rev-parse base:a)
+ 7:tree:child/:$(git rev-parse refs/tags/tree-tag:child)
+ 8:tree:left/:$(git rev-parse base:left)
+ 8:tree:left/:$(git rev-parse base~2:left)
+ 9:blob:left/b:$(git rev-parse base~2:left/b)
+ 9:blob:left/b:$(git rev-parse base:left/b)
+ 10:tree:right/:$(git rev-parse topic:right)
+ 10:tree:right/:$(git rev-parse base~1:right)
+ 10:tree:right/:$(git rev-parse base~2:right)
+ blobs:6
+ commits:4
+ tags:7
+ trees:13
+ EOF
+
+ test_cmp_sorted expect out
+'
+
+test_expect_success 'topic only, sparse:oid filter' '
+ test-tool path-walk --filter=sparse:oid=$sparse_oid -- topic >out &&
+
+ cat >expect <<-EOF &&
+ 0:commit::$(git rev-parse topic)
+ 0:commit::$(git rev-parse base~1)
+ 0:commit::$(git rev-parse base~2)
+ 1:tree::$(git rev-parse topic^{tree})
+ 1:tree::$(git rev-parse base~1^{tree})
+ 1:tree::$(git rev-parse base~2^{tree})
+ 2:blob:a:$(git rev-parse base~2:a)
+ 3:tree:left/:$(git rev-parse base~2:left)
+ 4:blob:left/b:$(git rev-parse base~2:left/b)
+ 5:tree:right/:$(git rev-parse topic:right)
+ 5:tree:right/:$(git rev-parse base~1:right)
+ 5:tree:right/:$(git rev-parse base~2:right)
+ blobs:2
+ commits:3
+ tags:0
+ trees:7
+ EOF
+
+ test_cmp_sorted expect out
+'
+
+# Demonstrate the SEEN flag ordering issue: when the same tree/blob OID
+# appears at two sibling paths where one is in-cone and the other is
+# out-of-cone, the path-walk must still discover blobs at the in-cone
+# path even when the shared tree OID was first encountered out-of-cone.
+# Since sparse:oid includes all trees, the out-of-cone tree (aaa/) is
+# walked first, and its blob is skipped. The path-walk then re-walks
+# the same tree OID at the in-cone path (zzz/) to find the blob there.
+
+test_expect_success 'setup shared tree OID across cone boundary' '
+ git checkout --orphan shared-tree &&
+ git rm -rf . &&
+ mkdir aaa zzz &&
+ echo "shared content" >aaa/file &&
+ echo "shared content" >zzz/file &&
+ echo "root file" >rootfile &&
+ git add aaa zzz rootfile &&
+ git commit -m "aaa and zzz have same tree OID" &&
+
+ # Verify they really share a tree OID
+ aaa_tree=$(git rev-parse HEAD:aaa) &&
+ zzz_tree=$(git rev-parse HEAD:zzz) &&
+ test "$aaa_tree" = "$zzz_tree" &&
+
+ # Cone pattern: include root + zzz/ (not aaa/)
+ cat >shared-patterns <<-\EOF &&
+ /*
+ !/*/
+ /zzz/
+ EOF
+ shared_sparse_oid=$(git hash-object -w -t blob shared-patterns)
+'
+
+test_expect_success 'sparse:oid with shared tree OID across cone boundary' '
+ test-tool path-walk \
+ --filter=sparse:oid=$shared_sparse_oid \
+ -- shared-tree >out &&
+
+ cat >expect <<-EOF &&
+ 0:commit::$(git rev-parse shared-tree)
+ 1:tree::$(git rev-parse shared-tree^{tree})
+ 2:blob:rootfile:$(git rev-parse shared-tree:rootfile)
+ 3:tree:aaa/:$(git rev-parse shared-tree:aaa)
+ 4:tree:zzz/:$(git rev-parse shared-tree:zzz)
+ 5:blob:zzz/file:$(git rev-parse shared-tree:zzz/file)
+ blobs:2
+ commits:1
+ tags:0
+ trees:3
+ EOF
+
+ test_cmp_sorted expect out
+'
+
test_done
--
gitgitgadget
^ permalink raw reply related
* [PATCH 6/7] path-walk: add pl_sparse_trees to control tree pruning
From: Derrick Stolee via GitGitGadget @ 2026-05-02 14:15 UTC (permalink / raw)
To: git
Cc: christian.couder, gitster, johannes.schindelin, johncai86,
karthik.188, kristofferhaugsbakk, me, newren, peff, ps,
Derrick Stolee, Derrick Stolee
In-Reply-To: <pull.2101.git.1777731354.gitgitgadget@gmail.com>
From: Derrick Stolee <stolee@gmail.com>
The path-walk API prunes trees and blobs when a sparse-checkout pattern
list is provided, which is the correct behavior for 'git backfill
--sparse' since it only needs to fill in objects at paths within the
sparse cone.
However, a future change will use the path-walk API with a sparse:<oid>
filter that restricts only blobs while retaining all reachable trees.
To support both behaviors, add a 'pl_sparse_trees' flag to
path_walk_info. When set (as in 'git backfill --sparse' and the
--stdin-pl test helper mode), the sparse patterns prune both trees and
blobs. When unset, only blobs are filtered and all trees are walked and
reported.
Additionally, move the SEEN flag assignment in add_tree_entries() to
after the sparse pattern and pathspec checks. Previously, SEEN was set
immediately upon discovering an object, before checking whether its path
matched the sparse patterns. When the same object ID appeared at
multiple paths (e.g. sibling directories with identical contents), the
first path to be visited would mark the object as SEEN. If that path was
outside the sparse cone, the object would be skipped there but also
never discovered at its in-cone path.
By deferring the SEEN flag until after the checks pass, objects that are
skipped due to sparse filtering remain discoverable at other paths where
they may be in scope.
Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
builtin/backfill.c | 1 +
path-walk.c | 5 +++--
path-walk.h | 6 ++++++
t/helper/test-path-walk.c | 6 +++++-
t/t6601-path-walk.sh | 37 +++++++++++++++++++++++++++++++++++++
5 files changed, 52 insertions(+), 3 deletions(-)
diff --git a/builtin/backfill.c b/builtin/backfill.c
index 7ef9dc305e..b661c34f7c 100644
--- a/builtin/backfill.c
+++ b/builtin/backfill.c
@@ -85,6 +85,7 @@ static int do_backfill(struct backfill_context *ctx)
if (ctx->sparse) {
CALLOC_ARRAY(info.pl, 1);
+ info.pl_sparse_trees = 1;
if (get_sparse_checkout_patterns(info.pl)) {
path_walk_info_clear(&info);
return error(_("problem loading sparse-checkout"));
diff --git a/path-walk.c b/path-walk.c
index 0e7dab7a6a..c25392b901 100644
--- a/path-walk.c
+++ b/path-walk.c
@@ -183,7 +183,6 @@ static int add_tree_entries(struct path_walk_context *ctx,
/* Skip this object if already seen. */
if (o->flags & SEEN)
continue;
- o->flags |= SEEN;
strbuf_setlen(&path, base_len);
strbuf_add(&path, entry.path, entry.pathlen);
@@ -204,7 +203,8 @@ static int add_tree_entries(struct path_walk_context *ctx,
ctx->repo->index);
if (ctx->info->pl->use_cone_patterns &&
- match == NOT_MATCHED)
+ match == NOT_MATCHED &&
+ (type == OBJ_BLOB || ctx->info->pl_sparse_trees))
continue;
else if (!ctx->info->pl->use_cone_patterns &&
type == OBJ_BLOB &&
@@ -239,6 +239,7 @@ static int add_tree_entries(struct path_walk_context *ctx,
continue;
}
+ o->flags |= SEEN;
add_path_to_list(ctx, path.buf, type, &entry.oid,
!(o->flags & UNINTERESTING));
diff --git a/path-walk.h b/path-walk.h
index bcb81b70a1..5fa3ff46b4 100644
--- a/path-walk.h
+++ b/path-walk.h
@@ -72,8 +72,14 @@ struct path_walk_info {
* of the cone. If not in cone mode, then all tree paths will be
* explored but the path_fn will only be called when the path matches
* the sparse-checkout patterns.
+ *
+ * When 'pl_sparse_trees' is zero, the sparse patterns only restrict
+ * blobs and all trees are included in the walk output. This matches
+ * the behavior of the sparse:oid object filter. When nonzero, trees
+ * are also pruned by the sparse patterns (as used by backfill).
*/
struct pattern_list *pl;
+ int pl_sparse_trees;
};
#define PATH_WALK_INFO_INIT { \
diff --git a/t/helper/test-path-walk.c b/t/helper/test-path-walk.c
index 88f86ae0dc..3f2b50a9aa 100644
--- a/t/helper/test-path-walk.c
+++ b/t/helper/test-path-walk.c
@@ -68,7 +68,7 @@ static int emit_block(const char *path, struct oid_array *oids,
int cmd__path_walk(int argc, const char **argv)
{
- int res, stdin_pl = 0;
+ int res, stdin_pl = 0, pl_sparse_trees = -1;
struct rev_info revs = REV_INFO_INIT;
struct path_walk_info info = PATH_WALK_INFO_INIT;
struct path_walk_test_data data = { 0 };
@@ -89,6 +89,8 @@ int cmd__path_walk(int argc, const char **argv)
N_("toggle aggressive edge walk")),
OPT_BOOL(0, "stdin-pl", &stdin_pl,
N_("read a pattern list over stdin")),
+ OPT_BOOL(0, "pl-sparse-trees", &pl_sparse_trees,
+ N_("toggle pruning of trees by sparse patterns")),
OPT_PARSE_LIST_OBJECTS_FILTER(&filter_options),
OPT_END(),
};
@@ -116,6 +118,8 @@ int cmd__path_walk(int argc, const char **argv)
if (stdin_pl) {
struct strbuf in = STRBUF_INIT;
CALLOC_ARRAY(info.pl, 1);
+ info.pl_sparse_trees = (pl_sparse_trees >= 0) ?
+ pl_sparse_trees : 1;
info.pl->use_cone_patterns = 1;
diff --git a/t/t6601-path-walk.sh b/t/t6601-path-walk.sh
index d9be7b9cd2..1126afaea1 100755
--- a/t/t6601-path-walk.sh
+++ b/t/t6601-path-walk.sh
@@ -206,6 +206,43 @@ test_expect_success 'base & topic, sparse' '
test_cmp_sorted expect out
'
+test_expect_success 'base & topic, sparse, no tree pruning' '
+ cat >patterns <<-EOF &&
+ /*
+ !/*/
+ /left/
+ EOF
+
+ test-tool path-walk --stdin-pl --no-pl-sparse-trees \
+ -- base topic <patterns >out &&
+
+ cat >expect <<-EOF &&
+ 0:commit::$(git rev-parse topic)
+ 0:commit::$(git rev-parse base)
+ 0:commit::$(git rev-parse base~1)
+ 0:commit::$(git rev-parse base~2)
+ 1:tree::$(git rev-parse topic^{tree})
+ 1:tree::$(git rev-parse base^{tree})
+ 1:tree::$(git rev-parse base~1^{tree})
+ 1:tree::$(git rev-parse base~2^{tree})
+ 2:blob:a:$(git rev-parse base~2:a)
+ 3:tree:a/:$(git rev-parse base:a)
+ 4:tree:left/:$(git rev-parse base:left)
+ 4:tree:left/:$(git rev-parse base~2:left)
+ 5:blob:left/b:$(git rev-parse base~2:left/b)
+ 5:blob:left/b:$(git rev-parse base:left/b)
+ 6:tree:right/:$(git rev-parse topic:right)
+ 6:tree:right/:$(git rev-parse base~1:right)
+ 6:tree:right/:$(git rev-parse base~2:right)
+ blobs:3
+ commits:4
+ tags:0
+ trees:10
+ EOF
+
+ test_cmp_sorted expect out
+'
+
test_expect_success 'topic only' '
test-tool path-walk -- topic >out &&
--
gitgitgadget
^ permalink raw reply related
* [PATCH 5/7] path-walk: support blob size limit filter
From: Derrick Stolee via GitGitGadget @ 2026-05-02 14:15 UTC (permalink / raw)
To: git
Cc: christian.couder, gitster, johannes.schindelin, johncai86,
karthik.188, kristofferhaugsbakk, me, newren, peff, ps,
Derrick Stolee, Derrick Stolee
In-Reply-To: <pull.2101.git.1777731354.gitgitgadget@gmail.com>
From: Derrick Stolee <stolee@gmail.com>
Extend the path-walk API to handle the 'blob:limit=<size>' object
filter natively. This filter omits blobs whose size is equal to or
greater than the given limit, matching the semantics used by the
list-objects-filter machinery.
When revs->filter.choice is LOFC_BLOB_LIMIT, the prepare_filters()
method stores the limit value in info->blob_limit and clears the filter
from revs. If the limit is zero, this degenerates to blob:none (all
blobs excluded), so info->blobs is set to 0 instead.
During walk_path(), blob batches are filtered before being delivered to
the callback: each blob's size is checked via odb_read_object_info(),
and only blobs strictly smaller than the limit are included. Blobs whose
size cannot be determined (e.g. missing in a partial clone) are
conservatively included, matching the existing filter behavior. Empty
batches after filtering are skipped entirely.
The check for inclusion in the path batch looks a little strange at
first glance. We use odb_read_object_info() to read the object's size.
Based on all of the assumptions to this point, this _should_ return
OBJ_BLOB. Since we are focused on the size filter, we use a
short-circuited OR (||) to skip the size check if that method returns a
different object type.
Notice that this inspection of object sizes requires the content to be
present in the repository. The odb_read_object_info() call will download
a missing blob on-demand. This means that the use of the path-walk API
within 'git backfill' would not operate nicely with this filter type.
The intention of that command is to download missing blobs in batches.
Downloading objects one-by-one would go against the point. Update the
validation in 'git backfill' to add its own compatibility check on top
of path_walk_filter_compatible().
Add tests for blob:limit=0 (equivalent to blob:none) and blob:limit=3
(which exercises partial filtering within a batch where some blobs are
kept and others are excluded).
Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
builtin/backfill.c | 2 ++
path-walk.c | 38 +++++++++++++++++++--
path-walk.h | 8 +++++
t/t5620-backfill.sh | 2 +-
t/t6601-path-walk.sh | 78 ++++++++++++++++++++++++++++++++++++++++++++
5 files changed, 124 insertions(+), 4 deletions(-)
diff --git a/builtin/backfill.c b/builtin/backfill.c
index 51eaa42169..7ef9dc305e 100644
--- a/builtin/backfill.c
+++ b/builtin/backfill.c
@@ -146,6 +146,8 @@ int cmd_backfill(int argc, const char **argv, const char *prefix, struct reposit
die(_("unrecognized argument: %s"), argv[1]);
if (!path_walk_filter_compatible(&ctx.revs.filter))
die(_("cannot backfill with these filter options"));
+ if (ctx.revs.filter.blob_limit_value)
+ die(_("cannot backfill with blob size limits"));
repo_config(repo, git_default_config, NULL);
diff --git a/path-walk.c b/path-walk.c
index a4dd197c37..0e7dab7a6a 100644
--- a/path-walk.c
+++ b/path-walk.c
@@ -10,6 +10,7 @@
#include "hex.h"
#include "list-objects.h"
#include "list-objects-filter-options.h"
+#include "odb.h"
#include "object.h"
#include "oid-array.h"
#include "path.h"
@@ -315,9 +316,29 @@ static int walk_path(struct path_walk_context *ctx,
/* Evaluate function pointer on this data, if requested. */
if ((list->type == OBJ_TREE && ctx->info->trees) ||
(list->type == OBJ_BLOB && ctx->info->blobs) ||
- (list->type == OBJ_TAG && ctx->info->tags))
- ret = ctx->info->path_fn(path, &list->oids, list->type,
- ctx->info->path_fn_data);
+ (list->type == OBJ_TAG && ctx->info->tags)) {
+ struct oid_array *oids = &list->oids;
+ struct oid_array filtered = OID_ARRAY_INIT;
+
+ if (list->type == OBJ_BLOB && ctx->info->blob_limit) {
+ for (size_t i = 0; i < list->oids.nr; i++) {
+ unsigned long size;
+
+ if (odb_read_object_info(ctx->repo->objects,
+ &list->oids.oid[i],
+ &size) != OBJ_BLOB ||
+ size < ctx->info->blob_limit)
+ oid_array_append(&filtered,
+ &list->oids.oid[i]);
+ }
+ oids = &filtered;
+ }
+
+ if (oids->nr)
+ ret = ctx->info->path_fn(path, oids, list->type,
+ ctx->info->path_fn_data);
+ oid_array_clear(&filtered);
+ }
/* Expand data for children. */
if (list->type == OBJ_TREE) {
@@ -500,6 +521,17 @@ static int prepare_filters(struct path_walk_info *info,
}
return 1;
+ case LOFC_BLOB_LIMIT:
+ if (info) {
+ if (!options->blob_limit_value) {
+ info->blobs = 0;
+ } else {
+ info->blob_limit = options->blob_limit_value;
+ }
+ list_objects_filter_release(options);
+ }
+ return 1;
+
default:
error(_("object filter '%s' not supported by the path-walk API"),
list_objects_filter_spec(options));
diff --git a/path-walk.h b/path-walk.h
index be8d27b398..bcb81b70a1 100644
--- a/path-walk.h
+++ b/path-walk.h
@@ -42,6 +42,14 @@ struct path_walk_info {
int blobs;
int tags;
+ /**
+ * If non-zero, specifies a maximum blob size. Blobs with a
+ * size equal to or greater than this limit will be omitted
+ * from the walk. Blobs smaller than the limit (or blobs
+ * whose size cannot be determined) are still visited.
+ */
+ unsigned long blob_limit;
+
/**
* When 'prune_all_uninteresting' is set and a path has all objects
* marked as UNINTERESTING, then the path-walk will not visit those
diff --git a/t/t5620-backfill.sh b/t/t5620-backfill.sh
index 3580e10b9c..3c8a75192a 100755
--- a/t/t5620-backfill.sh
+++ b/t/t5620-backfill.sh
@@ -20,7 +20,7 @@ test_expect_success 'backfill rejects incompatible filter options' '
test_grep "cannot backfill with these filter options" err &&
test_must_fail git backfill --objects --filter=blob:limit=10m 2>err &&
- test_grep "cannot backfill with these filter options" err
+ test_grep "cannot backfill with blob size limits" err
'
# We create objects in the 'src' repo.
diff --git a/t/t6601-path-walk.sh b/t/t6601-path-walk.sh
index 94df309987..d9be7b9cd2 100755
--- a/t/t6601-path-walk.sh
+++ b/t/t6601-path-walk.sh
@@ -475,4 +475,82 @@ test_expect_success 'topic only, blob:none filter' '
test_cmp_sorted expect out
'
+test_expect_success 'all, blob:limit=0 filter' '
+ test-tool path-walk --filter=blob:limit=0 -- --all >out &&
+
+ cat >expect <<-EOF &&
+ 0:commit::$(git rev-parse topic)
+ 0:commit::$(git rev-parse base)
+ 0:commit::$(git rev-parse base~1)
+ 0:commit::$(git rev-parse base~2)
+ 1:tag:/tags:$(git rev-parse refs/tags/first)
+ 1:tag:/tags:$(git rev-parse refs/tags/second.1)
+ 1:tag:/tags:$(git rev-parse refs/tags/second.2)
+ 1:tag:/tags:$(git rev-parse refs/tags/third)
+ 1:tag:/tags:$(git rev-parse refs/tags/fourth)
+ 1:tag:/tags:$(git rev-parse refs/tags/tree-tag)
+ 1:tag:/tags:$(git rev-parse refs/tags/blob-tag)
+ 2:tree::$(git rev-parse topic^{tree})
+ 2:tree::$(git rev-parse base^{tree})
+ 2:tree::$(git rev-parse base~1^{tree})
+ 2:tree::$(git rev-parse base~2^{tree})
+ 2:tree::$(git rev-parse refs/tags/tree-tag^{})
+ 2:tree::$(git rev-parse refs/tags/tree-tag2^{})
+ 3:tree:a/:$(git rev-parse base:a)
+ 4:tree:child/:$(git rev-parse refs/tags/tree-tag:child)
+ 5:tree:left/:$(git rev-parse base:left)
+ 5:tree:left/:$(git rev-parse base~2:left)
+ 6:tree:right/:$(git rev-parse topic:right)
+ 6:tree:right/:$(git rev-parse base~1:right)
+ 6:tree:right/:$(git rev-parse base~2:right)
+ blobs:0
+ commits:4
+ tags:7
+ trees:13
+ EOF
+
+ test_cmp_sorted expect out
+'
+
+test_expect_success 'all, blob:limit=3 filter' '
+ test-tool path-walk --filter=blob:limit=3 -- --all >out &&
+
+ cat >expect <<-EOF &&
+ 0:commit::$(git rev-parse topic)
+ 0:commit::$(git rev-parse base)
+ 0:commit::$(git rev-parse base~1)
+ 0:commit::$(git rev-parse base~2)
+ 1:tag:/tags:$(git rev-parse refs/tags/first)
+ 1:tag:/tags:$(git rev-parse refs/tags/second.1)
+ 1:tag:/tags:$(git rev-parse refs/tags/second.2)
+ 1:tag:/tags:$(git rev-parse refs/tags/third)
+ 1:tag:/tags:$(git rev-parse refs/tags/fourth)
+ 1:tag:/tags:$(git rev-parse refs/tags/tree-tag)
+ 1:tag:/tags:$(git rev-parse refs/tags/blob-tag)
+ 2:tree::$(git rev-parse topic^{tree})
+ 2:tree::$(git rev-parse base^{tree})
+ 2:tree::$(git rev-parse base~1^{tree})
+ 2:tree::$(git rev-parse base~2^{tree})
+ 2:tree::$(git rev-parse refs/tags/tree-tag^{})
+ 2:tree::$(git rev-parse refs/tags/tree-tag2^{})
+ 3:blob:a:$(git rev-parse base~2:a)
+ 4:tree:a/:$(git rev-parse base:a)
+ 5:tree:child/:$(git rev-parse refs/tags/tree-tag:child)
+ 6:tree:left/:$(git rev-parse base:left)
+ 6:tree:left/:$(git rev-parse base~2:left)
+ 7:blob:left/b:$(git rev-parse base~2:left/b)
+ 8:tree:right/:$(git rev-parse topic:right)
+ 8:tree:right/:$(git rev-parse base~1:right)
+ 8:tree:right/:$(git rev-parse base~2:right)
+ 9:blob:right/c:$(git rev-parse base~2:right/c)
+ 10:blob:right/d:$(git rev-parse base~1:right/d)
+ blobs:4
+ commits:4
+ tags:7
+ trees:13
+ EOF
+
+ test_cmp_sorted expect out
+'
+
test_done
--
gitgitgadget
^ permalink raw reply related
* [PATCH 4/7] backfill: die on incompatible filter options
From: Derrick Stolee via GitGitGadget @ 2026-05-02 14:15 UTC (permalink / raw)
To: git
Cc: christian.couder, gitster, johannes.schindelin, johncai86,
karthik.188, kristofferhaugsbakk, me, newren, peff, ps,
Derrick Stolee, Derrick Stolee
In-Reply-To: <pull.2101.git.1777731354.gitgitgadget@gmail.com>
From: Derrick Stolee <stolee@gmail.com>
The 'git backfill' command uses the path-walk API in a critical way: it
uses the objects output from the command to find the batches of missing
objects that should be requested from the server. Unlike 'git
pack-objects', we cannot fall back to another mechanism.
The previous change added the path_walk_filter_compatible() method that
we can reuse here. Use it during argument validation in cmd_backfill().
Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
builtin/backfill.c | 2 ++
t/t5620-backfill.sh | 8 ++++++++
2 files changed, 10 insertions(+)
diff --git a/builtin/backfill.c b/builtin/backfill.c
index d794dd842f..51eaa42169 100644
--- a/builtin/backfill.c
+++ b/builtin/backfill.c
@@ -144,6 +144,8 @@ int cmd_backfill(int argc, const char **argv, const char *prefix, struct reposit
if (argc > 1)
die(_("unrecognized argument: %s"), argv[1]);
+ if (!path_walk_filter_compatible(&ctx.revs.filter))
+ die(_("cannot backfill with these filter options"));
repo_config(repo, git_default_config, NULL);
diff --git a/t/t5620-backfill.sh b/t/t5620-backfill.sh
index f3b5e39493..3580e10b9c 100755
--- a/t/t5620-backfill.sh
+++ b/t/t5620-backfill.sh
@@ -15,6 +15,14 @@ test_expect_success 'backfill rejects unexpected arguments' '
test_grep "unrecognized argument: --unexpected-arg" err
'
+test_expect_success 'backfill rejects incompatible filter options' '
+ test_must_fail git backfill --objects --filter=tree:1 2>err &&
+ test_grep "cannot backfill with these filter options" err &&
+
+ test_must_fail git backfill --objects --filter=blob:limit=10m 2>err &&
+ test_grep "cannot backfill with these filter options" err
+'
+
# We create objects in the 'src' repo.
test_expect_success 'setup repo for object creation' '
echo "{print \$1}" >print_1.awk &&
--
gitgitgadget
^ permalink raw reply related
* [PATCH 3/7] path-walk: support blobless filter
From: Derrick Stolee via GitGitGadget @ 2026-05-02 14:15 UTC (permalink / raw)
To: git
Cc: christian.couder, gitster, johannes.schindelin, johncai86,
karthik.188, kristofferhaugsbakk, me, newren, peff, ps,
Derrick Stolee, Derrick Stolee
In-Reply-To: <pull.2101.git.1777731354.gitgitgadget@gmail.com>
From: Derrick Stolee <stolee@gmail.com>
The 'git pack-objects' command can opt-in to using the path-walk API for
scanning the objects. Currently, this option is dynamically disabled if
combined with '--filter=<X>', even when using a simple filter such as
'blob:none' to signal a blobless packfile. This is a common scenario for
repos at scale, so is worth integrating.
Also, users can opt-in to the '--path-walk' option by default through
the pack.usePathWalk=true config option. When using that in a blobless
partial clone, the following warning can appear even though the user did
not specify either option directly:
warning: cannot use --filter with --path-walk
Teach the path-walk API to handle the 'blob:none' object filter
natively. When revs->filter.choice is LOFC_BLOB_NONE, the path-walk
sets info->blobs to 0 (skipping all blob objects) and clears the
filter from revs so that prepare_revision_walk() does not reject the
configuration.
This check is implemented in the static prepare_filters() method, which
will simultaneously check if the input filters are compatible and will
make the appropriate mutations to the path_walk_info and filters if the
path_walk_info is non-NULL. This allows us to use this logic both in the
API method path_walk_filter_compatible() for use in
builtin/pack-objects.c and as a prep step in walk_objects_by_path().
Update the test helper (test-path-walk) to accept --filter=<spec>
as a test-tool option (before '--'), applying it to revs after
setup_revisions() to avoid the --objects requirement check.
Also switch test-path-walk from REV_INFO_INIT with manual repo
assignment to repo_init_revisions(), which properly initializes
the filter_spec strbuf needed for filter parsing.
Add tests for blob:none with --all and with a single branch.
The performance test p5315 shows the impact of this change when using
blobless filters:
Test HEAD~1 HEAD
---------------------------------------------------------------------
5315.6: repack (blob:none) 13.53 13.87 +2.5%
5315.7: repack size (blob:none) 137.7M 137.8M +0.1%
5315.8: repack (blob:none, --path-walk) 13.51 23.43 +73.4%
5315.9: repack size (blob:none, --path-walk) 137.7M 115.2M -16.3%
These performance tests were run on the Git repository. The --path-walk
feature shows meaningful space savings (16% smaller for blobless packs)
at the cost of increased computation time due to the two compression
passes. This data demonstrates that the feature is engaged and provides
real compression benefits when --no-reuse-delta forces fresh deltas.
Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
builtin/pack-objects.c | 2 +-
path-walk.c | 30 ++++++++++++++++++++
path-walk.h | 7 +++++
t/helper/test-path-walk.c | 11 ++++++-
t/t6601-path-walk.sh | 60 +++++++++++++++++++++++++++++++++++++++
5 files changed, 108 insertions(+), 2 deletions(-)
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index 4338962904..bc9fb5b457 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -5177,7 +5177,7 @@ int cmd_pack_objects(int argc,
if (path_walk) {
const char *option = NULL;
- if (filter_options.choice)
+ if (!path_walk_filter_compatible(&filter_options))
option = "--filter";
else if (use_delta_islands)
option = "--delta-islands";
diff --git a/path-walk.c b/path-walk.c
index 6e426af433..a4dd197c37 100644
--- a/path-walk.c
+++ b/path-walk.c
@@ -9,6 +9,7 @@
#include "hashmap.h"
#include "hex.h"
#include "list-objects.h"
+#include "list-objects-filter-options.h"
#include "object.h"
#include "oid-array.h"
#include "path.h"
@@ -485,6 +486,32 @@ static int setup_pending_objects(struct path_walk_info *info,
return 0;
}
+static int prepare_filters(struct path_walk_info *info,
+ struct list_objects_filter_options *options)
+{
+ switch (options->choice) {
+ case LOFC_DISABLED:
+ return 1;
+
+ case LOFC_BLOB_NONE:
+ if (info) {
+ info->blobs = 0;
+ list_objects_filter_release(options);
+ }
+ return 1;
+
+ default:
+ error(_("object filter '%s' not supported by the path-walk API"),
+ list_objects_filter_spec(options));
+ return 0;
+ }
+}
+
+int path_walk_filter_compatible(struct list_objects_filter_options *options)
+{
+ return prepare_filters(NULL, options);
+}
+
/**
* Given the configuration of 'info', walk the commits based on 'info->revs' and
* call 'info->path_fn' on each discovered path.
@@ -512,6 +539,9 @@ int walk_objects_by_path(struct path_walk_info *info)
trace2_region_enter("path-walk", "commit-walk", info->revs->repo);
+ if (!prepare_filters(info, &info->revs->filter))
+ return -1;
+
CALLOC_ARRAY(commit_list, 1);
commit_list->type = OBJ_COMMIT;
diff --git a/path-walk.h b/path-walk.h
index 5ef5a8440e..be8d27b398 100644
--- a/path-walk.h
+++ b/path-walk.h
@@ -85,3 +85,10 @@ void path_walk_info_clear(struct path_walk_info *info);
* Returns nonzero on an error.
*/
int walk_objects_by_path(struct path_walk_info *info);
+
+struct list_objects_filter_options;
+/**
+ * Given a set of options for filtering objects, return 1 if the options
+ * are compatible with the path-walk API and 0 otherwise.
+ */
+int path_walk_filter_compatible(struct list_objects_filter_options *options);
diff --git a/t/helper/test-path-walk.c b/t/helper/test-path-walk.c
index fe63002c2b..88f86ae0dc 100644
--- a/t/helper/test-path-walk.c
+++ b/t/helper/test-path-walk.c
@@ -4,6 +4,7 @@
#include "dir.h"
#include "environment.h"
#include "hex.h"
+#include "list-objects-filter-options.h"
#include "object-name.h"
#include "object.h"
#include "pretty.h"
@@ -71,6 +72,8 @@ int cmd__path_walk(int argc, const char **argv)
struct rev_info revs = REV_INFO_INIT;
struct path_walk_info info = PATH_WALK_INFO_INIT;
struct path_walk_test_data data = { 0 };
+ struct list_objects_filter_options filter_options =
+ LIST_OBJECTS_FILTER_INIT;
struct option options[] = {
OPT_BOOL(0, "blobs", &info.blobs,
N_("toggle inclusion of blob objects")),
@@ -86,11 +89,12 @@ int cmd__path_walk(int argc, const char **argv)
N_("toggle aggressive edge walk")),
OPT_BOOL(0, "stdin-pl", &stdin_pl,
N_("read a pattern list over stdin")),
+ OPT_PARSE_LIST_OBJECTS_FILTER(&filter_options),
OPT_END(),
};
setup_git_directory();
- revs.repo = the_repository;
+ repo_init_revisions(the_repository, &revs, NULL);
argc = parse_options(argc, argv, NULL,
options, path_walk_usage,
@@ -101,6 +105,10 @@ int cmd__path_walk(int argc, const char **argv)
else
usage(path_walk_usage[0]);
+ /* Apply the filter after setup_revisions to avoid the --objects check. */
+ if (filter_options.choice)
+ list_objects_filter_copy(&revs.filter, &filter_options);
+
info.revs = &revs;
info.path_fn = emit_block;
info.path_fn_data = &data;
@@ -129,6 +137,7 @@ int cmd__path_walk(int argc, const char **argv)
free(info.pl);
}
+ list_objects_filter_release(&filter_options);
release_revisions(&revs);
return res;
}
diff --git a/t/t6601-path-walk.sh b/t/t6601-path-walk.sh
index 56bd1e3c5b..94df309987 100755
--- a/t/t6601-path-walk.sh
+++ b/t/t6601-path-walk.sh
@@ -415,4 +415,64 @@ test_expect_success 'trees are reported exactly once' '
test_line_count = 1 out-filtered
'
+test_expect_success 'all, blob:none filter' '
+ test-tool path-walk --filter=blob:none -- --all >out &&
+
+ cat >expect <<-EOF &&
+ 0:commit::$(git rev-parse topic)
+ 0:commit::$(git rev-parse base)
+ 0:commit::$(git rev-parse base~1)
+ 0:commit::$(git rev-parse base~2)
+ 1:tag:/tags:$(git rev-parse refs/tags/first)
+ 1:tag:/tags:$(git rev-parse refs/tags/second.1)
+ 1:tag:/tags:$(git rev-parse refs/tags/second.2)
+ 1:tag:/tags:$(git rev-parse refs/tags/third)
+ 1:tag:/tags:$(git rev-parse refs/tags/fourth)
+ 1:tag:/tags:$(git rev-parse refs/tags/tree-tag)
+ 1:tag:/tags:$(git rev-parse refs/tags/blob-tag)
+ 2:tree::$(git rev-parse topic^{tree})
+ 2:tree::$(git rev-parse base^{tree})
+ 2:tree::$(git rev-parse base~1^{tree})
+ 2:tree::$(git rev-parse base~2^{tree})
+ 2:tree::$(git rev-parse refs/tags/tree-tag^{})
+ 2:tree::$(git rev-parse refs/tags/tree-tag2^{})
+ 3:tree:a/:$(git rev-parse base:a)
+ 4:tree:child/:$(git rev-parse refs/tags/tree-tag:child)
+ 5:tree:left/:$(git rev-parse base:left)
+ 5:tree:left/:$(git rev-parse base~2:left)
+ 6:tree:right/:$(git rev-parse topic:right)
+ 6:tree:right/:$(git rev-parse base~1:right)
+ 6:tree:right/:$(git rev-parse base~2:right)
+ blobs:0
+ commits:4
+ tags:7
+ trees:13
+ EOF
+
+ test_cmp_sorted expect out
+'
+
+test_expect_success 'topic only, blob:none filter' '
+ test-tool path-walk --filter=blob:none -- topic >out &&
+
+ cat >expect <<-EOF &&
+ 0:commit::$(git rev-parse topic)
+ 0:commit::$(git rev-parse base~1)
+ 0:commit::$(git rev-parse base~2)
+ 1:tree::$(git rev-parse topic^{tree})
+ 1:tree::$(git rev-parse base~1^{tree})
+ 1:tree::$(git rev-parse base~2^{tree})
+ 2:tree:left/:$(git rev-parse base~2:left)
+ 3:tree:right/:$(git rev-parse topic:right)
+ 3:tree:right/:$(git rev-parse base~1:right)
+ 3:tree:right/:$(git rev-parse base~2:right)
+ blobs:0
+ commits:3
+ tags:0
+ trees:7
+ EOF
+
+ test_cmp_sorted expect out
+'
+
test_done
--
gitgitgadget
^ permalink raw reply related
* [PATCH 2/7] t/perf: add pack-objects filter and path-walk benchmark
From: Derrick Stolee via GitGitGadget @ 2026-05-02 14:15 UTC (permalink / raw)
To: git
Cc: christian.couder, gitster, johannes.schindelin, johncai86,
karthik.188, kristofferhaugsbakk, me, newren, peff, ps,
Derrick Stolee, Derrick Stolee
In-Reply-To: <pull.2101.git.1777731354.gitgitgadget@gmail.com>
From: Derrick Stolee <stolee@gmail.com>
Add p5315-pack-objects-filter.sh to measure the performance of
'git pack-objects --revs --all' under different filter and traversal
combinations:
* no filter (baseline)
* --filter=blob:none (blobless)
* --filter=sparse:oid=<oid> (cone-mode sparse)
Each filter scenario is tested both with and without --path-walk,
producing paired measurements that show the impact of the path-walk
traversal for each filter type as we integrate the --path-walk feature
with different --filter options. It currently has no integration so
falls back to the standard revision walk. Thus, there are no significant
differences in the current results other than a full repack (and even
then, the --path-walk feature is not incredibly different for the
default Git repository):
Test HEAD
-----------------------------------------------------
5315.2: repack (no filter) 27.91
5315.3: repack size (no filter) 250.7M
5315.4: repack (no filter, --path-walk) 34.92
5315.5: repack size (no filter, --path-walk) 220.0M
5315.6: repack (blob:none) 13.63
5315.7: repack size (blob:none) 137.6M
5315.8: repack (blob:none, --path-walk) 13.48
5315.9: repack size (blob:none, --path-walk) 137.7M
5315.10: repack (sparse:oid) 72.67
5315.11: repack size (sparse:oid) 187.4M
5315.12: repack (sparse:oid, --path-walk) 72.47
5315.13: repack size (sparse:oid, --path-walk) 187.4M
The sparse filter definition is built automatically by sampling
depth-2 directories from the test repository, making the test work
on any repo passed via GIT_PERF_LARGE_REPO. For repos that lack
depth-2 directories, a single top-level directory is used; for flat
repos, the sparse tests are skipped via prerequisite.
Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
t/perf/p5315-pack-objects-filter.sh | 129 ++++++++++++++++++++++++++++
1 file changed, 129 insertions(+)
create mode 100755 t/perf/p5315-pack-objects-filter.sh
diff --git a/t/perf/p5315-pack-objects-filter.sh b/t/perf/p5315-pack-objects-filter.sh
new file mode 100755
index 0000000000..b009039c89
--- /dev/null
+++ b/t/perf/p5315-pack-objects-filter.sh
@@ -0,0 +1,129 @@
+#!/bin/sh
+
+test_description='Tests pack-objects performance with filters and --path-walk'
+. ./perf-lib.sh
+
+test_perf_large_repo
+
+test_expect_success 'setup filter inputs' '
+ # Sample a few depth-2 directories from the test repo to build
+ # a cone-mode sparse-checkout definition. The sampling picks
+ # directories at evenly-spaced positions so the choice is stable
+ # and scales to repos of any shape.
+
+ git ls-tree -d --name-only HEAD >top-dirs &&
+ top_nr=$(wc -l <top-dirs) &&
+
+ >depth2-dirs &&
+ while read tdir
+ do
+ git ls-tree -d --name-only "HEAD:$tdir" 2>/dev/null |
+ sed "s|^|$tdir/|" >>depth2-dirs || return 1
+ done <top-dirs &&
+
+ d2_nr=$(wc -l <depth2-dirs) &&
+
+ if test "$d2_nr" -ge 2
+ then
+ # Pick two directories from evenly-spaced positions.
+ first=$(sed -n "1p" depth2-dirs) &&
+ mid=$(sed -n "$((d2_nr / 2 + 1))p" depth2-dirs) &&
+
+ p1=$(dirname "$first") &&
+ p2=$(dirname "$mid") &&
+
+ # Build cone-mode sparse-checkout patterns.
+ {
+ echo "/*" &&
+ echo "!/*/" &&
+ echo "/$p1/" &&
+ echo "!/$p1/*/" &&
+ if test "$p1" != "$p2"
+ then
+ echo "/$p2/" &&
+ echo "!/$p2/*/"
+ fi &&
+ echo "/$first/" &&
+ if test "$first" != "$mid"
+ then
+ echo "/$mid/"
+ fi
+ } >sparse-patterns &&
+
+ git hash-object -w sparse-patterns >sparse-oid &&
+ echo "Sparse cone: $first $mid" &&
+ cat sparse-patterns &&
+ test_set_prereq SPARSE_OID
+ elif test "$top_nr" -ge 1
+ then
+ # Fallback: use a single top-level directory.
+ first=$(sed -n "1p" top-dirs) &&
+ {
+ echo "/*" &&
+ echo "!/*/" &&
+ echo "/$first/"
+ } >sparse-patterns &&
+
+ git hash-object -w sparse-patterns >sparse-oid &&
+ echo "Sparse cone: $first" &&
+ cat sparse-patterns &&
+ test_set_prereq SPARSE_OID
+ fi
+'
+
+test_perf 'repack (no filter)' '
+ git pack-objects --stdout --no-reuse-delta --revs --all </dev/null >pk
+'
+
+test_size 'repack size (no filter)' '
+ test_file_size pk
+'
+
+test_perf 'repack (no filter, --path-walk)' '
+ git pack-objects --stdout --no-reuse-delta --revs --all --path-walk </dev/null >pk
+'
+
+test_size 'repack size (no filter, --path-walk)' '
+ test_file_size pk
+'
+
+test_perf 'repack (blob:none)' '
+ git pack-objects --stdout --no-reuse-delta --revs --all --filter=blob:none </dev/null >pk
+'
+
+test_size 'repack size (blob:none)' '
+ test_file_size pk
+'
+
+test_perf 'repack (blob:none, --path-walk)' '
+ git pack-objects --stdout --no-reuse-delta --revs --all --path-walk \
+ --filter=blob:none </dev/null >pk
+'
+
+test_size 'repack size (blob:none, --path-walk)' '
+ test_file_size pk
+'
+
+test_perf 'repack (sparse:oid)' \
+ --prereq SPARSE_OID '
+ git pack-objects --stdout --no-reuse-delta --revs --all \
+ --filter=sparse:oid=$(cat sparse-oid) </dev/null >pk
+'
+
+test_size 'repack size (sparse:oid)' \
+ --prereq SPARSE_OID '
+ test_file_size pk
+'
+
+test_perf 'repack (sparse:oid, --path-walk)' \
+ --prereq SPARSE_OID '
+ git pack-objects --stdout --no-reuse-delta --revs --all --path-walk \
+ --filter=sparse:oid=$(cat sparse-oid) </dev/null >pk
+'
+
+test_size 'repack size (sparse:oid, --path-walk)' \
+ --prereq SPARSE_OID '
+ test_file_size pk
+'
+
+test_done
--
gitgitgadget
^ permalink raw reply related
* [PATCH 1/7] pack-objects: pass --objects with --path-walk
From: Derrick Stolee via GitGitGadget @ 2026-05-02 14:15 UTC (permalink / raw)
To: git
Cc: christian.couder, gitster, johannes.schindelin, johncai86,
karthik.188, kristofferhaugsbakk, me, newren, peff, ps,
Derrick Stolee, Derrick Stolee
In-Reply-To: <pull.2101.git.1777731354.gitgitgadget@gmail.com>
From: Derrick Stolee <stolee@gmail.com>
When 'git pack-objects' has the --path-walk option enabled, it uses a
different set of revision walk parameters than normal. For once,
--objects was previously assumed by the path-walk API and was not needed
to be added. We also needed --boundary to allow discovering
UNINTERESTING objects to use as delta bases.
We will be updating the path-walk API soon to work with some filter
options. However, the revision machinery will trigger a fatal error:
fatal: object filtering requires --objects
The fix is easy: add the --objects option as an argument. This has no
effect on the path-walk API but does simplify the revision option
parsing for the objects filter.
We can remove the comment about "removing" the options because they were
never removed and instead not added. We still need to disable using
bitmaps.
Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
builtin/pack-objects.c | 5 +----
1 file changed, 1 insertion(+), 4 deletions(-)
diff --git a/builtin/pack-objects.c b/builtin/pack-objects.c
index dd2480a73d..4338962904 100644
--- a/builtin/pack-objects.c
+++ b/builtin/pack-objects.c
@@ -5190,10 +5190,7 @@ int cmd_pack_objects(int argc,
}
if (path_walk) {
strvec_push(&rp, "--boundary");
- /*
- * We must disable the bitmaps because we are removing
- * the --objects / --objects-edge[-aggressive] options.
- */
+ strvec_push(&rp, "--objects");
use_bitmap_index = 0;
} else if (thin) {
use_internal_rev_list = 1;
--
gitgitgadget
^ permalink raw reply related
* [PATCH 0/7] pack-objects: integrate --path-walk and some --filter options
From: Derrick Stolee via GitGitGadget @ 2026-05-02 14:15 UTC (permalink / raw)
To: git
Cc: christian.couder, gitster, johannes.schindelin, johncai86,
karthik.188, kristofferhaugsbakk, me, newren, peff, ps,
Derrick Stolee
The 'git pack-objects' command has a '--path-walk' option that uses the
path-walk API instead of a typical revision walk to group objects into
chunks by path name instead of relying solely on name-hashes to group
similar files together. (It also does a second compression pass looking for
better deltas after the first pass that is focused within chunks per path.)
The '--path-walk' feature was not previously integrated with the '--filter'
feature, so a warning would appear and disable the path-walk API when a
filter is given. This patch series integrates these together in the
following ways:
* --filter=blob:none updates the path-walk API options to skip blobs.
* --filter=blob:limit=<size> adds a scan to a list of blob objects to
remove objects that are too large.
* --filter=sparse:<oid> adds a scan to the chunks to validate that the
paths match the sparse-checkout patterns.
In particular, this last check is significantly faster than the previous
algorithm because it can check all objects at a given path simultaneously
instead of checking all sparse-checkout patterns for each discovered blob
object.
A subtlety must be added here, in that we must change how we mark an object
as "seen" during the path-walk. We may need to add an object to multiple
paths and only mark it as "seen" if it indeed matched the sparse-checkout
patterns as the path is accepted for emitting to the callback. This adds a
new filter that the "seen" objects must also be removed from later chunks to
avoid sending the same object as grouped to multiple chunks.
There's also a subtle detail here in that the path-walk API also prunes tree
paths based on cone-mode sparse-checkouts, to enable 'git backfill --sparse'
operating quickly for small sparse-checkout scopes. But the
--filter=sparse:<oid> feature doesn't prune trees!
As a future step, I do plan to recommend that we add a treesparse:<oid>
setting that does allow us to trim the tree set by cone-mode sparse
patterns. At the time that partial clone filters were being created, cone
mode sparse-checkout didn't exist and neither did the sparse index. Those
features together make a smaller tree set possible, assuming the user never
needs to change their scope. This would be a significant change so it is not
implemented here, though the git pack-objects integration would be quick
after this series completes.
Neither the sparse:<oid> or hypothetical treesparse:<oid> options are or
should necessarily be supported by Git servers. It's too expensive to
compute dynamically and it doesn't work well with reachability bitmaps. What
becomes possible with this change is that it becomes reasonably fast to
construct bundles with these filters that can bootstrap a working
environment with the full history of all files within a given
sparse-checkout scope.
Performance Results
===================
Since the '--path-walk' option is ignored in today's Git version when a
'--filter' is added, the performance matches the behavior without
'--path-walk'. For the tables below, you can compare the rows against each
other (time and then packfile size) for the mode without and then with
'--path-walk' as a representation of "before" and "after". (These tables are
repeated in the commit messages as new implementations improve specific
rows.)
I chose a number of open source repositories of various sizes and shapes:
git/git
=======
Test HEAD
-------------------------------------------------------------------
5315.2: repack (no filter) 27.73
5315.3: repack size (no filter) 250.6M
5315.4: repack (no filter, --path-walk) 35.19
5315.5: repack size (no filter, --path-walk) 220.1M
5315.6: repack (blob:none) 13.42
5315.7: repack size (blob:none) 137.6M
5315.8: repack (blob:none, --path-walk) 20.98
5315.9: repack size (blob:none, --path-walk) 115.2M
5315.10: repack (sparse:oid) 72.53
5315.11: repack size (sparse:oid) 187.5M
5315.12: repack (sparse:oid, --path-walk) 29.00
5315.13: repack size (sparse:oid, --path-walk) 161.0M
nodejs/node
===========
Test HEAD
--------------------------------------------------------------------
5315.2: repack (no filter) 75.53
5315.3: repack size (no filter) 0.9G
5315.4: repack (no filter, --path-walk) 80.54
5315.5: repack size (no filter, --path-walk) 885.7M
5315.6: repack (blob:none) 12.65
5315.7: repack size (blob:none) 148.6M
5315.8: repack (blob:none, --path-walk) 17.60
5315.9: repack size (blob:none, --path-walk) 134.6M
5315.10: repack (sparse:oid) 518.84
5315.11: repack size (sparse:oid) 153.4M
5315.12: repack (sparse:oid, --path-walk) 27.99
5315.13: repack size (sparse:oid, --path-walk) 139.4M
microsoft/fluentui
==================
Test HEAD
--------------------------------------------------------------------
5315.2: repack (no filter) 146.77
5315.3: repack size (no filter) 562.1M
5315.4: repack (no filter, --path-walk) 72.82
5315.5: repack size (no filter, --path-walk) 172.6M
5315.6: repack (blob:none) 4.84
5315.7: repack size (blob:none) 62.7M
5315.8: repack (blob:none, --path-walk) 5.19
5315.9: repack size (blob:none, --path-walk) 59.9M
5315.10: repack (sparse:oid) 59.95
5315.11: repack size (sparse:oid) 85.6M
5315.12: repack (sparse:oid, --path-walk) 15.16
5315.13: repack size (sparse:oid, --path-walk) 72.5M
microsoftdocs/azure-devops-docs
===============================
Test HEAD
---------------------------------------------------------------------
5315.2: repack (no filter) 4.41
5315.3: repack size (no filter) 1.6G
5315.4: repack (no filter, --path-walk) 6.00
5315.5: repack size (no filter, --path-walk) 1.6G
5315.6: repack (blob:none) 1.35
5315.7: repack size (blob:none) 60.0M
5315.8: repack (blob:none, --path-walk) 1.23
5315.9: repack size (blob:none, --path-walk) 60.0M
5315.10: repack (sparse:oid) 138.24
5315.11: repack size (sparse:oid) 84.4M
5315.12: repack (sparse:oid, --path-walk) 1.86
5315.13: repack size (sparse:oid, --path-walk) 84.4M
Performance conclusions
=======================
As seen in earlier series around the '--path-walk' feature, the space
savings can be valuable but is not always guaranteed. When the space savings
doesn't happen, then the time spent is generally slower because of the
two-pass mechanism. The microsoftdocs/azure-devops-docs repo demonstrates
this case quite clearly.
However, even in these cases the 'sparse:<oid>' filters are much faster
because of the ability to check an entire set of objects against the
sparse-checkout patterns only once.
Thanks, -Stolee
P.S. I've CC'd the folks who were on the original path-walk feature thread
[1]
[1]
https://lore.kernel.org/git/pull.1819.git.1741571455.gitgitgadget@gmail.com/
Derrick Stolee (7):
pack-objects: pass --objects with --path-walk
t/perf: add pack-objects filter and path-walk benchmark
path-walk: support blobless filter
backfill: die on incompatible filter options
path-walk: support blob size limit filter
path-walk: add pl_sparse_trees to control tree pruning
pack-objects: support sparse:oid filter with path-walk
builtin/backfill.c | 5 +
builtin/pack-objects.c | 9 +-
path-walk.c | 154 ++++++++++++-
path-walk.h | 21 ++
t/helper/test-path-walk.c | 17 +-
t/perf/p5315-pack-objects-filter.sh | 129 +++++++++++
t/t5317-pack-objects-filter-objects.sh | 125 ++++++++++
t/t5620-backfill.sh | 8 +
t/t6601-path-walk.sh | 306 +++++++++++++++++++++++++
9 files changed, 758 insertions(+), 16 deletions(-)
create mode 100755 t/perf/p5315-pack-objects-filter.sh
base-commit: 94f057755b7941b321fd11fec1b2e3ca5313a4e0
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-2101%2Fderrickstolee%2Fpath-walk-filters-v1
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2101/derrickstolee/path-walk-filters-v1
Pull-Request: https://github.com/gitgitgadget/git/pull/2101
--
gitgitgadget
^ permalink raw reply
* Re: [PATCH v3 3/5] name-rev: factor code for sharing with a new command
From: Phillip Wood @ 2026-05-02 10:00 UTC (permalink / raw)
To: kristofferhaugsbakk; +Cc: ben.knoble, git, phillip.wood
In-Reply-To: <20260501172450.25037-2-kristofferhaugsbakk@fastmail.com>
On 01/05/2026 18:24, kristofferhaugsbakk@fastmail.com wrote:
> Hi Phillip. Thanks for taking a look.
>
> On Thu, Apr 30, 2026, at 15:54, Phillip Wood wrote:
>>> [snip]
>>> @@ -524,25 +543,32 @@ static void name_rev_line(char *p, struct name_ref_data *data)
>>> const char *name = NULL;
>>> char c = *(p + 1);
>>> int p_len = p - p_start + 1;
>>> + struct object *o = NULL;
>>> + int oid_ret = 1;
>>>
>>> counter = 0;
>>>
>>> *(p + 1) = 0;
>>> - if (!repo_get_oid(the_repository, p - (hexsz - 1), &oid)) {
>>> - struct object *o =
>>> - lookup_object(the_repository, &oid);
>>> + oid_ret = repo_get_oid(the_repository, p - (hexsz - 1), &oid);
>>
>> It would be safer to restore *(p + 1) here rather that relying on each
>> case block to do it.
>
> Yeah, I didn’t want to repeat that bookkeeping but in some iteration it
> looked necessary. But it’s good that it isn’t.
It looks like the printing code is shared between the two case blocks in
patch 5 as well so we should move that outside them as well and just set
"name" inside the switch statement.
>> *(p + 1) = c;
>>> +
>>> + switch (cmd->type) {
>>> + case NAME_REV:
>>> + if (!oid_ret)
>>> + o = lookup_object(the_repository, &oid);
>>> if (o)
>>> name = get_rev_name(o, &buf);
>>> + *(p + 1) = c;
>>> + if (!name)
>>> + goto start;
>>
>> The pre-image uses "continue" which will increment p - why the change in
>> behavior?
>
> They looked the same to me. So I will need to think about this some
> more. Just a lack of C experience on my part.
>
> Replacing the `continue` with a goto at the start of the loop was also
> unnecessary. Of course the `continue` breaks out of the loop and not the
> switch-block (unlike `break`).
>
> But I didn’t break `t6120-describe.sh`. So I’ll also take a look to see
> if there are any holes.
I think the difference only matters in pathological cases as "goto
start" means we end up looking at the same character twice but the loop
carries on as normal after that. We should just keep using "continue",
I'm not sure we need a new test case.
Thanks
Phillip
>
> Thanks again.
>
>> [snip]
>
^ permalink raw reply
* Re: [PATCH v3 5/5] format-rev: introduce builtin for on-demand pretty formatting
From: Phillip Wood @ 2026-05-02 10:00 UTC (permalink / raw)
To: kristofferhaugsbakk; +Cc: ben.knoble, git
In-Reply-To: <20260501182718.27853-2-kristofferhaugsbakk@fastmail.com>
Hi Kristoffer
On 01/05/2026 19:27, kristofferhaugsbakk@fastmail.com wrote:
> On Fri, May 1, 2026, at 12:16, Phillip Wood wrote:
>>> [snip]
>>> • You can’t feed commits piecemeal to these commands, one input
>>> for one output; they block until standard in is closed
>>
>> So you can feed them piecemeal but you don't get any output until you
>> close stdin. That can be helpful as it means the calling process can
>> write to "git log --stdin" and then read the output without worrying
>> about getting deadlocked.
>
> Okay. I don’t have much experience with concurrent programming.
>
>> The Implementation below works fine if there
>> are separate processes or threads writing to and reading from "git
>> format-rev", but if we want a single process to be able to read from and
>> write to "git format-rev --stdin-mode=text" there will need to be a way
>> to delimit message boundaries so that git knows where the input message
>> ends and the caller knows where the response ends.
>
> Okay, so I guess a null-terminator mode for output.
I think that's a good idea
>> We'll also need to be
>> careful about flushing the output at the end of a processed message.
>
> I don’t get why this takes special care. I’ll think about it.
Because the output from printf() is buffered, unless you explicitly
flush it you can get into a state where git thinks it has printed the
output and is waiting for the caller to write more input, but the caller
is still waiting to read git's output and so they are deadlocked.
Calling maybe_flush_or_die() is the usual way to handle this I think -
see 344a107b55 (merge-tree --stdin: flush stdout to avoid deadlock,
2025-02-18)
>> For "--stdin-mode=revs" the caller cannot know how many lines the output
>> will span because formats like %(trailers) will produce a variable
>> number of lines depending on which trailers are present. It is also
>> possible for a rev name to span more than one line. The following
>> example finds the most recent commit that mentions 'cherry-pick' in the
>> subject line
>>
>> :/^[^
>> ]cherry-pick
>>
>> so we need a way to delimit the input and output records there as well.
>
> Okay, so a null-terminator mode for input as well?
Yes I think "-z" should mean NUL terminated input and output.
>> I think the functionality implemented here is useful (transforming the
>> output of 'git blame' or 'git-last-modified' are convicing examples) and
>> it is probably better to do it as a command rather than adding a
>> "--format" option to name-rev.
>>
>>> • You can’t feed a list of possibly duplicate commits, like the output
>>> of git-last-modified(1); they effectively deduplicate the output
>>
>> That is definitely a problem
>
> Great, thanks.
>
>>> Beyond these two points there’s also the input massage problem: you
>>
>> s/massagge/message/?
>
> No. I meant massaging the input so that it can be processed by whatever
> tool you have. :) In this case splitting the object name column and file
> column because tools like git-log(1) can only deal with revision input.
Oh I see
Thanks
Phillip
> Thanks for reviewing the usability design.
>
^ permalink raw reply
* Re: git rename/moved status unreliable in ruby
From: sebastien.stettler @ 2026-05-02 9:34 UTC (permalink / raw)
To: phillip.wood@dunelm.org.uk, chris.torek@gmail.com
Cc: git@vger.kernel.org, j6t@kdbg.org
In-Reply-To: <CAPx1Gvd_VEWHrBWtUjNeWZ+wfmsAOTamKmL6fhBSQi=MbmXRcw@mail.gmail.com>
Thanks for all the responses and thoughts thus far.
> Which means that git sees that every line has changed because the
> indentation has changed. If you want git to realize that the file has
> been renamed you could move it in one commit and then add modify it in
> the next commit.
This is the minimal solution to the problem but as Johannes illustrated,
there cases where that "move commit doesn't solve the problem"
> Splitting into two commits helps only certain
> cases, in particular, when the commit that moves the files is compared
> to an earlier commit, such as `git log` does. However, if a commit after
> the change is compared to a commit before the move, the rename is still
> not detected.
Further examples:
The move commit helps if i want to blame a file if i use the -W option, but if it doesnt
help with git log since the changes are now considered modifications.
git blame -w ruby-example/lib/calculator.rb
(shows previous files changes and not the indentation)
```
303f25f5 ruby-example/lib/calculator.rb (nothing 2026-05-02 16:40:51 +0800 1) module Lib
^1cf274f ruby-example/calculator.rb (billybonks 2026-05-01 12:36:47 +0800 2) class Calculator
^1cf274f ruby-example/calculator.rb (billybonks 2026-05-01 12:36:47 +0800 3) def add(a, b)
3d22d303 ruby-example/calculator.rb (billybonks 2026-05-02 16:34:14 +0800 4) a + b
^1cf274f ruby-example/calculator.rb (billybonks 2026-05-01 12:36:47 +0800 5) end
^1cf274f ruby-example/calculator.rb (billybonks 2026-05-01 12:36:47 +0800 6)
^1cf274f ruby-example/calculator.rb (billybonks 2026-05-01 12:36:47 +0800 7) def subtract(a, b)
^1cf274f ruby-example/calculator.rb (billybonks 2026-05-01 12:36:47 +0800 8) a - b
^1cf274f ruby-example/calculator.rb (billybonks 2026-05-01 12:36:47 +0800 9) end
^1cf274f ruby-example/calculator.rb (billybonks 2026-05-01 12:36:47 +0800 10)
^1cf274f ruby-example/calculator.rb (billybonks 2026-05-01 12:36:47 +0800 11) def multiply(a, b)
^1cf274f ruby-example/calculator.rb (billybonks 2026-05-01 12:36:47 +0800 12) a * b
^1cf274f ruby-example/calculator.rb (billybonks 2026-05-01 12:36:47 +0800 13) end
^1cf274f ruby-example/calculator.rb (billybonks 2026-05-01 12:36:47 +0800 14)
^1cf274f ruby-example/calculator.rb (billybonks 2026-05-01 12:36:47 +0800 15) def divide(a, b)
^1cf274f ruby-example/calculator.rb (billybonks 2026-05-01 12:36:47 +0800 16) raise ZeroDivisionError, 'Cannot divide by zero' if b.zero?
^1cf274f ruby-example/calculator.rb (billybonks 2026-05-01 12:36:47 +0800 17)
^1cf274f ruby-example/calculator.rb (billybonks 2026-05-01 12:36:47 +0800 18) a / b
^1cf274f ruby-example/calculator.rb (billybonks 2026-05-01 12:36:47 +0800 19) end
^1cf274f ruby-example/calculator.rb (billybonks 2026-05-01 12:36:47 +0800 20) end
303f25f5 ruby-example/lib/calculator.rb (nothing 2026-05-02 16:40:51 +0800 21) end
```
git log --follow --diff-filter=ra -- ruby-example/lib/calculator.rb
( it does not show the move commit )
```
commit 303f25f50cde83c46f83bc3c337cd52b87b63d52 (HEAD -> example-2)
Author: nothing <nothing@contributed.com>
Date: Sat May 2 16:40:51 2026 +0800
update lib and require
commit 3d22d30304d836c07b3274689e5e2c536b29e1bb (example-3)
Author: billybonks <sebastienstettler@gmail.com>
Date: Sat May 2 16:34:14 2026 +0800
fix: sum was doing subtraction instead of addition
```
Using this approach does incur a heavy burnder on the user since most tools that do renaming etc
will move and rename, and that cost does not give a complete solution as there are still many thigns
that don't work in the most ideal sense.
As mentioned by Chris
> It *could* be improved. Doing so in a way that works for more than
> just some special cases -- e.g., in a way that works for ordinary
> text, or graphical images, for instance, rather than just for Ruby
> sources (or just C sources, or C++, or Swift, or Python, or whatever)
> -- seems particularly tricky. Some degree of ignoring white-space
> changes would probably help multiple cases, though.
Has there been explorations of ignoring white space for the similarity checker, i would
assume that majority of white space movements across many languages would result in a
semantically similar document in most cases.
- Sebastien
^ permalink raw reply
* Re: [Bug] fetch --deepen truncates history in v2.54.0
From: René Scharfe @ 2026-05-02 9:22 UTC (permalink / raw)
To: Owen Stephens, git; +Cc: Samo Pogačnik
In-Reply-To: <CANOh7gEEw+6146NN3JV8EYxQarj0KkyA7r3RZ6v-DxeqQZLrCA@mail.gmail.com>
On 4/29/26 1:27 PM, Owen Stephens wrote:
>> What did you do before the bug happened? (Steps to reproduce your issue)
>
> Repeatedy called `git fetch --deepen 2` inside a shallow repo that was a
> file:// clone of another repo. Once all commits had been fetched, a subsequent
> `fetch --deepen` appears to "reset" the repo back to being shallow with a depth
> of 2. A reproduction script is included below. This issue appears to have been
> introduced in v2.54.0.
>
>> What did you expect to happen? (Expected behavior)
>
> I expected `git fetch --deepen` in a non-shallow repo with no upstream commits
> to be a no-op.
>
>> What happened instead? (Actual behavior)
>
> `git log` history is truncated to two commits, and repo is considered shallow
> by `git rev-parse --is-shallow-repository`.
>
>> What's different between what you expected and what actually happened?
>
> The previously-present commits in `git log` are missing, and the repo is again
> considered shallow.
>
>> Anything else you want to add:
>
> Commit 3ef68ff seems relevant.
Indeed, bisect identifies 3ef68ff40e (shallow: handling fetch relative-deepen,
2026-02-15) and reverting it fixes the issue. Copying its author.
> The following script reproduces the issue in 2.54.0, and does not reproduce the
> issue in 2.53.0:
>
> ```
> mkdir repro.git
> cd repro.git
>
> git init
>
> for i in $(seq 1 4); do
> echo "$i" >> file.txt
> git add file.txt
> git commit -m "Change $i"
> done
>
> cd ..
>
> git clone --depth 2 "file://$PWD/repro.git" repro_clone.git
> cd repro_clone.git
>
> echo "Shallow repo? $(git rev-parse --is-shallow-repository)"
> git log --oneline
>
> for i in $(seq 1 3); do
> git fetch --deepen 2
> echo "Shallow repo? $(git rev-parse --is-shallow-repository)"
> git log --oneline
> done
> ```
Nice! Here's a test for that:
diff --git a/t/t5537-fetch-shallow.sh b/t/t5537-fetch-shallow.sh
index 6588ce6226..fdb1dd9823 100755
--- a/t/t5537-fetch-shallow.sh
+++ b/t/t5537-fetch-shallow.sh
@@ -251,6 +251,16 @@ test_expect_success '.git/shallow is edited by repack' '
origin "+refs/heads/*:refs/remotes/origin/*"
'
+test_expect_success 'fetch --deepen does not truncate' '
+ git clone --no-local .git full-clone &&
+ git rev-parse --is-shallow-repository >expect &&
+ git log --oneline >>expect &&
+ git -C full-clone fetch --deepen=1 &&
+ git -C full-clone rev-parse --is-shallow-repository >actual &&
+ git -C full-clone log --oneline >>actual &&
+ test_cmp expect actual
+'
+
. "$TEST_DIRECTORY"/lib-httpd.sh
start_httpd
^ permalink raw reply related
* Re: git rename/moved status unreliable in ruby
From: Chris Torek @ 2026-05-02 8:06 UTC (permalink / raw)
To: sebastien.stettler; +Cc: git@vger.kernel.org
In-Reply-To: <OsOzcjEwvHCQSghLE8LD_wHb_jDlil9I88OUuhpiRONnVd1o9p3gStbK1mx4q7OwY3ePtbZO-BBgTNOCeJ2DMyvBsdlMhRmDrTP894KP5xo=@proton.me>
On Thu, Apr 30, 2026 at 10:06 PM sebastien.stettler
<sebastien.stettler@proton.me> wrote:
> ... understanding the reasoning would be great
In my opinion, the key to understanding here is this:
Git Stores Snapshots.
What this means is that every commit is a full snapshot of all of the
files for that commit. There are no "changes" at all, there is only a
full snapshot, every time.
Now, internally, the storage format is more complicated (and
compressive, ultimately using the concept of changes as well, though
not exactly the way one might expect). But from the "what things look
like" point of view, and how you should think about what Git sees,
each commit is simply a full and complete snapshot of every file. So
if you have one commit where `foo.rb` exists, and `bar.br` does not,
that snapshot has a `foo.rb` but no `bar.rb`. If you make a second
snapshot, in which `foo.rb` no longer exists but `bar.rb` does now,
that second snapshot, well, has those files.
The tricky part is that you normally ask Git to *compare* two
snapshots (at least for "what changed" purposes). When you do that,
Git extracts both snapshots and, well, compares them. If `foo.rb` has
been removed and `bar.rb` has been added, Git then goes on to compare
the *contents* of those two files.
If the contents match exactly, and you've asked Git to "find renames",
Git will always say that the file that vanished from the first commit,
only to be created identically under a new name in the second, was
"renamed", rather than the one file being deleted and the second
added.
If the contents match "fuzzily" (for some value and algorithm of
fuzz-factor), Git may also say "renamed". You can control this with
`--find-renames=<value>`. The key idea here is that Git is *finding*
renames: either exact-same-contents, or "sufficiently similar"
contents, based on remove-and-add pairs.
Since Git only *stores* snapshots, you can get two different results
from comparing the same two commits. All you have to do to get this is
to adjust whether Git checks for renames at all, and if so, to what
extent.
These rules apply to `git show`, `git diff`, `git merge`, and even the
diffstat that `git commit` optionally shows after a commit. For this
reason, all the "compare some commits" commands -- including `git
merge` -- take this `--find-renames=<value>` option. Detection of
renames can be countermanded entirely with `--no-renames`.
This is why -- and when -- making two separate commits, one with
"exact same content for deleted-file-D vs added-file-A", followed by
later changes to new file A, helps: if you compare the commit that has
file D to the middle commit, the two files match exactly, and any
rename detection you have turned on finds that rename. If you then
compare the middle commit to the final commit, file A exists in both,
so Git shows changes to file A. But as soon as you compare the
original file-D-containing commit to the final file-A-updated commit,
you run into the original issue again: to detect this as a rename, you
may need to allow rather generous rename detection.
If, in the future, Git gets fancier rename detection, comparing the
original commit directly against the final one could find the rename
automatically. So:
> If this is considered something that can be improved ...
It *could* be improved. Doing so in a way that works for more than
just some special cases -- e.g., in a way that works for ordinary
text, or graphical images, for instance, rather than just for Ruby
sources (or just C sources, or C++, or Swift, or Python, or whatever)
-- seems particularly tricky. Some degree of ignoring white-space
changes would probably help multiple cases, though.
Chris
^ permalink raw reply
* Re: git rename/moved status unreliable in ruby
From: Johannes Sixt @ 2026-05-02 7:25 UTC (permalink / raw)
To: phillip.wood; +Cc: Git Mailing List, sebastien.stettler
In-Reply-To: <026b84f4-7052-4d5b-a9ae-c2487569d1ee@gmail.com>
Am 01.05.26 um 17:30 schrieb Phillip Wood:
> Rename detection is based on how similar the two files are. Looking at
> the example you linked to below you're changing a file that looks like
>
> ...
>
> Which means that git sees that every line has changed because the
> indentation has changed.
That's correct, of course, but...
> If you want git to realize that the file has
> been renamed you could move it in one commit and then add modify it in
> the next commit.
... this is a fallacy. Splitting into two commits helps only certain
cases, in particular, when the commit that moves the files is compared
to an earlier commit, such as `git log` does. However, if a commit after
the change is compared to a commit before the move, the rename is still
not detected.
-- Hannes
^ permalink raw reply
* [PATCH v3 8/8] t9904: add tests for the new url-parse builtin
From: Matheus Afonso Martins Moreira via GitGitGadget @ 2026-05-02 5:28 UTC (permalink / raw)
To: git
Cc: Torsten Bögershausen, Ghanshyam Thakkar, Matheus Moreira,
Matheus Afonso Martins Moreira
In-Reply-To: <pull.1715.v3.git.git.1777699722.gitgitgadget@gmail.com>
From: Matheus Afonso Martins Moreira <matheus@matheusmoreira.com>
Test git URL parsing, validation and component extraction
on all documented git URL schemes and syntaxes.
Add IPv6 host coverage in URL form:
ssh://[::1]/path
ssh://user@[::1]:1234/path
git://[::1]:9418/path
http://[2001:db8::1]/path
https://[2001:db8::1]/path
In URL form the brackets are kept in the host component (RFC 3986
syntax for IPv6 literals).
Also exercise the bracketed scp short forms that t5601-clone.sh
covers via parse_connect_url:
[host]:path
[host:port]:path
[::1]:repo
user@[::1]:repo
user@[host:port]:path
In scp form, brackets are kept for IPv6 literals (two or more inner
colons) and stripped for plain hostnames or host:port pairs.
Suggested-by: Torsten Bögershausen <tboegi@web.de>
Signed-off-by: Matheus Afonso Martins Moreira <matheus@matheusmoreira.com>
---
t/meson.build | 1 +
t/t9904-url-parse.sh | 319 +++++++++++++++++++++++++++++++++++++++++++
2 files changed, 320 insertions(+)
create mode 100755 t/t9904-url-parse.sh
diff --git a/t/meson.build b/t/meson.build
index 7528e5cda5..41b389a472 100644
--- a/t/meson.build
+++ b/t/meson.build
@@ -1114,6 +1114,7 @@ integration_tests = [
't9901-git-web--browse.sh',
't9902-completion.sh',
't9903-bash-prompt.sh',
+ 't9904-url-parse.sh',
]
benchmarks = [
diff --git a/t/t9904-url-parse.sh b/t/t9904-url-parse.sh
new file mode 100755
index 0000000000..8a369d2040
--- /dev/null
+++ b/t/t9904-url-parse.sh
@@ -0,0 +1,319 @@
+#!/bin/sh
+#
+# Copyright (c) 2024 Matheus Afonso Martins Moreira
+#
+
+test_description='git url-parse tests'
+
+. ./test-lib.sh
+
+test_expect_success 'git url-parse -- ssh syntax' '
+ git url-parse "ssh://user@example.com:1234/repository/path" &&
+ git url-parse "ssh://user@example.com/repository/path" &&
+ git url-parse "ssh://example.com:1234/repository/path" &&
+ git url-parse "ssh://example.com/repository/path"
+'
+
+test_expect_success 'git url-parse -- git syntax' '
+ git url-parse "git://example.com:1234/repository/path" &&
+ git url-parse "git://example.com/repository/path"
+'
+
+test_expect_success 'git url-parse -- http syntax' '
+ git url-parse "https://example.com:1234/repository/path" &&
+ git url-parse "https://example.com/repository/path" &&
+ git url-parse "http://example.com:1234/repository/path" &&
+ git url-parse "http://example.com/repository/path"
+'
+
+test_expect_success 'git url-parse -- scp syntax' '
+ git url-parse "user@example.com:/repository/path" &&
+ git url-parse "example.com:/repository/path"
+'
+
+test_expect_success 'git url-parse -- username expansion - ssh syntax' '
+ git url-parse "ssh://user@example.com:1234/~user/repository" &&
+ git url-parse "ssh://user@example.com/~user/repository" &&
+ git url-parse "ssh://example.com:1234/~user/repository" &&
+ git url-parse "ssh://example.com/~user/repository"
+'
+
+test_expect_success 'git url-parse -- username expansion - git syntax' '
+ git url-parse "git://example.com:1234/~user/repository" &&
+ git url-parse "git://example.com/~user/repository"
+'
+
+test_expect_success 'git url-parse -- username expansion - scp syntax' '
+ git url-parse "user@example.com:~user/repository" &&
+ git url-parse "example.com:~user/repository"
+'
+
+test_expect_success 'git url-parse -- file urls' '
+ git url-parse "file:///repository/path" &&
+ git url-parse "file://"
+'
+
+test_expect_success 'git url-parse -c scheme -- ssh syntax' '
+ test ssh = "$(git url-parse -c scheme "ssh://user@example.com:1234/repository/path")" &&
+ test ssh = "$(git url-parse -c scheme "ssh://user@example.com/repository/path")" &&
+ test ssh = "$(git url-parse -c scheme "ssh://example.com:1234/repository/path")" &&
+ test ssh = "$(git url-parse -c scheme "ssh://example.com/repository/path")"
+'
+
+test_expect_success 'git url-parse -c scheme -- git syntax' '
+ test git = "$(git url-parse -c scheme "git://example.com:1234/repository/path")" &&
+ test git = "$(git url-parse -c scheme "git://example.com/repository/path")"
+'
+
+test_expect_success 'git url-parse -c scheme -- http syntax' '
+ test https = "$(git url-parse -c scheme "https://example.com:1234/repository/path")" &&
+ test https = "$(git url-parse -c scheme "https://example.com/repository/path")" &&
+ test http = "$(git url-parse -c scheme "http://example.com:1234/repository/path")" &&
+ test http = "$(git url-parse -c scheme "http://example.com/repository/path")"
+'
+
+test_expect_success 'git url-parse -c scheme -- scp syntax' '
+ test ssh = "$(git url-parse -c scheme "user@example.com:/repository/path")" &&
+ test ssh = "$(git url-parse -c scheme "example.com:/repository/path")"
+'
+
+test_expect_success 'git url-parse -c user -- ssh syntax' '
+ test user = "$(git url-parse -c user "ssh://user@example.com:1234/repository/path")" &&
+ test user = "$(git url-parse -c user "ssh://user@example.com/repository/path")" &&
+ test "" = "$(git url-parse -c user "ssh://example.com:1234/repository/path")" &&
+ test "" = "$(git url-parse -c user "ssh://example.com/repository/path")"
+'
+
+test_expect_success 'git url-parse -c user -- git syntax' '
+ test "" = "$(git url-parse -c user "git://example.com:1234/repository/path")" &&
+ test "" = "$(git url-parse -c user "git://example.com/repository/path")"
+'
+
+test_expect_success 'git url-parse -c user -- http syntax' '
+ test "" = "$(git url-parse -c user "https://example.com:1234/repository/path")" &&
+ test "" = "$(git url-parse -c user "https://example.com/repository/path")" &&
+ test "" = "$(git url-parse -c user "http://example.com:1234/repository/path")" &&
+ test "" = "$(git url-parse -c user "http://example.com/repository/path")"
+'
+
+test_expect_success 'git url-parse -c user -- scp syntax' '
+ test user = "$(git url-parse -c user "user@example.com:/repository/path")" &&
+ test "" = "$(git url-parse -c user "example.com:/repository/path")"
+'
+
+test_expect_success 'git url-parse -c password -- http syntax' '
+ test secret = "$(git url-parse -c password "https://user:secret@example.com:1234/repository/path")" &&
+ test secret = "$(git url-parse -c password "http://user:secret@example.com/repository/path")" &&
+ test "" = "$(git url-parse -c password "https://user@example.com/repository/path")" &&
+ test "" = "$(git url-parse -c password "https://example.com/repository/path")"
+'
+
+test_expect_success 'git url-parse -c host -- ssh syntax' '
+ test example.com = "$(git url-parse -c host "ssh://user@example.com:1234/repository/path")" &&
+ test example.com = "$(git url-parse -c host "ssh://user@example.com/repository/path")" &&
+ test example.com = "$(git url-parse -c host "ssh://example.com:1234/repository/path")" &&
+ test example.com = "$(git url-parse -c host "ssh://example.com/repository/path")"
+'
+
+test_expect_success 'git url-parse -c host -- git syntax' '
+ test example.com = "$(git url-parse -c host "git://example.com:1234/repository/path")" &&
+ test example.com = "$(git url-parse -c host "git://example.com/repository/path")"
+'
+
+test_expect_success 'git url-parse -c host -- http syntax' '
+ test example.com = "$(git url-parse -c host "https://example.com:1234/repository/path")" &&
+ test example.com = "$(git url-parse -c host "https://example.com/repository/path")" &&
+ test example.com = "$(git url-parse -c host "http://example.com:1234/repository/path")" &&
+ test example.com = "$(git url-parse -c host "http://example.com/repository/path")"
+'
+
+test_expect_success 'git url-parse -c host -- scp syntax' '
+ test example.com = "$(git url-parse -c host "user@example.com:/repository/path")" &&
+ test example.com = "$(git url-parse -c host "example.com:/repository/path")"
+'
+
+test_expect_success 'git url-parse -c port -- ssh syntax' '
+ test 1234 = "$(git url-parse -c port "ssh://user@example.com:1234/repository/path")" &&
+ test "" = "$(git url-parse -c port "ssh://user@example.com/repository/path")" &&
+ test 1234 = "$(git url-parse -c port "ssh://example.com:1234/repository/path")" &&
+ test "" = "$(git url-parse -c port "ssh://example.com/repository/path")"
+'
+
+test_expect_success 'git url-parse -c port -- git syntax' '
+ test 1234 = "$(git url-parse -c port "git://example.com:1234/repository/path")" &&
+ test "" = "$(git url-parse -c port "git://example.com/repository/path")"
+'
+
+test_expect_success 'git url-parse -c port -- http syntax' '
+ test 1234 = "$(git url-parse -c port "https://example.com:1234/repository/path")" &&
+ test "" = "$(git url-parse -c port "https://example.com/repository/path")" &&
+ test 1234 = "$(git url-parse -c port "http://example.com:1234/repository/path")" &&
+ test "" = "$(git url-parse -c port "http://example.com/repository/path")"
+'
+
+test_expect_success 'git url-parse -c port -- scp syntax' '
+ test "" = "$(git url-parse -c port "user@example.com:/repository/path")" &&
+ test "" = "$(git url-parse -c port "example.com:/repository/path")"
+'
+
+test_expect_success 'git url-parse -c path -- ssh syntax' '
+ test "/repository/path" = "$(git url-parse -c path "ssh://user@example.com:1234/repository/path")" &&
+ test "/repository/path" = "$(git url-parse -c path "ssh://user@example.com/repository/path")" &&
+ test "/repository/path" = "$(git url-parse -c path "ssh://example.com:1234/repository/path")" &&
+ test "/repository/path" = "$(git url-parse -c path "ssh://example.com/repository/path")"
+'
+
+test_expect_success 'git url-parse -c path -- git syntax' '
+ test "/repository/path" = "$(git url-parse -c path "git://example.com:1234/repository/path")" &&
+ test "/repository/path" = "$(git url-parse -c path "git://example.com/repository/path")"
+'
+
+test_expect_success 'git url-parse -c path -- http syntax' '
+ test "/repository/path" = "$(git url-parse -c path "https://example.com:1234/repository/path")" &&
+ test "/repository/path" = "$(git url-parse -c path "https://example.com/repository/path")" &&
+ test "/repository/path" = "$(git url-parse -c path "http://example.com:1234/repository/path")" &&
+ test "/repository/path" = "$(git url-parse -c path "http://example.com/repository/path")"
+'
+
+test_expect_success 'git url-parse -c path -- scp syntax' '
+ test "/repository/path" = "$(git url-parse -c path "user@example.com:/repository/path")" &&
+ test "/repository/path" = "$(git url-parse -c path "example.com:/repository/path")"
+'
+
+test_expect_success 'git url-parse -c path -- username expansion - ssh syntax' '
+ test "~user/repository" = "$(git url-parse -c path "ssh://user@example.com:1234/~user/repository")" &&
+ test "~user/repository" = "$(git url-parse -c path "ssh://user@example.com/~user/repository")" &&
+ test "~user/repository" = "$(git url-parse -c path "ssh://example.com:1234/~user/repository")" &&
+ test "~user/repository" = "$(git url-parse -c path "ssh://example.com/~user/repository")"
+'
+
+test_expect_success 'git url-parse -c path -- username expansion - git syntax' '
+ test "~user/repository" = "$(git url-parse -c path "git://example.com:1234/~user/repository")" &&
+ test "~user/repository" = "$(git url-parse -c path "git://example.com/~user/repository")"
+'
+
+test_expect_success 'git url-parse -c path -- username expansion - scp syntax' '
+ test "~user/repository" = "$(git url-parse -c path "user@example.com:~user/repository")" &&
+ test "~user/repository" = "$(git url-parse -c path "example.com:~user/repository")"
+'
+
+test_expect_success 'git url-parse -c path -- username expansion strips query and fragment' '
+ test "~user/repository" = "$(git url-parse -c path "ssh://example.com/~user/repository?query")" &&
+ test "~user/repository" = "$(git url-parse -c path "ssh://example.com/~user/repository#fragment")" &&
+ test "~user/repository" = "$(git url-parse -c path "git://example.com/~user/repository?query")" &&
+ test "~user/repository" = "$(git url-parse -c path "user@example.com:~user/repository?query")"
+'
+
+test_expect_success 'git url-parse -- ssh syntax with IPv6' '
+ git url-parse "ssh://user@[::1]:1234/repository/path" &&
+ git url-parse "ssh://user@[::1]/repository/path" &&
+ git url-parse "ssh://[::1]:1234/repository/path" &&
+ git url-parse "ssh://[::1]/repository/path" &&
+ git url-parse "ssh://[2001:db8::1]/repository/path"
+'
+
+test_expect_success 'git url-parse -- git syntax with IPv6' '
+ git url-parse "git://[::1]:9418/repository/path" &&
+ git url-parse "git://[::1]/repository/path"
+'
+
+test_expect_success 'git url-parse -- http syntax with IPv6' '
+ git url-parse "https://[::1]:1234/repository/path" &&
+ git url-parse "https://[::1]/repository/path" &&
+ git url-parse "http://[2001:db8::1]/repository/path"
+'
+
+test_expect_success 'git url-parse -c host -- IPv6 in URL form' '
+ test "[::1]" = "$(git url-parse -c host "ssh://user@[::1]:1234/repository/path")" &&
+ test "[::1]" = "$(git url-parse -c host "ssh://[::1]/repository/path")" &&
+ test "[2001:db8::1]" = "$(git url-parse -c host "ssh://[2001:db8::1]/repository/path")" &&
+ test "[::1]" = "$(git url-parse -c host "git://[::1]/repository/path")" &&
+ test "[2001:db8::1]" = "$(git url-parse -c host "https://[2001:db8::1]/repository/path")"
+'
+
+test_expect_success 'git url-parse -c port -- IPv6 in URL form' '
+ test 1234 = "$(git url-parse -c port "ssh://user@[::1]:1234/repository/path")" &&
+ test "" = "$(git url-parse -c port "ssh://[::1]/repository/path")" &&
+ test 9418 = "$(git url-parse -c port "git://[::1]:9418/repository/path")"
+'
+
+test_expect_success 'git url-parse -- scp syntax with IPv6' '
+ git url-parse "[::1]:repository/path" &&
+ git url-parse "user@[::1]:repository/path" &&
+ git url-parse "[2001:db8::1]:repo"
+'
+
+test_expect_success 'git url-parse -- scp syntax with bracketed hostname' '
+ git url-parse "[myhost]:src" &&
+ git url-parse "user@[myhost]:src"
+'
+
+test_expect_success 'git url-parse -- scp syntax with bracketed host:port' '
+ git url-parse "[myhost:123]:src" &&
+ git url-parse "user@[myhost:123]:src"
+'
+
+test_expect_success 'git url-parse -c host -- scp+IPv6' '
+ test "[::1]" = "$(git url-parse -c host "[::1]:repository/path")" &&
+ test "[::1]" = "$(git url-parse -c host "user@[::1]:repository/path")" &&
+ test "[2001:db8::1]" = "$(git url-parse -c host "[2001:db8::1]:repo")"
+'
+
+test_expect_success 'git url-parse -c path -- scp+IPv6' '
+ test "/repository/path" = "$(git url-parse -c path "[::1]:/repository/path")" &&
+ test "/repository/path" = "$(git url-parse -c path "[::1]:repository/path")" &&
+ test "/repo" = "$(git url-parse -c path "[2001:db8::1]:repo")"
+'
+
+test_expect_success 'git url-parse -c host,port,path -- scp [host:port]:src' '
+ test myhost = "$(git url-parse -c host "[myhost:123]:src")" &&
+ test 123 = "$(git url-parse -c port "[myhost:123]:src")" &&
+ test "/src" = "$(git url-parse -c path "[myhost:123]:src")"
+'
+
+test_expect_success 'git url-parse -c host,path -- scp [host]:src' '
+ test myhost = "$(git url-parse -c host "[myhost]:src")" &&
+ test "/src" = "$(git url-parse -c path "[myhost]:src")"
+'
+
+test_expect_success 'git url-parse -c user -- scp with user@ and brackets' '
+ test user = "$(git url-parse -c user "user@[::1]:repo")" &&
+ test user = "$(git url-parse -c user "user@[myhost:123]:src")" &&
+ test user = "$(git url-parse -c user "user@[myhost]:src")"
+'
+
+test_expect_success 'git url-parse -- scp+IPv6 with username expansion' '
+ test "~user/repo" = "$(git url-parse -c path "[::1]:~user/repo")" &&
+ test "~user/repo" = "$(git url-parse -c path "user@[::1]:~user/repo")"
+'
+
+test_expect_success 'git url-parse fails on invalid URL' '
+ test_must_fail git url-parse "not a url"
+'
+
+test_expect_success 'git url-parse helpful error for absolute local path' '
+ test_must_fail git url-parse "/abs/path" 2>err &&
+ test_grep "is not a URL" err &&
+ test_grep "file:///" err
+'
+
+test_expect_success 'git url-parse helpful error for relative local path' '
+ test_must_fail git url-parse "./rel" 2>err &&
+ test_grep "is not a URL" err &&
+ test_grep "absolute path" err
+'
+
+test_expect_success 'git url-parse fails on unknown -c component name' '
+ test_must_fail git url-parse -c bogus "https://example.com/repo"
+'
+
+test_expect_success 'git url-parse fails on URL missing host' '
+ test_must_fail git url-parse "https://"
+'
+
+test_expect_success 'git url-parse with no URL prints usage' '
+ test_must_fail git url-parse 2>err &&
+ test_grep "usage:" err
+'
+
+test_done
--
gitgitgadget
^ permalink raw reply related
* [PATCH v3 7/8] doc: describe the url-parse builtin
From: Matheus Afonso Martins Moreira via GitGitGadget @ 2026-05-02 5:28 UTC (permalink / raw)
To: git
Cc: Torsten Bögershausen, Ghanshyam Thakkar, Matheus Moreira,
Matheus Afonso Martins Moreira
In-Reply-To: <pull.1715.v3.git.git.1777699722.gitgitgadget@gmail.com>
From: Matheus Afonso Martins Moreira <matheus@matheusmoreira.com>
The new url-parse builtin validates git URLs
and optionally extracts their components.
Helped-by: Ghanshyam Thakkar <shyamthakkar001@gmail.com>
Signed-off-by: Matheus Afonso Martins Moreira <matheus@matheusmoreira.com>
---
Documentation/git-url-parse.adoc | 80 ++++++++++++++++++++++++++++++++
Documentation/meson.build | 1 +
2 files changed, 81 insertions(+)
create mode 100644 Documentation/git-url-parse.adoc
diff --git a/Documentation/git-url-parse.adoc b/Documentation/git-url-parse.adoc
new file mode 100644
index 0000000000..9d0d93da4a
--- /dev/null
+++ b/Documentation/git-url-parse.adoc
@@ -0,0 +1,80 @@
+git-url-parse(1)
+================
+
+NAME
+----
+git-url-parse - Parse and extract git URL components
+
+SYNOPSIS
+--------
+[synopsis]
+git url-parse [-c <component>] [--] <url>...
+
+DESCRIPTION
+-----------
+
+Git supports many ways to specify URLs, some of them non-standard.
+For example, git supports the scp style [user@]host:[path] format.
+This command eases interoperability with git URLs by enabling the
+parsing and extraction of the components of all git URLs.
+
+Any syntactically valid URL is parsed, even if the scheme is not one
+git supports for fetching or pushing.
+
+OPTIONS
+-------
+
+`-c <component>`::
+`--component <component>`::
+ Extract the _<component>_ component from the given Git URLs.
+ _<component>_ can be one of:
+ `scheme`, `user`, `password`, `host`, `port`, `path`.
+
+OUTPUT
+------
+
+When `--component` is given, the requested component of each URL
+is printed on its own line, in the order the URLs were given. If
+the URL has no such component (for example, a port in a URL that
+does not specify one), an empty line is printed in its place.
+
+When `--component` is not given, no output is produced. The exit
+status is zero if every URL parses successfully and non-zero
+otherwise, allowing the command to be used purely as a validator.
+
+EXAMPLES
+--------
+
+* Print the host name:
++
+------------
+$ git url-parse --component host https://example.com/user/repo
+example.com
+------------
+
+* Print the path:
++
+------------
+$ git url-parse --component path https://example.com/user/repo
+/user/repo
+$ git url-parse --component path example.com:~user/repo
+~user/repo
+$ git url-parse --component path example.com:user/repo
+/user/repo
+------------
+
+* Validate URLs without outputting anything:
++
+------------
+$ git url-parse https://example.com/user/repo example.com:~user/repo
+------------
+
+SEE ALSO
+--------
+linkgit:git-clone[1],
+linkgit:git-fetch[1],
+linkgit:git-config[1]
+
+GIT
+---
+Part of the linkgit:git[1] suite
diff --git a/Documentation/meson.build b/Documentation/meson.build
index d6365b888b..32c8606a80 100644
--- a/Documentation/meson.build
+++ b/Documentation/meson.build
@@ -155,6 +155,7 @@ manpages = {
'git-update-server-info.adoc' : 1,
'git-upload-archive.adoc' : 1,
'git-upload-pack.adoc' : 1,
+ 'git-url-parse.adoc' : 1,
'git-var.adoc' : 1,
'git-verify-commit.adoc' : 1,
'git-verify-pack.adoc' : 1,
--
gitgitgadget
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox