* Re: What's cooking in git.git (May 2026, #04)
From: Jeff King @ 2026-05-19 19:19 UTC (permalink / raw)
To: Taylor Blau; +Cc: Junio C Hamano, git
In-Reply-To: <agyPJa3E2lPI9K/G@nand.local>
On Tue, May 19, 2026 at 12:26:13PM -0400, Taylor Blau wrote:
> > * tb/incremental-midx-part-3.3 (2026-04-29) 16 commits
> [...]
> Apologies, I didn't realize you were waiting on these until seeing this
> WC report. I sent an extremely tiny reroll
>
> https://lore.kernel.org/git/cover.1779206239.git.me@ttaylorr.com/
>
> that addresses the two outstanding comments you linked. They are very
> minor changes, and queueing either version of the series would be
> equally fine IMHO.
I peeked at the v4 range diff, and it looks good to me.
-Peff
^ permalink raw reply
* Re: [PATCH] http: fix memory leak in fetch_and_setup_pack_index()
From: Jeff King @ 2026-05-19 19:17 UTC (permalink / raw)
To: LorenzoPegorari; +Cc: git, Taylor Blau, Junio C Hamano, Patrick Steinhardt, fox
In-Reply-To: <agx5tblaCZNsYEBq@lorenzo-VM>
On Tue, May 19, 2026 at 04:54:45PM +0200, LorenzoPegorari wrote:
> Inside the function `fetch_and_setup_pack_index()`, when the pack
> obtained using `fetch_pack_index()` fails to be verified by
> `parse_pack_index()`, the function returns without closing and freeing
> said pack.
>
> Fix this by calling `close_pack_index()` to munmap the index file for
> the leaking pack (which might have been mmapped by `fetch_pack_index()`
> or `verify_pack_index()`), and then free it.
OK, I agree we are leaking here, but after reading the patch I'm left
with a few questions.
> ret = verify_pack_index(new_pack);
> - if (!ret)
> - close_pack_index(new_pack);
> +
> + close_pack_index(new_pack);
This part was a little confusing at first, because it looked like we are
already closing the index. But we were doing so on _success_, not on
failure. Which is a little funny since the point is to be able to read
from it later, but OK.
At any rate, that is an existing oddity, and I agree that closing it
before freeing the struct is obviously the right thing to do.
> free(tmp_idx);
> - if (ret)
> + if (ret) {
> + free(new_pack);
> return -1;
> + }
And here we free the actual struct. Good.
But this existing free(tmp_idx) is what puzzles me. We do not need the
filename anymore regardless of success or failure, so freeing it makes
sense. But earlier in the function we have:
new_pack = parse_pack_index(the_repository, sha1, tmp_idx);
if (!new_pack) {
unlink(tmp_idx);
free(tmp_idx);
return -1; /* parse_pack_index() already issued error message */
}
So on parse failure we actually unlink it, but not on verification
failure. Which seems like it would leave cruft after the process ends.
And I suspect we probably we did prior to 63aca3f7f1 (dumb-http: store
downloaded pack idx as tempfile, 2024-10-25), when we started
registering it as a tempfile to be deleted at process exit.
So I _think_ we could get away with dropping the existing unlink() call
and just let it get cleaned up at process exit. But if we are going to
keep it, do we want to also unlink() in this error path? At which point
it might make more sense to have an "out" label to consolidate all of
this cleanup.
If we are going to unlink() here it may also make sense to just return
the tempfile struct from fetch_pack_index(), and then we can call
delete_tempfile() on it. See the in-code comment in 63aca3f7f1 which
mentions this hackery.
So I dunno. I think your patch is doing the right thing as-is, but it
may be worth taking a moment to clean this up a bit further.
-Peff
^ permalink raw reply
* Re: [PATCH v1 10/11] git-gui: improve worktree discovery
From: Mark Levedahl @ 2026-05-19 19:00 UTC (permalink / raw)
To: Johannes Sixt; +Cc: egg_mushroomcow, bootaina702, git
In-Reply-To: <5081fcc5-19b5-49aa-a33c-2c13aba7edb1@kdbg.org>
On 5/19/26 4:16 AM, Johannes Sixt wrote:
> Am 16.05.26 um 17:28 schrieb Mark Levedahl:
>> On 5/16/26 4:16 AM, Johannes Sixt wrote:
>>> Am 14.05.26 um 16:33 schrieb Mark Levedahl:
>>>> + if {[is_gitvars_error $err]} {
>>>> + exit 1
>>>> + }
>>>> + set _gitworktree {}
>>>> + set _prefix {}
>>>> + if {[is_enabled bare]} {
>>>> + cd $_gitdir
>>> Why change the directory here? If we run `git gui browser master dir` we
>>> do not want to change the directory in an uncontrolled manner. The
>>> argument parser will want to check for the existence of files, and then
>>> we do not want to operate from a random directory.
>>>
>>> Also, I think that the check must be for [is_bare] and not [is_enabled
>>> bare].
>> [is_enabled_bare] is correct. This code handles the case:
>> - neither the startup directory nor GIT_WORK_TREE are useable worktrees, so [is_bare]
>> is currently true.
>> - the command given is browser or blame so a worktree is not needed. We can proceed.
> But in the case where the command is browser or blame, the argument
> parser must later check for the existence of files, provided that a
> worktree is present. But this conditional would change directory to
> somewhere that is not a worktree at all even though a worktree is
> available. So, I am still convinced that [is_bare] is correct.
I did change this. but...
I have reworked the blame/browser parser so it fully matches git blame parsing for the
single rev + path (or path rev) cases, all now do the same thing with or without a
worktree as they all work from git history, so having a worktree becomes almost moot (I
found issues with how git-gui handles --, it is dead wrong in some cases if the intent is
to match git-blame). What doesn't change is what blame displays based upon uncommitted or
staged changes in the worktree (if it does anything, but comments from 2007 suggest it
does, I haven't tested). I've done nothing to change that, only finding the args to pass
in to browser / blame. So, if a worktree exists, blame will still use the info there as it
does now.
Mark
^ permalink raw reply
* Re: [PATCH v1 11/11] git-gui: add gui and pick as explicit subcommands
From: Mark Levedahl @ 2026-05-19 18:45 UTC (permalink / raw)
To: Johannes Sixt; +Cc: egg_mushroomcow, bootaina702, git
In-Reply-To: <3b16fbc6-074b-410d-861e-6f77794b02a0@kdbg.org>
On 5/19/26 4:21 AM, Johannes Sixt wrote:
> Am 16.05.26 um 17:42
>
>
> I think I would be happier with the structure
>
> if not subcommand pick
> discover gitdir
> if error
> set subcommand pick
>
> if subcommand pick
> pick_repo
> set subcommand gui
>
> because this clarifies that pick_repo must erase all current traces of
> GIT_DIR and GIT_WORK_TREE from the envionment and must complete with a
> valid setup.
>
> With the structure in the proposed patch
>
> if subcommand pick
> pick_repo
> set subcommand gui
>
> discover gitdir
> if error
> pick_repo
>
> we still need the same operation of pick_repo, but after it runs due to
> a pick command, we go into "discover gitdir" mode in an already modified
> environment, something that does not happen if pick_repo runs due to the
> error in the gitdir discovery.
>
> -- Hannes
>
What I have now is
if (enabled gitdir discovery) {
discover gitdir
maybe an error occurs and gitdir remains {}
}
if (enabled pick && gitdir eq {}) {
unset GIT_DIR .. (just to be friendly, could throw an error instead...)
pick
discover gitdir to VALIDATE pick gave us a good thing
}
if no gitdir {
error No Repository
}
then on to worktree discovery (which also validates what pick returns as pick may not have
done so).
So, we can independently enable normal discovery or pick, and with both enabled pick can
be used to recover from an error in normal discovery. Either way, there is only one block
of code running pick, it is not a separate proc invoked multiple places. I hope this
scratches your itch.
Still scrubbing things, should send out in a day or two.
Mark
^ permalink raw reply
* Re: [PATCH v6 0/8] fetch: rework negotiation tip options
From: Matthew John Cheetham @ 2026-05-19 16:48 UTC (permalink / raw)
To: Derrick Stolee via GitGitGadget, git; +Cc: gitster, ps, Derrick Stolee
In-Reply-To: <pull.2085.v6.git.1779207896.gitgitgadget@gmail.com>
On 2026-05-19 17:24, Derrick Stolee via GitGitGadget wrote:
> Updates in v6
> =============
>
> Corrected reviewed-by annotations in commit messages.
>
> Thanks, -Stolee
>
v6 look good to me!
Thanks,
Matthew
^ permalink raw reply
* [PATCH 9/9] t: add tests for external notes command
From: Siddh Raman Pant @ 2026-05-19 16:30 UTC (permalink / raw)
To: git
Cc: Calvin Wan, Patrick Steinhardt, Elijah Newren,
Kristoffer Haugsbakk, Junio C Hamano
In-Reply-To: <cover.1779207350.git.siddh.raman.pant@oracle.com>
Assisted-by: Codex:gpt-5.5-xhigh-fast
Signed-off-by: Siddh Raman Pant <siddh.raman.pant@oracle.com>
---
t/helper/meson.build | 1 +
t/helper/test-external-notes | 64 +++
t/helper/test-notes-external-config-reset.c | 20 +
t/helper/test-tool.c | 1 +
t/helper/test-tool.h | 1 +
t/lib-notes.sh | 19 +
t/t3206-range-diff.sh | 68 +++
t/t3301-notes.sh | 437 ++++++++++++++++++++
t/t6120-describe.sh | 17 +
9 files changed, 628 insertions(+)
create mode 100755 t/helper/test-external-notes
create mode 100644 t/helper/test-notes-external-config-reset.c
create mode 100644 t/lib-notes.sh
diff --git a/t/helper/meson.build b/t/helper/meson.build
index 675e64c0101b..739614b90e78 100644
--- a/t/helper/meson.build
+++ b/t/helper/meson.build
@@ -35,6 +35,7 @@ test_tool_sources = [
'test-mergesort.c',
'test-mktemp.c',
'test-name-hash.c',
+ 'test-notes-external-config-reset.c',
'test-online-cpus.c',
'test-pack-deltas.c',
'test-pack-mtimes.c',
diff --git a/t/helper/test-external-notes b/t/helper/test-external-notes
new file mode 100755
index 000000000000..5e9dde3977ab
--- /dev/null
+++ b/t/helper/test-external-notes
@@ -0,0 +1,64 @@
+#!/bin/sh
+
+prefix=${TEST_EXTERNAL_NOTES_PREFIX:-external-notes}
+response=${TEST_EXTERNAL_NOTES_RESPONSE:-ok}
+line_ending=${TEST_EXTERNAL_NOTES_LINE_ENDING:-lf}
+exit_after_response=${TEST_EXTERNAL_NOTES_EXIT_AFTER_RESPONSE:-}
+exit_delay=${TEST_EXTERNAL_NOTES_EXIT_DELAY:-}
+delay=${TEST_EXTERNAL_NOTES_DELAY:-}
+char_delay=${TEST_EXTERNAL_NOTES_CHAR_DELAY:-}
+ignore_term=${TEST_EXTERNAL_NOTES_IGNORE_TERM:-}
+
+newline='\n'
+case "$line_ending" in
+crlf)
+ newline='\r\n'
+ ;;
+none)
+ newline=
+ ;;
+esac
+
+echo start >>"$prefix-starts"
+
+test "$ignore_term" = true && trap '' TERM
+
+emit_output() {
+ if test -n "$char_delay"
+ then
+ LC_ALL=C
+ payload=$(printf "$@"; printf .)
+ payload=${payload%.}
+
+ while test -n "$payload"
+ do
+ char=${payload%"${payload#?}"}
+ printf '%s' "$char" || return 1
+ payload=${payload#?}
+ sleep "$char_delay" || return 1
+ done
+ else
+ printf "$@"
+ fi
+}
+
+while IFS= read -r commit; do
+ if test "${TEST_EXTERNAL_NOTES_BODY+x}" = x
+ then
+ note=$TEST_EXTERNAL_NOTES_BODY
+ else
+ note=$commit
+ fi
+ printf "%s\n" "$commit" >>"$prefix-requests"
+ test -z "$delay" || sleep "$delay"
+ if test "$response" = missing
+ then
+ emit_output "%s missing%b" "$commit" "$newline"
+ else
+ emit_output "%s ok %d%b%s%b" \
+ "$commit" "${#note}" "$newline" "$note" "$newline"
+ fi
+ test "$exit_after_response" = true && break
+done
+
+test -z "$exit_delay" || sleep "$exit_delay"
diff --git a/t/helper/test-notes-external-config-reset.c b/t/helper/test-notes-external-config-reset.c
new file mode 100644
index 000000000000..1c6b26e3b49a
--- /dev/null
+++ b/t/helper/test-notes-external-config-reset.c
@@ -0,0 +1,20 @@
+#include "test-tool.h"
+#include "notes-external.h"
+
+int cmd__notes_external_config_reset(int argc, const char **argv UNUSED)
+{
+ if (argc != 1)
+ die("usage: test-tool notes-external-config-reset");
+
+ set_external_notes_command("helper");
+ set_external_notes_command_name("label");
+ set_external_notes_command_timeout_ms(250);
+ set_external_notes_for_grep(1);
+ reset_external_notes_command();
+
+ printf("configured=%d\n", external_notes_command_configured());
+ printf("name=%s\n", external_notes_command_name());
+ printf("timeout_ms=%d\n", external_notes_command_timeout_ms());
+ printf("grep=%d\n", external_notes_for_grep_enabled());
+ return 0;
+}
diff --git a/t/helper/test-tool.c b/t/helper/test-tool.c
index a7abc618b388..31bb2d1dca47 100644
--- a/t/helper/test-tool.c
+++ b/t/helper/test-tool.c
@@ -45,6 +45,7 @@ static struct test_cmd cmds[] = {
{ "mergesort", cmd__mergesort },
{ "mktemp", cmd__mktemp },
{ "name-hash", cmd__name_hash },
+ { "notes-external-config-reset", cmd__notes_external_config_reset },
{ "online-cpus", cmd__online_cpus },
{ "pack-deltas", cmd__pack_deltas },
{ "pack-mtimes", cmd__pack_mtimes },
diff --git a/t/helper/test-tool.h b/t/helper/test-tool.h
index 7f150fa1eb9a..ff25f0a29cf2 100644
--- a/t/helper/test-tool.h
+++ b/t/helper/test-tool.h
@@ -38,6 +38,7 @@ int cmd__match_trees(int argc, const char **argv);
int cmd__mergesort(int argc, const char **argv);
int cmd__mktemp(int argc, const char **argv);
int cmd__name_hash(int argc, const char **argv);
+int cmd__notes_external_config_reset(int argc, const char **argv);
int cmd__online_cpus(int argc, const char **argv);
int cmd__pack_deltas(int argc, const char **argv);
int cmd__pack_mtimes(int argc, const char **argv);
diff --git a/t/lib-notes.sh b/t/lib-notes.sh
new file mode 100644
index 000000000000..07422540d58f
--- /dev/null
+++ b/t/lib-notes.sh
@@ -0,0 +1,19 @@
+# Helpers for scripts testing notes behavior.
+
+# notes.externalCommand is run through a shell, so quote the path.
+external_notes_command=$(
+ printf "%s\n" "$TEST_DIRECTORY/helper/test-external-notes" |
+ sed "s/'/'\\\\''/g; s/^/'/; s/$/'/"
+)
+
+# The helper above is a shell script. Few Windows CI tests (3 out of 10
+# in matrix) are spending more than the production default timeout just
+# starting the shell and exchanging the first response, so tests that
+# are not about timeout behavior fail. So let us opt into a wider 1s
+# deadline for Windows instead of 100ms.
+external_notes_command_timeout_config=
+if test_have_prereq MINGW
+then
+ _timeout_config="notes.externalCommandTimeoutMs=1000"
+ external_notes_command_timeout_config="-c $_timeout_config"
+fi
diff --git a/t/t3206-range-diff.sh b/t/t3206-range-diff.sh
index 1e812df806bb..96adeb9bc4fe 100755
--- a/t/t3206-range-diff.sh
+++ b/t/t3206-range-diff.sh
@@ -6,6 +6,7 @@ GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
. ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-notes.sh
# Note that because of the range-diff's heuristics, test_commit does more
# harm than good. We need some real history.
@@ -690,6 +691,37 @@ test_expect_success 'range-diff with --notes=custom does not show default notes'
grep "## Notes (custom) ##" actual
'
+test_expect_success 'range-diff with --external-notes' '
+ topic_oid=$(git rev-parse topic) &&
+ unmodified_oid=$(git rev-parse unmodified) &&
+ git -c notes.externalCommand="$external_notes_command" \
+ $external_notes_command_timeout_config \
+ range-diff --no-color --external-notes \
+ main..topic main..unmodified >actual &&
+ test_grep "## Notes (external) ##" actual &&
+ test_grep "^ - $topic_oid$" actual &&
+ test_grep "^ + $unmodified_oid$" actual &&
+ ! grep "## Notes ##" actual
+'
+
+test_expect_success 'range-diff with disabled external notes' '
+ test_when_finished "git notes remove topic unmodified || :" &&
+ git notes add -m "topic note" topic &&
+ git notes add -m "unmodified note" unmodified &&
+ TEST_EXTERNAL_NOTES_PREFIX=range-diff-external-notes \
+ git -c notes.externalCommand="$external_notes_command" \
+ range-diff --no-color --external-notes --no-external-notes \
+ main..topic main..unmodified >actual &&
+ cat >expect <<-EOF &&
+ 1: $(test_oid t1) = 1: $(test_oid u1) s/5/A/
+ 2: $(test_oid t2) = 2: $(test_oid u2) s/4/A/
+ 3: $(test_oid t3) = 3: $(test_oid u3) s/11/B/
+ 4: $(test_oid t4) = 4: $(test_oid u4) s/12/B/
+ EOF
+ test_cmp expect actual &&
+ test_path_is_missing range-diff-external-notes-starts
+'
+
test_expect_success 'format-patch --range-diff does not compare notes by default' '
test_when_finished "git notes remove topic unmodified || :" &&
git notes add -m "topic note" topic &&
@@ -780,6 +812,42 @@ test_expect_success 'format-patch --range-diff with --notes' '
test_cmp expect actual
'
+test_expect_success 'format-patch --range-diff with --external-notes' '
+ topic_oid=$(git rev-parse topic) &&
+ unmodified_oid=$(git rev-parse unmodified) &&
+ test_when_finished "rm -f 000?-*" &&
+ git -c notes.externalCommand="$external_notes_command" \
+ $external_notes_command_timeout_config \
+ format-patch --external-notes --cover-letter --range-diff=$prev \
+ main..unmodified >actual &&
+ test_line_count = 5 actual &&
+ test_grep "^Range-diff:$" 0000-* &&
+ test_grep "## Notes (external) ##" 0000-* &&
+ test_grep "^ - $topic_oid$" 0000-* &&
+ test_grep "^ + $unmodified_oid$" 0000-* &&
+ ! grep "## Notes ##" 0000-*
+'
+
+test_expect_success 'format-patch --range-diff with disabled external notes' '
+ test_when_finished "git notes remove topic unmodified || :" &&
+ git notes add -m "topic note" topic &&
+ git notes add -m "unmodified note" unmodified &&
+ test_when_finished "rm -f 000?-*" &&
+ TEST_EXTERNAL_NOTES_PREFIX=range-diff-external-notes \
+ git -c notes.externalCommand="$external_notes_command" \
+ format-patch --external-notes --no-external-notes \
+ --cover-letter --range-diff=$prev main..unmodified >actual &&
+ test_line_count = 5 actual &&
+ test_grep "^Range-diff:$" 0000-* &&
+ grep "= 1: .* s/5/A" 0000-* &&
+ grep "= 2: .* s/4/A" 0000-* &&
+ grep "= 3: .* s/11/B" 0000-* &&
+ grep "= 4: .* s/12/B" 0000-* &&
+ ! grep "Notes" 0000-* &&
+ ! grep "note" 0000-* &&
+ test_path_is_missing range-diff-external-notes-starts
+'
+
test_expect_success 'format-patch --range-diff with format.notes config' '
test_when_finished "git notes remove topic unmodified || :" &&
git notes add -m "topic note" topic &&
diff --git a/t/t3301-notes.sh b/t/t3301-notes.sh
index 27439010dfbc..5a162dff3917 100755
--- a/t/t3301-notes.sh
+++ b/t/t3301-notes.sh
@@ -6,6 +6,7 @@
test_description='Test commit notes'
. ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-notes.sh
write_script fake_editor <<\EOF
echo "$MSG" >"$1"
@@ -16,6 +17,11 @@ export GIT_EDITOR
indent=" "
+run_with_limited_time () (
+ { set +x; } 2>/dev/null
+ "$PERL_PATH" -e 'alarm shift; exec @ARGV' -- "$@"
+)
+
test_expect_success 'cannot annotate non-existing HEAD' '
test_must_fail env MSG=3 git notes add
'
@@ -909,6 +915,437 @@ test_expect_success 'displayed notes are used for grep matching' '
test_must_be_empty actual
'
+test_expect_success 'notes.externalCommand shows external notes from protected config' '
+ commit=$(git rev-parse HEAD) &&
+ parent=$(git rev-parse HEAD^) &&
+ rm -f external-notes-starts external-notes-requests &&
+ git -c notes.externalCommand="$external_notes_command" \
+ $external_notes_command_timeout_config \
+ log -2 >actual &&
+ test_line_count = 1 external-notes-starts &&
+ {
+ printf "%s\n" "$commit" &&
+ printf "%s\n" "$parent"
+ } >expect-requests &&
+ test_cmp expect-requests external-notes-requests &&
+ test_grep "Notes (external):" actual &&
+ test_grep "^ $commit$" actual &&
+ test_grep "^ $parent$" actual
+'
+
+test_expect_success PERL,EXECKEEPSPID 'notes.externalCommand terminates helper during exit cleanup' '
+ commit=$(git rev-parse HEAD) &&
+ test_env TEST_EXTERNAL_NOTES_EXIT_DELAY=10 \
+ run_with_limited_time 2 \
+ git -c notes.externalCommand="$external_notes_command" \
+ $external_notes_command_timeout_config \
+ log --external-notes -1 >actual &&
+ test_grep "^Notes (external):$" actual &&
+ test_grep "^ $commit$" actual
+'
+
+test_expect_success 'notes.externalCommandName labels external notes' '
+ commit=$(git rev-parse HEAD) &&
+ git -c notes.externalCommand="$external_notes_command" \
+ $external_notes_command_timeout_config \
+ -c notes.externalCommandName=commit-id log -1 >actual &&
+ test_grep "Notes (commit-id):" actual &&
+ test_grep "^ $commit$" actual
+'
+
+test_expect_success 'notes.externalCommandName is rendered literally' '
+ commit=$(git rev-parse HEAD) &&
+ git -c notes.externalCommand="$external_notes_command" \
+ $external_notes_command_timeout_config \
+ -c notes.externalCommandName=refs/notes/commits \
+ log --external-notes -1 >actual &&
+ test_grep "^Notes (refs/notes/commits):$" actual &&
+ ! grep "^Notes:$" actual &&
+ test_grep "^ $commit$" actual
+'
+
+test_expect_success 'notes.externalCommandTimeoutMs rejects negative values' '
+ test_must_fail git -c notes.externalCommand="$external_notes_command" \
+ -c notes.externalCommandTimeoutMs=-1 log -1 2>err &&
+ test_grep "notes.externalCommandTimeoutMs must be non-negative" err
+'
+
+test_expect_success 'notes.externalCommandTimeoutMs times out delayed response' '
+ git log -1 >expect &&
+ test_env TEST_EXTERNAL_NOTES_DELAY=1 \
+ git -c notes.externalCommand="$external_notes_command" \
+ -c notes.externalCommandTimeoutMs=1 \
+ log -1 >actual 2>err &&
+ test_cmp expect actual &&
+ test_grep "notes.externalCommand failed" err &&
+ test_line_count = 1 err
+'
+
+test_expect_success 'notes.externalCommandTimeoutMs applies to whole response' '
+ git log -1 >expect &&
+ test_env TEST_EXTERNAL_NOTES_BODY=x \
+ TEST_EXTERNAL_NOTES_CHAR_DELAY=0.02 \
+ git -c notes.externalCommand="$external_notes_command" \
+ -c notes.externalCommandTimeoutMs=50 \
+ log -1 >actual 2>err &&
+ test_cmp expect actual &&
+ test_grep "notes.externalCommand failed" err &&
+ test_line_count = 1 err
+'
+
+test_expect_success PERL,EXECKEEPSPID 'notes.externalCommandTimeoutMs terminates timed-out helper' '
+ git log -1 >expect &&
+ test_env TEST_EXTERNAL_NOTES_DELAY=10 \
+ run_with_limited_time 2 \
+ git -c notes.externalCommand="$external_notes_command" \
+ -c notes.externalCommandTimeoutMs=1 \
+ log -1 >actual 2>err &&
+ test_cmp expect actual &&
+ test_grep "notes.externalCommand failed" err &&
+ test_line_count = 1 err
+'
+
+test_expect_success PERL,EXECKEEPSPID 'notes.externalCommandTimeoutMs force-kills timed-out helper' '
+ git log -1 >expect &&
+ test_env TEST_EXTERNAL_NOTES_DELAY=10 \
+ TEST_EXTERNAL_NOTES_IGNORE_TERM=true \
+ run_with_limited_time 2 \
+ git -c notes.externalCommand="$external_notes_command" \
+ -c notes.externalCommandTimeoutMs=1 \
+ log -1 >actual 2>err &&
+ test_cmp expect actual &&
+ test_grep "notes.externalCommand failed" err &&
+ test_line_count = 1 err
+'
+
+test_expect_success 'notes.externalCommandTimeoutMs=0 disables timeout' '
+ commit=$(git rev-parse HEAD) &&
+ test_env TEST_EXTERNAL_NOTES_DELAY=1 \
+ git -c notes.externalCommand="$external_notes_command" \
+ -c notes.externalCommandTimeoutMs=0 \
+ log --external-notes -1 >actual &&
+ test_grep "^Notes (external):$" actual &&
+ test_grep "^ $commit$" actual
+'
+
+test_expect_success 'notes.externalCommand handles CRLF note bodies' '
+ body=$(printf "A\r\nB") &&
+ test_env TEST_EXTERNAL_NOTES_BODY="$body" \
+ git -c notes.externalCommand="$external_notes_command" \
+ $external_notes_command_timeout_config \
+ log --external-notes -1 >actual &&
+ test_grep "^Notes (external):$" actual &&
+ test_grep "^ B$" actual
+'
+
+test_expect_success 'notes.externalCommand accepts CRLF missing response' '
+ git log -1 >expect &&
+ test_env TEST_EXTERNAL_NOTES_RESPONSE=missing \
+ TEST_EXTERNAL_NOTES_LINE_ENDING=crlf \
+ git -c notes.externalCommand="$external_notes_command" \
+ $external_notes_command_timeout_config \
+ log -1 >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'notes.externalCommand rejects unterminated missing response' '
+ git log -1 >expect &&
+ test_env TEST_EXTERNAL_NOTES_RESPONSE=missing \
+ TEST_EXTERNAL_NOTES_LINE_ENDING=none \
+ TEST_EXTERNAL_NOTES_EXIT_AFTER_RESPONSE=true \
+ git -c notes.externalCommand="$external_notes_command" \
+ log -1 >actual 2>err &&
+ test_cmp expect actual &&
+ test_grep "notes.externalCommand failed" err &&
+ test_line_count = 1 err
+'
+
+test_expect_success PERL,EXECKEEPSPID 'notes.externalCommand rejects unterminated live response without deadlock' '
+ git log -1 >expect &&
+ test_env TEST_EXTERNAL_NOTES_RESPONSE=missing \
+ TEST_EXTERNAL_NOTES_LINE_ENDING=none \
+ run_with_limited_time 2 \
+ git -c notes.externalCommand="$external_notes_command" \
+ log -1 >actual 2>err &&
+ test_cmp expect actual &&
+ test_grep "notes.externalCommand failed" err &&
+ test_line_count = 1 err
+'
+
+test_expect_success 'notes.externalCommand accepts CRLF protocol lines' '
+ commit=$(git rev-parse HEAD) &&
+ test_env TEST_EXTERNAL_NOTES_LINE_ENDING=crlf \
+ git -c notes.externalCommand="$external_notes_command" \
+ $external_notes_command_timeout_config \
+ log --external-notes -1 >actual &&
+ test_grep "^Notes (external):$" actual &&
+ test_grep "^ $commit$" actual
+'
+
+test_expect_success 'notes.externalCommand missing response shows no external notes' '
+ write_script external-notes-missing <<-\EOF &&
+ while IFS= read -r commit
+ do
+ printf "%s missing\n" "$commit"
+ done
+ EOF
+ git log -1 >expect &&
+ git -c notes.externalCommand=./external-notes-missing log -1 >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'notes.externalCommand empty note shows no external notes' '
+ write_script external-notes-empty <<-\EOF &&
+ while IFS= read -r commit
+ do
+ printf "%s ok 0\n\n" "$commit"
+ done
+ EOF
+ git log -1 >expect &&
+ git -c notes.externalCommand=./external-notes-empty log -1 >actual &&
+ test_cmp expect actual
+'
+
+test_expect_success 'notes.externalCommand rejects invalid note lengths' '
+ write_script external-notes-invalid-length <<-\EOF &&
+ while IFS= read -r commit
+ do
+ printf "%s ok %s\n" "$commit" "$1"
+ done
+ EOF
+ git log -2 >expect &&
+ for bad_length in -1 +1 1x x
+ do
+ git -c notes.externalCommand="./external-notes-invalid-length $bad_length" \
+ log -2 >actual 2>err &&
+ test_cmp expect actual &&
+ test_grep "notes.externalCommand failed" err &&
+ test_line_count = 1 err || return 1
+ done
+'
+
+test_expect_success 'notes.externalCommand is suppressed by --no-notes' '
+ rm -f external-notes-starts &&
+ git -c notes.externalCommand="$external_notes_command" log --no-notes -1 >actual &&
+ test_path_is_missing external-notes-starts &&
+ ! grep "Notes (external):" actual
+'
+
+test_expect_success 'notes.externalCommand is suppressed by --no-external-notes' '
+ rm -f external-notes-starts &&
+ git log -1 >expect &&
+ git -c notes.externalCommand="$external_notes_command" \
+ log --no-external-notes -1 >actual &&
+ test_cmp expect actual &&
+ test_path_is_missing external-notes-starts
+'
+
+test_expect_success 'notes.externalCommand combines with explicit notes ref' '
+ commit=$(git rev-parse HEAD) &&
+ rm -f external-notes-starts &&
+ git -c notes.externalCommand="$external_notes_command" \
+ $external_notes_command_timeout_config \
+ log --notes=other --external-notes -1 >actual &&
+ test_line_count = 1 external-notes-starts &&
+ test_grep "Notes (other):" actual &&
+ test_grep "^ other note$" actual &&
+ test_grep "Notes (external):" actual &&
+ test_grep "^ $commit$" actual &&
+ ! grep "^ order test$" actual
+'
+
+test_expect_success '--show-notes=ref remains additive after --external-notes' '
+ commit=$(git rev-parse HEAD) &&
+ rm -f external-notes-starts &&
+ git -c notes.externalCommand="$external_notes_command" \
+ $external_notes_command_timeout_config \
+ log --external-notes --show-notes=other -1 >actual &&
+ test_line_count = 1 external-notes-starts &&
+ test_grep "^Notes:$" actual &&
+ test_grep "^ order test$" actual &&
+ test_grep "^Notes (other):$" actual &&
+ test_grep "^ other note$" actual &&
+ test_grep "^Notes (external):$" actual &&
+ test_grep "^ $commit$" actual
+'
+
+test_expect_success 'notes.externalCommand can be enabled without default notes refs' '
+ commit=$(git rev-parse HEAD) &&
+ rm -f external-notes-starts &&
+ git -c notes.externalCommand="$external_notes_command" \
+ $external_notes_command_timeout_config \
+ log --external-notes -1 >actual &&
+ test_line_count = 1 external-notes-starts &&
+ test_grep "Notes (external):" actual &&
+ test_grep "^ $commit$" actual &&
+ ! grep "^ order test$" actual &&
+ ! grep "^ other note$" actual
+'
+
+test_expect_success 'notes.externalCommand combines with default notes refs' '
+ commit=$(git rev-parse HEAD) &&
+ rm -f external-notes-starts &&
+ git -c notes.externalCommand="$external_notes_command" \
+ $external_notes_command_timeout_config \
+ log --external-notes --notes -1 >actual &&
+ test_line_count = 1 external-notes-starts &&
+ test_grep "Notes:" actual &&
+ test_grep "^ order test$" actual &&
+ test_grep "Notes (external):" actual &&
+ test_grep "^ $commit$" actual &&
+ ! grep "^ other note$" actual
+'
+
+test_expect_success 'notes.externalCommand obeys last --external-notes option' '
+ commit=$(git rev-parse HEAD) &&
+ rm -f external-notes-starts &&
+ git log --no-notes -1 >expect &&
+ git -c notes.externalCommand="$external_notes_command" \
+ log --external-notes --no-external-notes -1 >actual &&
+ test_cmp expect actual &&
+ test_path_is_missing external-notes-starts &&
+ git -c notes.externalCommand="$external_notes_command" \
+ $external_notes_command_timeout_config \
+ log --notes=other --no-external-notes --external-notes -1 >actual &&
+ test_line_count = 1 external-notes-starts &&
+ test_grep "Notes (other):" actual &&
+ test_grep "^ other note$" actual &&
+ test_grep "Notes (external):" actual &&
+ test_grep "^ $commit$" actual &&
+ ! grep "^ order test$" actual
+'
+
+test_expect_success 'notes.externalCommand honors raw notes formatting' '
+ commit=$(git rev-parse HEAD) &&
+ git -c notes.externalCommand="$external_notes_command" \
+ $external_notes_command_timeout_config \
+ show -s --format=%N >actual &&
+ test_grep "^$commit$" actual &&
+ ! grep "Notes (external):" actual
+'
+
+test_expect_success 'format-patch --external-notes includes external notes only' '
+ commit=$(git rev-parse HEAD) &&
+ rm -f external-notes-starts &&
+ git -c notes.externalCommand="$external_notes_command" \
+ $external_notes_command_timeout_config \
+ format-patch --external-notes -1 --stdout >actual &&
+ test_line_count = 1 external-notes-starts &&
+ test_grep "^Notes (external):" actual &&
+ test_grep "^ $commit$" actual &&
+ ! grep "^ order test$" actual
+'
+
+test_expect_success 'notes.externalCommand is not used for grep matching' '
+ commit=$(git rev-parse HEAD) &&
+ rm -f external-notes-starts &&
+ git -c notes.externalCommand="$external_notes_command" \
+ log --grep="$commit" >actual &&
+ test_must_be_empty actual &&
+ test_path_is_missing external-notes-starts
+'
+
+test_expect_success 'notes.externalCommandForGrep includes external notes in grep matching' '
+ commit=$(git rev-parse HEAD) &&
+ rm -f external-notes-starts &&
+ git -c notes.externalCommand="$external_notes_command" \
+ $external_notes_command_timeout_config \
+ -c notes.externalCommandForGrep=true \
+ log --grep="$commit" -1 >actual &&
+ test_line_count = 1 external-notes-starts &&
+ test_grep "Notes (external):" actual
+'
+
+test_expect_success 'notes.externalCommandForGrep does not search hidden notes' '
+ commit=$(git rev-parse HEAD) &&
+ rm -f external-notes-starts &&
+ git -c notes.externalCommand="$external_notes_command" \
+ -c notes.externalCommandForGrep=true \
+ log --oneline --grep="$commit" -1 >actual &&
+ test_must_be_empty actual &&
+ test_path_is_missing external-notes-starts
+'
+
+test_expect_success 'notes.externalCommandForGrep honors --no-external-notes' '
+ commit=$(git rev-parse HEAD) &&
+ rm -f external-notes-starts &&
+ git -c notes.externalCommand="$external_notes_command" \
+ -c notes.externalCommandForGrep=true \
+ log --no-external-notes --grep="$commit" -1 >actual &&
+ test_must_be_empty actual &&
+ test_path_is_missing external-notes-starts
+'
+
+test_expect_success 'notes.externalCommandForGrep combines with explicit notes ref' '
+ commit=$(git rev-parse HEAD) &&
+ rm -f external-notes-starts &&
+ git -c notes.externalCommand="$external_notes_command" \
+ $external_notes_command_timeout_config \
+ -c notes.externalCommandForGrep=true \
+ log --notes=other --external-notes --grep="$commit" -1 >actual &&
+ test_line_count = 1 external-notes-starts &&
+ test_grep "Notes (external):" actual &&
+ test_grep "Notes (other):" actual &&
+ ! grep "^ order test$" actual
+'
+
+test_expect_success 'notes.externalCommandForGrep is ignored from local config' '
+ commit=$(git rev-parse HEAD) &&
+ rm -f external-notes-starts &&
+ test_config notes.externalCommandForGrep true &&
+ git -c notes.externalCommand="$external_notes_command" \
+ log --grep="$commit" >actual &&
+ test_must_be_empty actual &&
+ test_path_is_missing external-notes-starts
+'
+
+test_expect_success 'notes.externalCommand is not used with explicit notes ref' '
+ rm -f external-notes-starts &&
+ git -c notes.externalCommand="$external_notes_command" log --notes=other -1 >actual &&
+ test_path_is_missing external-notes-starts &&
+ ! grep "Notes (external):" actual
+'
+
+test_expect_success 'notes.externalCommand is ignored from local config' '
+ rm -f external-notes-starts &&
+ test_config notes.externalCommand "$external_notes_command" &&
+ git log -1 >actual &&
+ test_path_is_missing external-notes-starts &&
+ ! grep "Notes (external):" actual
+'
+
+test_expect_success 'notes.externalCommandName is ignored from local config' '
+ test_config notes.externalCommandName local &&
+ git -c notes.externalCommand="$external_notes_command" \
+ $external_notes_command_timeout_config \
+ log -1 >actual &&
+ test_grep "Notes (external):" actual &&
+ ! grep "Notes (local):" actual
+'
+
+test_expect_success 'reset_external_notes_command clears cached helper config' '
+ test-tool notes-external-config-reset >actual &&
+ cat >expect <<-\EOF &&
+ configured=0
+ name=external
+ timeout_ms=100
+ grep=0
+ EOF
+ test_cmp expect actual
+'
+
+test_expect_success 'notes.externalCommand warning is shown once' '
+ write_script external-notes-fail <<-\EOF &&
+ while IFS= read -r commit
+ do
+ printf "%s-mismatch missing\n" "$commit"
+ done
+ EOF
+ git -c notes.externalCommand=./external-notes-fail log -2 >actual 2>err &&
+ test_grep "notes.externalCommand failed" err &&
+ test_line_count = 1 err
+'
+
test_expect_success 'Allow notes on non-commits (trees, blobs, tags)' '
test_config core.notesRef refs/notes/other &&
echo "Note on a tree" >expect &&
diff --git a/t/t6120-describe.sh b/t/t6120-describe.sh
index 8ee3d2c37d02..abbdb42dc9f7 100755
--- a/t/t6120-describe.sh
+++ b/t/t6120-describe.sh
@@ -15,6 +15,7 @@ GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main
export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME
. ./test-lib.sh
+. "$TEST_DIRECTORY"/lib-notes.sh
check_describe () {
indir= &&
@@ -867,6 +868,22 @@ test_expect_success 'format-rev with %N (note)' '
test_cmp expect actual
'
+test_expect_success 'format-rev with %N uses external notes' '
+ commit=$(git -C repo-format rev-parse HEAD) &&
+ rm -f repo-format/format-rev-external-notes-starts \
+ repo-format/format-rev-external-notes-requests &&
+ printf "%s\n" "$commit" >input &&
+ printf "%s\n\n" "$commit" >expect &&
+ TEST_EXTERNAL_NOTES_PREFIX=format-rev-external-notes \
+ git -C repo-format -c notes.externalCommand="$external_notes_command" \
+ $external_notes_command_timeout_config \
+ format-rev --stdin-mode=text --format="tformat:%N" \
+ <input >actual &&
+ test_line_count = 1 repo-format/format-rev-external-notes-starts &&
+ test_cmp input repo-format/format-rev-external-notes-requests &&
+ test_cmp expect actual
+'
+
test_expect_success 'format-rev --notes<ref> (custom notes ref)' '
# One custom notes ref
test_when_finished "git -C repo-format notes remove" &&
--
2.53.0
^ permalink raw reply related
* [PATCH 8/9] Documentation: document external notes command options
From: Siddh Raman Pant @ 2026-05-19 16:30 UTC (permalink / raw)
To: git
Cc: Calvin Wan, Patrick Steinhardt, Elijah Newren,
Kristoffer Haugsbakk, Junio C Hamano
In-Reply-To: <cover.1779207350.git.siddh.raman.pant@oracle.com>
Assisted-by: Codex:gpt-5.5-xhigh-fast
Signed-off-by: Siddh Raman Pant <siddh.raman.pant@oracle.com>
---
Documentation/config/notes.adoc | 57 ++++++++++++++++++++++++++
Documentation/git-format-patch.adoc | 11 ++++-
Documentation/git-range-diff.adoc | 6 +++
Documentation/pretty-options.adoc | 9 ++++
contrib/completion/git-completion.bash | 4 +-
5 files changed, 84 insertions(+), 3 deletions(-)
diff --git a/Documentation/config/notes.adoc b/Documentation/config/notes.adoc
index b7e536496f51..b3ef3fa52950 100644
--- a/Documentation/config/notes.adoc
+++ b/Documentation/config/notes.adoc
@@ -34,6 +34,63 @@ The effective value of `core.notesRef` (possibly overridden by
`GIT_NOTES_REF`) is also implicitly added to the list of refs to be
displayed.
+`notes.externalCommand`::
+ Command to invoke as a long-lived helper when showing commit messages
+ with the `git log` family of commands. Git sends one commit object ID
+ per request on the command's standard input:
++
+------------
+<hex-commit-id>
+------------
++
+For each request, the command must respond on its standard output with either
+`<hex-commit-id> missing` followed by a newline, or `<hex-commit-id> ok <n>`
+followed by a newline and exactly `<n>` bytes of UTF-8 note text followed by a
+newline. The command must respond to each request as it is received; Git does
+not send all commit object IDs before reading responses. Empty note text is not
+displayed. If Git cannot start or communicate with the command, or the command
+sends an invalid response, Git warns once and disables it for the rest of the
+command. External notes are only used while formatting output by default; see
+`notes.externalCommandForGrep` to include them when matching commits.
++
+This setting is only respected in protected configuration (see
+linkgit:git-config[1]). This prevents untrusted repositories from running
+arbitrary commands when notes are displayed.
++
+This setting does not take effect when:
++
+--
+* the value is empty;
+* `--no-notes` is given;
+* `--no-external-notes` is given; or
+* `--notes=<ref>` is given by itself without `--external-notes` or `--notes`.
+--
+
+`notes.externalCommandName`::
+ Name to use in the `Notes (<name>):` header for notes returned by
+ `notes.externalCommand`. Defaults to `external`. This setting is only
+ respected in protected configuration.
+
+`notes.externalCommandTimeoutMs`::
+ Number of milliseconds to wait when reading each response from
+ `notes.externalCommand`. Defaults to `100`. If the command does not
+ produce the expected response in time, Git warns once and disables it
+ for the rest of the command. A value of `0` disables timeout handling,
+ so reads can block until the command writes output or exits. This
+ setting is only respected in protected configuration.
+
+`notes.externalCommandForGrep`::
+ Boolean indicating whether notes returned by `notes.externalCommand`
+ are included when matching commits with `--grep`, wherever notes would
+ normally participate in grep matching. Defaults to false. This does
+ not make hidden notes searchable in formats such as `--oneline` or
+ `--pretty=%s`; use `--notes` or `--external-notes` if those formats
+ should search notes too. When enabled, revision traversal may invoke
+ the external command for many commits that are not ultimately
+ displayed, which can be expensive for slow commands. The note output
+ can also change which commits match. This setting is only respected in
+ protected configuration.
+
`notes.rewrite.<command>`::
When rewriting commits with _<command>_ (currently `amend` or
`rebase`), if this variable is `false`, git will not copy
diff --git a/Documentation/git-format-patch.adoc b/Documentation/git-format-patch.adoc
index 566238245028..472b37e5237a 100644
--- a/Documentation/git-format-patch.adoc
+++ b/Documentation/git-format-patch.adoc
@@ -26,7 +26,7 @@ SYNOPSIS
[--[no-]cover-letter] [--quiet]
[--commit-list-format=<format-spec>]
[--[no-]encode-email-headers]
- [--no-notes | --notes[=<ref>]]
+ [--no-notes | --notes[=<ref>]] [--[no-]external-notes]
[--interdiff=<previous>]
[--range-diff=<previous> [--creation-factor=<percent>]]
[--filename-max-length=<n>]
@@ -395,6 +395,15 @@ configuration options in linkgit:git-notes[1] to use this workflow).
The default is `--no-notes`, unless the `format.notes` configuration is
set.
+--external-notes::
+--no-external-notes::
+ Invoke or do not invoke `notes.externalCommand` to obtain external
+ notes. Like `--notes=<ref>`, `--external-notes` names an explicit
+ note source and by itself does not include the default notes refs.
+ Use `--external-notes --notes` to include the default notes refs
+ too, or combine `--external-notes` with `--notes=<ref>` to include
+ external notes with specific notes refs.
+
--signature=<signature>::
--no-signature::
Add a signature to each message produced. Per RFC 3676 the signature
diff --git a/Documentation/git-range-diff.adoc b/Documentation/git-range-diff.adoc
index 5cc5e2ed5673..1de23f300517 100644
--- a/Documentation/git-range-diff.adoc
+++ b/Documentation/git-range-diff.adoc
@@ -12,6 +12,7 @@ git range-diff [--color=[<when>]] [--no-color] [<diff-options>]
[--no-dual-color] [--creation-factor=<factor>]
[--left-only | --right-only] [--diff-merges=<format>]
[--remerge-diff] [--no-notes | --notes[=<ref>]]
+ [--[no-]external-notes]
( <range1> <range2> | <rev1>...<rev2> | <base> <rev1> <rev2> )
[[--] <path>...]
@@ -101,6 +102,11 @@ diff.
This flag is passed to the `git log` program
(see linkgit:git-log[1]) that generates the patches.
+`--external-notes`::
+`--no-external-notes`::
+ This flag is passed to the `git log` program
+ (see linkgit:git-log[1]) that generates the patches.
+
`<range1> <range2>`::
Compare the commits specified by the two ranges, where
_<range1>_ is considered an older version of _<range2>_.
diff --git a/Documentation/pretty-options.adoc b/Documentation/pretty-options.adoc
index 658e462b2533..aad851c92cfd 100644
--- a/Documentation/pretty-options.adoc
+++ b/Documentation/pretty-options.adoc
@@ -93,6 +93,15 @@ being displayed. Examples: "`--notes=foo`" will show only notes from
"`--notes --notes=foo --no-notes --notes=bar`" will only show notes
from `refs/notes/bar`.
+`--external-notes`::
+`--no-external-notes`::
+ Invoke or do not invoke `notes.externalCommand` to obtain external
+ notes. Like `--notes=<ref>`, `--external-notes` names an explicit
+ note source and by itself does not include the default notes refs.
+ Use `--external-notes --notes` to include the default notes refs
+ too, or combine `--external-notes` with `--notes=<ref>` to include
+ external notes with specific notes refs.
+
`--show-notes-by-default`::
Show the default notes unless options for displaying specific
notes are given.
diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index a8e7c6ddbfb2..146444e65860 100644
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -2023,7 +2023,7 @@ _git_fetch ()
__git_format_patch_extra_options="
--full-index --not --all --no-prefix --src-prefix=
- --dst-prefix= --notes
+ --dst-prefix= --notes --external-notes --no-external-notes
"
_git_format_patch ()
@@ -2215,7 +2215,7 @@ __git_log_common_options="
__git_log_gitk_options="
--dense --sparse --full-history
--simplify-merges --simplify-by-decoration
- --left-right --notes --no-notes
+ --left-right --notes --no-notes --external-notes --no-external-notes
"
# Options that go well for log and shortlog (not gitk)
__git_log_shortlog_options="
--
2.53.0
^ permalink raw reply related
* [PATCH 3/9] wrapper: add sleep_nanosec
From: Siddh Raman Pant @ 2026-05-19 16:30 UTC (permalink / raw)
To: git
Cc: Calvin Wan, Patrick Steinhardt, Elijah Newren,
Kristoffer Haugsbakk, Junio C Hamano
In-Reply-To: <cover.1779207350.git.siddh.raman.pant@oracle.com>
Signed-off-by: Siddh Raman Pant <siddh.raman.pant@oracle.com>
---
wrapper.c | 49 +++++++++++++++++++++++++++++++++++++++++++++++++
wrapper.h | 1 +
2 files changed, 50 insertions(+)
diff --git a/wrapper.c b/wrapper.c
index 16f5a63fbb61..1349255f1eb4 100644
--- a/wrapper.c
+++ b/wrapper.c
@@ -10,6 +10,7 @@
#include "gettext.h"
#include "strbuf.h"
#include "trace2.h"
+#include <time.h>
#ifdef HAVE_RTLGENRANDOM
/* This is required to get access to RtlGenRandom. */
@@ -708,6 +709,54 @@ void sleep_millisec(int millisec)
poll(NULL, 0, millisec);
}
+#ifdef GIT_WINDOWS_NATIVE
+/* No nanosleep() on Windows, so fall-back to using sleep_millisec(). */
+int sleep_nanosec(uint64_t nanosec)
+{
+ uint64_t ns_in_1ms = 1000000ULL; /* 1 ms = 10^6 ns */
+
+ uint64_t millisec = nanosec / ns_in_1ms;
+ if (nanosec % ns_in_1ms)
+ millisec++;
+
+ /* Chunked sleep if we can't represent in integer. */
+ while (millisec > INT_MAX) {
+ sleep_millisec(INT_MAX);
+ millisec -= INT_MAX;
+ }
+
+ sleep_millisec((int)millisec);
+
+ return 0;
+}
+#else
+/* Not Windows, so use the more exact nanosleep(). */
+int sleep_nanosec(uint64_t nanosec)
+{
+ int ret;
+ struct timespec duration, remaining;
+
+ /* Construct the duration by dividing the given total (1s = 10^9ns). */
+ duration.tv_sec = nanosec / 1000000000ULL;
+ duration.tv_nsec = nanosec % 1000000000ULL;
+
+ while(1) {
+ ret = nanosleep(&duration, &remaining);
+
+ /* Continue sleeping if interrupted. */
+ if (ret == -1 && errno == EINTR) {
+ duration = remaining;
+ continue;
+ }
+
+ /* Either success or an error. */
+ break;
+ }
+
+ return ret;
+}
+#endif /* GIT_WINDOWS_NATIVE */
+
int xgethostname(char *buf, size_t len)
{
/*
diff --git a/wrapper.h b/wrapper.h
index 15ac3bab6e97..c39992893a81 100644
--- a/wrapper.h
+++ b/wrapper.h
@@ -130,6 +130,7 @@ int warn_on_fopen_errors(const char *path);
int open_nofollow(const char *path, int flags);
void sleep_millisec(int millisec);
+int sleep_nanosec(uint64_t nanosec);
enum {
/*
--
2.53.0
^ permalink raw reply related
* [PATCH 2/9] notes: convert raw arg in format_display_notes() to bool
From: Siddh Raman Pant @ 2026-05-19 16:30 UTC (permalink / raw)
To: git
Cc: Calvin Wan, Patrick Steinhardt, Elijah Newren,
Kristoffer Haugsbakk, Junio C Hamano
In-Reply-To: <cover.1779207350.git.siddh.raman.pant@oracle.com>
It's used as a boolean flag, let's not use an int.
Signed-off-by: Siddh Raman Pant <siddh.raman.pant@oracle.com>
---
log-tree.c | 3 +--
notes.c | 6 +++---
notes.h | 2 +-
revision.c | 2 +-
4 files changed, 6 insertions(+), 7 deletions(-)
diff --git a/log-tree.c b/log-tree.c
index 7e048701d0c5..4503a42dde6b 100644
--- a/log-tree.c
+++ b/log-tree.c
@@ -854,10 +854,9 @@ void show_log(struct rev_info *opt)
}
if (opt->show_notes) {
- int raw;
struct strbuf notebuf = STRBUF_INIT;
+ bool raw = (opt->commit_format == CMIT_FMT_USERFORMAT);
- raw = (opt->commit_format == CMIT_FMT_USERFORMAT);
format_display_notes(&commit->object.oid, ¬ebuf,
get_log_output_encoding(), raw);
ctx.notes_message = strbuf_detach(¬ebuf, NULL);
diff --git a/notes.c b/notes.c
index 8f315e2a00d2..201f1df3dc29 100644
--- a/notes.c
+++ b/notes.c
@@ -1273,11 +1273,11 @@ void free_notes(struct notes_tree *t)
* If the given notes_tree is NULL, the internal/default notes_tree will be
* used instead.
*
- * (raw != 0) gives the %N userformat; otherwise, the note message is given
+ * (raw == true) gives the %N userformat; otherwise, the note message is given
* for human consumption.
*/
static void format_note(struct notes_tree *t, const struct object_id *object_oid,
- struct strbuf *sb, const char *output_encoding, int raw)
+ struct strbuf *sb, const char *output_encoding, bool raw)
{
static const char utf8[] = "utf-8";
const struct object_id *oid;
@@ -1338,7 +1338,7 @@ static void format_note(struct notes_tree *t, const struct object_id *object_oid
}
void format_display_notes(const struct object_id *object_oid,
- struct strbuf *sb, const char *output_encoding, int raw)
+ struct strbuf *sb, const char *output_encoding, bool raw)
{
int i;
assert(display_notes_trees);
diff --git a/notes.h b/notes.h
index 6dc6d7b26548..f6410b31e1c9 100644
--- a/notes.h
+++ b/notes.h
@@ -313,7 +313,7 @@ void load_display_notes(struct display_notes_opt *opt);
* You *must* call load_display_notes() before using this function.
*/
void format_display_notes(const struct object_id *object_oid,
- struct strbuf *sb, const char *output_encoding, int raw);
+ struct strbuf *sb, const char *output_encoding, bool raw);
/*
* Load the notes tree from each ref listed in 'refs'. The output is
diff --git a/revision.c b/revision.c
index 599b3a66c369..cd9fcefa0a88 100644
--- a/revision.c
+++ b/revision.c
@@ -4107,7 +4107,7 @@ static int commit_match(struct commit *commit, struct rev_info *opt)
if (opt->show_notes) {
if (!buf.len)
strbuf_addstr(&buf, message);
- format_display_notes(&commit->object.oid, &buf, encoding, 1);
+ format_display_notes(&commit->object.oid, &buf, encoding, true);
}
/*
--
2.53.0
^ permalink raw reply related
* [PATCH 7/9] notes: support an external command to display notes
From: Siddh Raman Pant @ 2026-05-19 16:30 UTC (permalink / raw)
To: git
Cc: Calvin Wan, Patrick Steinhardt, Elijah Newren,
Kristoffer Haugsbakk, Junio C Hamano
In-Reply-To: <cover.1779207350.git.siddh.raman.pant@oracle.com>
git notes is a very very helpful feature to show user-supplied
information about a commit alongside its message transparently.
For distributed teams working on large git repos (huge number of
branches/refs, files, etc.) and using the notes feature to mark
information on git commits, a TOCTOU race can happen due to very
large size of the repo and notes ref:
- Person A updates a note for commit X.
- Person A pushes the notes but it takes some time.
- Person B fetches notes and doesn't find the updated note.
- Person B can come to know of it only when he overwrites it
and encounters a push failure.
This problem excaberates on scale.
One solution to this is a realtime fetch or faster updation via
external means, but unfortunately we lose the coherence in the
display of information, and the user would end up reinventing
git log.
So let's add support for an external command to display the notes.
We split the addition of documentation and tests from this commit for
easier review. The new help text added in Documentation/ in the next
commit should make the usage clear.
Assisted-by: Codex:gpt-5.5-xhigh-fast
Signed-off-by: Siddh Raman Pant <siddh.raman.pant@oracle.com>
---
Makefile | 2 +
builtin/log.c | 17 ++-
builtin/name-rev.c | 9 +-
builtin/range-diff.c | 2 +
log-tree.c | 7 +-
meson.build | 1 +
notes-external.c | 330 +++++++++++++++++++++++++++++++++++++++++++
notes-external.h | 19 +++
notes.c | 242 ++++++++++++++++++++++++-------
notes.h | 32 ++++-
revision.c | 32 ++++-
11 files changed, 633 insertions(+), 60 deletions(-)
create mode 100644 notes-external.c
create mode 100644 notes-external.h
diff --git a/Makefile b/Makefile
index c739ae78d0ef..a919bdd75f01 100644
--- a/Makefile
+++ b/Makefile
@@ -838,6 +838,7 @@ TEST_BUILTINS_OBJS += test-match-trees.o
TEST_BUILTINS_OBJS += test-mergesort.o
TEST_BUILTINS_OBJS += test-mktemp.o
TEST_BUILTINS_OBJS += test-name-hash.o
+TEST_BUILTINS_OBJS += test-notes-external-config-reset.o
TEST_BUILTINS_OBJS += test-online-cpus.o
TEST_BUILTINS_OBJS += test-pack-deltas.o
TEST_BUILTINS_OBJS += test-pack-mtimes.o
@@ -1209,6 +1210,7 @@ LIB_OBJS += negotiator/default.o
LIB_OBJS += negotiator/noop.o
LIB_OBJS += negotiator/skipping.o
LIB_OBJS += notes-cache.o
+LIB_OBJS += notes-external.o
LIB_OBJS += notes-merge.o
LIB_OBJS += notes-utils.o
LIB_OBJS += notes.o
diff --git a/builtin/log.c b/builtin/log.c
index 8c0939dd42ad..bed4c1576f2d 100644
--- a/builtin/log.c
+++ b/builtin/log.c
@@ -1337,9 +1337,24 @@ static void get_notes_args(struct strvec *arg, struct rev_info *rev)
(rev->notes_opt.use_default_notes == -1 &&
!rev->notes_opt.extra_notes_refs.nr)) {
strvec_push(arg, "--notes");
- } else {
+ } else if (rev->notes_opt.extra_notes_refs.nr) {
for_each_string_list(&rev->notes_opt.extra_notes_refs, get_notes_refs, arg);
+ } else if (rev->notes_opt.use_external_notes <= 0) {
+ /*
+ * rev->show_notes can stay set after
+ * --external-notes --no-external-notes.
+ *
+ * Since range-diff's child log starts with
+ * --show-notes-by-default, explicitly suppress
+ * notes when no notes source remains.
+ */
+ strvec_push(arg, "--no-notes");
}
+
+ if (rev->notes_opt.use_external_notes > 0)
+ strvec_push(arg, "--external-notes");
+ else if (rev->notes_opt.use_external_notes == 0)
+ strvec_push(arg, "--no-external-notes");
}
static void generate_shortlog_cover_letter(struct shortlog *log,
diff --git a/builtin/name-rev.c b/builtin/name-rev.c
index 60cbbfb4b7d1..95d5e076e24b 100644
--- a/builtin/name-rev.c
+++ b/builtin/name-rev.c
@@ -277,6 +277,7 @@ struct name_ref_data {
struct pretty_format {
struct pretty_print_context ctx;
struct userformat_want want;
+ bool show_external_notes;
};
enum command_type {
@@ -525,9 +526,9 @@ static const char *get_format_rev(const struct commit *c,
if (format_ctx->want.notes) {
struct strbuf notebuf = STRBUF_INIT;
- format_display_notes(&c->object.oid, ¬ebuf,
- get_log_output_encoding(),
- format_ctx->ctx.fmt == CMIT_FMT_USERFORMAT);
+ format_display_notes(c, ¬ebuf, get_log_output_encoding(),
+ format_ctx->ctx.fmt == CMIT_FMT_USERFORMAT,
+ format_ctx->show_external_notes);
format_ctx->ctx.notes_message = strbuf_detach(¬ebuf, NULL);
}
@@ -878,6 +879,8 @@ int cmd_format_rev(int argc,
enable_ref_display_notes(&format_notes_opt,
&ignore_show_notes,
n->string);
+ format_pp.show_external_notes =
+ display_notes_use_external(&format_notes_opt);
load_display_notes(&format_notes_opt);
}
diff --git a/builtin/range-diff.c b/builtin/range-diff.c
index e54c0f7fe156..41c27250404a 100644
--- a/builtin/range-diff.c
+++ b/builtin/range-diff.c
@@ -56,6 +56,8 @@ int cmd_range_diff(int argc,
OPT_PASSTHRU_ARGV(0, "notes", &log_arg,
N_("notes"), N_("passed to 'git log'"),
PARSE_OPT_OPTARG),
+ OPT_PASSTHRU_ARGV(0, "external-notes", &log_arg, NULL,
+ N_("passed to 'git log'"), PARSE_OPT_NOARG),
OPT_PASSTHRU_ARGV(0, "diff-merges", &diff_merges_arg,
N_("style"), N_("passed to 'git log'"), 0),
OPT_CALLBACK(0, "max-memory", &range_diff_opts.max_memory,
diff --git a/log-tree.c b/log-tree.c
index 4503a42dde6b..3289a085f66b 100644
--- a/log-tree.c
+++ b/log-tree.c
@@ -856,9 +856,12 @@ void show_log(struct rev_info *opt)
if (opt->show_notes) {
struct strbuf notebuf = STRBUF_INIT;
bool raw = (opt->commit_format == CMIT_FMT_USERFORMAT);
+ const struct display_notes_opt *notes_opt = &opt->notes_opt;
+
+ format_display_notes(commit, ¬ebuf,
+ get_log_output_encoding(), raw,
+ display_notes_use_external(notes_opt));
- format_display_notes(&commit->object.oid, ¬ebuf,
- get_log_output_encoding(), raw);
ctx.notes_message = strbuf_detach(¬ebuf, NULL);
}
diff --git a/meson.build b/meson.build
index de917bcf1146..21cdbc15aa18 100644
--- a/meson.build
+++ b/meson.build
@@ -397,6 +397,7 @@ libgit_sources = [
'notes-merge.c',
'notes-utils.c',
'notes.c',
+ 'notes-external.c',
'object-file-convert.c',
'object-file.c',
'object-name.c',
diff --git a/notes-external.c b/notes-external.c
new file mode 100644
index 000000000000..7d6b2c6060e0
--- /dev/null
+++ b/notes-external.c
@@ -0,0 +1,330 @@
+#include "git-compat-util.h"
+#include "gettext.h"
+#include "hex.h"
+#include "notes-external.h"
+#include "run-command.h"
+#include "sigchain.h"
+#include "strbuf.h"
+#include "trace.h"
+
+#define convert_ms_to_ns(ms) (uint64_t)(ms) * 1000000ULL
+#define convert_ns_to_ms(ns) (uint64_t)(ns) / 1000000ULL
+#define EXTERNAL_NOTES_TERMINATE_GRACE_NS 100000000ULL /* 100 ms = 10^8 ns */
+#define EXTERNAL_NOTES_READ_CHUNK_SIZE 16384 /* (16 * 1024) bytes */
+
+#ifdef GIT_WINDOWS_NATIVE
+#define EXTERNAL_NOTES_FORCE_KILL_SIGNAL SIGTERM
+#else
+#define EXTERNAL_NOTES_FORCE_KILL_SIGNAL SIGKILL
+#endif
+
+/* ------------------------------------------------------------------------- */
+
+/* Configuration helpers. */
+
+static char *external_notes_command;
+static char *external_notes_command_name_value;
+static uint64_t external_notes_read_timeout_ns = convert_ms_to_ns(100);
+static int external_notes_command_failed;
+static int external_notes_for_grep;
+static bool external_notes_started;
+
+void set_external_notes_command(const char *command)
+{
+ FREE_AND_NULL(external_notes_command);
+ if (command && *command)
+ external_notes_command = xstrdup(command);
+}
+
+void set_external_notes_command_name(const char *name)
+{
+ FREE_AND_NULL(external_notes_command_name_value);
+ if (name && *name)
+ external_notes_command_name_value = xstrdup(name);
+}
+
+void set_external_notes_command_timeout_ms(int timeout_ms)
+{
+ if (timeout_ms < 0)
+ BUG("negative notes.externalCommandTimeoutMs");
+
+ external_notes_read_timeout_ns = convert_ms_to_ns(timeout_ms);
+}
+
+void reset_external_notes_command(void)
+{
+ if (external_notes_started)
+ BUG("cannot reset external notes config while cmd is running");
+
+ FREE_AND_NULL(external_notes_command);
+ FREE_AND_NULL(external_notes_command_name_value);
+ external_notes_read_timeout_ns = convert_ms_to_ns(100);
+ external_notes_command_failed = 0;
+ external_notes_for_grep = 0;
+}
+
+int external_notes_command_configured(void)
+{
+ return external_notes_command && !external_notes_command_failed;
+}
+
+const char *external_notes_command_name(void)
+{
+ return external_notes_command_name_value ?
+ external_notes_command_name_value : "external";
+}
+
+int external_notes_command_timeout_ms(void)
+{
+ return (int)convert_ns_to_ms(external_notes_read_timeout_ns);
+}
+
+void set_external_notes_for_grep(int enabled)
+{
+ external_notes_for_grep = enabled;
+}
+
+int external_notes_for_grep_enabled(void)
+{
+ return external_notes_for_grep;
+}
+
+/* ------------------------------------------------------------------------- */
+
+/* Process management helpers. */
+
+static struct child_process external_notes_process = CHILD_PROCESS_INIT;
+static FILE *external_notes_in;
+static int external_notes_out_fd = -1;
+
+static void mute_routine(const char *msg UNUSED, va_list params UNUSED)
+{
+ /* do nothing */
+}
+
+static void close_external_notes_process_pipes(struct child_process *process)
+{
+ sigchain_push(SIGPIPE, SIG_IGN);
+
+ if (external_notes_in) {
+ fclose(external_notes_in);
+ external_notes_in = NULL;
+ } else {
+ close(process->in);
+ }
+
+ if (external_notes_out_fd >= 0) {
+ close(external_notes_out_fd);
+ external_notes_out_fd = -1;
+ } else {
+ close(process->out);
+ }
+
+ sigchain_pop(SIGPIPE);
+}
+
+/* We set this as callback later, so can't have void argument. */
+static void cleanup_external_notes_process(struct child_process *process)
+{
+ report_fn old_error = NULL;
+
+ /**
+ * The helper may still be sleeping with its pipes open, or may not
+ * exit promptly after EOF. Ask it to stop, then use a bounded wait
+ * that escalates if it ignores the signal.
+ */
+ kill(process->pid, SIGTERM);
+ old_error = get_error_routine();
+ set_error_routine(mute_routine);
+
+ close_external_notes_process_pipes(process);
+ finish_command_with_timeout(process, EXTERNAL_NOTES_TERMINATE_GRACE_NS,
+ EXTERNAL_NOTES_FORCE_KILL_SIGNAL);
+
+ if (old_error)
+ set_error_routine(old_error);
+
+ external_notes_started = false;
+}
+
+static void stop_external_notes_process(void)
+{
+ if (!external_notes_started)
+ return;
+
+ external_notes_process.clean_on_exit = 0;
+ cleanup_external_notes_process(&external_notes_process);
+ child_process_init(&external_notes_process);
+}
+
+static int fail_external_notes_command(void)
+{
+ if (!external_notes_command_failed)
+ warning(_("notes.externalCommand failed: %s"),
+ external_notes_command);
+
+ external_notes_command_failed = 1;
+ stop_external_notes_process();
+ return -1;
+}
+
+static int start_external_notes_command(void)
+{
+ struct child_process *cmd = &external_notes_process;
+
+ if (external_notes_started)
+ return 0;
+
+ if (!external_notes_command || external_notes_command_failed)
+ return -1;
+
+ child_process_init(cmd);
+ strvec_push(&cmd->args, external_notes_command);
+ cmd->use_shell = 1;
+ cmd->in = -1;
+ cmd->out = -1;
+ cmd->clean_on_exit = 1;
+ cmd->clean_on_exit_handler = cleanup_external_notes_process;
+ cmd->trace2_child_class = "notes-external";
+
+ if (start_command(cmd))
+ return fail_external_notes_command();
+
+ external_notes_in = xfdopen(cmd->in, "wb");
+ external_notes_out_fd = cmd->out;
+ external_notes_started = true;
+ return 0;
+}
+
+/* ------------------------------------------------------------------------- */
+
+/* Command parser. Essentially the main() function of this file. */
+int format_external_note(const struct object_id *object_oid,
+ struct strbuf *note_buf)
+{
+ struct strbuf status = STRBUF_INIT;
+ char commit_id_hex_str[GIT_MAX_HEXSZ + 1];
+ const char *arg;
+ char *end;
+ char ch;
+ unsigned long len;
+ uint64_t deadline_ns;
+ bool input_fail;
+ int ret = 0;
+
+ if (start_external_notes_command())
+ return -1;
+
+ /* Fetch the commit ID hex. */
+ oid_to_hex_r(commit_id_hex_str, object_oid);
+
+ /* Pass the input to the external command. */
+ sigchain_push(SIGPIPE, SIG_IGN);
+ input_fail = fprintf(external_notes_in, "%s\n", commit_id_hex_str) < 0
+ || fflush(external_notes_in) != 0;
+ sigchain_pop(SIGPIPE);
+
+ if (input_fail)
+ goto out_fail;
+
+ if (external_notes_read_timeout_ns == 0)
+ deadline_ns = 0;
+ else
+ deadline_ns = getnanotime() + external_notes_read_timeout_ns;
+
+ /**
+ * The output for each commit is either of the two:
+ * "{commit id} missing\n"
+ * "{commit id} ok {num_bytes}\n{str_of_num_bytes}\n"
+ *
+ * We can have "\r\n" instead of "\n" due to Windows.
+ */
+
+ /* Read the first line with its delimiter. */
+ if (strbuf_getwholeline_fd_deadline(&status, external_notes_out_fd,
+ '\n', deadline_ns) == EOF)
+ goto out_fail;
+
+ /* Reject EOF-terminated partial lines. */
+ if (!status.len || status.buf[status.len - 1] != '\n')
+ goto out_fail;
+
+ /**
+ * Strip LF and then optional CR so both LF and CRLF protocol lines
+ * are accepted.
+ */
+ strbuf_setlen(&status, status.len - 1);
+ strbuf_strip_suffix(&status, "\r");
+
+ /* Check if line starts with the commit ID. */
+ if (!skip_prefix(status.buf, commit_id_hex_str, &arg))
+ goto out_fail;
+
+ if (*arg++ != ' ') /* After commit ID there should be a space. */
+ goto out_fail;
+
+ if (strcmp(arg, "missing") == 0) /* No note available. */
+ goto out_success; /* Ending newline is already ensured. */
+
+ if (!skip_prefix(arg, "ok ", &arg)) /* Neither missing nor ok. */
+ goto out_fail;
+
+ /* We are in "ok" case. */
+
+ /* The next thing is length of the note. It must be unsigned digits. */
+ if (!isdigit(*arg))
+ goto out_fail;
+
+ /* Get the length of note. */
+ errno = 0;
+ len = strtoul(arg, &end, 10);
+ if (errno != 0 || *end != '\0' || end == arg)
+ goto out_fail;
+
+ /* Ending newline is already ensured. */
+
+ /* Read the trailing note in bounded-chunks. */
+ while (note_buf->len < len) {
+ ssize_t got;
+ size_t remaining = len - note_buf->len;
+ size_t want = remaining < EXTERNAL_NOTES_READ_CHUNK_SIZE ?
+ remaining : EXTERNAL_NOTES_READ_CHUNK_SIZE;
+
+ strbuf_grow(note_buf, want);
+
+ got = read_in_full_deadline(external_notes_out_fd,
+ note_buf->buf + note_buf->len,
+ want, deadline_ns);
+ if (got < 0 || (size_t)got != want)
+ goto out_fail;
+
+ strbuf_setlen(note_buf, note_buf->len + (size_t)got);
+ }
+
+ /* Ensure the ending newline (LF/CRLF) after the note. */
+ if (xread_deadline(external_notes_out_fd, &ch, 1, deadline_ns) != 1)
+ goto out_fail;
+
+ if (ch != '\n') { /* Not a LF. */
+ if (ch != '\r') /* Not a CRLF. */
+ goto out_fail;
+
+ /* We have '\r', let's read the next char. */
+ if (xread_deadline(external_notes_out_fd, &ch, 1,
+ deadline_ns) != 1)
+ goto out_fail;
+
+ if (ch != '\n') /* Not a CRLF. */
+ goto out_fail;
+ }
+
+ goto out_success;
+
+out_fail:
+ ret = fail_external_notes_command();
+out_success:
+ strbuf_release(&status);
+ return ret;
+}
+
+/* ------------------------------------------------------------------------- */
diff --git a/notes-external.h b/notes-external.h
new file mode 100644
index 000000000000..e49a50b09063
--- /dev/null
+++ b/notes-external.h
@@ -0,0 +1,19 @@
+#ifndef NOTES_EXTERNAL_H
+#define NOTES_EXTERNAL_H
+
+struct object_id;
+struct strbuf;
+
+void set_external_notes_command(const char *command);
+void set_external_notes_command_name(const char *name);
+void set_external_notes_command_timeout_ms(int timeout_ms);
+void set_external_notes_for_grep(int enabled);
+void reset_external_notes_command(void);
+int external_notes_command_configured(void);
+const char *external_notes_command_name(void);
+int external_notes_command_timeout_ms(void);
+int external_notes_for_grep_enabled(void);
+int format_external_note(const struct object_id *object_oid,
+ struct strbuf *out);
+
+#endif /* NOTES_EXTERNAL_H */
diff --git a/notes.c b/notes.c
index 201f1df3dc29..0ff8ba94afc5 100644
--- a/notes.c
+++ b/notes.c
@@ -3,9 +3,12 @@
#include "git-compat-util.h"
#include "config.h"
+#include "commit.h"
#include "environment.h"
+#include "gettext.h"
#include "hex.h"
#include "notes.h"
+#include "notes-external.h"
#include "object-file.h"
#include "object-name.h"
#include "odb.h"
@@ -983,18 +986,56 @@ void string_list_add_refs_from_colon_sep(struct string_list *list,
free(globs_copy);
}
+struct notes_display_config_data {
+ int load_refs;
+ int load_command;
+};
+
static int notes_display_config(const char *k, const char *v,
- const struct config_context *ctx UNUSED,
+ const struct config_context *ctx,
void *cb)
{
- int *load_refs = cb;
+ struct notes_display_config_data *data = cb;
- if (*load_refs && !strcmp(k, "notes.displayref")) {
+ if (data->load_refs && !strcmp(k, "notes.displayref")) {
if (!v)
return config_error_nonbool(k);
string_list_add_refs_by_glob(&display_notes_refs, v);
}
+ if (data->load_command && !strcmp(k, "notes.externalcommand")) {
+ if (!v)
+ return config_error_nonbool(k);
+
+ set_external_notes_command(v);
+ }
+
+ if (data->load_command && !strcmp(k, "notes.externalcommandname")) {
+ if (!v)
+ return config_error_nonbool(k);
+
+ if (strchr(v, '\n') || strchr(v, '\r'))
+ return error(_("notes.externalCommandName must not contain a newline"));
+
+ set_external_notes_command_name(v);
+ }
+
+ if (data->load_command && !strcmp(k, "notes.externalcommandtimeoutms")) {
+ int timeout_ms;
+
+ if (!v)
+ return config_error_nonbool(k);
+
+ timeout_ms = git_config_int(k, v, ctx->kvi);
+ if (timeout_ms < 0)
+ return error(_("notes.externalCommandTimeoutMs must be non-negative"));
+
+ set_external_notes_command_timeout_ms(timeout_ms);
+ }
+
+ if (data->load_command && !strcmp(k, "notes.externalcommandforgrep"))
+ set_external_notes_for_grep(git_config_bool(k, v));
+
return 0;
}
@@ -1075,6 +1116,7 @@ void init_display_notes(struct display_notes_opt *opt)
{
memset(opt, 0, sizeof(*opt));
opt->use_default_notes = -1;
+ opt->use_external_notes = -1;
string_list_init_dup(&opt->extra_notes_refs);
}
@@ -1086,6 +1128,7 @@ void release_display_notes(struct display_notes_opt *opt)
void enable_default_display_notes(struct display_notes_opt *opt, int *show_notes)
{
opt->use_default_notes = 1;
+ opt->default_notes_suppressed_by_external = 0;
*show_notes = 1;
}
@@ -1102,31 +1145,85 @@ void enable_ref_display_notes(struct display_notes_opt *opt, int *show_notes,
void disable_display_notes(struct display_notes_opt *opt, int *show_notes)
{
opt->use_default_notes = -1;
+ opt->use_external_notes = -1;
+ opt->default_notes_suppressed_by_external = 0;
string_list_clear(&opt->extra_notes_refs, 0);
*show_notes = 0;
}
+/*
+ * Resolve the default-notes tri-state in one place. Callers must not test
+ * use_default_notes directly unless they specifically need the unresolved
+ * command-line state.
+ */
+static bool display_notes_use_default(const struct display_notes_opt *opt)
+{
+ /* Options aren't specified, default to true. */
+ if (!opt)
+ return true;
+
+ /* Explicitly enabled. */
+ if (opt->use_default_notes > 0)
+ return true;
+
+ /* Undefined and no explicit notes-ref specified, default to true. */
+ if (opt->use_default_notes == -1 && !opt->extra_notes_refs.nr)
+ return true;
+
+ return false;
+}
+
+/*
+ * Resolve the external-notes tri-state. The unset value follows the resolved
+ * default-notes decision, which means "git log" runs the helper by default
+ * but "git log --notes=<ref>" does not.
+ */
+bool display_notes_use_external(const struct display_notes_opt *opt)
+{
+ /* Options aren't specified, default to true. */
+ if (!opt)
+ return true;
+
+ /* Explicitly enabled. */
+ if (opt->use_external_notes > 0)
+ return true;
+
+ /* Undefined and to use default notes set, default to true. */
+ if (opt->use_external_notes < 0 && display_notes_use_default(opt))
+ return true;
+
+ return false;
+}
+
void load_display_notes(struct display_notes_opt *opt)
{
char *display_ref_env;
- int load_config_refs = 0;
+ struct notes_display_config_data config = { 0, 0 };
+ struct notes_display_config_data protected_config = { 0, 0 };
+ bool use_default_notes = display_notes_use_default(opt);
+ bool use_external_notes = display_notes_use_external(opt);
+
display_notes_refs.strdup_strings = 1;
+ reset_external_notes_command();
assert(!display_notes_trees);
- if (!opt || opt->use_default_notes > 0 ||
- (opt->use_default_notes == -1 && !opt->extra_notes_refs.nr)) {
+ if (use_default_notes) {
string_list_append_nodup(&display_notes_refs, default_notes_ref(the_repository));
display_ref_env = getenv(GIT_NOTES_DISPLAY_REF_ENVIRONMENT);
if (display_ref_env) {
string_list_add_refs_from_colon_sep(&display_notes_refs,
display_ref_env);
- load_config_refs = 0;
+ config.load_refs = 0;
} else
- load_config_refs = 1;
+ config.load_refs = 1;
}
- repo_config(the_repository, notes_display_config, &load_config_refs);
+ if (use_external_notes)
+ protected_config.load_command = 1;
+
+ repo_config(the_repository, notes_display_config, &config);
+ git_protected_config(notes_display_config, &protected_config);
if (opt) {
struct string_list_item *item;
@@ -1266,47 +1363,31 @@ void free_notes(struct notes_tree *t)
}
/*
- * Fill the given strbuf with the notes associated with the given object.
+ * Append one already-loaded note message to the given strbuf.
*
- * If the given notes_tree structure is not initialized, it will be auto-
- * initialized to the default value (see documentation for init_notes() above).
- * If the given notes_tree is NULL, the internal/default notes_tree will be
- * used instead.
+ * Notes read from refs and notes obtained from notes.externalCommand both use
+ * this helper so they share the same encoding, header, and indentation rules.
*
* (raw == true) gives the %N userformat; otherwise, the note message is given
* for human consumption.
*/
-static void format_note(struct notes_tree *t, const struct object_id *object_oid,
- struct strbuf *sb, const char *output_encoding, bool raw)
+static void format_note_data(const char *ref, const char *msg, size_t msglen,
+ struct strbuf *sb, const char *output_encoding,
+ bool raw, bool literal_ref)
{
static const char utf8[] = "utf-8";
- const struct object_id *oid;
- char *msg, *msg_p;
- unsigned long linelen, msglen;
- enum object_type type;
-
- if (!t)
- t = &default_notes_tree;
- if (!t->initialized)
- init_notes(t, NULL, NULL, 0);
-
- oid = get_note(t, object_oid);
- if (!oid)
- return;
-
- if (!(msg = odb_read_object(the_repository->objects, oid, &type, &msglen)) ||
- type != OBJ_BLOB) {
- free(msg);
- return;
- }
+ char *reencoded = NULL;
+ const char *msg_p, *msg_end;
+ /* Convert the note text from UTF-8 to the requested output encoding. */
if (output_encoding && *output_encoding &&
!is_encoding_utf8(output_encoding)) {
- char *reencoded = reencode_string(msg, output_encoding, utf8);
+ size_t reencoded_len;
+ reencoded = reencode_string_len(msg, msglen, output_encoding,
+ utf8, &reencoded_len);
if (reencoded) {
- free(msg);
msg = reencoded;
- msglen = strlen(msg);
+ msglen = reencoded_len;
}
}
@@ -1314,37 +1395,100 @@ static void format_note(struct notes_tree *t, const struct object_id *object_oid
if (msglen && msg[msglen - 1] == '\n')
msglen--;
+ /* Raw mode is the %N userformat, so it omits the "Notes" header. */
if (!raw) {
- const char *ref = t->ref;
- if (!ref || !strcmp(ref, GIT_NOTES_DEFAULT_REF)) {
+ if (!ref)
strbuf_addstr(sb, "\nNotes:\n");
- } else {
- skip_prefix(ref, "refs/", &ref);
- skip_prefix(ref, "notes/", &ref);
+ else if (!literal_ref && !strcmp(ref, GIT_NOTES_DEFAULT_REF))
+ strbuf_addstr(sb, "\nNotes:\n");
+ else {
+ if (!literal_ref) {
+ skip_prefix(ref, "refs/", &ref);
+ skip_prefix(ref, "notes/", &ref);
+ }
strbuf_addf(sb, "\nNotes (%s):\n", ref);
}
}
- for (msg_p = msg; msg_p < msg + msglen; msg_p += linelen + 1) {
- linelen = strchrnul(msg_p, '\n') - msg_p;
+ msg_end = msg + msglen;
+ for (msg_p = msg; msg_p < msg_end; ) {
+ const char *eol = memchr(msg_p, '\n', msg_end - msg_p);
+ size_t linelen = eol ? eol - msg_p : msg_end - msg_p;
+ /* Human output indents note body lines under the header. */
if (!raw)
strbuf_addstr(sb, " ");
+
strbuf_add(sb, msg_p, linelen);
strbuf_addch(sb, '\n');
+
+ msg_p += linelen;
+ if (msg_p < msg_end)
+ msg_p++;
+ }
+
+ free(reencoded);
+}
+
+/*
+ * Fill the given strbuf with the notes associated with the given object.
+ *
+ * If the given notes_tree structure is not initialized, it will be auto-
+ * initialized to the default value (see documentation for init_notes() above).
+ * If the given notes_tree is NULL, the internal/default notes_tree will be
+ * used instead.
+ */
+static void format_note_from_tree(struct notes_tree *t,
+ const struct object_id *object_oid,
+ struct strbuf *sb,
+ const char *output_encoding, bool raw)
+{
+ const struct object_id *oid;
+ char *msg;
+ unsigned long msglen;
+ enum object_type type;
+
+ if (!t)
+ t = &default_notes_tree;
+ if (!t->initialized)
+ init_notes(t, NULL, NULL, 0);
+
+ oid = get_note(t, object_oid);
+ if (!oid)
+ return;
+
+ if (!(msg = odb_read_object(the_repository->objects, oid, &type, &msglen)) ||
+ type != OBJ_BLOB) {
+ free(msg);
+ return;
}
+ format_note_data(t->ref, msg, msglen, sb, output_encoding, raw, false);
+
free(msg);
}
-void format_display_notes(const struct object_id *object_oid,
- struct strbuf *sb, const char *output_encoding, bool raw)
+void format_display_notes(const struct commit *commit,
+ struct strbuf *sb, const char *output_encoding,
+ bool raw, bool show_external)
{
int i;
+ const struct object_id *commit_oid = &commit->object.oid;
+
assert(display_notes_trees);
for (i = 0; display_notes_trees[i]; i++)
- format_note(display_notes_trees[i], object_oid, sb,
- output_encoding, raw);
+ format_note_from_tree(display_notes_trees[i], commit_oid, sb,
+ output_encoding, raw);
+
+ if (show_external && external_notes_command_configured()) {
+ struct strbuf out = STRBUF_INIT;
+
+ if (format_external_note(commit_oid, &out) == 0 && out.len)
+ format_note_data(external_notes_command_name(),
+ out.buf, out.len, sb,
+ output_encoding, raw, true);
+ strbuf_release(&out);
+ }
}
int copy_note(struct notes_tree *t,
diff --git a/notes.h b/notes.h
index f6410b31e1c9..748af70e34af 100644
--- a/notes.h
+++ b/notes.h
@@ -6,6 +6,7 @@
struct object_id;
struct repository;
struct strbuf;
+struct commit;
/*
* Function type for combining two notes annotating the same object.
@@ -264,6 +265,20 @@ struct display_notes_opt {
*/
int use_default_notes;
+ /*
+ * Less than `0` is "unset", which means external notes are shown iff
+ * the default notes are shown. Otherwise, treat it like a boolean.
+ */
+ int use_external_notes;
+
+ /*
+ * Tracks the synthetic "default notes off" state introduced by
+ * `--external-notes`, so a later deprecated `--show-notes=<ref>`
+ * can still preserve its historical additive behavior without
+ * overriding an explicit `--no-standard-notes`.
+ */
+ int default_notes_suppressed_by_external;
+
/*
* A list of globs (in the same style as notes.displayRef) where
* notes should be loaded from.
@@ -304,16 +319,27 @@ void disable_display_notes(struct display_notes_opt *opt, int *show_notes);
void load_display_notes(struct display_notes_opt *opt);
/*
- * Append notes for the given 'object_sha1' from all trees set up by
+ * Return true if notes.externalCommand should be used for 'opt'.
+ *
+ * 'opt' may be NULL.
+ */
+bool display_notes_use_external(const struct display_notes_opt *opt);
+
+/*
+ * Append notes for the given commit from all trees set up by
* load_display_notes() to 'sb'.
*
* If 'raw' is false the note will be indented by 4 places and
* a 'Notes (refname):' header added.
*
+ * If 'show_external' is true then notes.externalCommand will be used to append
+ * the note from external source.
+ *
* You *must* call load_display_notes() before using this function.
*/
-void format_display_notes(const struct object_id *object_oid,
- struct strbuf *sb, const char *output_encoding, bool raw);
+void format_display_notes(const struct commit *commit,
+ struct strbuf *sb, const char *output_encoding,
+ bool raw, bool show_external);
/*
* Load the notes tree from each ref listed in 'refs'. The output is
diff --git a/revision.c b/revision.c
index cd9fcefa0a88..84d9af961988 100644
--- a/revision.c
+++ b/revision.c
@@ -6,6 +6,7 @@
#include "environment.h"
#include "gettext.h"
#include "hex.h"
+#include "notes-external.h"
#include "object-name.h"
#include "object-file.h"
#include "odb.h"
@@ -2583,18 +2584,40 @@ static int handle_revision_opt(struct rev_info *revs, int argc, const char **arg
} else if (skip_prefix(arg, "--show-notes=", &optarg) ||
skip_prefix(arg, "--notes=", &optarg)) {
if (starts_with(arg, "--show-notes=") &&
- revs->notes_opt.use_default_notes < 0)
+ (revs->notes_opt.use_default_notes < 0 ||
+ revs->notes_opt.default_notes_suppressed_by_external)) {
revs->notes_opt.use_default_notes = 1;
+ revs->notes_opt.default_notes_suppressed_by_external = 0;
+ }
enable_ref_display_notes(&revs->notes_opt, &revs->show_notes, optarg);
revs->show_notes_given = 1;
} else if (!strcmp(arg, "--no-notes")) {
disable_display_notes(&revs->notes_opt, &revs->show_notes);
revs->show_notes_given = 1;
+ } else if (!strcmp(arg, "--external-notes")) {
+ revs->notes_opt.use_external_notes = 1;
+ revs->show_notes = 1;
+ revs->show_notes_given = 1;
+ /*
+ * `--external-notes` names a note source on its own. If the
+ * default notes ref is still undecided, settle it to "off" so
+ * this option does not also trigger the "no explicit notes
+ * refs" fallback. A later use of `--notes` or the deprecated
+ * `--show-notes=<ref>` can still turn the default ref on.
+ */
+ if (revs->notes_opt.use_default_notes < 0) {
+ revs->notes_opt.use_default_notes = 0;
+ revs->notes_opt.default_notes_suppressed_by_external = 1;
+ }
+ } else if (!strcmp(arg, "--no-external-notes")) {
+ revs->notes_opt.use_external_notes = 0;
} else if (!strcmp(arg, "--standard-notes")) {
revs->show_notes_given = 1;
revs->notes_opt.use_default_notes = 1;
+ revs->notes_opt.default_notes_suppressed_by_external = 0;
} else if (!strcmp(arg, "--no-standard-notes")) {
revs->notes_opt.use_default_notes = 0;
+ revs->notes_opt.default_notes_suppressed_by_external = 0;
} else if (!strcmp(arg, "--oneline")) {
revs->verbose_header = 1;
get_commit_format("oneline", revs);
@@ -4105,9 +4128,14 @@ static int commit_match(struct commit *commit, struct rev_info *opt)
/* Append "fake" message parts as needed */
if (opt->show_notes) {
+ const struct display_notes_opt *notes_opt = &opt->notes_opt;
+
if (!buf.len)
strbuf_addstr(&buf, message);
- format_display_notes(&commit->object.oid, &buf, encoding, true);
+
+ format_display_notes(commit, &buf, encoding, true,
+ (display_notes_use_external(notes_opt)
+ && external_notes_for_grep_enabled()));
}
/*
--
2.53.0
^ permalink raw reply related
* [PATCH 6/9] t3301: cover generic displayed notes behavior
From: Siddh Raman Pant @ 2026-05-19 16:30 UTC (permalink / raw)
To: git
Cc: Calvin Wan, Patrick Steinhardt, Elijah Newren,
Kristoffer Haugsbakk, Junio C Hamano
In-Reply-To: <cover.1779207350.git.siddh.raman.pant@oracle.com>
Displayed notes already participate in common log behavior.
Add explicit coverage for raw notes formatting, --no-notes
suppression, explicit notes refs, and --grep matching before
teaching external notes to feed the same display path.
Assisted-by: Codex:gpt-5.5-xhigh-fast
Signed-off-by: Siddh Raman Pant <siddh.raman.pant@oracle.com>
---
t/t3301-notes.sh | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)
diff --git a/t/t3301-notes.sh b/t/t3301-notes.sh
index d6c50460d086..27439010dfbc 100755
--- a/t/t3301-notes.sh
+++ b/t/t3301-notes.sh
@@ -885,6 +885,30 @@ test_expect_success '--show-notes=ref accumulates' '
test_cmp expect-both-reversed actual
'
+test_expect_success 'displayed notes honor raw notes formatting' '
+ git show -s --format=%N >actual &&
+ test_grep "^order test$" actual &&
+ ! grep "Notes" actual
+'
+
+test_expect_success 'displayed notes are suppressed by --no-notes' '
+ git log --no-notes -1 >actual &&
+ test_cmp expect-not-other actual
+'
+
+test_expect_success 'explicit notes ref replaces default displayed notes' '
+ git log --notes=other -1 >actual &&
+ test_cmp expect-other actual
+'
+
+test_expect_success 'displayed notes are used for grep matching' '
+ commit=$(git rev-parse HEAD) &&
+ git log --grep="order test" -1 >actual &&
+ test_grep "^commit $commit$" actual &&
+ git log --no-notes --grep="order test" -1 >actual &&
+ test_must_be_empty actual
+'
+
test_expect_success 'Allow notes on non-commits (trees, blobs, tags)' '
test_config core.notesRef refs/notes/other &&
echo "Note on a tree" >expect &&
--
2.53.0
^ permalink raw reply related
* [PATCH 5/9] wrapper: add support for timeout and deadline in read helpers
From: Siddh Raman Pant @ 2026-05-19 16:30 UTC (permalink / raw)
To: git
Cc: Calvin Wan, Patrick Steinhardt, Elijah Newren,
Kristoffer Haugsbakk, Junio C Hamano
In-Reply-To: <cover.1779207350.git.siddh.raman.pant@oracle.com>
Add read helpers which allow a caller to enforce a timeout per read,
and a deadline for the read in case multiple reads have to be done
under a common timeout.
Assisted-by: Codex:gpt-5.5-xhigh-fast
Signed-off-by: Siddh Raman Pant <siddh.raman.pant@oracle.com>
---
strbuf.c | 26 +++++++++-
strbuf.h | 4 ++
wrapper.c | 139 ++++++++++++++++++++++++++++++++++++++++++++++++++----
wrapper.h | 23 +++++++++
4 files changed, 182 insertions(+), 10 deletions(-)
diff --git a/strbuf.c b/strbuf.c
index 3e04addc22fe..b3fc7c624aa2 100644
--- a/strbuf.c
+++ b/strbuf.c
@@ -749,13 +749,15 @@ int strbuf_getline_nul(struct strbuf *sb, FILE *fp)
return strbuf_getdelim(sb, fp, '\0');
}
-int strbuf_getwholeline_fd(struct strbuf *sb, int fd, int term)
+static int strbuf_getwholeline_fd_with(struct strbuf *sb, int fd, int term,
+ xread_cb_t xread_cb,
+ void *cb_data)
{
strbuf_reset(sb);
while (1) {
char ch;
- ssize_t len = xread(fd, &ch, 1);
+ ssize_t len = xread_cb(fd, &ch, 1, cb_data);
if (len <= 0)
return EOF;
strbuf_addch(sb, ch);
@@ -765,6 +767,26 @@ int strbuf_getwholeline_fd(struct strbuf *sb, int fd, int term)
return 0;
}
+int strbuf_getwholeline_fd_deadline(struct strbuf *sb, int fd, int term,
+ uint64_t deadline_ns)
+{
+ return strbuf_getwholeline_fd_with(sb, fd, term, xread_deadline_fn,
+ &deadline_ns);
+}
+
+int strbuf_getwholeline_fd_timeout(struct strbuf *sb, int fd, int term,
+ int timeout_ms)
+{
+ return strbuf_getwholeline_fd_with(sb, fd, term, xread_timeout_fn,
+ &timeout_ms);
+}
+
+/* Non-timeout version for compatibility. */
+int strbuf_getwholeline_fd(struct strbuf *sb, int fd, int term)
+{
+ return strbuf_getwholeline_fd_timeout(sb, fd, term, 0);
+}
+
ssize_t strbuf_read_file(struct strbuf *sb, const char *path, size_t hint)
{
int fd;
diff --git a/strbuf.h b/strbuf.h
index 06e284f9cca4..f896da1277a6 100644
--- a/strbuf.h
+++ b/strbuf.h
@@ -535,6 +535,10 @@ int strbuf_appendwholeline(struct strbuf *sb, FILE *file, int term);
* descriptor.
*/
int strbuf_getwholeline_fd(struct strbuf *sb, int fd, int term);
+int strbuf_getwholeline_fd_timeout(struct strbuf *sb, int fd, int term,
+ int timeout_ms);
+int strbuf_getwholeline_fd_deadline(struct strbuf *sb, int fd, int term,
+ uint64_t deadline_ns);
/**
* Set the buffer to the path of the current working directory.
diff --git a/wrapper.c b/wrapper.c
index 1349255f1eb4..3e0d65724e47 100644
--- a/wrapper.c
+++ b/wrapper.c
@@ -9,6 +9,7 @@
#include "parse.h"
#include "gettext.h"
#include "strbuf.h"
+#include "trace.h"
#include "trace2.h"
#include <time.h>
@@ -221,28 +222,129 @@ static int handle_nonblock(int fd, short poll_events, int err)
return 1;
}
-/*
- * xread() is the same a read(), but it automatically restarts read()
- * operations with a recoverable error (EAGAIN and EINTR). xread()
+static int wait_for_fd(int fd, short poll_events, int timeout_ms)
+{
+ struct pollfd pfd;
+
+ if (timeout_ms < 0) {
+ /* Negative timeout makes no sense. */
+ errno = EINVAL;
+ return -1;
+ }
+
+ pfd.fd = fd;
+ pfd.events = poll_events;
+
+ while(1) {
+ int ret = poll(&pfd, 1, timeout_ms);
+
+ if (ret <= 0) {
+ /* Retry if interrupted. */
+ if (ret < 0 && errno == EINTR)
+ continue;
+
+ /* Set errno if timeout happened. */
+ if (ret == 0)
+ errno = ETIMEDOUT;
+
+ return -1;
+ }
+
+ /* Invalid FD passed. */
+ if (pfd.revents & POLLNVAL) {
+ errno = EBADF;
+ return -1;
+ }
+
+ /* Some error happened. */
+ if (pfd.revents & POLLERR) {
+ errno = EIO;
+ return -1;
+ }
+
+ /* HangUp => We are ready to consume output till EOF. */
+ if (pfd.revents & (poll_events | POLLHUP))
+ return 0;
+ }
+}
+
+/**
+ * xread_timeout() is the same as read(), but it automatically restarts read()
+ * operations with a recoverable error (EAGAIN and EINTR). xread_timeout()
* DOES NOT GUARANTEE that "len" bytes is read even if the data is available.
+ *
+ * Fails with ETIMEDOUT when no bytes become available within timeout_ms
+ * milliseconds. A zero timeout disables timeout handling, so reads can
+ * block until the file descriptor is readable. Negative timeouts are invalid.
*/
-ssize_t xread(int fd, void *buf, size_t len)
+ssize_t xread_timeout(int fd, void *buf, size_t len, int timeout_ms)
{
ssize_t nr;
+
if (len > MAX_IO_SIZE)
len = MAX_IO_SIZE;
+
while (1) {
+ if (timeout_ms && wait_for_fd(fd, POLLIN, timeout_ms))
+ return -1;
+
nr = read(fd, buf, len);
+
if (nr < 0) {
if (errno == EINTR)
continue;
- if (handle_nonblock(fd, POLLIN, errno))
- continue;
+
+ if (timeout_ms) {
+ if (errno == EAGAIN || errno == EWOULDBLOCK)
+ continue;
+ } else {
+ if (handle_nonblock(fd, POLLIN, errno))
+ continue;
+ }
}
+
return nr;
}
}
+/* Non-timeout version for compatibility. */
+ssize_t xread(int fd, void *buf, size_t len)
+{
+ return xread_timeout(fd, buf, len, 0);
+}
+
+static int remaining_timeout_ms(uint64_t deadline_ns)
+{
+ uint64_t now, remaining_ns;
+
+ if (!deadline_ns)
+ return 0;
+
+ now = getnanotime();
+ if (now >= deadline_ns) {
+ errno = ETIMEDOUT;
+ return -1;
+ }
+
+ remaining_ns = deadline_ns - now;
+ return (int)((remaining_ns + 999999ULL) / 1000000ULL);
+}
+
+/* (deadline_ns = 0) disables the deadline and short-circuits to xread(). */
+ssize_t xread_deadline(int fd, void *buf, size_t len, uint64_t deadline_ns)
+{
+ int timeout_ms;
+
+ if (deadline_ns == 0)
+ return xread(fd, buf, len);
+
+ timeout_ms = remaining_timeout_ms(deadline_ns);
+ if (timeout_ms < 0)
+ return -1;
+
+ return xread_timeout(fd, buf, len, timeout_ms);
+}
+
/*
* xwrite() is the same a write(), but it automatically restarts write()
* operations with a recoverable error (EAGAIN and EINTR). xwrite() DOES NOT
@@ -284,13 +386,15 @@ ssize_t xpread(int fd, void *buf, size_t len, off_t offset)
}
}
-ssize_t read_in_full(int fd, void *buf, size_t count)
+static ssize_t read_in_full_with(int fd, void *buf, size_t count,
+ xread_cb_t xread_cb,
+ void *cb_data)
{
char *p = buf;
ssize_t total = 0;
while (count > 0) {
- ssize_t loaded = xread(fd, p, count);
+ ssize_t loaded = xread_cb(fd, p, count, cb_data);
if (loaded < 0)
return -1;
if (loaded == 0)
@@ -303,6 +407,25 @@ ssize_t read_in_full(int fd, void *buf, size_t count)
return total;
}
+ssize_t read_in_full_deadline(int fd, void *buf, size_t count,
+ uint64_t deadline_ns)
+{
+ return read_in_full_with(fd, buf, count, xread_deadline_fn,
+ &deadline_ns);
+}
+
+ssize_t read_in_full_timeout(int fd, void *buf, size_t count, int timeout_ms)
+{
+ return read_in_full_with(fd, buf, count, xread_timeout_fn,
+ &timeout_ms);
+}
+
+/* Non-timeout version for compatibility. */
+ssize_t read_in_full(int fd, void *buf, size_t count)
+{
+ return read_in_full_timeout(fd, buf, count, 0);
+}
+
ssize_t write_in_full(int fd, const void *buf, size_t count)
{
const char *p = buf;
diff --git a/wrapper.h b/wrapper.h
index c39992893a81..f8592599216a 100644
--- a/wrapper.h
+++ b/wrapper.h
@@ -15,6 +15,8 @@ const char *mmap_os_err(void);
void *xmmap_gently(void *start, size_t length, int prot, int flags, int fd, off_t offset);
int xopen(const char *path, int flags, ...);
ssize_t xread(int fd, void *buf, size_t len);
+ssize_t xread_timeout(int fd, void *buf, size_t len, int timeout_ms);
+ssize_t xread_deadline(int fd, void *buf, size_t len, uint64_t deadline_ns);
ssize_t xwrite(int fd, const void *buf, size_t len);
ssize_t xpread(int fd, void *buf, size_t len, off_t offset);
int xdup(int fd);
@@ -44,9 +46,30 @@ int git_mkstemps_mode(char *pattern, int suffix_len, int mode);
int git_mkstemp_mode(char *pattern, int mode);
ssize_t read_in_full(int fd, void *buf, size_t count);
+ssize_t read_in_full_timeout(int fd, void *buf, size_t count, int timeout_ms);
+ssize_t read_in_full_deadline(int fd, void *buf, size_t count,
+ uint64_t deadline_ns);
ssize_t write_in_full(int fd, const void *buf, size_t count);
ssize_t pread_in_full(int fd, void *buf, size_t count, off_t offset);
+typedef ssize_t xread_cb_t(int fd, void *buf, size_t len, const void *cb_data);
+
+static inline ssize_t xread_timeout_fn(int fd, void *buf, size_t len,
+ const void *cb_data)
+{
+ const int *timeout_ms = cb_data;
+
+ return xread_timeout(fd, buf, len, *timeout_ms);
+}
+
+static inline ssize_t xread_deadline_fn(int fd, void *buf, size_t len,
+ const void *cb_data)
+{
+ const uint64_t *deadline_ns = cb_data;
+
+ return xread_deadline(fd, buf, len, *deadline_ns);
+}
+
static inline ssize_t write_str_in_full(int fd, const char *str)
{
return write_in_full(fd, str, strlen(str));
--
2.53.0
^ permalink raw reply related
* [PATCH 4/9] run-command: add support for timeout in command finisher
From: Siddh Raman Pant @ 2026-05-19 16:30 UTC (permalink / raw)
To: git
Cc: Calvin Wan, Patrick Steinhardt, Elijah Newren,
Kristoffer Haugsbakk, Junio C Hamano
In-Reply-To: <cover.1779207350.git.siddh.raman.pant@oracle.com>
A called command may not respond to the initial signal and will get
stuck in finish_command() -> wait_or_whine().
So let's add timeout support into the finisher so that if a deadline
occurs, we can send a force-kill signal.
The force-kill signal is in the argument because a program may trap a
signal, so it is the responsibility of caller to pass the correct kill
signal.
Assisted-by: Codex:gpt-5.5-xhigh-fast
Signed-off-by: Siddh Raman Pant <siddh.raman.pant@oracle.com>
---
run-command.c | 92 +++++++++++++++++++++++++++++++++++++++++++++++----
run-command.h | 13 ++++++++
2 files changed, 98 insertions(+), 7 deletions(-)
diff --git a/run-command.c b/run-command.c
index c146a56532a1..60b84610d1f0 100644
--- a/run-command.c
+++ b/run-command.c
@@ -554,16 +554,63 @@ static inline void set_cloexec(int fd)
fcntl(fd, F_SETFD, flags | FD_CLOEXEC);
}
-static int wait_or_whine(pid_t pid, const char *argv0, int in_signal)
+#define NS_IN_10MS 10000000ULL /* 10 ms = 10^-2 s = 10^(9-2) ns = 10^7 ns */
+
+/* If timeout_ns == 0, no timeout happens (the timeout path is not taken). */
+static int wait_or_whine_timeout(pid_t pid, const char *argv0, int in_signal,
+ uint64_t timeout_ns)
{
int status, code = -1;
pid_t waiting;
int failed_errno = 0;
+ int flags = timeout_ns ? WNOHANG : 0;
+ bool timed_out = false;
+ uint64_t deadline_ns = getnanotime() + timeout_ns;
+
+ while(1) {
+ uint64_t current_time_ns, remaining_ns;
+ waiting = waitpid(pid, &status, flags);
+
+ /* Retry if interrupted. */
+ if (waiting < 0 && errno == EINTR)
+ continue;
+
+ /* Break if exited. */
+ if (waiting)
+ break;
+
+ /* If no timeout is specified, retry till it exits. */
+ if (!timeout_ns)
+ continue;
- while ((waiting = waitpid(pid, &status, 0)) < 0 && errno == EINTR)
- ; /* nothing */
+ current_time_ns = getnanotime();
+
+ /* If we are past the deadline, set errno and break. */
+ if (deadline_ns <= current_time_ns) {
+ errno = ETIMEDOUT;
+ timed_out = true;
+ break;
+ }
+
+ /**
+ * Retry after a sleep(min(remaining, default_chunk)).
+ *
+ * We don't blindly sleep for the entire remaining time because
+ * the process can exit early.
+ *
+ * The subtraction of uint64_t is safe here since we have
+ * already established that deadline_ns > current_time_ns.
+ */
+ remaining_ns = deadline_ns - current_time_ns;
+ sleep_nanosec(remaining_ns < NS_IN_10MS ?
+ remaining_ns : NS_IN_10MS);
+ }
- if (waiting < 0) {
+ if (timed_out) {
+ failed_errno = errno;
+ if (!in_signal)
+ error_errno("waitpid for %s timed out", argv0);
+ } else if (waiting < 0) {
failed_errno = errno;
if (!in_signal)
error_errno("waitpid for %s failed", argv0);
@@ -587,13 +634,28 @@ static int wait_or_whine(pid_t pid, const char *argv0, int in_signal)
error("waitpid is confused (%s)", argv0);
}
- if (!in_signal)
+ /**
+ * Signal handlers use the cleanup list while reaping children, so only
+ * non-signal waiters (in_signal != 0) should update it.
+ *
+ * In case of a timeout, we keep the child registered since it is
+ * actually not reaped so removing would be wrong. It is the
+ * responsibility of the caller to detect the timeout and do cleanup,
+ * like sending a kill signal using this function without a timeout.
+ */
+ if (!in_signal && !timed_out)
clear_child_for_cleanup(pid);
errno = failed_errno;
return code;
}
+/* Non-timeout wrapper for compatibility. */
+static int wait_or_whine(pid_t pid, const char *argv0, int in_signal)
+{
+ return wait_or_whine_timeout(pid, argv0, in_signal, 0);
+}
+
static void trace_add_env(struct strbuf *dst, const char *const *deltaenv)
{
struct string_list envs = STRING_LIST_INIT_DUP;
@@ -989,15 +1051,31 @@ int start_command(struct child_process *cmd)
return 0;
}
-int finish_command(struct child_process *cmd)
+/* See comment in the header file for executive summary. */
+int finish_command_with_timeout(struct child_process *cmd, uint64_t timeout_ns,
+ int signal_on_timeout)
{
- int ret = wait_or_whine(cmd->pid, cmd->args.v[0], 0);
+ int ret = wait_or_whine_timeout(cmd->pid, cmd->args.v[0], 0,
+ timeout_ns);
+
+ if (timeout_ns && ret < 0 && errno == ETIMEDOUT) {
+ kill(cmd->pid, signal_on_timeout);
+ ret = wait_or_whine(cmd->pid, cmd->args.v[0], 0);
+ }
+
trace2_child_exit(cmd, ret);
child_process_clear(cmd);
invalidate_lstat_cache();
return ret;
}
+/* Non-timeout wrapper for compatibility. */
+int finish_command(struct child_process *cmd)
+{
+ return finish_command_with_timeout(cmd, 0, 0);
+}
+
+
int finish_command_in_signal(struct child_process *cmd)
{
int ret = wait_or_whine(cmd->pid, cmd->args.v[0], 1);
diff --git a/run-command.h b/run-command.h
index 8ca496d7bdeb..cb1c8ba4ec01 100644
--- a/run-command.h
+++ b/run-command.h
@@ -215,6 +215,19 @@ int start_command(struct child_process *);
*/
int finish_command(struct child_process *);
+/**
+ * Wait for the completion of a sub-process that was started with
+ * start_command(), but uptil a given timeout duration timeout_ns.
+ *
+ * If it has not exited after timeout_ns, signal_on_timeout is sent to the
+ * process. We don't enforce a timeout for the second wait after sending
+ * the signal (as the process cleanup needs to happen), so it will block there.
+ *
+ * If timeout_ns == 0, no timeout happens and signal_on_timeout is ignored.
+ */
+int finish_command_with_timeout(struct child_process *cmd, uint64_t timeout_ns,
+ int signal_on_timeout);
+
int finish_command_in_signal(struct child_process *);
/**
--
2.53.0
^ permalink raw reply related
* [PATCH 1/9] Documentation/git-range-diff: add missing notes options in synopsis
From: Siddh Raman Pant @ 2026-05-19 16:30 UTC (permalink / raw)
To: git
Cc: Calvin Wan, Patrick Steinhardt, Elijah Newren,
Kristoffer Haugsbakk, Junio C Hamano
In-Reply-To: <cover.1779207350.git.siddh.raman.pant@oracle.com>
Signed-off-by: Siddh Raman Pant <siddh.raman.pant@oracle.com>
---
Documentation/git-range-diff.adoc | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/Documentation/git-range-diff.adoc b/Documentation/git-range-diff.adoc
index 880557084533..5cc5e2ed5673 100644
--- a/Documentation/git-range-diff.adoc
+++ b/Documentation/git-range-diff.adoc
@@ -11,7 +11,7 @@ SYNOPSIS
git range-diff [--color=[<when>]] [--no-color] [<diff-options>]
[--no-dual-color] [--creation-factor=<factor>]
[--left-only | --right-only] [--diff-merges=<format>]
- [--remerge-diff]
+ [--remerge-diff] [--no-notes | --notes[=<ref>]]
( <range1> <range2> | <rev1>...<rev2> | <base> <rev1> <rev2> )
[[--] <path>...]
--
2.53.0
^ permalink raw reply related
* [PATCH 0/9] Add support for an external command for fetching notes
From: Siddh Raman Pant @ 2026-05-19 16:30 UTC (permalink / raw)
To: git
Cc: Calvin Wan, Patrick Steinhardt, Elijah Newren,
Kristoffer Haugsbakk, Junio C Hamano
Hi,
This series teaches the notes display machinery to obtain note text from a
long-lived external helper configured by `notes.externalCommand`.
The motivation is mentioned in the main commit message (PATCH 7/9).
The helper protocol is intentionally narrow. Git starts the command once,
sends one commit object ID per request, and expects either:
<object-id> missing
<object-id> ok <n>
<n bytes of UTF-8 note text>
with the documented trailing newlines. The command is read only from protected
configuration, so an untrusted repository cannot make ordinary note display run
arbitrary commands. If the helper cannot be started, times out, exits, or sends
an invalid response, Git warns once, disables it for the rest of the process,
and continues without external notes.
Users can control from command line too with `--external-notes` and
`--no-external-notes`. The semantics are close to `--notes=<ref>`:
`--external-notes` implies naming an explicit notes source by itself, while
`--external-notes --notes` combines it with the default notes refs, and
`--external-notes --notes=<ref>` combines it with specific notes refs. The
series also adds `notes.externalCommandName`, `notes.externalCommandTimeoutMs`,
and the opt-in `notes.externalCommandForGrep` knob for installations that want
external notes to participate in `--grep` matching.
Because this puts an external process on the log-formatting path, the series
also adds the small support pieces needed to keep that boundary bounded:
timeout/deadline variants of the read helpers, a timeout-aware command
finisher, and cleanup that escalates if the helper does not exit promptly.
Testing: https://github.com/siddhpant/git/actions/runs/26107938855
Thanks,
Siddh
Siddh Raman Pant (9):
Documentation/git-range-diff: add missing notes options in synopsis
notes: convert raw arg in format_display_notes() to bool
wrapper: add sleep_nanosec
run-command: add support for timeout in command finisher
wrapper: add support for timeout and deadline in read helpers
t3301: cover generic displayed notes behavior
notes: support an external command to display notes
Documentation: document external notes command options
t: add tests for external notes command
Documentation/config/notes.adoc | 57 +++
Documentation/git-format-patch.adoc | 11 +-
Documentation/git-range-diff.adoc | 8 +-
Documentation/pretty-options.adoc | 9 +
Makefile | 2 +
builtin/log.c | 17 +-
builtin/name-rev.c | 9 +-
builtin/range-diff.c | 2 +
contrib/completion/git-completion.bash | 4 +-
log-tree.c | 10 +-
meson.build | 1 +
notes-external.c | 330 ++++++++++++++
notes-external.h | 19 +
notes.c | 244 ++++++++---
notes.h | 32 +-
revision.c | 32 +-
run-command.c | 92 +++-
run-command.h | 13 +
strbuf.c | 26 +-
strbuf.h | 4 +
t/helper/meson.build | 1 +
t/helper/test-external-notes | 64 +++
t/helper/test-notes-external-config-reset.c | 20 +
t/helper/test-tool.c | 1 +
t/helper/test-tool.h | 1 +
t/lib-notes.sh | 19 +
t/t3206-range-diff.sh | 68 +++
t/t3301-notes.sh | 461 ++++++++++++++++++++
t/t6120-describe.sh | 17 +
wrapper.c | 188 +++++++-
wrapper.h | 24 +
31 files changed, 1702 insertions(+), 84 deletions(-)
create mode 100644 notes-external.c
create mode 100644 notes-external.h
create mode 100755 t/helper/test-external-notes
create mode 100644 t/helper/test-notes-external-config-reset.c
create mode 100644 t/lib-notes.sh
--
2.53.0
^ permalink raw reply
* Re: What's cooking in git.git (May 2026, #04)
From: Taylor Blau @ 2026-05-19 16:26 UTC (permalink / raw)
To: Junio C Hamano; +Cc: git
In-Reply-To: <xmqqv7clbizy.fsf@gitster.g>
On Mon, May 18, 2026 at 10:32:01AM +0900, Junio C Hamano wrote:
> * tb/incremental-midx-part-3.3 (2026-04-29) 16 commits
> - repack: allow `--write-midx=incremental` without `--geometric`
> - repack: introduce `--write-midx=incremental`
> - repack: implement incremental MIDX repacking
> - packfile: ensure `close_pack_revindex()` frees in-memory revindex
> - builtin/repack.c: convert `--write-midx` to an `OPT_CALLBACK`
> - repack-geometry: prepare for incremental MIDX repacking
> - repack-midx: extract `repack_fill_midx_stdin_packs()`
> - repack-midx: factor out `repack_prepare_midx_command()`
> - midx: expose `midx_layer_contains_pack()`
> - repack: track the ODB source via existing_packs
> - midx: support custom `--base` for incremental MIDX writes
> - midx: introduce `--no-write-chain-file` for incremental MIDX writes
> - midx: use `strvec` for `keep_hashes`
> - midx: build `keep_hashes` array in order
> - midx: use `strset` for retained MIDX files
> - midx-write: handle noop writes when converting incremental chains
>
> The repacking code has been refactored and compaction of MIDX layers
> have been implemented, and incremental strategy that does not require
> all-into-one repacking has been introduced.
>
> Waiting for response(s) to review comment(s).
> cf. <agTw579yuy4iHoMq@szeder.dev>
> cf. <20260513230825.GA1378716@coredump.intra.peff.net>
> source: <cover.1777507303.git.me@ttaylorr.com>
Apologies, I didn't realize you were waiting on these until seeing this
WC report. I sent an extremely tiny reroll
https://lore.kernel.org/git/cover.1779206239.git.me@ttaylorr.com/
that addresses the two outstanding comments you linked. They are very
minor changes, and queueing either version of the series would be
equally fine IMHO.
Thanks,
Taylor
^ permalink raw reply
* [PATCH v6 8/8] send-pack: pass negotiation config in push
From: Derrick Stolee via GitGitGadget @ 2026-05-19 16:24 UTC (permalink / raw)
To: git; +Cc: gitster, ps, Matthew John Cheetham, Derrick Stolee,
Derrick Stolee
In-Reply-To: <pull.2085.v6.git.1779207896.gitgitgadget@gmail.com>
From: Derrick Stolee <stolee@gmail.com>
When push.negotiate is enabled, 'git push' spawns a child 'git fetch
--negotiate-only' process to find common commits. Pass
--negotiation-include and --negotiation-restrict options from the
'remote.<name>.negotiationInclude' and
'remote.<name>.negotiationRestrict' config keys to this child process.
When negotiationRestrict is configured, it replaces the default
behavior of using all remote refs as negotiation tips. This allows
the user to control which local refs are used for push negotiation.
When negotiationInclude is configured, the specified ref patterns
are passed as --negotiation-include to ensure their tips are always
sent as 'have' lines during push negotiation.
Reviewed-by: Matthew John Cheetham <mjcheetham@outlook.com>
Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
Documentation/config/remote.adoc | 6 ++++++
send-pack.c | 37 ++++++++++++++++++++++++++------
send-pack.h | 2 ++
t/t5516-fetch-push.sh | 30 ++++++++++++++++++++++++++
transport.c | 2 ++
5 files changed, 70 insertions(+), 7 deletions(-)
diff --git a/Documentation/config/remote.adoc b/Documentation/config/remote.adoc
index 1951df154e..eb9c8a3c48 100644
--- a/Documentation/config/remote.adoc
+++ b/Documentation/config/remote.adoc
@@ -122,6 +122,9 @@ command-line option. If `--negotiation-restrict` (or its synonym
`--negotiation-tip`) is specified on the command line, then the config
values are not used.
+
+These values also influence negotiation during `git push` if
+`push.negotiate` is enabled.
++
Blank values signal to ignore all previous values, allowing a reset of
the list from broader config scenarios.
@@ -147,6 +150,9 @@ negotiation algorithm still runs and advertises its own selected commits,
but the refs matching `remote.<name>.negotiationInclude` are sent
unconditionally on top of those heuristically selected commits.
+
+These values also influence negotiation during `git push` if
+`push.negotiate` is enabled.
++
Blank values signal to ignore all previous values, allowing a reset of
the list from broader config scenarios.
diff --git a/send-pack.c b/send-pack.c
index 3d5d36ba3b..d18e030ce8 100644
--- a/send-pack.c
+++ b/send-pack.c
@@ -433,28 +433,48 @@ static void reject_invalid_nonce(const char *nonce, int len)
static void get_commons_through_negotiation(struct repository *r,
const char *url,
+ const struct string_list *negotiation_include,
+ const struct string_list *negotiation_restrict,
const struct ref *remote_refs,
struct oid_array *commons)
{
struct child_process child = CHILD_PROCESS_INIT;
const struct ref *ref;
int len = r->hash_algo->hexsz + 1; /* hash + NL */
- int nr_negotiation_tip = 0;
+ int nr_negotiation = 0;
child.git_cmd = 1;
child.no_stdin = 1;
child.out = -1;
strvec_pushl(&child.args, "fetch", "--negotiate-only", NULL);
- for (ref = remote_refs; ref; ref = ref->next) {
- if (!is_null_oid(&ref->new_oid)) {
+
+ if (negotiation_restrict && negotiation_restrict->nr) {
+ struct string_list_item *item;
+ for_each_string_list_item(item, negotiation_restrict)
strvec_pushf(&child.args, "--negotiation-restrict=%s",
- oid_to_hex(&ref->new_oid));
- nr_negotiation_tip++;
+ item->string);
+ nr_negotiation = negotiation_restrict->nr;
+ } else {
+ for (ref = remote_refs; ref; ref = ref->next) {
+ if (!is_null_oid(&ref->new_oid)) {
+ strvec_pushf(&child.args, "--negotiation-restrict=%s",
+ oid_to_hex(&ref->new_oid));
+ nr_negotiation++;
+ }
}
}
+
+ if (negotiation_include && negotiation_include->nr) {
+ struct string_list_item *item;
+ for_each_string_list_item(item, negotiation_include)
+ strvec_pushf(&child.args, "--negotiation-include=%s",
+ item->string);
+ nr_negotiation += negotiation_include->nr;
+ }
+
strvec_push(&child.args, url);
- if (!nr_negotiation_tip) {
+ if (!nr_negotiation) {
child_process_clear(&child);
return;
}
@@ -528,7 +548,10 @@ int send_pack(struct repository *r,
repo_config_get_bool(r, "push.negotiate", &push_negotiate);
if (push_negotiate) {
trace2_region_enter("send_pack", "push_negotiate", r);
- get_commons_through_negotiation(r, args->url, remote_refs, &commons);
+ get_commons_through_negotiation(r, args->url,
+ args->negotiation_include,
+ args->negotiation_restrict,
+ remote_refs, &commons);
trace2_region_leave("send_pack", "push_negotiate", r);
}
diff --git a/send-pack.h b/send-pack.h
index c5ded2d200..13850c98bb 100644
--- a/send-pack.h
+++ b/send-pack.h
@@ -18,6 +18,8 @@ struct repository;
struct send_pack_args {
const char *url;
+ const struct string_list *negotiation_include;
+ const struct string_list *negotiation_restrict;
unsigned verbose:1,
quiet:1,
porcelain:1,
diff --git a/t/t5516-fetch-push.sh b/t/t5516-fetch-push.sh
index ac8447f21e..177cbc6c75 100755
--- a/t/t5516-fetch-push.sh
+++ b/t/t5516-fetch-push.sh
@@ -254,6 +254,36 @@ test_expect_success 'push with negotiation does not attempt to fetch submodules'
! grep "Fetching submodule" err
'
+test_expect_success 'push with negotiation and remote.<name>.negotiationInclude' '
+ test_when_finished rm -rf negotiation_include &&
+ mk_empty negotiation_include &&
+ git push negotiation_include $the_first_commit:refs/remotes/origin/first_commit &&
+ test_commit -C negotiation_include unrelated_commit &&
+ git -C negotiation_include config receive.hideRefs refs/remotes/origin/first_commit &&
+ test_when_finished "rm event" &&
+ GIT_TRACE2_EVENT="$(pwd)/event" \
+ git -c protocol.version=2 -c push.negotiate=1 \
+ -c remote.negotiation_include.negotiationInclude=refs/heads/main \
+ push negotiation_include refs/heads/main:refs/remotes/origin/main &&
+ test_grep \"key\":\"total_rounds\" event &&
+ grep_wrote 2 event # 1 commit, 1 tree
+'
+
+test_expect_success 'push with negotiation and remote.<name>.negotiationRestrict' '
+ test_when_finished rm -rf negotiation_restrict &&
+ mk_empty negotiation_restrict &&
+ git push negotiation_restrict $the_first_commit:refs/remotes/origin/first_commit &&
+ test_commit -C negotiation_restrict unrelated_commit &&
+ git -C negotiation_restrict config receive.hideRefs refs/remotes/origin/first_commit &&
+ test_when_finished "rm event" &&
+ GIT_TRACE2_EVENT="$(pwd)/event" \
+ git -c protocol.version=2 -c push.negotiate=1 \
+ -c remote.negotiation_restrict.negotiationRestrict=refs/heads/main \
+ push negotiation_restrict refs/heads/main:refs/remotes/origin/main &&
+ test_grep \"key\":\"total_rounds\" event &&
+ grep_wrote 2 event # 1 commit, 1 tree
+'
+
test_expect_success 'push without wildcard' '
mk_empty testrepo &&
diff --git a/transport.c b/transport.c
index fa54928966..a2d8958cb8 100644
--- a/transport.c
+++ b/transport.c
@@ -921,6 +921,8 @@ static int git_transport_push(struct transport *transport, struct ref *remote_re
args.atomic = !!(flags & TRANSPORT_PUSH_ATOMIC);
args.push_options = transport->push_options;
args.url = transport->url;
+ args.negotiation_include = &transport->remote->negotiation_include;
+ args.negotiation_restrict = &transport->remote->negotiation_restrict;
if (flags & TRANSPORT_PUSH_CERT_ALWAYS)
args.push_cert = SEND_PACK_PUSH_CERT_ALWAYS;
--
gitgitgadget
^ permalink raw reply related
* [PATCH v6 7/8] remote: add remote.*.negotiationInclude config
From: Derrick Stolee via GitGitGadget @ 2026-05-19 16:24 UTC (permalink / raw)
To: git; +Cc: gitster, ps, Matthew John Cheetham, Derrick Stolee,
Derrick Stolee
In-Reply-To: <pull.2085.v6.git.1779207896.gitgitgadget@gmail.com>
From: Derrick Stolee <stolee@gmail.com>
Add a new 'remote.<name>.negotiationInclude' multi-valued config option that
provides default values for --negotiation-include when no
--negotiation-include arguments are specified over the command line. This
is a mirror of how 'remote.<name>.negotiationRestrict' specifies defaults
for the --negotiation-restrict arguments.
Each value is either an exact ref name or a glob pattern whose tips should
always be sent as 'have' lines during negotiation. The config values are
resolved through the same resolve_negotiation_include() codepath as the CLI
options.
This option is additive with the normal negotiation process: the negotiation
algorithm still runs and advertises its own selected commits, but the refs
matching the config are sent unconditionally on top of those heuristically
selected commits.
Similar to the negotiationRestrict config, an empty value resets the value
list to allow ignoring earlier config values, such as those that might be
set in system or global config.
Reviewed-by: Matthew John Cheetham <mjcheetham@outlook.com>
Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
Documentation/config/remote.adoc | 25 ++++++++++++++++
Documentation/fetch-options.adoc | 4 +++
builtin/fetch.c | 12 ++++++++
remote.c | 5 ++++
remote.h | 1 +
t/t5510-fetch.sh | 49 ++++++++++++++++++++++++++++++++
6 files changed, 96 insertions(+)
diff --git a/Documentation/config/remote.adoc b/Documentation/config/remote.adoc
index 4dcf81fbce..1951df154e 100644
--- a/Documentation/config/remote.adoc
+++ b/Documentation/config/remote.adoc
@@ -125,6 +125,31 @@ values are not used.
Blank values signal to ignore all previous values, allowing a reset of
the list from broader config scenarios.
+remote.<name>.negotiationInclude::
+ When negotiating with this remote during `git fetch`, the client
+ advertises a list of commits that exist locally. In repos with
+ many references, this list of "haves" can be truncated. Depending
+ on data shape, dropping certain references may be expensive. This
+ multi-valued config option specifies references, commit hashes,
+ or ref pattern globs whose tips should always be sent as "have"
+ commits during fetch negotiation with this remote.
++
+Each value is either an exact ref name (e.g. `refs/heads/release`), a
+commit hash, or a glob pattern (e.g. `refs/heads/release/*`). The
+pattern syntax is the same as for `--negotiation-include`.
++
+These config values are used as defaults for the `--negotiation-include`
+command-line option. If `--negotiation-include` is specified on the
+command line, then the config values are not used.
++
+This option is additive with the normal negotiation process: the
+negotiation algorithm still runs and advertises its own selected commits,
+but the refs matching `remote.<name>.negotiationInclude` are sent
+unconditionally on top of those heuristically selected commits.
++
+Blank values signal to ignore all previous values, allowing a reset of
+the list from broader config scenarios.
+
remote.<name>.followRemoteHEAD::
How linkgit:git-fetch[1] should handle updates to `remotes/<name>/HEAD`
when fetching using the configured refspecs of a remote.
diff --git a/Documentation/fetch-options.adoc b/Documentation/fetch-options.adoc
index 7b897a7202..8074004377 100644
--- a/Documentation/fetch-options.adoc
+++ b/Documentation/fetch-options.adoc
@@ -91,6 +91,10 @@ The pattern syntax is the same as for `--negotiation-restrict`.
If `--negotiation-restrict` is used, the have set is first restricted by
that option and then increased to include the tips specified by
`--negotiation-include`.
++
+If this option is not specified on the command line, then any
+`remote.<name>.negotiationInclude` config values for the current remote
+are used instead.
`--negotiate-only`::
Do not fetch anything from the server, and instead print the
diff --git a/builtin/fetch.c b/builtin/fetch.c
index ba56e9022b..1af6500c1d 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1634,6 +1634,18 @@ static struct transport *prepare_transport(struct remote *remote, int deepen,
else
warning(_("ignoring %s because the protocol does not support it"),
"--negotiation-include");
+ } else if (remote->negotiation_include.nr) {
+ if (transport->smart_options) {
+ add_negotiation_tips(&remote->negotiation_include,
+ &transport->smart_options->negotiation_include_tips,
+ "--negotiation-include");
+ } else {
+ struct strbuf config_name = STRBUF_INIT;
+ strbuf_addf(&config_name, "remote.%s.negotiationInclude", remote->name);
+ warning(_("ignoring %s because the protocol does not support it"),
+ config_name.buf);
+ strbuf_release(&config_name);
+ }
}
return transport;
}
diff --git a/remote.c b/remote.c
index 620086e16e..6fb5758820 100644
--- a/remote.c
+++ b/remote.c
@@ -153,6 +153,7 @@ static struct remote *make_remote(struct remote_state *remote_state,
refspec_init_fetch(&ret->fetch);
string_list_init_dup(&ret->server_options);
string_list_init_dup(&ret->negotiation_restrict);
+ string_list_init_dup(&ret->negotiation_include);
ALLOC_GROW(remote_state->remotes, remote_state->remotes_nr + 1,
remote_state->remotes_alloc);
@@ -181,6 +182,7 @@ static void remote_clear(struct remote *remote)
FREE_AND_NULL(remote->http_proxy_authmethod);
string_list_clear(&remote->server_options, 0);
string_list_clear(&remote->negotiation_restrict, 0);
+ string_list_clear(&remote->negotiation_include, 0);
}
static void add_merge(struct branch *branch, const char *name)
@@ -567,6 +569,9 @@ static int handle_config(const char *key, const char *value,
} else if (!strcmp(subkey, "negotiationrestrict")) {
return parse_transport_option(key, value,
&remote->negotiation_restrict);
+ } else if (!strcmp(subkey, "negotiationinclude")) {
+ return parse_transport_option(key, value,
+ &remote->negotiation_include);
} else if (!strcmp(subkey, "followremotehead")) {
const char *no_warn_branch;
if (!strcmp(value, "never"))
diff --git a/remote.h b/remote.h
index e6ec37c393..d8809b6991 100644
--- a/remote.h
+++ b/remote.h
@@ -118,6 +118,7 @@ struct remote {
struct string_list server_options;
struct string_list negotiation_restrict;
+ struct string_list negotiation_include;
enum follow_remote_head_settings follow_remote_head;
const char *no_warn_branch;
diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index bc2e2af959..33f61ac12a 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -1587,6 +1587,55 @@ test_expect_success '--negotiation-include avoids duplicates with negotiator' '
test_line_count = 1 matches
'
+test_expect_success 'remote.<name>.negotiationInclude used as default for --negotiation-include' '
+ test_when_finished rm -f trace &&
+ setup_negotiation_tip server server 0 &&
+
+ # test the reset of the list on an empty value
+ git -C client config --add remote.origin.negotiationInclude refs/tags/alpha_1 &&
+ git -C client config --add remote.origin.negotiationInclude "" &&
+ git -C client config --add remote.origin.negotiationInclude refs/tags/beta_1 &&
+ GIT_TRACE_PACKET="$(pwd)/trace" git -C client fetch \
+ --negotiation-restrict=beta_2 \
+ origin alpha_s beta_s &&
+
+ ALPHA_1=$(git -C client rev-parse alpha_1) &&
+ test_grep ! "fetch> have $ALPHA_1" trace &&
+ BETA_1=$(git -C client rev-parse beta_1) &&
+ test_grep "fetch> have $BETA_1" trace
+'
+
+test_expect_success 'remote.<name>.negotiationInclude works with glob patterns' '
+ test_when_finished rm -f trace &&
+ setup_negotiation_tip server server 0 &&
+
+ git -C client config --add remote.origin.negotiationInclude "refs/tags/beta_*" &&
+ GIT_TRACE_PACKET="$(pwd)/trace" git -C client fetch \
+ --negotiation-restrict=alpha_1 \
+ origin alpha_s beta_s &&
+
+ BETA_1=$(git -C client rev-parse beta_1) &&
+ test_grep "fetch> have $BETA_1" trace &&
+ BETA_2=$(git -C client rev-parse beta_2) &&
+ test_grep "fetch> have $BETA_2" trace
+'
+
+test_expect_success 'CLI --negotiation-include overrides remote.<name>.negotiationInclude' '
+ test_when_finished rm -f trace &&
+ setup_negotiation_tip server server 0 &&
+
+ git -C client config --add remote.origin.negotiationInclude refs/tags/beta_2 &&
+ GIT_TRACE_PACKET="$(pwd)/trace" git -C client fetch \
+ --negotiation-restrict=alpha_1 \
+ --negotiation-include=refs/tags/beta_1 \
+ origin alpha_s beta_s &&
+
+ BETA_1=$(git -C client rev-parse beta_1) &&
+ test_grep "fetch> have $BETA_1" trace &&
+ BETA_2=$(git -C client rev-parse beta_2) &&
+ test_grep ! "fetch> have $BETA_2" trace
+'
+
test_expect_success '--negotiation-include avoids duplicates with v0' '
test_when_finished rm -f trace &&
setup_negotiation_tip server server 0 &&
--
gitgitgadget
^ permalink raw reply related
* [PATCH v6 6/8] fetch: add --negotiation-include option for negotiation
From: Derrick Stolee via GitGitGadget @ 2026-05-19 16:24 UTC (permalink / raw)
To: git; +Cc: gitster, ps, Matthew John Cheetham, Derrick Stolee,
Derrick Stolee
In-Reply-To: <pull.2085.v6.git.1779207896.gitgitgadget@gmail.com>
From: Derrick Stolee <stolee@gmail.com>
Add a new --negotiation-include option to 'git fetch', which ensures
that certain ref tips are always sent as 'have' lines during fetch
negotiation, regardless of what the negotiation algorithm selects.
This is useful when the repository has a large number of references, so
the normal negotiation algorithm truncates the list. This is especially
important in repositories with long parallel commit histories. For
example, a repo could have a 'dev' branch for development and a
'release' branch for released versions. If the 'dev' branch isn't
selected for negotiation, then it's not a big deal because there are
many in-progress development branches with a shared history. However, if
'release' is not selected for negotiation, then the server may think
that this is the first time the client has asked for that reference,
causing a full download of its parallel commit history (and any extra
data that may be unique to that branch). This is based on a real example
where certain fetches would grow to 60+ GB when a release branch
updated.
This option is a complement to --negotiation-restrict, which reduces the
negotiation ref set to a specific list. In the earlier example, using
--negotiation-restrict to focus the negotiation to 'dev' and 'release'
would avoid those problematic downloads, but would still not allow
advertising potentially-relevant user branches. In this way, the
'include' version solves the problem I mention while allowing
negotiation to pick other references opportunistically. The two options
can also be combined to allow the best of both worlds.
The argument may be an exact ref name or a glob pattern. Non-existent
refs are silently ignored. This behavior is also updated in the ref matching
logic for the related --negotiation-restrict option to match.
The implementation outputs the requested objects as haves before the
negotiator performs its own algorithm to choose the next haves. Use the new
have_sent() interface to signal these have commits were sent before engaging
with the negotiator's next() iterator.
Also add --negotiation-include to 'git pull' passthrough options.
Reviewed-by: Matthew John Cheetham <mjcheetham@outlook.com>
Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
Documentation/fetch-options.adoc | 19 +++++++
builtin/fetch.c | 38 ++++++++++---
builtin/pull.c | 3 ++
fetch-pack.c | 81 +++++++++++++++++++++++++---
fetch-pack.h | 6 ++-
t/t5510-fetch.sh | 91 ++++++++++++++++++++++++++++++++
transport.c | 8 ++-
transport.h | 5 +-
8 files changed, 232 insertions(+), 19 deletions(-)
diff --git a/Documentation/fetch-options.adoc b/Documentation/fetch-options.adoc
index d39cecb446..7b897a7202 100644
--- a/Documentation/fetch-options.adoc
+++ b/Documentation/fetch-options.adoc
@@ -73,6 +73,25 @@ See also the `fetch.negotiationAlgorithm` and `push.negotiate`
configuration variables documented in linkgit:git-config[1], and the
`--negotiate-only` option below.
+`--negotiation-include=(<commit>|<glob>)`::
+ Ensure that the commits at the given tips are always sent as "have"
+ lines during fetch negotiation, regardless of what the negotiation
+ algorithm selects. This is useful to guarantee that common
+ history reachable from specific refs is always considered, even
+ when `--negotiation-restrict` restricts the set of tips or when
+ the negotiation algorithm would otherwise skip them.
++
+This option may be specified more than once; if so, each commit is sent
+unconditionally.
++
+The argument may be an exact ref name (e.g. `refs/heads/release`), an
+object hash, or a glob pattern (e.g. `refs/heads/release/{asterisk}`).
+The pattern syntax is the same as for `--negotiation-restrict`.
++
+If `--negotiation-restrict` is used, the have set is first restricted by
+that option and then increased to include the tips specified by
+`--negotiation-include`.
+
`--negotiate-only`::
Do not fetch anything from the server, and instead print the
ancestors of the provided `--negotiation-restrict=` arguments,
diff --git a/builtin/fetch.c b/builtin/fetch.c
index a957739f37..ba56e9022b 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -99,6 +99,7 @@ static struct transport *gsecondary;
static struct refspec refmap = REFSPEC_INIT_FETCH;
static struct string_list server_options = STRING_LIST_INIT_DUP;
static struct string_list negotiation_restrict = STRING_LIST_INIT_NODUP;
+static struct string_list negotiation_include = STRING_LIST_INIT_NODUP;
struct fetch_config {
enum display_format display_format;
@@ -1534,23 +1535,29 @@ static int add_oid(const struct reference *ref, void *cb_data)
return 0;
}
-static void add_negotiation_restrict_tips(struct git_transport_options *smart_options)
+static void add_negotiation_tips(struct string_list *input_list,
+ struct oid_array **output_list,
+ const char *argname)
{
struct oid_array *oids = xcalloc(1, sizeof(*oids));
int i;
- for (i = 0; i < negotiation_restrict.nr; i++) {
- const char *s = negotiation_restrict.items[i].string;
+ for (i = 0; i < input_list->nr; i++) {
+ const char *s = input_list->items[i].string;
struct refs_for_each_ref_options opts = {
.pattern = s,
};
int old_nr;
if (!has_glob_specials(s)) {
struct object_id oid;
+
+ /* Ignore missing reference. */
if (repo_get_oid(the_repository, s, &oid))
- die(_("%s is not a valid object"), s);
+ continue;
+ /* Fail on missing object pointed by ref. */
if (!odb_has_object(the_repository->objects, &oid, 0))
die(_("the object %s does not exist"), s);
+
oid_array_append(oids, &oid);
continue;
}
@@ -1559,9 +1566,9 @@ static void add_negotiation_restrict_tips(struct git_transport_options *smart_op
add_oid, oids, &opts);
if (old_nr == oids->nr)
warning(_("ignoring %s=%s because it does not match any refs"),
- "--negotiation-restrict", s);
+ argname, s);
}
- smart_options->negotiation_restrict_tips = oids;
+ *output_list = oids;
}
static struct transport *prepare_transport(struct remote *remote, int deepen,
@@ -1597,7 +1604,9 @@ static struct transport *prepare_transport(struct remote *remote, int deepen,
}
if (negotiation_restrict.nr) {
if (transport->smart_options)
- add_negotiation_restrict_tips(transport->smart_options);
+ add_negotiation_tips(&negotiation_restrict,
+ &transport->smart_options->negotiation_restrict_tips,
+ "--negotiation-restrict");
else
warning(_("ignoring %s because the protocol does not support it"),
"--negotiation-restrict");
@@ -1606,7 +1615,9 @@ static struct transport *prepare_transport(struct remote *remote, int deepen,
for_each_string_list_item(item, &remote->negotiation_restrict)
string_list_append(&negotiation_restrict, item->string);
if (transport->smart_options)
- add_negotiation_restrict_tips(transport->smart_options);
+ add_negotiation_tips(&negotiation_restrict,
+ &transport->smart_options->negotiation_restrict_tips,
+ "--negotiation-restrict");
else {
struct strbuf config_name = STRBUF_INIT;
strbuf_addf(&config_name, "remote.%s.negotiationRestrict", remote->name);
@@ -1615,6 +1626,15 @@ static struct transport *prepare_transport(struct remote *remote, int deepen,
strbuf_release(&config_name);
}
}
+ if (negotiation_include.nr) {
+ if (transport->smart_options)
+ add_negotiation_tips(&negotiation_include,
+ &transport->smart_options->negotiation_include_tips,
+ "--negotiation-include");
+ else
+ warning(_("ignoring %s because the protocol does not support it"),
+ "--negotiation-include");
+ }
return transport;
}
@@ -2582,6 +2602,8 @@ int cmd_fetch(int argc,
OPT_STRING_LIST(0, "negotiation-restrict", &negotiation_restrict, N_("revision"),
N_("report that we have only objects reachable from this object")),
OPT_ALIAS(0, "negotiation-tip", "negotiation-restrict"),
+ OPT_STRING_LIST(0, "negotiation-include", &negotiation_include, N_("revision"),
+ N_("ensure this ref is always sent as a negotiation have")),
OPT_BOOL(0, "negotiate-only", &negotiate_only,
N_("do not fetch a packfile; instead, print ancestors of negotiation tips")),
OPT_PARSE_LIST_OBJECTS_FILTER(&filter_options),
diff --git a/builtin/pull.c b/builtin/pull.c
index cc6ce485fc..d49b09114a 100644
--- a/builtin/pull.c
+++ b/builtin/pull.c
@@ -1000,6 +1000,9 @@ int cmd_pull(int argc,
N_("report that we have only objects reachable from this object"),
0),
OPT_ALIAS(0, "negotiation-tip", "negotiation-restrict"),
+ OPT_PASSTHRU_ARGV(0, "negotiation-include", &opt_fetch, N_("revision"),
+ N_("ensure this ref is always sent as a negotiation have"),
+ 0),
OPT_BOOL(0, "show-forced-updates", &opt_show_forced_updates,
N_("check for forced-updates on all updated branches")),
OPT_PASSTHRU(0, "set-upstream", &set_upstream, NULL,
diff --git a/fetch-pack.c b/fetch-pack.c
index baf239adf9..96071434b8 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -25,6 +25,7 @@
#include "oidset.h"
#include "packfile.h"
#include "odb.h"
+#include "object-name.h"
#include "path.h"
#include "connected.h"
#include "fetch-negotiator.h"
@@ -332,6 +333,21 @@ static void send_filter(struct fetch_pack_args *args,
}
}
+static void add_oids_to_set(const struct oid_array *array,
+ struct oidset *set)
+{
+ if (!array)
+ return;
+
+ for (size_t i = 0; i < array->nr; i++) {
+ struct object_id *oid = &array->oid[i];
+ if (!odb_has_object(the_repository->objects, oid, 0))
+ die(_("the object %s does not exist"), oid_to_hex(oid));
+
+ oidset_insert(set, oid);
+ }
+}
+
static int find_common(struct fetch_negotiator *negotiator,
struct fetch_pack_args *args,
int fd[2], struct object_id *result_oid,
@@ -347,6 +363,7 @@ static int find_common(struct fetch_negotiator *negotiator,
struct strbuf req_buf = STRBUF_INIT;
size_t state_len = 0;
struct packet_reader reader;
+ struct oidset negotiation_include_oids = OIDSET_INIT;
if (args->stateless_rpc && multi_ack == 1)
die(_("the option '%s' requires '%s'"), "--stateless-rpc", "multi_ack_detailed");
@@ -474,6 +491,27 @@ static int find_common(struct fetch_negotiator *negotiator,
trace2_region_enter("fetch-pack", "negotiation_v0_v1", the_repository);
flushes = 0;
retval = -1;
+
+ /* Send unconditional haves from --negotiation-include */
+ add_oids_to_set(args->negotiation_include_tips,
+ &negotiation_include_oids);
+ if (oidset_size(&negotiation_include_oids)) {
+ struct oidset_iter iter;
+ oidset_iter_init(&negotiation_include_oids, &iter);
+
+ while ((oid = oidset_iter_next(&iter))) {
+ struct commit *commit;
+ packet_buf_write(&req_buf, "have %s\n",
+ oid_to_hex(oid));
+ print_verbose(args, "have %s", oid_to_hex(oid));
+ count++;
+
+ commit = lookup_commit(the_repository, oid);
+ if (commit)
+ negotiator->have_sent(negotiator, commit);
+ }
+ }
+
while ((oid = negotiator->next(negotiator))) {
packet_buf_write(&req_buf, "have %s\n", oid_to_hex(oid));
print_verbose(args, "have %s", oid_to_hex(oid));
@@ -584,6 +622,7 @@ done:
flushes++;
}
strbuf_release(&req_buf);
+ oidset_clear(&negotiation_include_oids);
if (!got_ready || !no_done)
consume_shallow_list(args, &reader);
@@ -1305,11 +1344,27 @@ static void add_common(struct strbuf *req_buf, struct oidset *common)
static int add_haves(struct fetch_negotiator *negotiator,
struct strbuf *req_buf,
- int *haves_to_send)
+ int *haves_to_send,
+ struct oidset *negotiation_include_oids)
{
int haves_added = 0;
const struct object_id *oid;
+ /* Send unconditional haves from --negotiation-include */
+ if (negotiation_include_oids) {
+ struct oidset_iter iter;
+ oidset_iter_init(negotiation_include_oids, &iter);
+
+ while ((oid = oidset_iter_next(&iter))) {
+ struct commit *commit = lookup_commit(the_repository, oid);
+ if (commit) {
+ packet_buf_write(req_buf, "have %s\n",
+ oid_to_hex(oid));
+ negotiator->have_sent(negotiator, commit);
+ }
+ }
+ }
+
while ((oid = negotiator->next(negotiator))) {
packet_buf_write(req_buf, "have %s\n", oid_to_hex(oid));
if (++haves_added >= *haves_to_send)
@@ -1358,7 +1413,8 @@ static int send_fetch_request(struct fetch_negotiator *negotiator, int fd_out,
struct fetch_pack_args *args,
const struct ref *wants, struct oidset *common,
int *haves_to_send, int *in_vain,
- int sideband_all, int seen_ack)
+ int sideband_all, int seen_ack,
+ struct oidset *negotiation_include_oids)
{
int haves_added;
int done_sent = 0;
@@ -1413,7 +1469,8 @@ static int send_fetch_request(struct fetch_negotiator *negotiator, int fd_out,
/* Add all of the common commits we've found in previous rounds */
add_common(&req_buf, common);
- haves_added = add_haves(negotiator, &req_buf, haves_to_send);
+ haves_added = add_haves(negotiator, &req_buf, haves_to_send,
+ negotiation_include_oids);
*in_vain += haves_added;
trace2_data_intmax("negotiation_v2", the_repository, "haves_added", haves_added);
trace2_data_intmax("negotiation_v2", the_repository, "in_vain", *in_vain);
@@ -1657,6 +1714,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,
struct ref *ref = copy_ref_list(orig_ref);
enum fetch_state state = FETCH_CHECK_LOCAL;
struct oidset common = OIDSET_INIT;
+ struct oidset negotiation_include_oids = OIDSET_INIT;
struct packet_reader reader;
int in_vain = 0, negotiation_started = 0;
int negotiation_round = 0;
@@ -1729,6 +1787,8 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,
state = FETCH_SEND_REQUEST;
mark_tips(negotiator, args->negotiation_restrict_tips);
+ add_oids_to_set(args->negotiation_include_tips,
+ &negotiation_include_oids);
for_each_cached_alternate(negotiator,
insert_one_alternate_object);
break;
@@ -1747,7 +1807,8 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,
&common,
&haves_to_send, &in_vain,
reader.use_sideband,
- seen_ack)) {
+ seen_ack,
+ &negotiation_include_oids)) {
trace2_region_leave_printf("negotiation_v2", "round",
the_repository, "%d",
negotiation_round);
@@ -1883,6 +1944,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,
negotiator->release(negotiator);
oidset_clear(&common);
+ oidset_clear(&negotiation_include_oids);
return ref;
}
@@ -2181,12 +2243,14 @@ void negotiate_using_fetch(const struct oid_array *negotiation_restrict_tips,
const struct string_list *server_options,
int stateless_rpc,
int fd[],
- struct oidset *acked_commits)
+ struct oidset *acked_commits,
+ const struct oid_array *negotiation_include_tips)
{
struct fetch_negotiator negotiator;
struct packet_reader reader;
struct object_array nt_object_array = OBJECT_ARRAY_INIT;
struct strbuf req_buf = STRBUF_INIT;
+ struct oidset negotiation_include_oids = OIDSET_INIT;
int haves_to_send = INITIAL_FLUSH;
int in_vain = 0;
int seen_ack = 0;
@@ -2197,6 +2261,9 @@ void negotiate_using_fetch(const struct oid_array *negotiation_restrict_tips,
fetch_negotiator_init(the_repository, &negotiator);
mark_tips(&negotiator, negotiation_restrict_tips);
+ add_oids_to_set(negotiation_include_tips,
+ &negotiation_include_oids);
+
packet_reader_init(&reader, fd[0], NULL, 0,
PACKET_READ_CHOMP_NEWLINE |
PACKET_READ_DIE_ON_ERR_PACKET);
@@ -2221,7 +2288,8 @@ void negotiate_using_fetch(const struct oid_array *negotiation_restrict_tips,
packet_buf_write(&req_buf, "wait-for-done");
- haves_added = add_haves(&negotiator, &req_buf, &haves_to_send);
+ haves_added = add_haves(&negotiator, &req_buf, &haves_to_send,
+ &negotiation_include_oids);
in_vain += haves_added;
if (!haves_added || (seen_ack && in_vain >= MAX_IN_VAIN))
last_iteration = 1;
@@ -2273,6 +2341,7 @@ void negotiate_using_fetch(const struct oid_array *negotiation_restrict_tips,
clear_common_flag(acked_commits);
object_array_clear(&nt_object_array);
+ oidset_clear(&negotiation_include_oids);
negotiator.release(&negotiator);
strbuf_release(&req_buf);
}
diff --git a/fetch-pack.h b/fetch-pack.h
index 6c70c942c2..6d0dec7f41 100644
--- a/fetch-pack.h
+++ b/fetch-pack.h
@@ -19,9 +19,10 @@ struct fetch_pack_args {
/*
* If not NULL, during packfile negotiation, fetch-pack will send "have"
- * lines only with these tips and their ancestors.
+ * lines for all _include_ tips and then a subset of the _restrict_ tips.
*/
const struct oid_array *negotiation_restrict_tips;
+ const struct oid_array *negotiation_include_tips;
unsigned deepen_relative:1;
unsigned quiet:1;
@@ -93,7 +94,8 @@ void negotiate_using_fetch(const struct oid_array *negotiation_restrict_tips,
const struct string_list *server_options,
int stateless_rpc,
int fd[],
- struct oidset *acked_commits);
+ struct oidset *acked_commits,
+ const struct oid_array *negotiation_include_tips);
/*
* Print an appropriate error message for each sought ref that wasn't
diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index eff3ce8e2d..bc2e2af959 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -1460,6 +1460,16 @@ EOF
test_cmp fatal-expect fatal-actual
'
+test_expect_success '--negotiation-tip ignores missing refs and invalid hashes' '
+ setup_negotiation_tip server server 0 &&
+ GIT_TRACE_PACKET="$(pwd)/trace" git -C client fetch \
+ --negotiation-tip=alpha_1 --negotiation-tip=beta_1 \
+ --negotiation-tip=no-such-ref \
+ --negotiation-tip=invalid-hash \
+ origin alpha_s beta_s &&
+ check_negotiation_tip
+'
+
test_expect_success '--negotiation-restrict limits "have" lines sent' '
setup_negotiation_tip server server 0 &&
GIT_TRACE_PACKET="$(pwd)/trace" git -C client fetch \
@@ -1511,6 +1521,87 @@ test_expect_success 'CLI --negotiation-restrict overrides remote config' '
test_grep ! "fetch> have $BETA_1" trace
'
+test_expect_success '--negotiation-include includes configured refs as haves' '
+ test_when_finished rm -f trace &&
+ setup_negotiation_tip server server 0 &&
+
+ GIT_TRACE_PACKET="$(pwd)/trace" git -C client fetch \
+ --negotiation-restrict=alpha_1 \
+ --negotiation-include=refs/tags/beta_1 \
+ origin alpha_s beta_s &&
+
+ ALPHA_1=$(git -C client rev-parse alpha_1) &&
+ test_grep "fetch> have $ALPHA_1" trace &&
+ BETA_1=$(git -C client rev-parse beta_1) &&
+ test_grep "fetch> have $BETA_1" trace
+'
+
+test_expect_success '--negotiation-include works with glob patterns' '
+ test_when_finished rm -f trace &&
+ setup_negotiation_tip server server 0 &&
+
+ GIT_TRACE_PACKET="$(pwd)/trace" git -C client fetch \
+ --negotiation-restrict=alpha_1 \
+ --negotiation-include="refs/tags/beta_*" \
+ origin alpha_s beta_s &&
+
+ BETA_1=$(git -C client rev-parse beta_1) &&
+ test_grep "fetch> have $BETA_1" trace &&
+ BETA_2=$(git -C client rev-parse beta_2) &&
+ test_grep "fetch> have $BETA_2" trace
+'
+
+test_expect_success '--negotiation-include is additive with negotiation' '
+ test_when_finished rm -f trace &&
+ setup_negotiation_tip server server 0 &&
+
+ GIT_TRACE_PACKET="$(pwd)/trace" git -C client fetch \
+ --negotiation-include=refs/tags/beta_1 \
+ origin alpha_s beta_s &&
+
+ BETA_1=$(git -C client rev-parse beta_1) &&
+ test_grep "fetch> have $BETA_1" trace
+'
+
+test_expect_success '--negotiation-include ignores non-existent refs silently' '
+ setup_negotiation_tip server server 0 &&
+
+ git -C client fetch --quiet \
+ --negotiation-restrict=alpha_1 \
+ --negotiation-include=refs/tags/nonexistent \
+ origin alpha_s beta_s 2>err &&
+ test_must_be_empty err
+'
+
+test_expect_success '--negotiation-include avoids duplicates with negotiator' '
+ test_when_finished rm -f trace &&
+ setup_negotiation_tip server server 0 &&
+
+ ALPHA_1=$(git -C client rev-parse alpha_1) &&
+ GIT_TRACE_PACKET="$(pwd)/trace" git -C client fetch \
+ --negotiation-restrict=alpha_1 \
+ --negotiation-include=refs/tags/alpha_1 \
+ origin alpha_s beta_s &&
+
+ test_grep "fetch> have $ALPHA_1" trace >matches &&
+ test_line_count = 1 matches
+'
+
+test_expect_success '--negotiation-include avoids duplicates with v0' '
+ test_when_finished rm -f trace &&
+ setup_negotiation_tip server server 0 &&
+
+ ALPHA_1=$(git -C client rev-parse alpha_1) &&
+ GIT_TRACE_PACKET="$(pwd)/trace" git -C client \
+ -c protocol.version=0 fetch \
+ --negotiation-restrict=alpha_1 \
+ --negotiation-include=refs/tags/alpha_1 \
+ origin alpha_s beta_s &&
+
+ test_grep "fetch> have $ALPHA_1" trace >matches &&
+ test_line_count = 1 matches
+'
+
test_expect_success SYMLINKS 'clone does not get confused by a D/F conflict' '
git init df-conflict &&
(
diff --git a/transport.c b/transport.c
index a3051f6733..fa54928966 100644
--- a/transport.c
+++ b/transport.c
@@ -464,6 +464,7 @@ static int fetch_refs_via_pack(struct transport *transport,
args.stateless_rpc = transport->stateless_rpc;
args.server_options = transport->server_options;
args.negotiation_restrict_tips = data->options.negotiation_restrict_tips;
+ args.negotiation_include_tips = data->options.negotiation_include_tips;
args.reject_shallow_remote = transport->smart_options->reject_shallow;
if (!data->finished_handshake) {
@@ -495,7 +496,8 @@ static int fetch_refs_via_pack(struct transport *transport,
transport->server_options,
transport->stateless_rpc,
data->fd,
- data->options.acked_commits);
+ data->options.acked_commits,
+ data->options.negotiation_include_tips);
ret = 0;
}
goto cleanup;
@@ -983,6 +985,10 @@ static int disconnect_git(struct transport *transport)
oid_array_clear(data->options.negotiation_restrict_tips);
free(data->options.negotiation_restrict_tips);
}
+ if (data->options.negotiation_include_tips) {
+ oid_array_clear(data->options.negotiation_include_tips);
+ free(data->options.negotiation_include_tips);
+ }
list_objects_filter_release(&data->options.filter_options);
oid_array_clear(&data->extra_have);
oid_array_clear(&data->shallow);
diff --git a/transport.h b/transport.h
index cdeb33c16f..97d905ecc0 100644
--- a/transport.h
+++ b/transport.h
@@ -40,13 +40,14 @@ struct git_transport_options {
/*
* This is only used during fetch. See the documentation of
- * negotiation_restrict_tips in struct fetch_pack_args.
+ * these member names in struct fetch_pack_args.
*
- * This field is only supported by transports that support connect or
+ * These fields are only supported by transports that support connect or
* stateless_connect. Set this field directly instead of using
* transport_set_option().
*/
struct oid_array *negotiation_restrict_tips;
+ struct oid_array *negotiation_include_tips;
/*
* If allocated, whenever transport_fetch_refs() is called, add known
--
gitgitgadget
^ permalink raw reply related
* [PATCH v6 5/8] negotiator: add have_sent() interface
From: Derrick Stolee via GitGitGadget @ 2026-05-19 16:24 UTC (permalink / raw)
To: git; +Cc: gitster, ps, Matthew John Cheetham, Derrick Stolee,
Derrick Stolee
In-Reply-To: <pull.2085.v6.git.1779207896.gitgitgadget@gmail.com>
From: Derrick Stolee <stolee@gmail.com>
In a future change, we will introduce a capability to choose specific commit
OIDs as 'have's in fetch negotiation, with the ability to have the
negotiator choose more 'have's to increase coverage beyond that required
core set. The negotiator works to avoid emitting 'have's that can reach each
other, but that logic is hidden beneath the negotiator's iterator function
pointer ('next'). We need a way to communicate to the negotiator that we
have picked a 'have' so it could incorporate that into its logic.
Add a have_sent() method to the fetch_negotiator interface. This is the
signal that allows the negotiator to track the commit as already shown and
can perform the proper bookkeeping to avoid emitting those objects or
anything they can reach.
For our non-trivial negotiators, it is sufficient to mark these commits as
common, so the implementation is quite simple. This logic will be exercised
in the next change.
Reviewed-by: Matthew John Cheetham <mjcheetham@outlook.com>
Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
fetch-negotiator.h | 9 +++++++++
negotiator/default.c | 8 ++++++++
negotiator/noop.c | 7 +++++++
negotiator/skipping.c | 8 ++++++++
4 files changed, 32 insertions(+)
diff --git a/fetch-negotiator.h b/fetch-negotiator.h
index e348905a1f..6ca422a064 100644
--- a/fetch-negotiator.h
+++ b/fetch-negotiator.h
@@ -47,6 +47,15 @@ struct fetch_negotiator {
*/
int (*ack)(struct fetch_negotiator *, struct commit *);
+ /*
+ * Inform the negotiator that this commit has already been sent as
+ * a "have" line outside of the negotiator's control. The negotiator
+ * should avoid outputting it from next() and may use it to optimize
+ * further negotiation (e.g., by treating it and its ancestors as
+ * common).
+ */
+ void (*have_sent)(struct fetch_negotiator *, struct commit *);
+
void (*release)(struct fetch_negotiator *);
/* internal use */
diff --git a/negotiator/default.c b/negotiator/default.c
index 116dedcf83..05ab616f39 100644
--- a/negotiator/default.c
+++ b/negotiator/default.c
@@ -175,6 +175,13 @@ static int ack(struct fetch_negotiator *n, struct commit *c)
return known_to_be_common;
}
+static void have_sent(struct fetch_negotiator *n, struct commit *c)
+{
+ if (repo_parse_commit(the_repository, c))
+ return;
+ mark_common(n->data, c, 0, 0);
+}
+
static void release(struct fetch_negotiator *n)
{
clear_prio_queue(&((struct negotiation_state *)n->data)->rev_list);
@@ -188,6 +195,7 @@ void default_negotiator_init(struct fetch_negotiator *negotiator)
negotiator->add_tip = add_tip;
negotiator->next = next;
negotiator->ack = ack;
+ negotiator->have_sent = have_sent;
negotiator->release = release;
negotiator->data = CALLOC_ARRAY(ns, 1);
ns->rev_list.compare = compare_commits_by_commit_date;
diff --git a/negotiator/noop.c b/negotiator/noop.c
index 65e3c20008..edf1b456f3 100644
--- a/negotiator/noop.c
+++ b/negotiator/noop.c
@@ -29,6 +29,12 @@ static int ack(struct fetch_negotiator *n UNUSED, struct commit *c UNUSED)
return 0;
}
+static void have_sent(struct fetch_negotiator *n UNUSED,
+ struct commit *c UNUSED)
+{
+ /* nothing to do */
+}
+
static void release(struct fetch_negotiator *n UNUSED)
{
/* nothing to release */
@@ -40,6 +46,7 @@ void noop_negotiator_init(struct fetch_negotiator *negotiator)
negotiator->add_tip = add_tip;
negotiator->next = next;
negotiator->ack = ack;
+ negotiator->have_sent = have_sent;
negotiator->release = release;
negotiator->data = NULL;
}
diff --git a/negotiator/skipping.c b/negotiator/skipping.c
index 0a272130fb..69472c58e1 100644
--- a/negotiator/skipping.c
+++ b/negotiator/skipping.c
@@ -243,6 +243,13 @@ static int ack(struct fetch_negotiator *n, struct commit *c)
return known_to_be_common;
}
+static void have_sent(struct fetch_negotiator *n, struct commit *c)
+{
+ if (repo_parse_commit(the_repository, c))
+ return;
+ mark_common(n->data, c);
+}
+
static void release(struct fetch_negotiator *n)
{
struct data *data = n->data;
@@ -259,6 +266,7 @@ void skipping_negotiator_init(struct fetch_negotiator *negotiator)
negotiator->add_tip = add_tip;
negotiator->next = next;
negotiator->ack = ack;
+ negotiator->have_sent = have_sent;
negotiator->release = release;
negotiator->data = CALLOC_ARRAY(data, 1);
data->rev_list.compare = compare;
--
gitgitgadget
^ permalink raw reply related
* [PATCH v6 4/8] remote: add remote.*.negotiationRestrict config
From: Derrick Stolee via GitGitGadget @ 2026-05-19 16:24 UTC (permalink / raw)
To: git; +Cc: gitster, ps, Matthew John Cheetham, Derrick Stolee,
Derrick Stolee
In-Reply-To: <pull.2085.v6.git.1779207896.gitgitgadget@gmail.com>
From: Derrick Stolee <stolee@gmail.com>
In a previous change, the --negotiation-restrict command-line option of 'git
fetch' was added as a synonym of --negotiation-tip. Both of these options
restrict the set of 'haves' the client can send as part of negotiation.
This was previously not available via a configuration option. Add a new
'remote.<name>.negotiationRestrict' multi-valued config option that updates
'git fetch <name>' to use these restrictions by default.
If the user provides even one --negotiation-restrict argument, then the
config is ignored.
An empty value resets the value list to allow ignoring earlier config
values, such as those that might be set in system or global config.
Reviewed-by: Matthew John Cheetham <mjcheetham@outlook.com>
Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
Documentation/config/remote.adoc | 18 ++++++++++++++++++
builtin/fetch.c | 28 +++++++++++++++++++++-------
remote.c | 5 +++++
remote.h | 1 +
t/t5510-fetch.sh | 26 ++++++++++++++++++++++++++
5 files changed, 71 insertions(+), 7 deletions(-)
diff --git a/Documentation/config/remote.adoc b/Documentation/config/remote.adoc
index 91e46f66f5..4dcf81fbce 100644
--- a/Documentation/config/remote.adoc
+++ b/Documentation/config/remote.adoc
@@ -107,6 +107,24 @@ priority configuration file (e.g. `.git/config` in a repository) to clear
the values inherited from a lower priority configuration files (e.g.
`$HOME/.gitconfig`).
+remote.<name>.negotiationRestrict::
+ When negotiating with this remote during `git fetch`, restrict the
+ commits advertised as "have" lines to only those reachable from refs
+ matching the given patterns. This multi-valued config option behaves
+ like `--negotiation-restrict` on the command line.
++
+Each value is either an exact ref name (e.g. `refs/heads/release`) or a
+glob pattern (e.g. `refs/heads/release/*`). The pattern syntax is the
+same as for `--negotiation-restrict`.
++
+These config values are used as defaults for the `--negotiation-restrict`
+command-line option. If `--negotiation-restrict` (or its synonym
+`--negotiation-tip`) is specified on the command line, then the config
+values are not used.
++
+Blank values signal to ignore all previous values, allowing a reset of
+the list from broader config scenarios.
+
remote.<name>.followRemoteHEAD::
How linkgit:git-fetch[1] should handle updates to `remotes/<name>/HEAD`
when fetching using the configured refspecs of a remote.
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 2ba0051d52..a957739f37 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1601,6 +1601,19 @@ static struct transport *prepare_transport(struct remote *remote, int deepen,
else
warning(_("ignoring %s because the protocol does not support it"),
"--negotiation-restrict");
+ } else if (remote->negotiation_restrict.nr) {
+ struct string_list_item *item;
+ for_each_string_list_item(item, &remote->negotiation_restrict)
+ string_list_append(&negotiation_restrict, item->string);
+ if (transport->smart_options)
+ add_negotiation_restrict_tips(transport->smart_options);
+ else {
+ struct strbuf config_name = STRBUF_INIT;
+ strbuf_addf(&config_name, "remote.%s.negotiationRestrict", remote->name);
+ warning(_("ignoring %s because the protocol does not support it"),
+ config_name.buf);
+ strbuf_release(&config_name);
+ }
}
return transport;
}
@@ -2658,10 +2671,6 @@ int cmd_fetch(int argc,
config.display_format = DISPLAY_FORMAT_PORCELAIN;
}
- if (negotiate_only && !negotiation_restrict.nr)
- die(_("%s needs one or more %s"), "--negotiate-only",
- "--negotiation-restrict=*");
-
if (deepen_relative) {
if (deepen_relative < 0)
die(_("negative depth in --deepen is not supported"));
@@ -2749,14 +2758,19 @@ int cmd_fetch(int argc,
if (!remote)
die(_("must supply remote when using --negotiate-only"));
gtransport = prepare_transport(remote, 1, &filter_options);
- if (gtransport->smart_options) {
- gtransport->smart_options->acked_commits = &acked_commits;
- } else {
+
+ if (!gtransport->smart_options) {
warning(_("protocol does not support --negotiate-only, exiting"));
result = 1;
trace2_region_leave("fetch", "negotiate-only", the_repository);
goto cleanup;
}
+ if (!gtransport->smart_options->negotiation_restrict_tips)
+ die(_("%s needs one or more %s"), "--negotiate-only",
+ "--negotiation-restrict=*");
+
+ gtransport->smart_options->acked_commits = &acked_commits;
+
if (server_options.nr)
gtransport->server_options = &server_options;
result = transport_fetch_refs(gtransport, NULL);
diff --git a/remote.c b/remote.c
index 7ca2a6501b..620086e16e 100644
--- a/remote.c
+++ b/remote.c
@@ -152,6 +152,7 @@ static struct remote *make_remote(struct remote_state *remote_state,
refspec_init_push(&ret->push);
refspec_init_fetch(&ret->fetch);
string_list_init_dup(&ret->server_options);
+ string_list_init_dup(&ret->negotiation_restrict);
ALLOC_GROW(remote_state->remotes, remote_state->remotes_nr + 1,
remote_state->remotes_alloc);
@@ -179,6 +180,7 @@ static void remote_clear(struct remote *remote)
FREE_AND_NULL(remote->http_proxy);
FREE_AND_NULL(remote->http_proxy_authmethod);
string_list_clear(&remote->server_options, 0);
+ string_list_clear(&remote->negotiation_restrict, 0);
}
static void add_merge(struct branch *branch, const char *name)
@@ -562,6 +564,9 @@ static int handle_config(const char *key, const char *value,
} else if (!strcmp(subkey, "serveroption")) {
return parse_transport_option(key, value,
&remote->server_options);
+ } else if (!strcmp(subkey, "negotiationrestrict")) {
+ return parse_transport_option(key, value,
+ &remote->negotiation_restrict);
} else if (!strcmp(subkey, "followremotehead")) {
const char *no_warn_branch;
if (!strcmp(value, "never"))
diff --git a/remote.h b/remote.h
index fc052945ee..e6ec37c393 100644
--- a/remote.h
+++ b/remote.h
@@ -117,6 +117,7 @@ struct remote {
char *http_proxy_authmethod;
struct string_list server_options;
+ struct string_list negotiation_restrict;
enum follow_remote_head_settings follow_remote_head;
const char *no_warn_branch;
diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index dc3ce56d84..eff3ce8e2d 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -1485,6 +1485,32 @@ test_expect_success '--negotiation-restrict and --negotiation-tip can be mixed'
check_negotiation_tip
'
+test_expect_success 'remote.<name>.negotiationRestrict used as default' '
+ setup_negotiation_tip server server 0 &&
+
+ # test the reset of the list on an empty value
+ git -C client config --add remote.origin.negotiationRestrict alpha_2 &&
+ git -C client config --add remote.origin.negotiationRestrict "" &&
+ git -C client config --add remote.origin.negotiationRestrict alpha_1 &&
+ git -C client config --add remote.origin.negotiationRestrict beta_1 &&
+ GIT_TRACE_PACKET="$(pwd)/trace" git -C client fetch \
+ origin alpha_s beta_s &&
+ check_negotiation_tip
+'
+
+test_expect_success 'CLI --negotiation-restrict overrides remote config' '
+ setup_negotiation_tip server server 0 &&
+ git -C client config --add remote.origin.negotiationRestrict alpha_1 &&
+ git -C client config --add remote.origin.negotiationRestrict beta_1 &&
+ ALPHA_1=$(git -C client rev-parse alpha_1) &&
+ GIT_TRACE_PACKET="$(pwd)/trace" git -C client fetch \
+ --negotiation-restrict=alpha_1 \
+ origin alpha_s beta_s &&
+ test_grep "fetch> have $ALPHA_1" trace &&
+ BETA_1=$(git -C client rev-parse beta_1) &&
+ test_grep ! "fetch> have $BETA_1" trace
+'
+
test_expect_success SYMLINKS 'clone does not get confused by a D/F conflict' '
git init df-conflict &&
(
--
gitgitgadget
^ permalink raw reply related
* [PATCH v6 3/8] transport: rename negotiation_tips
From: Derrick Stolee via GitGitGadget @ 2026-05-19 16:24 UTC (permalink / raw)
To: git; +Cc: gitster, ps, Matthew John Cheetham, Derrick Stolee,
Derrick Stolee
In-Reply-To: <pull.2085.v6.git.1779207896.gitgitgadget@gmail.com>
From: Derrick Stolee <stolee@gmail.com>
The previous change added the --negotiation-restrict synonym for the
--negotiation-tip option for 'git fetch'. In anticipation of adding a new
option that behaves similarly but with distinct changes to its behavior,
rename the internal representation of this data from 'negotiation_tips' to
'negotiation_restrict_tips'.
The 'tips' part is kept because this is an oid_array in the transport layer.
This requires the builtin to handle parsing refs into collections of oids so
the transport layer can handle this cleaner form of the data.
Also update the string_list used to store the inputs from command-line
options.
Reviewed-by: Matthew John Cheetham <mjcheetham@outlook.com>
Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
builtin/fetch.c | 18 +++++++++---------
fetch-pack.c | 18 +++++++++---------
fetch-pack.h | 4 ++--
transport-helper.c | 2 +-
transport.c | 10 +++++-----
transport.h | 4 ++--
6 files changed, 28 insertions(+), 28 deletions(-)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index fc950fe35b..2ba0051d52 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -98,7 +98,7 @@ static struct transport *gtransport;
static struct transport *gsecondary;
static struct refspec refmap = REFSPEC_INIT_FETCH;
static struct string_list server_options = STRING_LIST_INIT_DUP;
-static struct string_list negotiation_tip = STRING_LIST_INIT_NODUP;
+static struct string_list negotiation_restrict = STRING_LIST_INIT_NODUP;
struct fetch_config {
enum display_format display_format;
@@ -1534,13 +1534,13 @@ static int add_oid(const struct reference *ref, void *cb_data)
return 0;
}
-static void add_negotiation_tips(struct git_transport_options *smart_options)
+static void add_negotiation_restrict_tips(struct git_transport_options *smart_options)
{
struct oid_array *oids = xcalloc(1, sizeof(*oids));
int i;
- for (i = 0; i < negotiation_tip.nr; i++) {
- const char *s = negotiation_tip.items[i].string;
+ for (i = 0; i < negotiation_restrict.nr; i++) {
+ const char *s = negotiation_restrict.items[i].string;
struct refs_for_each_ref_options opts = {
.pattern = s,
};
@@ -1561,7 +1561,7 @@ static void add_negotiation_tips(struct git_transport_options *smart_options)
warning(_("ignoring %s=%s because it does not match any refs"),
"--negotiation-restrict", s);
}
- smart_options->negotiation_tips = oids;
+ smart_options->negotiation_restrict_tips = oids;
}
static struct transport *prepare_transport(struct remote *remote, int deepen,
@@ -1595,9 +1595,9 @@ static struct transport *prepare_transport(struct remote *remote, int deepen,
set_option(transport, TRANS_OPT_LIST_OBJECTS_FILTER, spec);
set_option(transport, TRANS_OPT_FROM_PROMISOR, "1");
}
- if (negotiation_tip.nr) {
+ if (negotiation_restrict.nr) {
if (transport->smart_options)
- add_negotiation_tips(transport->smart_options);
+ add_negotiation_restrict_tips(transport->smart_options);
else
warning(_("ignoring %s because the protocol does not support it"),
"--negotiation-restrict");
@@ -2566,7 +2566,7 @@ int cmd_fetch(int argc,
N_("specify fetch refmap"), PARSE_OPT_NONEG, parse_refmap_arg),
OPT_STRING_LIST('o', "server-option", &server_options, N_("server-specific"), N_("option to transmit")),
OPT_IPVERSION(&family),
- OPT_STRING_LIST(0, "negotiation-restrict", &negotiation_tip, N_("revision"),
+ OPT_STRING_LIST(0, "negotiation-restrict", &negotiation_restrict, N_("revision"),
N_("report that we have only objects reachable from this object")),
OPT_ALIAS(0, "negotiation-tip", "negotiation-restrict"),
OPT_BOOL(0, "negotiate-only", &negotiate_only,
@@ -2658,7 +2658,7 @@ int cmd_fetch(int argc,
config.display_format = DISPLAY_FORMAT_PORCELAIN;
}
- if (negotiate_only && !negotiation_tip.nr)
+ if (negotiate_only && !negotiation_restrict.nr)
die(_("%s needs one or more %s"), "--negotiate-only",
"--negotiation-restrict=*");
diff --git a/fetch-pack.c b/fetch-pack.c
index 6ecd468ef7..baf239adf9 100644
--- a/fetch-pack.c
+++ b/fetch-pack.c
@@ -291,21 +291,21 @@ static int next_flush(int stateless_rpc, int count)
}
static void mark_tips(struct fetch_negotiator *negotiator,
- const struct oid_array *negotiation_tips)
+ const struct oid_array *negotiation_restrict_tips)
{
struct refs_for_each_ref_options opts = {
.flags = REFS_FOR_EACH_INCLUDE_BROKEN,
};
int i;
- if (!negotiation_tips) {
+ if (!negotiation_restrict_tips) {
refs_for_each_ref_ext(get_main_ref_store(the_repository),
rev_list_insert_ref_oid, negotiator, &opts);
return;
}
- for (i = 0; i < negotiation_tips->nr; i++)
- rev_list_insert_ref(negotiator, &negotiation_tips->oid[i]);
+ for (i = 0; i < negotiation_restrict_tips->nr; i++)
+ rev_list_insert_ref(negotiator, &negotiation_restrict_tips->oid[i]);
return;
}
@@ -355,7 +355,7 @@ static int find_common(struct fetch_negotiator *negotiator,
PACKET_READ_CHOMP_NEWLINE |
PACKET_READ_DIE_ON_ERR_PACKET);
- mark_tips(negotiator, args->negotiation_tips);
+ mark_tips(negotiator, args->negotiation_restrict_tips);
for_each_cached_alternate(negotiator, insert_one_alternate_object);
fetching = 0;
@@ -1728,7 +1728,7 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args,
else
state = FETCH_SEND_REQUEST;
- mark_tips(negotiator, args->negotiation_tips);
+ mark_tips(negotiator, args->negotiation_restrict_tips);
for_each_cached_alternate(negotiator,
insert_one_alternate_object);
break;
@@ -2177,7 +2177,7 @@ static void clear_common_flag(struct oidset *s)
}
}
-void negotiate_using_fetch(const struct oid_array *negotiation_tips,
+void negotiate_using_fetch(const struct oid_array *negotiation_restrict_tips,
const struct string_list *server_options,
int stateless_rpc,
int fd[],
@@ -2195,13 +2195,13 @@ void negotiate_using_fetch(const struct oid_array *negotiation_tips,
timestamp_t min_generation = GENERATION_NUMBER_INFINITY;
fetch_negotiator_init(the_repository, &negotiator);
- mark_tips(&negotiator, negotiation_tips);
+ mark_tips(&negotiator, negotiation_restrict_tips);
packet_reader_init(&reader, fd[0], NULL, 0,
PACKET_READ_CHOMP_NEWLINE |
PACKET_READ_DIE_ON_ERR_PACKET);
- oid_array_for_each((struct oid_array *) negotiation_tips,
+ oid_array_for_each((struct oid_array *) negotiation_restrict_tips,
add_to_object_array,
&nt_object_array);
diff --git a/fetch-pack.h b/fetch-pack.h
index 9d3470366f..6c70c942c2 100644
--- a/fetch-pack.h
+++ b/fetch-pack.h
@@ -21,7 +21,7 @@ struct fetch_pack_args {
* If not NULL, during packfile negotiation, fetch-pack will send "have"
* lines only with these tips and their ancestors.
*/
- const struct oid_array *negotiation_tips;
+ const struct oid_array *negotiation_restrict_tips;
unsigned deepen_relative:1;
unsigned quiet:1;
@@ -89,7 +89,7 @@ struct ref *fetch_pack(struct fetch_pack_args *args,
* In the capability advertisement that has happened prior to invoking this
* function, the "wait-for-done" capability must be present.
*/
-void negotiate_using_fetch(const struct oid_array *negotiation_tips,
+void negotiate_using_fetch(const struct oid_array *negotiation_restrict_tips,
const struct string_list *server_options,
int stateless_rpc,
int fd[],
diff --git a/transport-helper.c b/transport-helper.c
index dd78d40668..f4388da766 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -754,7 +754,7 @@ static int fetch_refs(struct transport *transport,
set_helper_option(transport, "filter", spec);
}
- if (data->transport_options.negotiation_tips)
+ if (data->transport_options.negotiation_restrict_tips)
warning(_("ignoring %s because the protocol does not support it."),
"--negotiation-restrict");
diff --git a/transport.c b/transport.c
index 107f4fa5dc..a3051f6733 100644
--- a/transport.c
+++ b/transport.c
@@ -463,7 +463,7 @@ static int fetch_refs_via_pack(struct transport *transport,
args.refetch = data->options.refetch;
args.stateless_rpc = transport->stateless_rpc;
args.server_options = transport->server_options;
- args.negotiation_tips = data->options.negotiation_tips;
+ args.negotiation_restrict_tips = data->options.negotiation_restrict_tips;
args.reject_shallow_remote = transport->smart_options->reject_shallow;
if (!data->finished_handshake) {
@@ -491,7 +491,7 @@ static int fetch_refs_via_pack(struct transport *transport,
warning(_("server does not support wait-for-done"));
ret = -1;
} else {
- negotiate_using_fetch(data->options.negotiation_tips,
+ negotiate_using_fetch(data->options.negotiation_restrict_tips,
transport->server_options,
transport->stateless_rpc,
data->fd,
@@ -979,9 +979,9 @@ static int disconnect_git(struct transport *transport)
finish_connect(data->conn);
}
- if (data->options.negotiation_tips) {
- oid_array_clear(data->options.negotiation_tips);
- free(data->options.negotiation_tips);
+ if (data->options.negotiation_restrict_tips) {
+ oid_array_clear(data->options.negotiation_restrict_tips);
+ free(data->options.negotiation_restrict_tips);
}
list_objects_filter_release(&data->options.filter_options);
oid_array_clear(&data->extra_have);
diff --git a/transport.h b/transport.h
index 892f19454a..cdeb33c16f 100644
--- a/transport.h
+++ b/transport.h
@@ -40,13 +40,13 @@ struct git_transport_options {
/*
* This is only used during fetch. See the documentation of
- * negotiation_tips in struct fetch_pack_args.
+ * negotiation_restrict_tips in struct fetch_pack_args.
*
* This field is only supported by transports that support connect or
* stateless_connect. Set this field directly instead of using
* transport_set_option().
*/
- struct oid_array *negotiation_tips;
+ struct oid_array *negotiation_restrict_tips;
/*
* If allocated, whenever transport_fetch_refs() is called, add known
--
gitgitgadget
^ permalink raw reply related
* [PATCH v6 2/8] fetch: add --negotiation-restrict option
From: Derrick Stolee via GitGitGadget @ 2026-05-19 16:24 UTC (permalink / raw)
To: git; +Cc: gitster, ps, Matthew John Cheetham, Derrick Stolee,
Derrick Stolee
In-Reply-To: <pull.2085.v6.git.1779207896.gitgitgadget@gmail.com>
From: Derrick Stolee <stolee@gmail.com>
The --negotiation-tip option to 'git fetch' and 'git pull' allows users
to specify that they want to focus negotiation on a small set of
references. This is a _restriction_ on the negotiation set, helping to
focus the negotiation when the ref count is high. However, it doesn't
allow for the ability to opportunistically select references beyond that
list.
This subtle detail that this is a 'maximum set' and not a 'minimum set'
is not immediately clear from the option name. This makes it more
complicated to add a new option that provides the complementary behavior
of a minimum set.
For now, create a new synonym option, --negotiation-restrict, that
behaves identically to --negotiation-tip. Update the documentation to
make it clear that this new name is the preferred option, but we keep
the old name for compatibility. Mark --negotiation-tip as an alias of the
new, preferred option.
Update a few warning messages with the new option, but also make them
translatable with the option name inserted by formatting. At least one
of these messages will be reused later for a new option.
Reviewed-by: Matthew John Cheetham <mjcheetham@outlook.com>
Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
Documentation/config/fetch.adoc | 2 +-
Documentation/fetch-options.adoc | 6 +++++-
builtin/fetch.c | 13 ++++++++-----
builtin/pull.c | 3 ++-
send-pack.c | 2 +-
t/t5510-fetch.sh | 25 +++++++++++++++++++++++++
t/t5702-protocol-v2.sh | 4 ++--
transport-helper.c | 3 ++-
8 files changed, 46 insertions(+), 12 deletions(-)
diff --git a/Documentation/config/fetch.adoc b/Documentation/config/fetch.adoc
index cd40db0cad..04ac90912d 100644
--- a/Documentation/config/fetch.adoc
+++ b/Documentation/config/fetch.adoc
@@ -76,7 +76,7 @@
default is `skipping`. Unknown values will cause `git fetch` to
error out.
+
-See also the `--negotiate-only` and `--negotiation-tip` options to
+See also the `--negotiate-only` and `--negotiation-restrict` options to
linkgit:git-fetch[1].
`fetch.showForcedUpdates`::
diff --git a/Documentation/fetch-options.adoc b/Documentation/fetch-options.adoc
index 81a9d7f9bb..d39cecb446 100644
--- a/Documentation/fetch-options.adoc
+++ b/Documentation/fetch-options.adoc
@@ -49,6 +49,7 @@ the current repository has the same history as the source repository.
`.git/shallow`. This option updates `.git/shallow` and accepts such
refs.
+`--negotiation-restrict=(<commit>|<glob>)`::
`--negotiation-tip=(<commit>|<glob>)`::
By default, Git will report, to the server, commits reachable
from all local refs to find common commits in an attempt to
@@ -58,6 +59,9 @@ the current repository has the same history as the source repository.
local ref is likely to have commits in common with the
upstream ref being fetched.
+
+`--negotiation-restrict` is the preferred name for this option;
+`--negotiation-tip` is accepted as a synonym.
++
This option may be specified more than once; if so, Git will report
commits reachable from any of the given commits.
+
@@ -71,7 +75,7 @@ configuration variables documented in linkgit:git-config[1], and the
`--negotiate-only`::
Do not fetch anything from the server, and instead print the
- ancestors of the provided `--negotiation-tip=` arguments,
+ ancestors of the provided `--negotiation-restrict=` arguments,
which we have in common with the server.
+
This is incompatible with `--recurse-submodules=(yes|on-demand)`.
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 4795b2a13c..fc950fe35b 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -1558,8 +1558,8 @@ static void add_negotiation_tips(struct git_transport_options *smart_options)
refs_for_each_ref_ext(get_main_ref_store(the_repository),
add_oid, oids, &opts);
if (old_nr == oids->nr)
- warning("ignoring --negotiation-tip=%s because it does not match any refs",
- s);
+ warning(_("ignoring %s=%s because it does not match any refs"),
+ "--negotiation-restrict", s);
}
smart_options->negotiation_tips = oids;
}
@@ -1599,7 +1599,8 @@ static struct transport *prepare_transport(struct remote *remote, int deepen,
if (transport->smart_options)
add_negotiation_tips(transport->smart_options);
else
- warning("ignoring --negotiation-tip because the protocol does not support it");
+ warning(_("ignoring %s because the protocol does not support it"),
+ "--negotiation-restrict");
}
return transport;
}
@@ -2565,8 +2566,9 @@ int cmd_fetch(int argc,
N_("specify fetch refmap"), PARSE_OPT_NONEG, parse_refmap_arg),
OPT_STRING_LIST('o', "server-option", &server_options, N_("server-specific"), N_("option to transmit")),
OPT_IPVERSION(&family),
- OPT_STRING_LIST(0, "negotiation-tip", &negotiation_tip, N_("revision"),
+ OPT_STRING_LIST(0, "negotiation-restrict", &negotiation_tip, N_("revision"),
N_("report that we have only objects reachable from this object")),
+ OPT_ALIAS(0, "negotiation-tip", "negotiation-restrict"),
OPT_BOOL(0, "negotiate-only", &negotiate_only,
N_("do not fetch a packfile; instead, print ancestors of negotiation tips")),
OPT_PARSE_LIST_OBJECTS_FILTER(&filter_options),
@@ -2657,7 +2659,8 @@ int cmd_fetch(int argc,
}
if (negotiate_only && !negotiation_tip.nr)
- die(_("--negotiate-only needs one or more --negotiation-tip=*"));
+ die(_("%s needs one or more %s"), "--negotiate-only",
+ "--negotiation-restrict=*");
if (deepen_relative) {
if (deepen_relative < 0)
diff --git a/builtin/pull.c b/builtin/pull.c
index 7e67fdce97..cc6ce485fc 100644
--- a/builtin/pull.c
+++ b/builtin/pull.c
@@ -996,9 +996,10 @@ int cmd_pull(int argc,
OPT_PASSTHRU('6', "ipv6", &opt_ipv6, NULL,
N_("use IPv6 addresses only"),
PARSE_OPT_NOARG),
- OPT_PASSTHRU_ARGV(0, "negotiation-tip", &opt_fetch, N_("revision"),
+ OPT_PASSTHRU_ARGV(0, "negotiation-restrict", &opt_fetch, N_("revision"),
N_("report that we have only objects reachable from this object"),
0),
+ OPT_ALIAS(0, "negotiation-tip", "negotiation-restrict"),
OPT_BOOL(0, "show-forced-updates", &opt_show_forced_updates,
N_("check for forced-updates on all updated branches")),
OPT_PASSTHRU(0, "set-upstream", &set_upstream, NULL,
diff --git a/send-pack.c b/send-pack.c
index 67d6987b1c..3d5d36ba3b 100644
--- a/send-pack.c
+++ b/send-pack.c
@@ -447,7 +447,7 @@ static void get_commons_through_negotiation(struct repository *r,
strvec_pushl(&child.args, "fetch", "--negotiate-only", NULL);
for (ref = remote_refs; ref; ref = ref->next) {
if (!is_null_oid(&ref->new_oid)) {
- strvec_pushf(&child.args, "--negotiation-tip=%s",
+ strvec_pushf(&child.args, "--negotiation-restrict=%s",
oid_to_hex(&ref->new_oid));
nr_negotiation_tip++;
}
diff --git a/t/t5510-fetch.sh b/t/t5510-fetch.sh
index 5dcb4b51a4..dc3ce56d84 100755
--- a/t/t5510-fetch.sh
+++ b/t/t5510-fetch.sh
@@ -1460,6 +1460,31 @@ EOF
test_cmp fatal-expect fatal-actual
'
+test_expect_success '--negotiation-restrict limits "have" lines sent' '
+ setup_negotiation_tip server server 0 &&
+ GIT_TRACE_PACKET="$(pwd)/trace" git -C client fetch \
+ --negotiation-restrict=alpha_1 --negotiation-restrict=beta_1 \
+ origin alpha_s beta_s &&
+ check_negotiation_tip
+'
+
+test_expect_success '--negotiation-restrict understands globs' '
+ setup_negotiation_tip server server 0 &&
+ GIT_TRACE_PACKET="$(pwd)/trace" git -C client fetch \
+ --negotiation-restrict=*_1 \
+ origin alpha_s beta_s &&
+ check_negotiation_tip
+'
+
+test_expect_success '--negotiation-restrict and --negotiation-tip can be mixed' '
+ setup_negotiation_tip server server 0 &&
+ GIT_TRACE_PACKET="$(pwd)/trace" git -C client fetch \
+ --negotiation-restrict=alpha_1 \
+ --negotiation-tip=beta_1 \
+ origin alpha_s beta_s &&
+ check_negotiation_tip
+'
+
test_expect_success SYMLINKS 'clone does not get confused by a D/F conflict' '
git init df-conflict &&
(
diff --git a/t/t5702-protocol-v2.sh b/t/t5702-protocol-v2.sh
index f826ac46a5..9f6cf4142d 100755
--- a/t/t5702-protocol-v2.sh
+++ b/t/t5702-protocol-v2.sh
@@ -869,14 +869,14 @@ setup_negotiate_only () {
test_commit -C client three
}
-test_expect_success 'usage: --negotiate-only without --negotiation-tip' '
+test_expect_success 'usage: --negotiate-only without --negotiation-restrict' '
SERVER="server" &&
URI="file://$(pwd)/server" &&
setup_negotiate_only "$SERVER" "$URI" &&
cat >err.expect <<-\EOF &&
- fatal: --negotiate-only needs one or more --negotiation-tip=*
+ fatal: --negotiate-only needs one or more --negotiation-restrict=*
EOF
test_must_fail git -c protocol.version=2 -C client fetch \
diff --git a/transport-helper.c b/transport-helper.c
index 4d95d84f9e..dd78d40668 100644
--- a/transport-helper.c
+++ b/transport-helper.c
@@ -755,7 +755,8 @@ static int fetch_refs(struct transport *transport,
}
if (data->transport_options.negotiation_tips)
- warning("Ignoring --negotiation-tip because the protocol does not support it.");
+ warning(_("ignoring %s because the protocol does not support it."),
+ "--negotiation-restrict");
if (data->fetch)
return fetch_with_fetch(transport, nr_heads, to_fetch);
--
gitgitgadget
^ permalink raw reply related
* [PATCH v6 1/8] t5516: fix test order flakiness
From: Derrick Stolee via GitGitGadget @ 2026-05-19 16:24 UTC (permalink / raw)
To: git; +Cc: gitster, ps, Matthew John Cheetham, Derrick Stolee,
Derrick Stolee
In-Reply-To: <pull.2085.v6.git.1779207896.gitgitgadget@gmail.com>
From: Derrick Stolee <stolee@gmail.com>
The 'fetch follows tags by default' test sorts using 'sort -k 4', but
for-each-ref output only has 3 columns. This relies on sort treating records
with fewer fields as having an empty fourth field, which may produce
unstable results depending on locale. This appears to be an accident added
in 3f763ddf28 (fetch: set remote/HEAD if it does not exist, 2024-11-22).
Use 'sort -k 3' to match the actual number of columns in the output.
Reviewed-by: Matthew John Cheetham <mjcheetham@outlook.com>
Signed-off-by: Derrick Stolee <stolee@gmail.com>
---
t/t5516-fetch-push.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/t/t5516-fetch-push.sh b/t/t5516-fetch-push.sh
index 29e2f17608..ac8447f21e 100755
--- a/t/t5516-fetch-push.sh
+++ b/t/t5516-fetch-push.sh
@@ -1349,7 +1349,7 @@ test_expect_success 'fetch follows tags by default' '
git for-each-ref >tmp1 &&
sed -n "p; s|refs/heads/main$|refs/remotes/origin/main|p" tmp1 |
sed -n "p; s|refs/heads/main$|refs/remotes/origin/HEAD|p" |
- sort -k 4 >../expect
+ sort -k 3 >../expect
) &&
test_when_finished "rm -rf dst" &&
git init dst &&
--
gitgitgadget
^ permalink raw reply related
* [PATCH v6 0/8] fetch: rework negotiation tip options
From: Derrick Stolee via GitGitGadget @ 2026-05-19 16:24 UTC (permalink / raw)
To: git; +Cc: gitster, ps, Matthew John Cheetham, Derrick Stolee
In-Reply-To: <pull.2085.v5.git.1779135575.gitgitgadget@gmail.com>
Fetch negotiation aims to find enough information from haves and wants such
that the server can be reasonably confident that it will send all necessary
objects and not too many "extra" objects that the client already has.
However, this can break down if there are too many references, since Git
truncates the list of haves based on a few factors (a 256 count limit or the
server sending an ACK at the right time).
We already have the --negotiation-tip feature to focus the set of references
that are used in negotiation, but I feel like this is designed backwards.
I'd rather that we have a way to say "this is an important set of refs, but
feel free to add more refs if needed" than "only use these refs for
negotiation".
Here's an example that demonstrates the problem. In an internal monorepo,
developers work off of the 'main' branch so there are thousands of user
branches that each add a few commits different from the 'main' branch.
However, there is also a long-lived 'release' branch. This branch has a
first-parent history that is parallel to 'main' and each of those commits is
a merge whose second parent is a commit from 'main' that had a successful CI
run. There are additional changes in the 'release' branch merge commits that
add some changelog data, so there is a nontrivial set of novel blob content
in that branch and not just a different set of commits.
The problem we had was that our georeplication system was regularly fetching
from the origin and trying to get all data from all reachable branches. When
the 'release' branch updated, the client would run out of haves before
advertising its copy of the 'release' branch, but it would still list the
new 'release' tip as a want. The server would then think that the client had
never fetched that branch before and would send all of the changelog data
from the whole history of the repo. (This led to a lot of downstream
problems; we mitigated by setting a refspec that stopped fetching the
'release' branch, but this is not ideal.)
What I'd like is a mechanism to say "always advertise the client's version
of 'main' and 'release' but also opportunistically include some user
branches".
Based on my understanding, the '--negotiation-tip' option is close but not
quite what I want. I could have the client only advertise 'release' and
'main' and never advertise any user branches. But then we'd download all
content from each user branch every time it updates. Perhaps this would
happen even with opportunistic inclusion of more haves, but I'd like to
explore this area more.
There's also an issue that the '--negotiation-tip' feature doesn't seem to
have a config key that enables it without CLI arguments. This is something
that we could consider independently.
This patch series adds a new '--negotiation-include' option that does what I
want: it makes sure that these references are included as 'have's during
negotiation. In order to help clarify the difference between this and
'--negotiation-tip', I first create a synonym called
'--negotiation-restrict'.
Both of these options get 'remote.*.negotiation(Include|Restrict)' config
options that enable their behavior by default.
During development, I had briefly considered only using config values, but
that required some strange changes to care about the remote name in the
transport layer. This was most different in the 'git push' integration. When
I discovered the '--negotiation-tip' feature during the process, that gave
me a clear pattern to follow with the addition of a config on top.
Updates in v2
=============
This version is a near-complete rewrite based on feedback around the names
of the previous option and config. The --negotiation-restrict option is new
and the ability to set it via config is also new.
I did try to be more careful around translatable error messages, too.
Updates in v3
=============
* --negotiation-tip is now an alias of --negotiation-restrict.
* More translatable strings use %s to isolate non-translatable options from
translatable words.
* The string_list named negotiation_tip is now renamed to
negotiation_restrict.
* The config options now allow an empty value to reset the list.
* The --negotiation-require option is now called --negotiation-include.
* Similarly, the config option is renamed and all code references.
* The included haves now mark their commits as COMMON so commits that they
can reach are not included in the negotiation walk if they are reached
from the restricted commits.
* The ref iterators are more careful about failing on bad references (ref
exists but object doesn't) and ignoring missing references (perhaps
config is erroneous?).
* When sending tips during push negotiation, use the --negotiation-restrict
option instead of -tip.
Updates in v4
=============
Thanks, Matthew, for the detailed review! There are some big changes in this
version.
* Expanded commit message to cite the commit that introduced the bug
(3f763ddf28).
* Renamed --negotiation-tip to --negotiation-restrict throughout docs/code
(including send-pack.c, transport-helper.c, builtin/pull.c). Added
OPT_ALIAS in git-pull.
* Switched config parsing to use parse_transport_option() helper. Removed
git push from docs (not implemented yet). Restructured --negotiate-only
validation flow.
* NEW Patch 5: Added have_sent() interface to negotiators, so included
haves can be de-duplicated properly by the negotiation algorithm.
* Replaced COMMON flag hack with negotiator->have_sent() calls. Moved
ref-pattern resolution into builtin/fetch.c (add_negotiation_tips()) so
fetch-pack receives pre-resolved oid_array instead of string_list. Added
test for --negotiation-tip ignoring missing refs. Added
duplicate-avoidance test for v0. Accepts commit hashes in addition to ref
names/globs.
* Use parse_transport_option() for config. Updated docs to mention commit
hashes. Removed git push from config docs. Fixed test to use correct
restrict/include combinations.
* In the last patch, add doc notes that remote config values also apply
during git push with push.negotiate, now that they are integrated by that
change.
Updates in v5
=============
Responded to small comments.
Updates in v6
=============
Corrected reviewed-by annotations in commit messages.
Thanks, -Stolee
Derrick Stolee (8):
t5516: fix test order flakiness
fetch: add --negotiation-restrict option
transport: rename negotiation_tips
remote: add remote.*.negotiationRestrict config
negotiator: add have_sent() interface
fetch: add --negotiation-include option for negotiation
remote: add remote.*.negotiationInclude config
send-pack: pass negotiation config in push
Documentation/config/fetch.adoc | 2 +-
Documentation/config/remote.adoc | 49 ++++++++
Documentation/fetch-options.adoc | 29 ++++-
builtin/fetch.c | 87 +++++++++++---
builtin/pull.c | 6 +-
fetch-negotiator.h | 9 ++
fetch-pack.c | 99 +++++++++++++---
fetch-pack.h | 10 +-
negotiator/default.c | 8 ++
negotiator/noop.c | 7 ++
negotiator/skipping.c | 8 ++
remote.c | 10 ++
remote.h | 2 +
send-pack.c | 39 +++++--
send-pack.h | 2 +
t/t5510-fetch.sh | 191 +++++++++++++++++++++++++++++++
t/t5516-fetch-push.sh | 32 +++++-
t/t5702-protocol-v2.sh | 4 +-
transport-helper.c | 5 +-
transport.c | 20 +++-
transport.h | 7 +-
21 files changed, 564 insertions(+), 62 deletions(-)
base-commit: 6e8d538aab8fe4dd07ba9fb87b5c7edcfa5706ad
Published-As: https://github.com/gitgitgadget/git/releases/tag/pr-2085%2Fderrickstolee%2Fmust-have-v6
Fetch-It-Via: git fetch https://github.com/gitgitgadget/git pr-2085/derrickstolee/must-have-v6
Pull-Request: https://github.com/gitgitgadget/git/pull/2085
Range-diff vs v5:
1: 538913a327 ! 1: c8c422f646 t5516: fix test order flakiness
@@ Commit message
Use 'sort -k 3' to match the actual number of columns in the output.
- Reviewed-by: Matthew John Cheetham <mcheetham@outlook.com>
+ Reviewed-by: Matthew John Cheetham <mjcheetham@outlook.com>
Signed-off-by: Derrick Stolee <stolee@gmail.com>
## t/t5516-fetch-push.sh ##
2: 580aa58943 ! 2: ac3e8f74d9 fetch: add --negotiation-restrict option
@@ Commit message
translatable with the option name inserted by formatting. At least one
of these messages will be reused later for a new option.
- Reviewed-by: Matthew John Cheetham <mcheetham@outlook.com>
+ Reviewed-by: Matthew John Cheetham <mjcheetham@outlook.com>
Signed-off-by: Derrick Stolee <stolee@gmail.com>
## Documentation/config/fetch.adoc ##
3: eee0543647 ! 3: 5206640b8b transport: rename negotiation_tips
@@ Commit message
Also update the string_list used to store the inputs from command-line
options.
- Reviewed-by: Matthew John Cheetham <mcheetham@outlook.com>
+ Reviewed-by: Matthew John Cheetham <mjcheetham@outlook.com>
Signed-off-by: Derrick Stolee <stolee@gmail.com>
## builtin/fetch.c ##
4: 63c675e93e ! 4: eec0f90e02 remote: add remote.*.negotiationRestrict config
@@ Commit message
An empty value resets the value list to allow ignoring earlier config
values, such as those that might be set in system or global config.
- Reviewed-by: Matthew John Cheetham <mcheetham@outlook.com>
+ Reviewed-by: Matthew John Cheetham <mjcheetham@outlook.com>
Signed-off-by: Derrick Stolee <stolee@gmail.com>
## Documentation/config/remote.adoc ##
5: d423c56283 ! 5: 840db1d957 negotiator: add have_sent() interface
@@ Commit message
common, so the implementation is quite simple. This logic will be exercised
in the next change.
- Reviewed-by: Matthew John Cheetham <mcheetham@outlook.com>
+ Reviewed-by: Matthew John Cheetham <mjcheetham@outlook.com>
Signed-off-by: Derrick Stolee <stolee@gmail.com>
## fetch-negotiator.h ##
6: e86c9791e2 ! 6: 62e5ef1a4b fetch: add --negotiation-include option for negotiation
@@ Commit message
Also add --negotiation-include to 'git pull' passthrough options.
- Reviewed-by: Matthew John Cheetham <mcheetham@outlook.com>
+ Reviewed-by: Matthew John Cheetham <mjcheetham@outlook.com>
Signed-off-by: Derrick Stolee <stolee@gmail.com>
## Documentation/fetch-options.adoc ##
7: e5714115b5 ! 7: 05a4b69b9b remote: add remote.*.negotiationInclude config
@@ Commit message
list to allow ignoring earlier config values, such as those that might be
set in system or global config.
- Reviewed-by: Matthew John Cheetham <mcheetham@outlook.com>
+ Reviewed-by: Matthew John Cheetham <mjcheetham@outlook.com>
Signed-off-by: Derrick Stolee <stolee@gmail.com>
## Documentation/config/remote.adoc ##
8: ed0be32e2c ! 8: c69ca2e919 send-pack: pass negotiation config in push
@@ Commit message
are passed as --negotiation-include to ensure their tips are always
sent as 'have' lines during push negotiation.
- Reviewed-by: Matthew John Cheetham <mcheetham@outlook.com>
+ Reviewed-by: Matthew John Cheetham <mjcheetham@outlook.com>
Signed-off-by: Derrick Stolee <stolee@gmail.com>
## Documentation/config/remote.adoc ##
--
gitgitgadget
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox