[RFC PATCH v3 0/2] push: add support for pushing to remote groups

[Usman Akinyemi <usmanakinyemi202@gmail.com>]

This RFC series adds support for `git push` to accept a remote group
name (as configured via `remotes.<name>` in config) in addition to a
single remote name, mirroring the behaviour that `git fetch` has
supported for some time.

A user with multiple remotes configured as a group can now do:

    git push all-remotes

instead of pushing to each remote individually, in the same way that:

    git fetch all-remotes

already works.

The series is split into two patches:

  - Patch 1 moves `get_remote_group`, `add_remote_or_group`, and the
    `remote_group_data` struct out of builtin/fetch.c and into
    remote.c/remote.h, making them part of the public remote API.

  - Patch 2 extends builtin/push.c to use the newly public
    `add_remote_or_group()` to resolve the repository argument as
    either a single remote or a group, and pushes to each member of
    the group in turn.

Changes in v3:

- Clarify documentation for remote group pushes:
  * describe behavior on partial failures (rejection vs fatal errors)
  * emphasize that group push is equivalent to running independent
    pushes to each member remote

- Simplify tests:
  * remove redundant comparisons between remotes
  * verify pushed commits against the expected value using `rev-parse`

- Add tests for failure scenarios:
  * ensure push continues on non-fast-forward rejection
  * ensure push stops on fatal transport errors

- Use `size_t` for loop index to match type of `remote_group.nr`
  and avoid compiler warnings

Range-diff v2 -> v3:

1:  dd370a19e7 = 1:  dd370a19e7 remote: move remote group resolution to remote.c
2:  ba5801cee1 ! 2:  6a7957e61c push: support pushing to a remote group
    @@ Commit message
     
             git push <group>
     
    -    When the argument resolves to a single remote the behaviour is
    +    When the argument resolves to a single remote, the behaviour is
         identical to before. When it resolves to a group, each member remote
         is pushed in sequence.
     
    @@ Commit message
         a copy of the flags, so that a mirror remote in the group cannot set
         TRANSPORT_PUSH_FORCE on subsequent non-mirror remotes in the same group.
     
    -    A known interaction: push.default = simple will die when the current
    -    branch has no upstream configured, because setup_default_push_refspecs()
    -    requires an upstream for that mode. Users pushing to a group should set
    -    push.default = current or supply explicit refspecs. This is consistent
    -    with how fetch handles default refspec resolution per remote.
    -
         Suggested-by: Junio C Hamano <gitster@pobox.com>
         Signed-off-by: Usman Akinyemi <usmanakinyemi202@gmail.com>
     
    @@ Documentation/git-push.adoc: further recursion will occur. In this case, `only`
     +behaviour is added or removed — the group is purely a shorthand for
     +running the same push command against each member remote individually.
     +
    ++The behaviour upon failure depends on the kind of error encountered:
    ++
    ++If a member remote rejects the push, for example due to a
    ++non-fast-forward update, force needed but not given, an existing tag,
    ++or a server-side hook refusing a ref, Git reports the error and continues
    ++pushing to the remaining remotes in the group. The overall exit code is
    ++non-zero if any member push fails.
    ++
    ++If a member remote cannot be contacted at all, for example because the
    ++repository does not exist, authentication fails, or the network is
    ++unreachable, the push stops at that point and the remaining remotes
    ++are not attempted.
    ++
     +This means the user is responsible for ensuring that the sequence of
    -+individual pushes makes sense.  For example, if `push.default = simple`
    -+is set and the current branch has no upstream configured, then
    -+`git push r1` may fail.  `git push all-remotes` will fail in the same
    -+way, on whichever member remote triggers the condition first.  Setting
    -+`push.default = current` or supplying explicit refspecs is recommended
    -+when pushing to a remote group.
    -+
    -+Similarly, if `--force-with-lease` is given without an explicit expected
    -+commit, Git will guess the expected commit for each remote independently
    -+from that remote's own remote-tracking branch, the same way it would if
    -+each push were run separately.  If an explicit commit is given with
    -+`--force-with-lease=<refname>:<expect>`, that same value is forwarded
    -+to every member remote, as if each of
    -+`git push --force-with-lease=<refname>:<expect> r1`,
    -+`git push --force-with-lease=<refname>:<expect> r2`, ...,
    -+`git push --force-with-lease=<refname>:<expect> rN` had been invoked.
    -+
    -+Each member remote is pushed using its own push mapping configuration
    -+(`remote.<name>.push`), so a refspec that maps differently on r1 than
    -+on r2 is resolved correctly for each one.
    ++individual pushes makes sense. If `git push r1`` would fail for a given
    ++set of options and arguments, then `git push all-remotes` will fail in
    ++the same way when it reaches r1. The group push does not do anything
    ++special to make a failing individual push succeed.
     +
      OUTPUT
      ------
      
     
      ## builtin/push.c ##
    -@@ builtin/push.c: static int git_push_config(const char *k, const char *v,
    - 
    - 	return git_default_config(k, v, ctx, NULL);
    - }
    --
    - int cmd_push(int argc,
    - 	     const char **argv,
    - 	     const char *prefix,
     @@ builtin/push.c: int cmd_push(int argc,
      	int flags = 0;
      	int tags = 0;
    @@ builtin/push.c: int cmd_push(int argc,
     +	 * same group.
     +	 */
     +	base_flags = flags;
    -+	for (int i = 0; i < remote_group.nr; i++) {
    ++	for (size_t i = 0; i < remote_group.nr; i++) {
     +		int iter_flags = base_flags;
     +		struct remote *r = pushremote_get(remote_group.items[i].string);
     +		if (!r)
    @@ t/t5566-push-group.sh (new)
     +test_expect_success 'setup' '
     +	for i in 1 2 3
     +	do
    -+		git init --bare dest-$i.git &&
    -+		git -C dest-$i.git symbolic-ref HEAD refs/heads/not-a-branch ||
    -+		return 1
    ++		git init --bare dest-$i.git || return 1
     +	done &&
     +	test_tick &&
     +	git commit --allow-empty -m "initial" &&
    @@ t/t5566-push-group.sh (new)
     +	git config set remotes.all-remotes "remote-1 remote-2 remote-3"
     +'
     +
    -+test_expect_success 'push to remote group pushes to all members' '
    ++test_expect_success 'push to remote group updates all members correctly' '
     +	git push all-remotes HEAD:refs/heads/main &&
    -+	j= &&
    ++	git rev-parse HEAD >expect &&
     +	for i in 1 2 3
     +	do
    -+		git -C dest-$i.git for-each-ref >actual-$i &&
    -+		if test -n "$j"
    -+		then
    -+			test_cmp actual-$j actual-$i
    -+		else
    -+			cat actual-$i
    -+		fi &&
    -+		j=$i ||
    ++		git -C dest-$i.git rev-parse refs/heads/main >actual ||
     +		return 1
    ++		test_cmp expect actual || return 1
     +	done
     +'
     +
    @@ t/t5566-push-group.sh (new)
     +	test_tick &&
     +	git commit --allow-empty -m "second" &&
     +	git push all-remotes HEAD:refs/heads/main &&
    ++	git rev-parse HEAD >expect &&
     +	for i in 1 2 3
     +	do
    -+		git -C dest-$i.git rev-parse refs/heads/main >hash-$i ||
    ++		git -C dest-$i.git rev-parse refs/heads/main >actual ||
     +		return 1
    -+	done &&
    -+	test_cmp hash-1 hash-2 &&
    -+	test_cmp hash-2 hash-3
    ++		test_cmp expect actual || return 1
    ++	done
     +'
     +
     +test_expect_success 'push to single remote in group does not affect others' '
    @@ t/t5566-push-group.sh (new)
     +	! test_cmp hash-after-1 hash-after-2
     +'
     +
    -+test_expect_success 'push to nonexistent group fails with error' '
    -+	test_must_fail git push no-such-group HEAD:refs/heads/main
    -+'
    -+
    -+test_expect_success 'push explicit refspec to group' '
    -+	test_tick &&
    -+	git commit --allow-empty -m "fourth" &&
    -+	git push all-remotes HEAD:refs/heads/other &&
    -+	for i in 1 2 3
    -+	do
    -+		git -C dest-$i.git rev-parse refs/heads/other >other-hash-$i ||
    -+		return 1
    -+	done &&
    -+	test_cmp other-hash-1 other-hash-2 &&
    -+	test_cmp other-hash-2 other-hash-3
    -+'
    -+
     +test_expect_success 'mirror remote in group with refspec fails' '
     +	git config set remote.remote-1.mirror true &&
     +	test_must_fail git push all-remotes HEAD:refs/heads/main 2>err &&
    -+	grep "mirror" err &&
    ++	test_grep "mirror" err &&
     +	git config unset remote.remote-1.mirror
     +'
     +test_expect_success 'push.default=current works with group push' '
    @@ t/t5566-push-group.sh (new)
     +	git config unset push.default
     +'
     +
    ++test_expect_success 'push continues past rejection to remaining remotes' '
    ++	for i in c1 c2 c3
    ++	do
    ++		git init --bare dest-$i.git || return 1
    ++	done &&
    ++	git config set remote.c1.url "file://$(pwd)/dest-c1.git" &&
    ++	git config set remote.c2.url "file://$(pwd)/dest-c2.git" &&
    ++	git config set remote.c3.url "file://$(pwd)/dest-c3.git" &&
    ++	git config set remotes.continue-group "c1 c2 c3" &&
    ++
    ++	test_tick &&
    ++	git commit --allow-empty -m "base for continue test" &&
    ++
    ++	# initial sync
    ++	git push continue-group HEAD:refs/heads/main &&
    ++
    ++  # advance c2 independently
    ++  git clone dest-c2.git tmp-c2 &&
    ++  (
    ++    cd tmp-c2 &&
    ++    git checkout -b main origin/main &&
    ++    test_commit c2_independent &&
    ++    git push origin HEAD:refs/heads/main
    ++  ) &&
    ++  rm -rf tmp-c2 &&
    ++
    ++	test_tick &&
    ++	git commit --allow-empty -m "local diverging commit" &&
    ++
    ++	# push: c2 rejects, others succeed
    ++	test_must_fail git push continue-group HEAD:refs/heads/main &&
    ++
    ++	git rev-parse HEAD >expect &&
    ++	git -C dest-c1.git rev-parse refs/heads/main >actual-c1 &&
    ++	git -C dest-c3.git rev-parse refs/heads/main >actual-c3 &&
    ++	test_cmp expect actual-c1 &&
    ++	test_cmp expect actual-c3 &&
    ++
    ++	# c2 should not have the new commit
    ++	git -C dest-c2.git rev-parse refs/heads/main >actual-c2 &&
    ++	! test_cmp expect actual-c2
    ++'
    ++
    ++test_expect_success 'fatal connection error stops remaining remotes' '
    ++	for i in f1 f2 f3
    ++	do
    ++		git init --bare dest-$i.git || return 1
    ++	done &&
    ++	git config set remote.f1.url "file://$(pwd)/dest-f1.git" &&
    ++	git config set remote.f2.url "file://$(pwd)/dest-f2.git" &&
    ++	git config set remote.f3.url "file://$(pwd)/dest-f3.git" &&
    ++	git config set remotes.fatal-group "f1 f2 f3" &&
    ++
    ++	test_tick &&
    ++	git commit --allow-empty -m "base for fatal test" &&
    ++
    ++	# initial sync
    ++	git push fatal-group HEAD:refs/heads/main &&
    ++
    ++	# break f2
    ++	git config set remote.f2.url "file:///tmp/does-not-exist-$$" &&
    ++
    ++	test_tick &&
    ++	git commit --allow-empty -m "after fatal setup" &&
    ++
    ++	test_must_fail git push fatal-group HEAD:refs/heads/main &&
    ++
    ++	git rev-parse HEAD >expect &&
    ++	git -C dest-f1.git rev-parse refs/heads/main >actual-f1 &&
    ++	test_cmp expect actual-f1 &&
    ++
    ++	# f3 should not be updated
    ++	git -C dest-f3.git rev-parse refs/heads/main >actual-f3 &&
    ++	! test_cmp expect actual-f3 &&
    ++
    ++	git config set remote.f2.url "file://$(pwd)/dest-f2.git"
    ++'
    ++
     +test_done

Usman Akinyemi (2):
  remote: move remote group resolution to remote.c
  push: support pushing to a remote group

 Documentation/git-push.adoc |  73 ++++++++++++++++--
 builtin/fetch.c             |  42 ----------
 builtin/push.c              | 123 +++++++++++++++++++++--------
 remote.c                    |  37 +++++++++
 remote.h                    |  12 +++
 t/meson.build               |   1 +
 t/t5566-push-group.sh       | 150 ++++++++++++++++++++++++++++++++++++
 7 files changed, 355 insertions(+), 83 deletions(-)
 create mode 100755 t/t5566-push-group.sh

-- 
2.48.0.rc0.4242.g73eb647d24.dirty