Re: [PATCH v2 7/7] update-ref: add --allow-partial flag for stdin mode

[Phillip Wood <phillip.wood123@gmail.com>]

Hi Karthik

On 25/02/2025 09:29, Karthik Nayak wrote:
> When updating multiple references through stdin, Git's update-ref
> command normally aborts the entire transaction if any single update
> fails. While this atomic behavior prevents partial updates by default,
> there are cases where applying successful updates while reporting
> failures is desirable.
> 
> Add a new `--allow-partial` flag that allows the transaction to continue
> even when individual reference updates fail. This flag can only be used
> in `--stdin` mode and builds upon the partial transaction support added
> to the refs subsystem.

As '--stdin' allows a single instance of "git update-ref" to create more 
than one transaction perhaps we should instead allow the caller to 
specify which transactions they want to allow to fail by passing an 
argument to "start", similar to how we support "no-deref" with "update"

> following format:
> 
>    rejected SP (<old-oid> | <old-target>) SP (<new-oid> | <new-target>) SP <rejection-reason> LF
> 
> or with `-z`:
> 
>    rejected NUL (<old-oid> | <old-target>) NUL (<new-oid> | <new-target>) NUL <rejection-reason> NUL

What's the reason for the different output with '-z'? In the list of 
options '-z' is documented as only applying to the input stream. Looking 
at the code the existing messages generated by report_ok() are all 
printed to stdout with a LF terminator.

> +static void print_rejected_refs(const char *refname,
> +				const struct object_id *old_oid,
> +				const struct object_id *new_oid,
> +				const char *old_target,
> +				const char *new_target,
> +				enum transaction_error err,
> +				void *cb_data UNUSED)
> +{
> +	struct strbuf sb = STRBUF_INIT;
> +	char space = ' ';
> +	const char *reason = "";
> +
> +	switch (err) {
> +	case TRANSACTION_NAME_CONFLICT:
> +		reason = _("refname conflict");
> +		break;
> +	case TRANSACTION_CREATE_EXISTS:
> +		reason = _("reference already exists");
> +		break;
> +	case TRANSACTION_NONEXISTENT_REF:
> +		reason = _("reference does not exist");
> +		break;
> +	case TRANSACTION_INCORRECT_OLD_VALUE:
> +		reason = _("incorrect old value provided");
> +		break;
> +	case TRANSACTION_INVALID_NEW_VALUE:
> +		reason = _("invalid new value provided");
> +		break;
> +	case TRANSACTION_EXPECTED_SYMREF:
> +		reason = _("expected symref but found regular ref");
> +		break;
> +	default:
> +		reason = _("unkown failure");
> +	}

I agree with Patrick that these messages should not be translated.

> +	if (!line_termination)
> +		space = line_termination;
> +
> +	strbuf_addf(&sb, "rejected%c%s%c%s%c%c%s%c%s%c", space,
> +		    refname, space, new_oid ? oid_to_hex(new_oid) : new_target,
> +		    space, space, old_oid ? oid_to_hex(old_oid) : old_target,
> +		    space, reason, line_termination);
> +
> +	fwrite(sb.buf, sb.len, 1, stdout);
> +	strbuf_release(&sb);
> +	fflush(stdout);

There is no need to flush after each line, we'll flush all the error 
messages when we call report_ok() in parse_cmd_commit() or when the 
program exits. The caller has no way to know how many error messages 
there are to read so flushing each one individually does not help the 
reader avoid deadlocks.

> +}
> +
>   static void parse_cmd_commit(struct ref_transaction *transaction,
>   			     const char *next, const char *end UNUSED)
>   {
> @@ -573,6 +622,10 @@ static void parse_cmd_commit(struct ref_transaction *transaction,
>   		die("commit: extra input: %s", next);
>   	if (ref_transaction_commit(transaction, &error))
>   		die("commit: %s", error.buf);
> +
> +	ref_transaction_for_each_rejected_update(transaction,
> +						 print_rejected_refs, NULL);
> +
>   	report_ok("commit");

This is good, the caller knows to stop reading when they see "commit: ok"


Best Wishes

Phillip

>   	ref_transaction_free(transaction);
>   }
> @@ -609,7 +662,7 @@ static const struct parse_cmd {
>   	{ "commit",        parse_cmd_commit,        0, UPDATE_REFS_CLOSED },
>   };
>   
> -static void update_refs_stdin(void)
> +static void update_refs_stdin(unsigned int flags)
>   {
>   	struct strbuf input = STRBUF_INIT, err = STRBUF_INIT;
>   	enum update_refs_state state = UPDATE_REFS_OPEN;
> @@ -617,7 +670,7 @@ static void update_refs_stdin(void)
>   	int i, j;
>   
>   	transaction = ref_store_transaction_begin(get_main_ref_store(the_repository),
> -						  0, &err);
> +						  flags, &err);
>   	if (!transaction)
>   		die("%s", err.buf);
>   
> @@ -685,7 +738,7 @@ static void update_refs_stdin(void)
>   			 */
>   			state = cmd->state;
>   			transaction = ref_store_transaction_begin(get_main_ref_store(the_repository),
> -								  0, &err);
> +								  flags, &err);
>   			if (!transaction)
>   				die("%s", err.buf);
>   
> @@ -701,6 +754,8 @@ static void update_refs_stdin(void)
>   		/* Commit by default if no transaction was requested. */
>   		if (ref_transaction_commit(transaction, &err))
>   			die("%s", err.buf);
> +		ref_transaction_for_each_rejected_update(transaction,
> +						 print_rejected_refs, NULL);
>   		ref_transaction_free(transaction);
>   		break;
>   	case UPDATE_REFS_STARTED:
> @@ -726,7 +781,9 @@ int cmd_update_ref(int argc,
>   	const char *refname, *oldval;
>   	struct object_id oid, oldoid;
>   	int delete = 0, no_deref = 0, read_stdin = 0, end_null = 0;
> -	int create_reflog = 0;
> +	int create_reflog = 0, allow_partial = 0;
> +	unsigned int flags = 0;
> +
>   	struct option options[] = {
>   		OPT_STRING( 'm', NULL, &msg, N_("reason"), N_("reason of the update")),
>   		OPT_BOOL('d', NULL, &delete, N_("delete the reference")),
> @@ -735,6 +792,8 @@ int cmd_update_ref(int argc,
>   		OPT_BOOL('z', NULL, &end_null, N_("stdin has NUL-terminated arguments")),
>   		OPT_BOOL( 0 , "stdin", &read_stdin, N_("read updates from stdin")),
>   		OPT_BOOL( 0 , "create-reflog", &create_reflog, N_("create a reflog")),
> +		OPT_BIT('0', "allow-partial", &flags, N_("allow partial transactions"),
> +			REF_TRANSACTION_ALLOW_PARTIAL),
>   		OPT_END(),
>   	};
>   
> @@ -756,9 +815,10 @@ int cmd_update_ref(int argc,
>   			usage_with_options(git_update_ref_usage, options);
>   		if (end_null)
>   			line_termination = '\0';
> -		update_refs_stdin();
> +		update_refs_stdin(flags);
>   		return 0;
> -	}
> +	} else if (allow_partial)
> +		die("--allow-partial can only be used with --stdin");
>   
>   	if (end_null)
>   		usage_with_options(git_update_ref_usage, options);
> diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
> index 29045aad43..fb9442982e 100755
> --- a/t/t1400-update-ref.sh
> +++ b/t/t1400-update-ref.sh
> @@ -2066,6 +2066,222 @@ do
>   		grep "$(git rev-parse $a) $(git rev-parse $a)" actual
>   	'
>   
> +	test_expect_success "stdin $type allow-partial" '
> +		git init repo &&
> +		test_when_finished "rm -fr repo" &&
> +		(
> +			cd repo &&
> +			test_commit commit &&
> +			head=$(git rev-parse HEAD) &&
> +
> +			format_command $type "update refs/heads/ref1" "$head" "$Z" >stdin &&
> +			format_command $type "update refs/heads/ref2" "$head" "$Z" >>stdin &&
> +			git update-ref $type --stdin --allow-partial <stdin &&
> +			echo $head >expect &&
> +			git rev-parse refs/heads/ref1 >actual &&
> +			test_cmp expect actual &&
> +			git rev-parse refs/heads/ref2 >actual &&
> +			test_cmp expect actual
> +		)
> +	'
> +
> +	test_expect_success "stdin $type allow-partial with invalid new_oid" '
> +		git init repo &&
> +		test_when_finished "rm -fr repo" &&
> +		(
> +			cd repo &&
> +			test_commit one &&
> +			old_head=$(git rev-parse HEAD) &&
> +			test_commit two &&
> +			head=$(git rev-parse HEAD) &&
> +			git update-ref refs/heads/ref1 $head &&
> +			git update-ref refs/heads/ref2 $head &&
> +
> +			format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
> +			format_command $type "update refs/heads/ref2" "$(test_oid 001)" "$head" >>stdin &&
> +			git update-ref $type --stdin --allow-partial <stdin >stdout &&
> +			echo $old_head >expect &&
> +			git rev-parse refs/heads/ref1 >actual &&
> +			test_cmp expect actual &&
> +			echo $head >expect &&
> +			git rev-parse refs/heads/ref2 >actual &&
> +			test_cmp expect actual &&
> +			test_grep -q "invalid new value provided" stdout
> +		)
> +	'
> +
> +	test_expect_success "stdin $type allow-partial with non-commit new_oid" '
> +		git init repo &&
> +		test_when_finished "rm -fr repo" &&
> +		(
> +			cd repo &&
> +			test_commit one &&
> +			old_head=$(git rev-parse HEAD) &&
> +			test_commit two &&
> +			head=$(git rev-parse HEAD) &&
> +			head_tree=$(git rev-parse HEAD^{tree}) &&
> +			git update-ref refs/heads/ref1 $head &&
> +			git update-ref refs/heads/ref2 $head &&
> +
> +			format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
> +			format_command $type "update refs/heads/ref2" "$head_tree" "$head" >>stdin &&
> +			git update-ref $type --stdin --allow-partial <stdin >stdout &&
> +			echo $old_head >expect &&
> +			git rev-parse refs/heads/ref1 >actual &&
> +			test_cmp expect actual &&
> +			echo $head >expect &&
> +			git rev-parse refs/heads/ref2 >actual &&
> +			test_cmp expect actual &&
> +			test_grep -q "invalid new value provided" stdout
> +		)
> +	'
> +
> +	test_expect_success "stdin $type allow-partial with non-existent ref" '
> +		git init repo &&
> +		test_when_finished "rm -fr repo" &&
> +		(
> +			cd repo &&
> +			test_commit one &&
> +			old_head=$(git rev-parse HEAD) &&
> +			test_commit two &&
> +			head=$(git rev-parse HEAD) &&
> +			git update-ref refs/heads/ref1 $head &&
> +
> +			format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
> +			format_command $type "update refs/heads/ref2" "$old_head" "$head" >>stdin &&
> +			git update-ref $type --stdin --allow-partial <stdin >stdout &&
> +			echo $old_head >expect &&
> +			git rev-parse refs/heads/ref1 >actual &&
> +			test_cmp expect actual &&
> +			test_must_fail git rev-parse refs/heads/ref2 &&
> +			test_grep -q "reference does not exist" stdout
> +		)
> +	'
> +
> +	test_expect_success "stdin $type allow-partial with dangling symref" '
> +		git init repo &&
> +		test_when_finished "rm -fr repo" &&
> +		(
> +			cd repo &&
> +			test_commit one &&
> +			old_head=$(git rev-parse HEAD) &&
> +			test_commit two &&
> +			head=$(git rev-parse HEAD) &&
> +			git update-ref refs/heads/ref1 $head &&
> +			git symbolic-ref refs/heads/ref2 refs/heads/nonexistent &&
> +
> +			format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
> +			format_command $type "update refs/heads/ref2" "$old_head" "$head" >>stdin &&
> +			git update-ref $type --no-deref --stdin --allow-partial <stdin >stdout &&
> +			echo $old_head >expect &&
> +			git rev-parse refs/heads/ref1 >actual &&
> +			test_cmp expect actual &&
> +			echo $head >expect &&
> +			test_must_fail git rev-parse refs/heads/ref2 &&
> +			test_grep -q "reference does not exist" stdout
> +		)
> +	'
> +
> +	test_expect_success "stdin $type allow-partial with regular ref as symref" '
> +		git init repo &&
> +		test_when_finished "rm -fr repo" &&
> +		(
> +			cd repo &&
> +			test_commit one &&
> +			old_head=$(git rev-parse HEAD) &&
> +			test_commit two &&
> +			head=$(git rev-parse HEAD) &&
> +			git update-ref refs/heads/ref1 $head &&
> +			git update-ref refs/heads/ref2 $head &&
> +
> +			format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
> +			format_command $type "symref-update refs/heads/ref2" "$old_head" "ref" "refs/heads/nonexistent" >>stdin &&
> +			git update-ref $type --no-deref --stdin --allow-partial <stdin >stdout &&
> +			echo $old_head >expect &&
> +			git rev-parse refs/heads/ref1 >actual &&
> +			test_cmp expect actual &&
> +			echo $head >expect &&
> +			echo $head >expect &&
> +			git rev-parse refs/heads/ref2 >actual &&
> +			test_cmp expect actual &&
> +			test_grep -q "expected symref but found regular ref" stdout
> +		)
> +	'
> +
> +	test_expect_success "stdin $type allow-partial with invalid old_oid" '
> +		git init repo &&
> +		test_when_finished "rm -fr repo" &&
> +		(
> +			cd repo &&
> +			test_commit one &&
> +			old_head=$(git rev-parse HEAD) &&
> +			test_commit two &&
> +			head=$(git rev-parse HEAD) &&
> +			git update-ref refs/heads/ref1 $head &&
> +			git update-ref refs/heads/ref2 $head &&
> +
> +			format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
> +			format_command $type "update refs/heads/ref2" "$old_head" "$Z" >>stdin &&
> +			git update-ref $type --stdin --allow-partial <stdin >stdout &&
> +			echo $old_head >expect &&
> +			git rev-parse refs/heads/ref1 >actual &&
> +			test_cmp expect actual &&
> +			echo $head >expect &&
> +			git rev-parse refs/heads/ref2 >actual &&
> +			test_cmp expect actual &&
> +			test_grep -q "reference already exists" stdout
> +		)
> +	'
> +
> +	test_expect_success "stdin $type allow-partial with incorrect old oid" '
> +		git init repo &&
> +		test_when_finished "rm -fr repo" &&
> +		(
> +			cd repo &&
> +			test_commit one &&
> +			old_head=$(git rev-parse HEAD) &&
> +			test_commit two &&
> +			head=$(git rev-parse HEAD) &&
> +			git update-ref refs/heads/ref1 $head &&
> +			git update-ref refs/heads/ref2 $head &&
> +
> +			format_command $type "update refs/heads/ref1" "$old_head" "$head" >stdin &&
> +			format_command $type "update refs/heads/ref2" "$head" "$old_head" >>stdin &&
> +			git update-ref $type --stdin --allow-partial <stdin >stdout &&
> +			echo $old_head >expect &&
> +			git rev-parse refs/heads/ref1 >actual &&
> +			test_cmp expect actual &&
> +			echo $head >expect &&
> +			git rev-parse refs/heads/ref2 >actual &&
> +			test_cmp expect actual &&
> +			test_grep -q "incorrect old value provided" stdout
> +		)
> +	'
> +
> +	# F/D conflicts on the files backend are resolved on an individual
> +	# update level since refs are stored as files. On the reftable backend
> +	# this check is batched to optimize for performance, so failures cannot
> +	# be isolated to a single update.
> +	test_expect_success REFFILES "stdin $type allow-partial refname conflict" '
> +		git init repo &&
> +		test_when_finished "rm -fr repo" &&
> +		(
> +			cd repo &&
> +			test_commit one &&
> +			old_head=$(git rev-parse HEAD) &&
> +			test_commit two &&
> +			head=$(git rev-parse HEAD) &&
> +			git update-ref refs/heads/ref/foo $head &&
> +
> +			format_command $type "update refs/heads/ref/foo" "$old_head" "$head" >stdin &&
> +			format_command $type "update refs/heads/ref" "$old_head" "" >>stdin &&
> +			git update-ref $type --stdin --allow-partial <stdin >stdout &&
> +			echo $old_head >expect &&
> +			git rev-parse refs/heads/ref/foo >actual &&
> +			test_cmp expect actual &&
> +			test_grep -q "refname conflict" stdout
> +		)
> +	'
>   done
>   
>   test_expect_success 'update-ref should also create reflog for HEAD' '
>