[PATCH/RFC 0/7] Multiple simultaneously locked ref updates

[PATCH/RFC 0/7] Multiple simultaneously locked ref updates

[Brad King]

Hi Folks,

While thinking about some how some server-side branch management
services might work, I came across a need to update multiple refs
locked with verified old values simultaneously.  For example, to
transfer ownership of some commits by rewinding a branch and creating
a new branch at the original head, one must lock both refs.
Otherwise, depending on the order of updates another process could
create the new branch after we've rewound the original, or add commits
to the original after we've created the new branch.

This series teaches update-ref a new --stdin option to read update and
delete instructions from lines of standard input, lock all refs up
front with verified old values, and then perform the modifications.
This is still work in progress, but it is ready for comments and
feedback.  The series is based on master as of v1.8.4.

Notable unfinished work:

* I propose a format for stdin lines in the last commit of the series
  as a proof-of-concept but I invite suggestions of better formats.
  The format must be able to specify updates and deletes with optional
  old values and optional no-deref.

* No tests for new features, though existing tests pass for me.

* No check for duplicate refs in input.  Currently a duplicate ref
  will result in a failure message like:

   fatal: Unable to create '....lock': File exists.
   If no other git process is currently running, this probably means a
   git process crashed in this repository earlier. Make sure no other git
   process is running and remove the file manually to continue.

  Instead we should reject duplicate ref names up front.  I would
  appreciate suggestions about an efficient data structure already
  available in Git to perform this lookup.

I welcome feedback on the approach, interface, and implementation.

Thanks,
-Brad

Brad King (7):
  reset: rename update_refs to reset_refs
  refs: report ref type from lock_any_ref_for_update
  refs: factor update_ref steps into helpers
  refs: factor delete_ref loose ref step into a helper
  refs: add function to repack without multiple refs
  refs: add update_refs for multiple simultaneous updates
  update-ref: support multiple simultaneous updates

 Documentation/git-update-ref.txt |   19 ++++-
 branch.c                         |    2 +-
 builtin/commit.c                 |    2 +-
 builtin/fetch.c                  |    2 +-
 builtin/receive-pack.c           |    2 +-
 builtin/reflog.c                 |    2 +-
 builtin/replace.c                |    2 +-
 builtin/reset.c                  |    4 +-
 builtin/tag.c                    |    2 +-
 builtin/update-ref.c             |   93 ++++++++++++++++++++++-
 fast-import.c                    |    2 +-
 refs.c                           |  150 ++++++++++++++++++++++++++++++++------
 refs.h                           |   13 +++-
 sequencer.c                      |    2 +-
 14 files changed, 262 insertions(+), 35 deletions(-)

-- 
1.7.10.4

[PATCH/RFC 1/7] reset: rename update_refs to reset_refs

[Brad King]

Get it out of the way for a future refs.h function.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 builtin/reset.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/builtin/reset.c b/builtin/reset.c
index afa6e02..789ee48 100644
--- a/builtin/reset.c
+++ b/builtin/reset.c
@@ -219,7 +219,7 @@ static const char **parse_args(const char **argv, const char *prefix, const char
 	return argv[0] ? get_pathspec(prefix, argv) : NULL;
 }
 
-static int update_refs(const char *rev, const unsigned char *sha1)
+static int reset_refs(const char *rev, const unsigned char *sha1)
 {
 	int update_ref_status;
 	struct strbuf msg = STRBUF_INIT;
@@ -350,7 +350,7 @@ int cmd_reset(int argc, const char **argv, const char *prefix)
 	if (!pathspec && !unborn) {
 		/* Any resets without paths update HEAD to the head being
 		 * switched to, saving the previous head in ORIG_HEAD before. */
-		update_ref_status = update_refs(rev, sha1);
+		update_ref_status = reset_refs(rev, sha1);
 
 		if (reset_type == HARD && !update_ref_status && !quiet)
 			print_new_head_line(lookup_commit_reference(sha1));
-- 
1.7.10.4

Re: [PATCH/RFC 1/7] reset: rename update_refs to reset_refs

[Junio C Hamano]

Brad King <brad.king@kitware.com> writes:

> Get it out of the way for a future refs.h function.

Readers do not know if "update_refs()" is a good name for that
future refs.h function at this point, so "evict squatter" is not a
very good justification by itself.  I do agree that this static
function is resetting a ref, not doing an arbitrary update, and the
new name is a better match than the old one for what it does, though.

> Signed-off-by: Brad King <brad.king@kitware.com>
> ---
>  builtin/reset.c |    4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/builtin/reset.c b/builtin/reset.c
> index afa6e02..789ee48 100644
> --- a/builtin/reset.c
> +++ b/builtin/reset.c
> @@ -219,7 +219,7 @@ static const char **parse_args(const char **argv, const char *prefix, const char
>  	return argv[0] ? get_pathspec(prefix, argv) : NULL;
>  }
>  
> -static int update_refs(const char *rev, const unsigned char *sha1)
> +static int reset_refs(const char *rev, const unsigned char *sha1)
>  {
>  	int update_ref_status;
>  	struct strbuf msg = STRBUF_INIT;
> @@ -350,7 +350,7 @@ int cmd_reset(int argc, const char **argv, const char *prefix)
>  	if (!pathspec && !unborn) {
>  		/* Any resets without paths update HEAD to the head being
>  		 * switched to, saving the previous head in ORIG_HEAD before. */
> -		update_ref_status = update_refs(rev, sha1);
> +		update_ref_status = reset_refs(rev, sha1);
>  
>  		if (reset_type == HARD && !update_ref_status && !quiet)
>  			print_new_head_line(lookup_commit_reference(sha1));

Re: [PATCH/RFC 1/7] reset: rename update_refs to reset_refs

[Brad King]

On 08/29/2013 01:17 PM, Junio C Hamano wrote:
> Brad King <brad.king@kitware.com> writes:
> 
>> Get it out of the way for a future refs.h function.
> 
> Readers do not know if "update_refs()" is a good name for that
> future refs.h function at this point, so "evict squatter" is not a
> very good justification by itself.  I do agree that this static
> function is resetting a ref, not doing an arbitrary update, and the
> new name is a better match than the old one for what it does, though.

Okay, I've revised the commit message for the next revision.

-Brad

[PATCH/RFC 2/7] refs: report ref type from lock_any_ref_for_update

[Brad King]

Expose lock_ref_sha1_basic's type_p argument to callers of
lock_any_ref_for_update.  Update all call sites to ignore it; we will
use it later.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 branch.c               |    2 +-
 builtin/commit.c       |    2 +-
 builtin/fetch.c        |    2 +-
 builtin/receive-pack.c |    2 +-
 builtin/reflog.c       |    2 +-
 builtin/replace.c      |    2 +-
 builtin/tag.c          |    2 +-
 fast-import.c          |    2 +-
 refs.c                 |    7 ++++---
 refs.h                 |    2 +-
 sequencer.c            |    2 +-
 11 files changed, 14 insertions(+), 13 deletions(-)

diff --git a/branch.c b/branch.c
index c5c6984..c244483 100644
--- a/branch.c
+++ b/branch.c
@@ -291,7 +291,7 @@ void create_branch(const char *head,
 	hashcpy(sha1, commit->object.sha1);
 
 	if (!dont_change_ref) {
-		lock = lock_any_ref_for_update(ref.buf, NULL, 0);
+		lock = lock_any_ref_for_update(ref.buf, NULL, 0, 0);
 		if (!lock)
 			die_errno(_("Failed to lock ref for update"));
 	}
diff --git a/builtin/commit.c b/builtin/commit.c
index 10acc53..78d773f 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1618,7 +1618,7 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 					   !current_head
 					   ? NULL
 					   : current_head->object.sha1,
-					   0);
+					   0, 0);
 
 	nl = strchr(sb.buf, '\n');
 	if (nl)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index d784b2e..34903ef 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -246,7 +246,7 @@ static int s_update_ref(const char *action,
 		rla = default_rla.buf;
 	snprintf(msg, sizeof(msg), "%s: %s", rla, action);
 	lock = lock_any_ref_for_update(ref->name,
-				       check_old ? ref->old_sha1 : NULL, 0);
+				       check_old ? ref->old_sha1 : NULL, 0, 0);
 	if (!lock)
 		return errno == ENOTDIR ? STORE_REF_ERROR_DF_CONFLICT :
 					  STORE_REF_ERROR_OTHER;
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index e3eb5fc..dd61234 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -524,7 +524,7 @@ static const char *update(struct command *cmd)
 		return NULL; /* good */
 	}
 	else {
-		lock = lock_any_ref_for_update(namespaced_name, old_sha1, 0);
+		lock = lock_any_ref_for_update(namespaced_name, old_sha1, 0, 0);
 		if (!lock) {
 			rp_error("failed to lock %s", name);
 			return "failed to lock";
diff --git a/builtin/reflog.c b/builtin/reflog.c
index 54184b3..11b30f9 100644
--- a/builtin/reflog.c
+++ b/builtin/reflog.c
@@ -366,7 +366,7 @@ static int expire_reflog(const char *ref, const unsigned char *sha1, int unused,
 	 * we take the lock for the ref itself to prevent it from
 	 * getting updated.
 	 */
-	lock = lock_any_ref_for_update(ref, sha1, 0);
+	lock = lock_any_ref_for_update(ref, sha1, 0, 0);
 	if (!lock)
 		return error("cannot lock ref '%s'", ref);
 	log_file = git_pathdup("logs/%s", ref);
diff --git a/builtin/replace.c b/builtin/replace.c
index 59d3115..e2e2002 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -105,7 +105,7 @@ static int replace_object(const char *object_ref, const char *replace_ref,
 	else if (!force)
 		die("replace ref '%s' already exists", ref);
 
-	lock = lock_any_ref_for_update(ref, prev, 0);
+	lock = lock_any_ref_for_update(ref, prev, 0, 0);
 	if (!lock)
 		die("%s: cannot lock the ref", ref);
 	if (write_ref_sha1(lock, repl, NULL) < 0)
diff --git a/builtin/tag.c b/builtin/tag.c
index af3af3f..c261469 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -577,7 +577,7 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
 	if (annotate)
 		create_tag(object, tag, &buf, &opt, prev, object);
 
-	lock = lock_any_ref_for_update(ref.buf, prev, 0);
+	lock = lock_any_ref_for_update(ref.buf, prev, 0, 0);
 	if (!lock)
 		die(_("%s: cannot lock the ref"), ref.buf);
 	if (write_ref_sha1(lock, object, NULL) < 0)
diff --git a/fast-import.c b/fast-import.c
index 23f625f..5f7ef82 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -1678,7 +1678,7 @@ static int update_branch(struct branch *b)
 		return 0;
 	if (read_ref(b->name, old_sha1))
 		hashclr(old_sha1);
-	lock = lock_any_ref_for_update(b->name, old_sha1, 0);
+	lock = lock_any_ref_for_update(b->name, old_sha1, 0, 0);
 	if (!lock)
 		return error("Unable to lock %s", b->name);
 	if (!force_update && !is_null_sha1(old_sha1)) {
diff --git a/refs.c b/refs.c
index 7922261..3a7b597 100644
--- a/refs.c
+++ b/refs.c
@@ -2121,11 +2121,12 @@ struct ref_lock *lock_ref_sha1(const char *refname, const unsigned char *old_sha
 }
 
 struct ref_lock *lock_any_ref_for_update(const char *refname,
-					 const unsigned char *old_sha1, int flags)
+					 const unsigned char *old_sha1,
+					 int flags, int *type_p)
 {
 	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL))
 		return NULL;
-	return lock_ref_sha1_basic(refname, old_sha1, flags, NULL);
+	return lock_ref_sha1_basic(refname, old_sha1, flags, type_p);
 }
 
 /*
@@ -3174,7 +3175,7 @@ int update_ref(const char *action, const char *refname,
 		int flags, enum action_on_err onerr)
 {
 	static struct ref_lock *lock;
-	lock = lock_any_ref_for_update(refname, oldval, flags);
+	lock = lock_any_ref_for_update(refname, oldval, flags, 0);
 	if (!lock) {
 		const char *str = "Cannot lock the ref '%s'.";
 		switch (onerr) {
diff --git a/refs.h b/refs.h
index 9e5db3a..2cd307a 100644
--- a/refs.h
+++ b/refs.h
@@ -137,7 +137,7 @@ extern struct ref_lock *lock_ref_sha1(const char *refname, const unsigned char *
 #define REF_NODEREF	0x01
 extern struct ref_lock *lock_any_ref_for_update(const char *refname,
 						const unsigned char *old_sha1,
-						int flags);
+						int flags, int *type_p);
 
 /** Close the file descriptor owned by a lock and return the status */
 extern int close_ref(struct ref_lock *lock);
diff --git a/sequencer.c b/sequencer.c
index 351548f..4180d58 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -279,7 +279,7 @@ static int fast_forward_to(const unsigned char *to, const unsigned char *from,
 	read_cache();
 	if (checkout_fast_forward(from, to, 1))
 		exit(1); /* the callee should have complained already */
-	ref_lock = lock_any_ref_for_update("HEAD", unborn ? null_sha1 : from, 0);
+	ref_lock = lock_any_ref_for_update("HEAD", unborn ? null_sha1 : from, 0, 0);
 	strbuf_addf(&sb, "%s: fast-forward", action_name(opts));
 	ret = write_ref_sha1(ref_lock, to, sb.buf);
 	strbuf_release(&sb);
-- 
1.7.10.4

Re: [PATCH/RFC 2/7] refs: report ref type from lock_any_ref_for_update

[Junio C Hamano]

Brad King <brad.king@kitware.com> writes:

> Expose lock_ref_sha1_basic's type_p argument to callers of
> lock_any_ref_for_update.  Update all call sites to ignore it; we will
> use it later.
> ...
> diff --git a/branch.c b/branch.c
> index c5c6984..c244483 100644
> --- a/branch.c
> +++ b/branch.c
> @@ -291,7 +291,7 @@ void create_branch(const char *head,
>  	hashcpy(sha1, commit->object.sha1);
>  
>  	if (!dont_change_ref) {
> -		lock = lock_any_ref_for_update(ref.buf, NULL, 0);
> +		lock = lock_any_ref_for_update(ref.buf, NULL, 0, 0);

If you are passing an NULL as a new parameter, please spell it
"NULL", not "0".

Re: [PATCH/RFC 2/7] refs: report ref type from lock_any_ref_for_update

[Brad King]

On 08/29/2013 01:22 PM, Junio C Hamano wrote:
> If you are passing an NULL as a new parameter, please spell it
> "NULL", not "0".

Fixed at all updated call sites.

-Brad

[PATCH/RFC 3/7] refs: factor update_ref steps into helpers

[Brad King]

Factor the lock and write steps and error handling into helper functions
update_ref_lock and update_ref_write to allow later use elsewhere.
Expose lock_any_ref_for_update's type_p to update_ref_lock callers.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 refs.c |   28 +++++++++++++++++++++++-----
 1 file changed, 23 insertions(+), 5 deletions(-)

diff --git a/refs.c b/refs.c
index 3a7b597..2e755b4 100644
--- a/refs.c
+++ b/refs.c
@@ -3170,12 +3170,13 @@ int for_each_reflog(each_ref_fn fn, void *cb_data)
 	return retval;
 }
 
-int update_ref(const char *action, const char *refname,
-		const unsigned char *sha1, const unsigned char *oldval,
-		int flags, enum action_on_err onerr)
+static struct ref_lock *update_ref_lock(const char *refname,
+					const unsigned char *oldval,
+					int flags, int *type_p,
+					enum action_on_err onerr)
 {
 	static struct ref_lock *lock;
-	lock = lock_any_ref_for_update(refname, oldval, flags, 0);
+	lock = lock_any_ref_for_update(refname, oldval, flags, type_p);
 	if (!lock) {
 		const char *str = "Cannot lock the ref '%s'.";
 		switch (onerr) {
@@ -3183,8 +3184,14 @@ int update_ref(const char *action, const char *refname,
 		case DIE_ON_ERR: die(str, refname); break;
 		case QUIET_ON_ERR: break;
 		}
-		return 1;
 	}
+	return lock;
+}
+
+static int update_ref_write(const char *action, const char *refname,
+			    const unsigned char *sha1, struct ref_lock *lock,
+			    enum action_on_err onerr)
+{
 	if (write_ref_sha1(lock, sha1, action) < 0) {
 		const char *str = "Cannot update the ref '%s'.";
 		switch (onerr) {
@@ -3197,6 +3204,17 @@ int update_ref(const char *action, const char *refname,
 	return 0;
 }
 
+int update_ref(const char *action, const char *refname,
+	       const unsigned char *sha1, const unsigned char *oldval,
+	       int flags, enum action_on_err onerr)
+{
+	static struct ref_lock *lock;
+	lock = update_ref_lock(refname, oldval, flags, 0, onerr);
+	if (!lock)
+		return 1;
+	return update_ref_write(action, refname, sha1, lock, onerr);
+}
+
 struct ref *find_ref_by_name(const struct ref *list, const char *name)
 {
 	for ( ; list; list = list->next)
-- 
1.7.10.4

[PATCH/RFC 4/7] refs: factor delete_ref loose ref step into a helper

[Brad King]

Factor loose ref deletion into helper function delete_ref_loose to allow
later use elsewhere.  While at it, rename local names 'flag => type' and
'delopt => flags' for consistency with callers and called functions.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 refs.c |   24 ++++++++++++++++--------
 1 file changed, 16 insertions(+), 8 deletions(-)

diff --git a/refs.c b/refs.c
index 2e755b4..5908648 100644
--- a/refs.c
+++ b/refs.c
@@ -2450,15 +2450,10 @@ static int repack_without_ref(const char *refname)
 	return commit_packed_refs();
 }
 
-int delete_ref(const char *refname, const unsigned char *sha1, int delopt)
+static int delete_ref_loose(struct ref_lock *lock, int type)
 {
-	struct ref_lock *lock;
-	int err, i = 0, ret = 0, flag = 0;
-
-	lock = lock_ref_sha1_basic(refname, sha1, delopt, &flag);
-	if (!lock)
-		return 1;
-	if (!(flag & REF_ISPACKED) || flag & REF_ISSYMREF) {
+	int err, i, ret = 0;
+	if (!(type & REF_ISPACKED) || type & REF_ISSYMREF) {
 		/* loose */
 		i = strlen(lock->lk->filename) - 5; /* .lock */
 		lock->lk->filename[i] = 0;
@@ -2468,6 +2463,19 @@ int delete_ref(const char *refname, const unsigned char *sha1, int delopt)
 
 		lock->lk->filename[i] = '.';
 	}
+	return ret;
+}
+
+int delete_ref(const char *refname, const unsigned char *sha1, int flags)
+{
+	struct ref_lock *lock;
+	int ret = 0, type = 0;
+
+	lock = lock_ref_sha1_basic(refname, sha1, flags, &type);
+	if (!lock)
+		return 1;
+	ret |= delete_ref_loose(lock, type);
+
 	/* removing the loose one could have resurrected an earlier
 	 * packed one.  Also, if it was not loose we need to repack
 	 * without it.
-- 
1.7.10.4

Re: [PATCH/RFC 4/7] refs: factor delete_ref loose ref step into a helper

[Junio C Hamano]

Brad King <brad.king@kitware.com> writes:

> Factor loose ref deletion into helper function delete_ref_loose to allow
> later use elsewhere.  While at it, rename local names 'flag => type' and
> 'delopt => flags' for consistency with callers and called functions.
>
> Signed-off-by: Brad King <brad.king@kitware.com>
> ---
>  refs.c |   24 ++++++++++++++++--------
>  1 file changed, 16 insertions(+), 8 deletions(-)
>
> diff --git a/refs.c b/refs.c
> index 2e755b4..5908648 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -2450,15 +2450,10 @@ static int repack_without_ref(const char *refname)
>  	return commit_packed_refs();
>  }
>  
> -int delete_ref(const char *refname, const unsigned char *sha1, int delopt)
> +static int delete_ref_loose(struct ref_lock *lock, int type)
>  {
> -	struct ref_lock *lock;
> -	int err, i = 0, ret = 0, flag = 0;
> -
> -	lock = lock_ref_sha1_basic(refname, sha1, delopt, &flag);
> -	if (!lock)
> -		return 1;
> -	if (!(flag & REF_ISPACKED) || flag & REF_ISSYMREF) {
> +	int err, i, ret = 0;
> +	if (!(type & REF_ISPACKED) || type & REF_ISSYMREF) {

Hits from "git grep REF_IS" tell me that all users of REF_IS* symbol
that check if a bit is on in a word does so against "flag" (either a
variable called "flag", "flags", or a structure member ".flag").

This change is making things less consistent, not more, I am afraid.

>  		/* loose */
>  		i = strlen(lock->lk->filename) - 5; /* .lock */
>  		lock->lk->filename[i] = 0;
> @@ -2468,6 +2463,19 @@ int delete_ref(const char *refname, const unsigned char *sha1, int delopt)
>  
>  		lock->lk->filename[i] = '.';
>  	}
> +	return ret;
> +}
> +
> +int delete_ref(const char *refname, const unsigned char *sha1, int flags)
> +{
> +	struct ref_lock *lock;
> +	int ret = 0, type = 0;
> +
> +	lock = lock_ref_sha1_basic(refname, sha1, flags, &type);
> +	if (!lock)
> +		return 1;
> +	ret |= delete_ref_loose(lock, type);
> +
>  	/* removing the loose one could have resurrected an earlier
>  	 * packed one.  Also, if it was not loose we need to repack
>  	 * without it.

Re: [PATCH/RFC 4/7] refs: factor delete_ref loose ref step into a helper

[Brad King]

On 08/29/2013 01:28 PM, Junio C Hamano wrote:
> Brad King <brad.king@kitware.com> writes:
>> -	if (!(flag & REF_ISPACKED) || flag & REF_ISSYMREF) {
>> +	if (!(type & REF_ISPACKED) || type & REF_ISSYMREF) {
> 
> Hits from "git grep REF_IS" tell me that all users of REF_IS* symbol
> that check if a bit is on in a word does so against "flag" (either a
> variable called "flag", "flags", or a structure member ".flag").
> 
> This change is making things less consistent, not more, I am afraid.

Okay, I removed this part of the change.  It makes the commit
simpler anyway.

-Brad

[PATCH/RFC 5/7] refs: add function to repack without multiple refs

[Brad King]

Generalize repack_without_ref as repack_without_refs to support a list
of refs and implement the former in terms of the latter.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 refs.c |   29 ++++++++++++++++++++++-------
 1 file changed, 22 insertions(+), 7 deletions(-)

diff --git a/refs.c b/refs.c
index 5908648..5a6c14e 100644
--- a/refs.c
+++ b/refs.c
@@ -2414,25 +2414,35 @@ static int curate_packed_ref_fn(struct ref_entry *entry, void *cb_data)
 	return 0;
 }
 
-static int repack_without_ref(const char *refname)
+static int repack_without_refs(const char **refnames, int n)
 {
 	struct ref_dir *packed;
 	struct string_list refs_to_delete = STRING_LIST_INIT_DUP;
 	struct string_list_item *ref_to_delete;
+	int i, removed = 0;
+
+	/* Look for a packed ref: */
+	for (i = 0; i < n; ++i)
+		if (get_packed_ref(refnames[i]))
+			break;
 
-	if (!get_packed_ref(refname))
-		return 0; /* refname does not exist in packed refs */
+	/* Avoid locking if we have nothing to do: */
+	if(i == n)
+		return 0; /* no refname exists in packed refs */
 
 	if (lock_packed_refs(0)) {
 		unable_to_lock_error(git_path("packed-refs"), errno);
-		return error("cannot delete '%s' from packed refs", refname);
+		return error("cannot delete '%s' from packed refs", refnames[i]);
 	}
 	packed = get_packed_refs(&ref_cache);
 
-	/* Remove refname from the cache: */
-	if (remove_entry(packed, refname) == -1) {
+	/* Remove refnames from the cache: */
+	for (i = 0; i < n; ++i)
+		if (remove_entry(packed, refnames[i]) != -1)
+			removed = 1;
+	if (!removed) {
 		/*
-		 * The packed entry disappeared while we were
+		 * All packed entries disappeared while we were
 		 * acquiring the lock.
 		 */
 		rollback_packed_refs();
@@ -2450,6 +2460,11 @@ static int repack_without_ref(const char *refname)
 	return commit_packed_refs();
 }
 
+static int repack_without_ref(const char *refname)
+{
+	return repack_without_refs(&refname, 1);
+}
+
 static int delete_ref_loose(struct ref_lock *lock, int type)
 {
 	int err, i, ret = 0;
-- 
1.7.10.4

Re: [PATCH/RFC 5/7] refs: add function to repack without multiple refs

[Junio C Hamano]

Brad King <brad.king@kitware.com> writes:

> Generalize repack_without_ref as repack_without_refs to support a list
> of refs and implement the former in terms of the latter.
>
> Signed-off-by: Brad King <brad.king@kitware.com>
> ---
>  refs.c |   29 ++++++++++++++++++++++-------
>  1 file changed, 22 insertions(+), 7 deletions(-)
>
> diff --git a/refs.c b/refs.c
> index 5908648..5a6c14e 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -2414,25 +2414,35 @@ static int curate_packed_ref_fn(struct ref_entry *entry, void *cb_data)
>  	return 0;
>  }
>  
> -static int repack_without_ref(const char *refname)
> +static int repack_without_refs(const char **refnames, int n)
>  {
>  	struct ref_dir *packed;
>  	struct string_list refs_to_delete = STRING_LIST_INIT_DUP;
>  	struct string_list_item *ref_to_delete;
> +	int i, removed = 0;
> +
> +	/* Look for a packed ref: */
> +	for (i = 0; i < n; ++i)
> +		if (get_packed_ref(refnames[i]))
> +			break;
>  
> -	if (!get_packed_ref(refname))
> -		return 0; /* refname does not exist in packed refs */
> +	/* Avoid locking if we have nothing to do: */
> +	if(i == n)

Style:
	if (i == n)

> +		return 0; /* no refname exists in packed refs */
>  
>  	if (lock_packed_refs(0)) {
>  		unable_to_lock_error(git_path("packed-refs"), errno);
> -		return error("cannot delete '%s' from packed refs", refname);
> +		return error("cannot delete '%s' from packed refs", refnames[i]);
>  	}
>  	packed = get_packed_refs(&ref_cache);
>  
> -	/* Remove refname from the cache: */
> -	if (remove_entry(packed, refname) == -1) {
> +	/* Remove refnames from the cache: */
> +	for (i = 0; i < n; ++i)
> +		if (remove_entry(packed, refnames[i]) != -1)
> +			removed = 1;
> +	if (!removed) {
>  		/*
> -		 * The packed entry disappeared while we were
> +		 * All packed entries disappeared while we were
>  		 * acquiring the lock.
>  		 */
>  		rollback_packed_refs();

... and this is not an error; somebody else did the work we wanted
to do for us, which is good ;-)

> @@ -2450,6 +2460,11 @@ static int repack_without_ref(const char *refname)
>  	return commit_packed_refs();
>  }
>  
> +static int repack_without_ref(const char *refname)
> +{
> +	return repack_without_refs(&refname, 1);
> +}
> +
>  static int delete_ref_loose(struct ref_lock *lock, int type)
>  {
>  	int err, i, ret = 0;

Re: [PATCH/RFC 5/7] refs: add function to repack without multiple refs

[Brad King]

On 08/29/2013 01:34 PM, Junio C Hamano wrote:
> Brad King <brad.king@kitware.com> writes:
>> +	if(i == n)
> 
> Style:
> 	if (i == n)

Fixed in next revision.

-Brad

[PATCH/RFC 6/7] refs: add update_refs for multiple simultaneous updates

[Brad King]

Add 'struct ref_update' to encode the information needed to update or
delete a ref (name, new sha1, optional old sha1, no-deref flag).  Add
function 'update_refs' accepting an array of updates to perform.  First
acquire locks on all refs with verified old values.  Then update or
delete all refs accordingly.  Fail if any one lock cannot be obtained or
any one old value does not match.

Though the refs themeselves cannot be modified together in a single
atomic transaction, this function does enable some useful semantics.
For example, a caller may create a new branch starting from the head of
another branch and rewind the original branch at the same time.  This
transfers ownership of commits between branches without risk of losing
commits added to the original branch by a concurrent process, or risk of
a concurrent process creating the new branch first.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 refs.c |   66 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 refs.h |   11 +++++++++++
 2 files changed, 77 insertions(+)

diff --git a/refs.c b/refs.c
index 5a6c14e..0a0c19e 100644
--- a/refs.c
+++ b/refs.c
@@ -3238,6 +3238,72 @@ int update_ref(const char *action, const char *refname,
 	return update_ref_write(action, refname, sha1, lock, onerr);
 }
 
+int update_refs(const char *action, struct ref_update *updates,
+		int n, enum action_on_err onerr)
+{
+	int ret = 0, delnum = 0, i;
+	int *types;
+	struct ref_lock **locks;
+	const char **delnames;
+
+	if (!updates || !n)
+		return 0;
+
+	/* Allocate work space: */
+	types = xmalloc(sizeof(int)*n);
+	locks = xmalloc(sizeof(struct ref_lock*)*n);
+	delnames = xmalloc(sizeof(const char*)*n);
+
+	/* Acquire all locks while verifying old values: */
+	for (i=0; i < n; ++i) {
+		locks[i] = update_ref_lock(updates[i].ref_name,
+					   updates[i].old_sha1,
+					   updates[i].flags,
+					   &types[i], onerr);
+		if (!locks[i])
+			break;
+	}
+
+	/* Abort if we did not get all locks: */
+	if (i < n) {
+		while (--i >= 0)
+			unlock_ref(locks[i]);
+		free(types);
+		free(locks);
+		free(delnames);
+		return 1;
+	}
+
+	/* Perform updates first so live commits remain referenced: */
+	for (i=0; i < n; ++i)
+		if (!is_null_sha1(updates[i].new_sha1)) {
+			ret |= update_ref_write(action,
+						updates[i].ref_name,
+						updates[i].new_sha1,
+						locks[i], onerr);
+			locks[i] = 0; /* freed by update_ref_write */
+		}
+
+	/* Perform deletes now that updates are safely completed: */
+	for (i=0; i < n; ++i)
+		if (locks[i]) {
+			delnames[delnum++] = locks[i]->ref_name;
+			ret |= delete_ref_loose(locks[i], types[i]);
+		}
+	ret |= repack_without_refs(delnames, delnum);
+	for (i=0; i < delnum; ++i)
+		unlink_or_warn(git_path("logs/%s", delnames[i]));
+	clear_loose_ref_cache(&ref_cache);
+	for (i=0; i < n; ++i)
+		if (locks[i])
+			unlock_ref(locks[i]);
+
+	free(types);
+	free(locks);
+	free(delnames);
+	return ret;
+}
+
 struct ref *find_ref_by_name(const struct ref *list, const char *name)
 {
 	for ( ; list; list = list->next)
diff --git a/refs.h b/refs.h
index 2cd307a..5763f3a 100644
--- a/refs.h
+++ b/refs.h
@@ -214,6 +214,17 @@ int update_ref(const char *action, const char *refname,
 		const unsigned char *sha1, const unsigned char *oldval,
 		int flags, enum action_on_err onerr);
 
+struct ref_update {
+	const char *ref_name;
+	unsigned char new_sha1[20];
+	unsigned char *old_sha1;
+	int flags;
+};
+
+/** lock all refs and then write all of them */
+int update_refs(const char *action, struct ref_update *updates,
+		int n, enum action_on_err onerr);
+
 extern int parse_hide_refs_config(const char *var, const char *value, const char *);
 extern int ref_is_hidden(const char *);
 
-- 
1.7.10.4

Re: [PATCH/RFC 6/7] refs: add update_refs for multiple simultaneous updates

[Junio C Hamano]

Brad King <brad.king@kitware.com> writes:

> Add 'struct ref_update' to encode the information needed to update or
> delete a ref (name, new sha1, optional old sha1, no-deref flag).  Add
> function 'update_refs' accepting an array of updates to perform.  First
> acquire locks on all refs with verified old values.  Then update or
> delete all refs accordingly.  Fail if any one lock cannot be obtained or
> any one old value does not match.
>
> Though the refs themeselves cannot be modified together in a single
> atomic transaction, this function does enable some useful semantics.
> For example, a caller may create a new branch starting from the head of
> another branch and rewind the original branch at the same time.  This
> transfers ownership of commits between branches without risk of losing
> commits added to the original branch by a concurrent process, or risk of
> a concurrent process creating the new branch first.
>
> Signed-off-by: Brad King <brad.king@kitware.com>
> ---
>  refs.c |   66 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  refs.h |   11 +++++++++++
>  2 files changed, 77 insertions(+)
>
> diff --git a/refs.c b/refs.c
> index 5a6c14e..0a0c19e 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -3238,6 +3238,72 @@ int update_ref(const char *action, const char *refname,
>  	return update_ref_write(action, refname, sha1, lock, onerr);
>  }
>  
> +int update_refs(const char *action, struct ref_update *updates,
> +		int n, enum action_on_err onerr)
> +{
> +	int ret = 0, delnum = 0, i;
> +	int *types;
> +	struct ref_lock **locks;
> +	const char **delnames;
> +
> +	if (!updates || !n)
> +		return 0;
> +
> +	/* Allocate work space: */
> +	types = xmalloc(sizeof(int)*n);
> +	locks = xmalloc(sizeof(struct ref_lock*)*n);
> +	delnames = xmalloc(sizeof(const char*)*n);
> +
> +	/* Acquire all locks while verifying old values: */
> +	for (i=0; i < n; ++i) {

Style:

	for (i = 0; i < n; i++) {

> +		locks[i] = update_ref_lock(updates[i].ref_name,
> +					   updates[i].old_sha1,
> +					   updates[i].flags,
> +					   &types[i], onerr);
> +		if (!locks[i])
> +			break;
> +	}

Is it asking for AB-BA deadlock?  If so, is the caller responsible
for avoiding it?

> +	/* Abort if we did not get all locks: */
> +	if (i < n) {
> +		while (--i >= 0)
> +			unlock_ref(locks[i]);
> +		free(types);
> +		free(locks);
> +		free(delnames);
> +		return 1;
> +	}
> +
> +	/* Perform updates first so live commits remain referenced: */
> +	for (i=0; i < n; ++i)
> +		if (!is_null_sha1(updates[i].new_sha1)) {
> +			ret |= update_ref_write(action,
> +						updates[i].ref_name,
> +						updates[i].new_sha1,
> +						locks[i], onerr);
> +			locks[i] = 0; /* freed by update_ref_write */
> +		}
> +
> +	/* Perform deletes now that updates are safely completed: */
> +	for (i=0; i < n; ++i)
> +		if (locks[i]) {
> +			delnames[delnum++] = locks[i]->ref_name;
> +			ret |= delete_ref_loose(locks[i], types[i]);
> +		}
> +	ret |= repack_without_refs(delnames, delnum);
> +	for (i=0; i < delnum; ++i)
> +		unlink_or_warn(git_path("logs/%s", delnames[i]));
> +	clear_loose_ref_cache(&ref_cache);
> +	for (i=0; i < n; ++i)
> +		if (locks[i])
> +			unlock_ref(locks[i]);
> +
> +	free(types);
> +	free(locks);
> +	free(delnames);
> +	return ret;
> +}
> +
>  struct ref *find_ref_by_name(const struct ref *list, const char *name)
>  {
>  	for ( ; list; list = list->next)
> diff --git a/refs.h b/refs.h
> index 2cd307a..5763f3a 100644
> --- a/refs.h
> +++ b/refs.h
> @@ -214,6 +214,17 @@ int update_ref(const char *action, const char *refname,
>  		const unsigned char *sha1, const unsigned char *oldval,
>  		int flags, enum action_on_err onerr);
>  
> +struct ref_update {
> +	const char *ref_name;
> +	unsigned char new_sha1[20];
> +	unsigned char *old_sha1;

This asymmetry is rather curious and will later become problematic
when we have more users of this API.  Maybe your callers happen have
static storage to keep old_sha1[] outside this structure but do not
have it for new_sha1[] for some unknown reason, which may not apply
to later callers we may want to add.

> +	int flags;
> +};
> +
> +/** lock all refs and then write all of them */
> +int update_refs(const char *action, struct ref_update *updates,
> +		int n, enum action_on_err onerr);
> +
>  extern int parse_hide_refs_config(const char *var, const char *value, const char *);
>  extern int ref_is_hidden(const char *);

Re: [PATCH/RFC 6/7] refs: add update_refs for multiple simultaneous updates

[Brad King]

On 08/29/2013 01:39 PM, Junio C Hamano wrote:
> Brad King <brad.king@kitware.com> writes:
>> +	for (i=0; i < n; ++i) {
> 
> Style:
> 
> 	for (i = 0; i < n; i++) {

Fixed.

> Is it asking for AB-BA deadlock?  If so, is the caller responsible
> for avoiding it?

Since we don't actually block waiting for locks we won't really
deadlock.  However, if two competing callers keep repeating they
might never finish.  In order to avoid this one must sort the refs
so that locks are always acquired in a consistent order.

For Git's internal API I think we can document this in a comment so
that update_refs does not have to sort.  Then we can add a new
ref_update_sort function to sort an array of struct ref_update.
The user-facing "update-ref --stdin" can then use ref_update_sort.

The sort logic may subsume duplicate ref update detection too.
After sorting a simple linear-time scan can detect duplicates.

>> +	unsigned char new_sha1[20];
>> +	unsigned char *old_sha1;
> 
> This asymmetry is rather curious and will later become problematic
> when we have more users of this API.  Maybe your callers happen have
> static storage to keep old_sha1[] outside this structure but do not
> have it for new_sha1[] for some unknown reason, which may not apply
> to later callers we may want to add.

I wasn't happy with the asymmetry either but forgot to raise it in
the cover letter.  We need a way to represent "old value not given"
as different from "old value is_null_sha1".

One approach is to use a bit in the "flags" member that already
stores REF_NODEREF.  However, that will be inconsistent with the
other ref update APIs that check whether old_sha1 is NULL.  We could
still do it and document the bit to mean "ignore old_sha1 member of
struct ref_update".

Another approach is to add a dedicated member to struct ref_update.

Comments?
-Brad

Re: [PATCH/RFC 6/7] refs: add update_refs for multiple simultaneous updates

[Junio C Hamano]

Brad King <brad.king@kitware.com> writes:

> On 08/29/2013 01:39 PM, Junio C Hamano wrote:
>> Brad King <brad.king@kitware.com> writes:
>>> +	for (i=0; i < n; ++i) {
>> 
>> Style:
>> 
>> 	for (i = 0; i < n; i++) {
>
> Fixed.
>
>> Is it asking for AB-BA deadlock?  If so, is the caller responsible
>> for avoiding it?
>
> Since we don't actually block waiting for locks we won't really
> deadlock.

Ahh, OK.

> For Git's internal API I think we can document this in a comment so
> that update_refs does not have to sort.  Then we can add a new
> ref_update_sort function to sort an array of struct ref_update.
> The user-facing "update-ref --stdin" can then use ref_update_sort.

My immediate reaction was "is there a case where the caller knows
that it already has a sorted collection?".  The single caller you
are envisioning could collect the proposed updates to a string list
and dedup, I think, and the resulting list would then be already
sorted.

But it may not be a bad idea to keep the callers dumb and have this
function always sort, dedup, *and* fail inconsistent request.  Then
your original caller that just collects --stdin input can pass
possibly unsorted, duplicated and/or inconsistent request to the
function and have it do the sanity check.

Re: [PATCH/RFC 6/7] refs: add update_refs for multiple simultaneous updates

[Brad King]

On 08/29/2013 02:32 PM, Junio C Hamano wrote:
> But it may not be a bad idea to keep the callers dumb and have this
> function always sort, dedup, *and* fail inconsistent request.

I agree.  I was just starting to write the comment for update_refs
and it basically would have said "always use ref_update_sort and
check for duplicates first".  We might was well build it into the
function.  If some call site in the future wants to optimize a case
known to be sorted it can be refactored later.

Thanks,
-Brad

Re: [PATCH/RFC 6/7] refs: add update_refs for multiple simultaneous updates

[Brad King]

On 08/29/2013 02:20 PM, Brad King wrote:
> I wasn't happy with the asymmetry either but forgot to raise it in
> the cover letter.  We need a way to represent "old value not given"
> as different from "old value is_null_sha1".
[snip]
> Another approach is to add a dedicated member to struct ref_update.

FYI, I chose the latter approach for simplicity.  The member
can be folded into the flags by future refactoring if needed.

-Brad

[PATCH/RFC 7/7] update-ref: support multiple simultaneous updates

[Brad King]

Add a --stdin signature to read update instructions from standard input
and apply multiple ref updates and deletes together.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 Documentation/git-update-ref.txt |   19 +++++++-
 builtin/update-ref.c             |   93 +++++++++++++++++++++++++++++++++++++-
 2 files changed, 110 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
index 0df13ff..a79afe8 100644
--- a/Documentation/git-update-ref.txt
+++ b/Documentation/git-update-ref.txt
@@ -8,7 +8,7 @@ git-update-ref - Update the object name stored in a ref safely
 SYNOPSIS
 --------
 [verse]
-'git update-ref' [-m <reason>] (-d <ref> [<oldvalue>] | [--no-deref] <ref> <newvalue> [<oldvalue>])
+'git update-ref' [-m <reason>] (-d <ref> [<oldvalue>] | [--no-deref] <ref> <newvalue> [<oldvalue>] | --stdin)
 
 DESCRIPTION
 -----------
@@ -58,6 +58,23 @@ archive by creating a symlink tree).
 With `-d` flag, it deletes the named <ref> after verifying it
 still contains <oldvalue>.
 
+With `--stdin`, update-ref reads instructions from standard input
+and performs all modifications together.  Specify updates with
+lines of the form:
+
+	[ --no-deref SP ] <ref> SP <newvalue> [ SP <oldvalue> ] LF
+
+and deletes with lines of the form:
+
+	[ --no-deref SP ] -d SP <ref> [ SP <oldvalue> ] LF
+
+or as updates with 40 "0" as <newvalue>.  Blank lines are ignored.
+Lines of any other format or a repeated <ref> produce an error.
+If all <ref>s can be locked with matching <oldvalue>s
+simultaneously all modifications are performed.  Otherwise, no
+modifications are performed.  Note that while each individual
+<ref> is updated or deleted atomically, a concurrent reader may
+still see a subset of the modifications.
 
 Logging Updates
 ---------------
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 51d2684..2f0d34c 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -6,19 +6,102 @@
 static const char * const git_update_ref_usage[] = {
 	N_("git update-ref [options] -d <refname> [<oldval>]"),
 	N_("git update-ref [options]    <refname> <newval> [<oldval>]"),
+	N_("git update-ref [options] --stdin"),
 	NULL
 };
 
+static const char blank[] = " \t\r\n";
+
+static int updates_size;
+static int updates_count;
+static struct ref_update *updates;
+
+static void update_refs_stdin(const char *line)
+{
+	int delete = 0, i;
+	const char *c, *s, *oldvalue, *value[2] = {0,0};
+	struct ref_update *update;
+	c = line;
+
+	/* Skip blank lines: */
+	if (*c == '\n')
+		return;
+
+	/* Allocate a ref_update struct: */
+	if (updates_count == updates_size) {
+		updates_size = updates_size? updates_size*2 : 16;
+		updates = xrealloc(updates, sizeof(*updates)*updates_size);
+		memset(updates + updates_count, 0,
+		       sizeof(*updates)*(updates_size-updates_count));
+	}
+	update = &updates[updates_count++];
+
+	/* --no-deref SP */
+	if (strncmp(c, "--no-deref ", 11) == 0) {
+		c += 11;
+		update->flags |= REF_NODEREF;
+	}
+
+	/* -d SP */
+	if (strncmp(c, "-d ", 3) == 0) {
+		c += 3;
+		delete = 1;
+	}
+
+	/* <ref> */
+	s = c;
+	c = s + strcspn(s, blank);
+	update->ref_name = xstrndup(s, c-s);
+
+	/* [ SP <value> ]... */
+	for (i=0; i < 2; ++i) {
+		if (*c != ' ')
+			break;
+		++c;
+		s = c;
+		c = s + strcspn(s, blank);
+		value[i] = xstrndup(s, c-s);
+	}
+
+	if (*c && *c != '\n')
+		die("unrecognized input line: %s", line);
+
+	if (check_refname_format(update->ref_name, REFNAME_ALLOW_ONELEVEL))
+		die("invalid <ref> format on input line: %s", line);
+
+	if (delete) {
+		hashclr(update->new_sha1);
+		oldvalue = value[0];
+		if (value[1])
+			die("both <newvalue> and <oldvalue> on delete line: %s",
+			    line);
+	} else {
+		if (!value[0])
+			die("missing <newvalue> on update line: %s", line);
+		if (get_sha1(value[0], update->new_sha1))
+			die("invalid <newvalue> on update line: %s", line);
+		oldvalue = value[1];
+	}
+	if (oldvalue) {
+		update->old_sha1 = xmalloc(20);
+		if (get_sha1(oldvalue, update->old_sha1))
+			die("invalid <oldvalue> on %s line: %s",
+			    delete? "delete":"update", line);
+	}
+}
+
 int cmd_update_ref(int argc, const char **argv, const char *prefix)
 {
 	const char *refname, *oldval, *msg = NULL;
 	unsigned char sha1[20], oldsha1[20];
-	int delete = 0, no_deref = 0, flags = 0;
+	int delete = 0, no_deref = 0, read_stdin = 0, flags = 0;
+	char line[1000];
 	struct option options[] = {
 		OPT_STRING( 'm', NULL, &msg, N_("reason"), N_("reason of the update")),
 		OPT_BOOLEAN('d', NULL, &delete, N_("delete the reference")),
 		OPT_BOOLEAN( 0 , "no-deref", &no_deref,
 					N_("update <refname> not the one it points to")),
+		OPT_BOOLEAN( 0 , "stdin", &read_stdin, N_("read updates from stdin")),
 		OPT_END(),
 	};
 
@@ -28,6 +111,14 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)
 	if (msg && !*msg)
 		die("Refusing to perform update with empty message.");
 
+	if (read_stdin) {
+		if (delete || no_deref || argc > 0)
+			usage_with_options(git_update_ref_usage, options);
+		while (fgets(line, sizeof(line), stdin))
+			update_refs_stdin(line);
+		return update_refs(msg, updates, updates_count, DIE_ON_ERR);
+	}
+
 	if (delete) {
 		if (argc < 1 || argc > 2)
 			usage_with_options(git_update_ref_usage, options);
-- 
1.7.10.4

Re: [PATCH/RFC 7/7] update-ref: support multiple simultaneous updates

[Junio C Hamano]

Brad King <brad.king@kitware.com> writes:

> +	const char *c, *s, *oldvalue, *value[2] = {0,0};

This patch has many style issues of the same kind, lack of a SP at
places where there should be between operators and after comma.

Re: [PATCH/RFC 7/7] update-ref: support multiple simultaneous updates

[Brad King]

On 08/29/2013 02:34 PM, Junio C Hamano wrote:
> Brad King <brad.king@kitware.com> writes:
> 
>> +	const char *c, *s, *oldvalue, *value[2] = {0,0};
> 
> This patch has many style issues of the same kind, lack of a SP at
> places where there should be between operators and after comma.

Okay, I can fix those.  However, for this patch I'm particularly
interested in suggestions for the proposed stdin format.  Right
now it just looks like the command line, but it feels strange to
parse "-options" from a formatted input stream that is not an
options response file.

Thanks,
-Brad

Re: [PATCH/RFC 0/7] Multiple simultaneously locked ref updates

[Martin Fick]

On Thursday, August 29, 2013 08:11:48 am Brad King wrote:
> 
>    fatal: Unable to create '....lock': File exists.
>    If no other git process is currently running, this
> probably means a git process crashed in this repository
> earlier. Make sure no other git process is running and
> remove the file manually to continue.

I don't believe git currently tries to do any form of stale 
lock recovery since it is racy and unreliable (both single 
server or on a multi-server shared repo),


-Martin

Re: [PATCH/RFC 0/7] Multiple simultaneously locked ref updates

[Brad King]

On 08/29/2013 11:32 AM, Martin Fick wrote:
> On Thursday, August 29, 2013 08:11:48 am Brad King wrote:
>>
>>    fatal: Unable to create '....lock': File exists.
>>    If no other git process is currently running, this
>> probably means a git process crashed in this repository
>> earlier. Make sure no other git process is running and
>> remove the file manually to continue.
> 
> I don't believe git currently tries to do any form of stale 
> lock recovery since it is racy and unreliable (both single 
> server or on a multi-server shared repo),

Nor should it in this case.  I was saying that the front-end
needs to reject duplicate ref names from the stdin lines before
trying to lock the ref twice to avoid this message.  I'm asking
for a suggestion for existing data structure capabilities in
Git's source to efficiently detect the duplicate name.

-Brad

Re: [PATCH/RFC 0/7] Multiple simultaneously locked ref updates

[Junio C Hamano]

Brad King <brad.king@kitware.com> writes:

> Nor should it in this case.  I was saying that the front-end
> needs to reject duplicate ref names from the stdin lines before
> trying to lock the ref twice to avoid this message.

How about trying not to feed duplicates?

Re: [PATCH/RFC 0/7] Multiple simultaneously locked ref updates

[Brad King]

On 08/29/2013 12:21 PM, Junio C Hamano wrote:
> Brad King <brad.king@kitware.com> writes:
>> needs to reject duplicate ref names from the stdin lines before
>> trying to lock the ref twice to avoid this message.
> 
> How about trying not to feed duplicates?

Sure, perhaps it is simplest to push the responsibility on the user
to avoid duplicates.  However, the error message will need to be
re-worded to distinguish this case from a stale lock or competing
process since both locks may come from the same update-ref process.

Without checking the input for duplicates ourselves we cannot
distinguish these cases to provide a more informative error message.
However, such a check would add runtime overhead even for valid input.
If we prefer to avoid input validation then here is proposed new
wording for the lock failure message:

--------------------------------------------------------------------
fatal: Unable to create '....lock': File exists.

The lock file may exist because:
- another running git process already has the lock, or
- this process already has the lock because it was asked to
  update the same file multiple times simultaneously, or
- a stale lock is left from a git process that crashed earlier.
In the last case, make sure no other git process is running and
remove the file manually to continue.
--------------------------------------------------------------------

IIUC the message cannot say anything about a 'ref' because it is
used for other file type lock failures too.

Comments?
-Brad

Re: [PATCH/RFC 0/7] Multiple simultaneously locked ref updates

[Junio C Hamano]

Brad King <brad.king@kitware.com> writes:

> On 08/29/2013 12:21 PM, Junio C Hamano wrote:
>> Brad King <brad.king@kitware.com> writes:
>>> needs to reject duplicate ref names from the stdin lines before
>>> trying to lock the ref twice to avoid this message.
>> 
>> How about trying not to feed duplicates?
>
> Sure, perhaps it is simplest to push the responsibility on the user
> to avoid duplicates.

I didn't mean to force the caller of new "update-ref --stdin"; the
new code you wrote for it is what feeds the input to update_refs()
function, and that is one place you can make sure you do not lock
yourself out.

Besides, if you get two updates to the same ref from --stdin, you
would need to verify these are identical (i.e. updating to the same
new object name from the same old object name; otherwise the requests
are conflicting and do not make sense), so the code to collect the
requests from stdin needs to match potential duplicates anyway.

One clean way to do so may be to put an update request (old and new
sha1) in a structure, and use a string_list to hold list of refs,
with the util field pointing at the update request data.

> The lock file may exist because:
> - another running git process already has the lock, or
> - this process already has the lock because it was asked to
>   update the same file multiple times simultaneously, or
> - a stale lock is left from a git process that crashed earlier.
> In the last case, make sure no other git process is running and
> remove the file manually to continue.

The second case is like left hand not knowing what right hand is
doing, no?  It should not happen if we code it right.

> --------------------------------------------------------------------
>
> IIUC the message cannot say anything about a 'ref' because it is
> used for other file type lock failures too.
>
> Comments?
> -Brad

Re: [PATCH/RFC 0/7] Multiple simultaneously locked ref updates

[Brad King]

On 08/29/2013 02:07 PM, Junio C Hamano wrote:
> I didn't mean to force the caller of new "update-ref --stdin"; the
> new code you wrote for it is what feeds the input to update_refs()
> function, and that is one place you can make sure you do not lock
> yourself out.
> 
> Besides, if you get two updates to the same ref from --stdin, you
> would need to verify these are identical (i.e. updating to the same
> new object name from the same old object name; otherwise the requests
> are conflicting and do not make sense), so the code to collect the
> requests from stdin needs to match potential duplicates anyway.
> 
> One clean way to do so may be to put an update request (old and new
> sha1) in a structure, and use a string_list to hold list of refs,
> with the util field pointing at the update request data.
> 
>> - this process already has the lock because it was asked to
>>   update the same file multiple times simultaneously, or
> 
> The second case is like left hand not knowing what right hand is
> doing, no?  It should not happen if we code it right.

Yes.  All of the above is what I originally intended when asking
the question in the cover letter.  Using string_list and its util
field may be useful.  However, see also my response at
$gmane/233260 about how it may fold into sorting.

Thanks for the reviews!
-Brad

[PATCH v2 0/8] Multiple simultaneously locked ref updates

[Brad King]

Hi Folks,

Here is the second revision of a series to support locking multiple
refs at the same time to update all of them consistently.  The first
series can be found at $gmane/233260.  This revision is ready to
consider for integration.

Updates since the previous revision of the series:

* Incorporated style fixes and cleanups suggested by Junio.

* In patch 6, the new update_refs function now sorts the updates
  so that locks are acquired in a consistent order by competing
  processes.  Then it uses a simple linear scan to reject input
  containing duplicate refs (which are adjacent after sorting).
  Also, struct ref_update now has a symmetric representation
  for new_sha1 and old_sha1.

* In patch 7, I propose a new format for instructions read from
  standard input that is much more robust and extensible.

* Patch 8 is new and adds test cases covering new features
  and error cases.

-Brad

Brad King (8):
  reset: rename update_refs to reset_refs
  refs: report ref type from lock_any_ref_for_update
  refs: factor update_ref steps into helpers
  refs: factor delete_ref loose ref step into a helper
  refs: add function to repack without multiple refs
  refs: add update_refs for multiple simultaneous updates
  update-ref: support multiple simultaneous updates
  update-ref: add test cases covering --stdin signature

 Documentation/git-update-ref.txt |   21 +++-
 branch.c                         |    2 +-
 builtin/commit.c                 |    2 +-
 builtin/fetch.c                  |    3 +-
 builtin/receive-pack.c           |    3 +-
 builtin/reflog.c                 |    2 +-
 builtin/replace.c                |    2 +-
 builtin/reset.c                  |    4 +-
 builtin/tag.c                    |    2 +-
 builtin/update-ref.c             |  121 +++++++++++++++++++++-
 fast-import.c                    |    2 +-
 refs.c                           |  203 +++++++++++++++++++++++++++++++++----
 refs.h                           |   16 ++-
 sequencer.c                      |    3 +-
 t/t1400-update-ref.sh            |  206 ++++++++++++++++++++++++++++++++++++++
 15 files changed, 558 insertions(+), 34 deletions(-)

-- 
1.7.10.4

[PATCH v2 1/8] reset: rename update_refs to reset_refs

[Brad King]

The function resets refs rather than doing arbitrary updates.
Rename it to allow a future general-purpose update_refs function
to be added.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 builtin/reset.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/builtin/reset.c b/builtin/reset.c
index afa6e02..789ee48 100644
--- a/builtin/reset.c
+++ b/builtin/reset.c
@@ -219,7 +219,7 @@ static const char **parse_args(const char **argv, const char *prefix, const char
 	return argv[0] ? get_pathspec(prefix, argv) : NULL;
 }
 
-static int update_refs(const char *rev, const unsigned char *sha1)
+static int reset_refs(const char *rev, const unsigned char *sha1)
 {
 	int update_ref_status;
 	struct strbuf msg = STRBUF_INIT;
@@ -350,7 +350,7 @@ int cmd_reset(int argc, const char **argv, const char *prefix)
 	if (!pathspec && !unborn) {
 		/* Any resets without paths update HEAD to the head being
 		 * switched to, saving the previous head in ORIG_HEAD before. */
-		update_ref_status = update_refs(rev, sha1);
+		update_ref_status = reset_refs(rev, sha1);
 
 		if (reset_type == HARD && !update_ref_status && !quiet)
 			print_new_head_line(lookup_commit_reference(sha1));
-- 
1.7.10.4

[PATCH v2 2/8] refs: report ref type from lock_any_ref_for_update

[Brad King]

Expose lock_ref_sha1_basic's type_p argument to callers of
lock_any_ref_for_update.  Update all call sites to ignore it by passing
NULL for now.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 branch.c               |    2 +-
 builtin/commit.c       |    2 +-
 builtin/fetch.c        |    3 ++-
 builtin/receive-pack.c |    3 ++-
 builtin/reflog.c       |    2 +-
 builtin/replace.c      |    2 +-
 builtin/tag.c          |    2 +-
 fast-import.c          |    2 +-
 refs.c                 |    7 ++++---
 refs.h                 |    2 +-
 sequencer.c            |    3 ++-
 11 files changed, 17 insertions(+), 13 deletions(-)

diff --git a/branch.c b/branch.c
index c5c6984..f2d383f 100644
--- a/branch.c
+++ b/branch.c
@@ -291,7 +291,7 @@ void create_branch(const char *head,
 	hashcpy(sha1, commit->object.sha1);
 
 	if (!dont_change_ref) {
-		lock = lock_any_ref_for_update(ref.buf, NULL, 0);
+		lock = lock_any_ref_for_update(ref.buf, NULL, 0, NULL);
 		if (!lock)
 			die_errno(_("Failed to lock ref for update"));
 	}
diff --git a/builtin/commit.c b/builtin/commit.c
index 10acc53..be08f41 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1618,7 +1618,7 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 					   !current_head
 					   ? NULL
 					   : current_head->object.sha1,
-					   0);
+					   0, NULL);
 
 	nl = strchr(sb.buf, '\n');
 	if (nl)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index d784b2e..28e4025 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -246,7 +246,8 @@ static int s_update_ref(const char *action,
 		rla = default_rla.buf;
 	snprintf(msg, sizeof(msg), "%s: %s", rla, action);
 	lock = lock_any_ref_for_update(ref->name,
-				       check_old ? ref->old_sha1 : NULL, 0);
+				       check_old ? ref->old_sha1 : NULL,
+				       0, NULL);
 	if (!lock)
 		return errno == ENOTDIR ? STORE_REF_ERROR_DF_CONFLICT :
 					  STORE_REF_ERROR_OTHER;
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index e3eb5fc..a323070 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -524,7 +524,8 @@ static const char *update(struct command *cmd)
 		return NULL; /* good */
 	}
 	else {
-		lock = lock_any_ref_for_update(namespaced_name, old_sha1, 0);
+		lock = lock_any_ref_for_update(namespaced_name, old_sha1,
+					       0, NULL);
 		if (!lock) {
 			rp_error("failed to lock %s", name);
 			return "failed to lock";
diff --git a/builtin/reflog.c b/builtin/reflog.c
index 54184b3..28d756a 100644
--- a/builtin/reflog.c
+++ b/builtin/reflog.c
@@ -366,7 +366,7 @@ static int expire_reflog(const char *ref, const unsigned char *sha1, int unused,
 	 * we take the lock for the ref itself to prevent it from
 	 * getting updated.
 	 */
-	lock = lock_any_ref_for_update(ref, sha1, 0);
+	lock = lock_any_ref_for_update(ref, sha1, 0, NULL);
 	if (!lock)
 		return error("cannot lock ref '%s'", ref);
 	log_file = git_pathdup("logs/%s", ref);
diff --git a/builtin/replace.c b/builtin/replace.c
index 59d3115..1ecae8d 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -105,7 +105,7 @@ static int replace_object(const char *object_ref, const char *replace_ref,
 	else if (!force)
 		die("replace ref '%s' already exists", ref);
 
-	lock = lock_any_ref_for_update(ref, prev, 0);
+	lock = lock_any_ref_for_update(ref, prev, 0, NULL);
 	if (!lock)
 		die("%s: cannot lock the ref", ref);
 	if (write_ref_sha1(lock, repl, NULL) < 0)
diff --git a/builtin/tag.c b/builtin/tag.c
index af3af3f..2c867d2 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -577,7 +577,7 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
 	if (annotate)
 		create_tag(object, tag, &buf, &opt, prev, object);
 
-	lock = lock_any_ref_for_update(ref.buf, prev, 0);
+	lock = lock_any_ref_for_update(ref.buf, prev, 0, NULL);
 	if (!lock)
 		die(_("%s: cannot lock the ref"), ref.buf);
 	if (write_ref_sha1(lock, object, NULL) < 0)
diff --git a/fast-import.c b/fast-import.c
index 23f625f..5c329f6 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -1678,7 +1678,7 @@ static int update_branch(struct branch *b)
 		return 0;
 	if (read_ref(b->name, old_sha1))
 		hashclr(old_sha1);
-	lock = lock_any_ref_for_update(b->name, old_sha1, 0);
+	lock = lock_any_ref_for_update(b->name, old_sha1, 0, NULL);
 	if (!lock)
 		return error("Unable to lock %s", b->name);
 	if (!force_update && !is_null_sha1(old_sha1)) {
diff --git a/refs.c b/refs.c
index 7922261..c69fd68 100644
--- a/refs.c
+++ b/refs.c
@@ -2121,11 +2121,12 @@ struct ref_lock *lock_ref_sha1(const char *refname, const unsigned char *old_sha
 }
 
 struct ref_lock *lock_any_ref_for_update(const char *refname,
-					 const unsigned char *old_sha1, int flags)
+					 const unsigned char *old_sha1,
+					 int flags, int *type_p)
 {
 	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL))
 		return NULL;
-	return lock_ref_sha1_basic(refname, old_sha1, flags, NULL);
+	return lock_ref_sha1_basic(refname, old_sha1, flags, type_p);
 }
 
 /*
@@ -3174,7 +3175,7 @@ int update_ref(const char *action, const char *refname,
 		int flags, enum action_on_err onerr)
 {
 	static struct ref_lock *lock;
-	lock = lock_any_ref_for_update(refname, oldval, flags);
+	lock = lock_any_ref_for_update(refname, oldval, flags, NULL);
 	if (!lock) {
 		const char *str = "Cannot lock the ref '%s'.";
 		switch (onerr) {
diff --git a/refs.h b/refs.h
index 9e5db3a..2cd307a 100644
--- a/refs.h
+++ b/refs.h
@@ -137,7 +137,7 @@ extern struct ref_lock *lock_ref_sha1(const char *refname, const unsigned char *
 #define REF_NODEREF	0x01
 extern struct ref_lock *lock_any_ref_for_update(const char *refname,
 						const unsigned char *old_sha1,
-						int flags);
+						int flags, int *type_p);
 
 /** Close the file descriptor owned by a lock and return the status */
 extern int close_ref(struct ref_lock *lock);
diff --git a/sequencer.c b/sequencer.c
index 351548f..06e52b4 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -279,7 +279,8 @@ static int fast_forward_to(const unsigned char *to, const unsigned char *from,
 	read_cache();
 	if (checkout_fast_forward(from, to, 1))
 		exit(1); /* the callee should have complained already */
-	ref_lock = lock_any_ref_for_update("HEAD", unborn ? null_sha1 : from, 0);
+	ref_lock = lock_any_ref_for_update("HEAD", unborn ? null_sha1 : from,
+					   0, NULL);
 	strbuf_addf(&sb, "%s: fast-forward", action_name(opts));
 	ret = write_ref_sha1(ref_lock, to, sb.buf);
 	strbuf_release(&sb);
-- 
1.7.10.4

[PATCH v2 3/8] refs: factor update_ref steps into helpers

[Brad King]

Factor the lock and write steps and error handling into helper functions
update_ref_lock and update_ref_write to allow later use elsewhere.
Expose lock_any_ref_for_update's type_p to update_ref_lock callers.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 refs.c |   28 +++++++++++++++++++++++-----
 1 file changed, 23 insertions(+), 5 deletions(-)

diff --git a/refs.c b/refs.c
index c69fd68..2e755b4 100644
--- a/refs.c
+++ b/refs.c
@@ -3170,12 +3170,13 @@ int for_each_reflog(each_ref_fn fn, void *cb_data)
 	return retval;
 }
 
-int update_ref(const char *action, const char *refname,
-		const unsigned char *sha1, const unsigned char *oldval,
-		int flags, enum action_on_err onerr)
+static struct ref_lock *update_ref_lock(const char *refname,
+					const unsigned char *oldval,
+					int flags, int *type_p,
+					enum action_on_err onerr)
 {
 	static struct ref_lock *lock;
-	lock = lock_any_ref_for_update(refname, oldval, flags, NULL);
+	lock = lock_any_ref_for_update(refname, oldval, flags, type_p);
 	if (!lock) {
 		const char *str = "Cannot lock the ref '%s'.";
 		switch (onerr) {
@@ -3183,8 +3184,14 @@ int update_ref(const char *action, const char *refname,
 		case DIE_ON_ERR: die(str, refname); break;
 		case QUIET_ON_ERR: break;
 		}
-		return 1;
 	}
+	return lock;
+}
+
+static int update_ref_write(const char *action, const char *refname,
+			    const unsigned char *sha1, struct ref_lock *lock,
+			    enum action_on_err onerr)
+{
 	if (write_ref_sha1(lock, sha1, action) < 0) {
 		const char *str = "Cannot update the ref '%s'.";
 		switch (onerr) {
@@ -3197,6 +3204,17 @@ int update_ref(const char *action, const char *refname,
 	return 0;
 }
 
+int update_ref(const char *action, const char *refname,
+	       const unsigned char *sha1, const unsigned char *oldval,
+	       int flags, enum action_on_err onerr)
+{
+	static struct ref_lock *lock;
+	lock = update_ref_lock(refname, oldval, flags, 0, onerr);
+	if (!lock)
+		return 1;
+	return update_ref_write(action, refname, sha1, lock, onerr);
+}
+
 struct ref *find_ref_by_name(const struct ref *list, const char *name)
 {
 	for ( ; list; list = list->next)
-- 
1.7.10.4

Re: [PATCH v2 3/8] refs: factor update_ref steps into helpers

[Junio C Hamano]

Brad King <brad.king@kitware.com> writes:

> Factor the lock and write steps and error handling into helper functions
> update_ref_lock and update_ref_write to allow later use elsewhere.
> Expose lock_any_ref_for_update's type_p to update_ref_lock callers.
>
> Signed-off-by: Brad King <brad.king@kitware.com>
> ---
>  refs.c |   28 +++++++++++++++++++++++-----
>  1 file changed, 23 insertions(+), 5 deletions(-)
>
> diff --git a/refs.c b/refs.c
> index c69fd68..2e755b4 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -3170,12 +3170,13 @@ int for_each_reflog(each_ref_fn fn, void *cb_data)
>  	return retval;
>  }
>  
> -int update_ref(const char *action, const char *refname,
> -		const unsigned char *sha1, const unsigned char *oldval,
> -		int flags, enum action_on_err onerr)
> +static struct ref_lock *update_ref_lock(const char *refname,
> +					const unsigned char *oldval,
> +					int flags, int *type_p,
> +					enum action_on_err onerr)
>  {
>  	static struct ref_lock *lock;

Not the fault of this patch, as the original update_ref() had it
this way, but it is not necessary to keep the value of this variable
across invocations.  Let's drop "static" from here, and also the
corresponding variable in the new update_ref().

Will locally tweak while queuing.

> -	lock = lock_any_ref_for_update(refname, oldval, flags, NULL);
> +	lock = lock_any_ref_for_update(refname, oldval, flags, type_p);
>  	if (!lock) {
>  		const char *str = "Cannot lock the ref '%s'.";
>  		switch (onerr) {
> @@ -3183,8 +3184,14 @@ int update_ref(const char *action, const char *refname,
>  		case DIE_ON_ERR: die(str, refname); break;
>  		case QUIET_ON_ERR: break;
>  		}
> -		return 1;
>  	}
> +	return lock;
> +}
> +
> +static int update_ref_write(const char *action, const char *refname,
> +			    const unsigned char *sha1, struct ref_lock *lock,
> +			    enum action_on_err onerr)
> +{
>  	if (write_ref_sha1(lock, sha1, action) < 0) {
>  		const char *str = "Cannot update the ref '%s'.";
>  		switch (onerr) {
> @@ -3197,6 +3204,17 @@ int update_ref(const char *action, const char *refname,
>  	return 0;
>  }
>  
> +int update_ref(const char *action, const char *refname,
> +	       const unsigned char *sha1, const unsigned char *oldval,
> +	       int flags, enum action_on_err onerr)
> +{
> +	static struct ref_lock *lock;
> +	lock = update_ref_lock(refname, oldval, flags, 0, onerr);
> +	if (!lock)
> +		return 1;
> +	return update_ref_write(action, refname, sha1, lock, onerr);
> +}
> +
>  struct ref *find_ref_by_name(const struct ref *list, const char *name)
>  {
>  	for ( ; list; list = list->next)

Re: [PATCH v2 3/8] refs: factor update_ref steps into helpers

[Brad King]

On 09/01/2013 02:08 AM, Junio C Hamano wrote:
> Brad King <brad.king@kitware.com> writes:
>>  	static struct ref_lock *lock;
> 
> Not the fault of this patch, as the original update_ref() had it
> this way, but it is not necessary to keep the value of this variable
> across invocations.  Let's drop "static" from here, and also the
> corresponding variable in the new update_ref().

Fixed in next revision.

Thanks,
-Brad

[PATCH v2 4/8] refs: factor delete_ref loose ref step into a helper

[Brad King]

Factor loose ref deletion into helper function delete_ref_loose to allow
later use elsewhere.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 refs.c |   22 +++++++++++++++-------
 1 file changed, 15 insertions(+), 7 deletions(-)

diff --git a/refs.c b/refs.c
index 2e755b4..5dd86ee 100644
--- a/refs.c
+++ b/refs.c
@@ -2450,14 +2450,9 @@ static int repack_without_ref(const char *refname)
 	return commit_packed_refs();
 }
 
-int delete_ref(const char *refname, const unsigned char *sha1, int delopt)
+static int delete_ref_loose(struct ref_lock *lock, int flag)
 {
-	struct ref_lock *lock;
-	int err, i = 0, ret = 0, flag = 0;
-
-	lock = lock_ref_sha1_basic(refname, sha1, delopt, &flag);
-	if (!lock)
-		return 1;
+	int err, i, ret = 0;
 	if (!(flag & REF_ISPACKED) || flag & REF_ISSYMREF) {
 		/* loose */
 		i = strlen(lock->lk->filename) - 5; /* .lock */
@@ -2468,6 +2463,19 @@ int delete_ref(const char *refname, const unsigned char *sha1, int delopt)
 
 		lock->lk->filename[i] = '.';
 	}
+	return ret;
+}
+
+int delete_ref(const char *refname, const unsigned char *sha1, int delopt)
+{
+	struct ref_lock *lock;
+	int ret = 0, flag = 0;
+
+	lock = lock_ref_sha1_basic(refname, sha1, delopt, &flag);
+	if (!lock)
+		return 1;
+	ret |= delete_ref_loose(lock, flag);
+
 	/* removing the loose one could have resurrected an earlier
 	 * packed one.  Also, if it was not loose we need to repack
 	 * without it.
-- 
1.7.10.4

Re: [PATCH v2 4/8] refs: factor delete_ref loose ref step into a helper

[Michael Haggerty]

On 08/30/2013 08:12 PM, Brad King wrote:
> Factor loose ref deletion into helper function delete_ref_loose to allow
> later use elsewhere.
> 
> Signed-off-by: Brad King <brad.king@kitware.com>
> ---
>  refs.c |   22 +++++++++++++++-------
>  1 file changed, 15 insertions(+), 7 deletions(-)
> 
> diff --git a/refs.c b/refs.c
> index 2e755b4..5dd86ee 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -2450,14 +2450,9 @@ static int repack_without_ref(const char *refname)
>  	return commit_packed_refs();
>  }
>  
> -int delete_ref(const char *refname, const unsigned char *sha1, int delopt)
> +static int delete_ref_loose(struct ref_lock *lock, int flag)
>  {
> -	struct ref_lock *lock;
> -	int err, i = 0, ret = 0, flag = 0;
> -
> -	lock = lock_ref_sha1_basic(refname, sha1, delopt, &flag);
> -	if (!lock)
> -		return 1;
> +	int err, i, ret = 0;
>  	if (!(flag & REF_ISPACKED) || flag & REF_ISSYMREF) {
>  		/* loose */
>  		i = strlen(lock->lk->filename) - 5; /* .lock */
> @@ -2468,6 +2463,19 @@ int delete_ref(const char *refname, const unsigned char *sha1, int delopt)
>  
>  		lock->lk->filename[i] = '.';
>  	}
> +	return ret;
> +}
> +


At first glance it is odd that delete_ref_loose() takes a (struct
ref_lock *) argument but only actually uses lock->lk->filename.  But I
guess that the function is so specific to the contents of struct
ref_lock and indeed struct lock_file that it wouldn't make sense to pass
it only the filename attribute.  So OK.

Given that ret is only returned, you could restore the filename before
the if statement and replace the ret variable with an immediate return
statement:

static int delete_ref_loose(struct ref_lock *lock, int flag)
{
	if (!(flag & REF_ISPACKED) || flag & REF_ISSYMREF) {
		/* loose */
		int err, i = strlen(lock->lk->filename) - 5; /* .lock */

		lock->lk->filename[i] = 0;
		err = unlink_or_warn(lock->lk->filename);
		lock->lk->filename[i] = '.';
		if (err && errno != ENOENT)
			return 1;
	}
	return 0;
}

> +int delete_ref(const char *refname, const unsigned char *sha1, int delopt)
> +{
> +	struct ref_lock *lock;
> +	int ret = 0, flag = 0;
> +
> +	lock = lock_ref_sha1_basic(refname, sha1, delopt, &flag);
> +	if (!lock)
> +		return 1;
> +	ret |= delete_ref_loose(lock, flag);
> +
>  	/* removing the loose one could have resurrected an earlier
>  	 * packed one.  Also, if it was not loose we need to repack
>  	 * without it.
> 

Otherwise looks good.

Michael

-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/

Re: [PATCH v2 4/8] refs: factor delete_ref loose ref step into a helper

[Brad King]

On 08/31/2013 12:30 PM, Michael Haggerty wrote:
> Given that ret is only returned, you could restore the filename before
> the if statement and replace the ret variable with an immediate return
> statement:

Good idea.  Fixed in next revision.

Thanks,
-Brad

[PATCH v2 5/8] refs: add function to repack without multiple refs

[Brad King]

Generalize repack_without_ref as repack_without_refs to support a list
of refs and implement the former in terms of the latter.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 refs.c |   29 ++++++++++++++++++++++-------
 1 file changed, 22 insertions(+), 7 deletions(-)

diff --git a/refs.c b/refs.c
index 5dd86ee..3bcd26e 100644
--- a/refs.c
+++ b/refs.c
@@ -2414,25 +2414,35 @@ static int curate_packed_ref_fn(struct ref_entry *entry, void *cb_data)
 	return 0;
 }
 
-static int repack_without_ref(const char *refname)
+static int repack_without_refs(const char **refnames, int n)
 {
 	struct ref_dir *packed;
 	struct string_list refs_to_delete = STRING_LIST_INIT_DUP;
 	struct string_list_item *ref_to_delete;
+	int i, removed = 0;
+
+	/* Look for a packed ref: */
+	for (i = 0; i < n; ++i)
+		if (get_packed_ref(refnames[i]))
+			break;
 
-	if (!get_packed_ref(refname))
-		return 0; /* refname does not exist in packed refs */
+	/* Avoid locking if we have nothing to do: */
+	if (i == n)
+		return 0; /* no refname exists in packed refs */
 
 	if (lock_packed_refs(0)) {
 		unable_to_lock_error(git_path("packed-refs"), errno);
-		return error("cannot delete '%s' from packed refs", refname);
+		return error("cannot delete '%s' from packed refs", refnames[i]);
 	}
 	packed = get_packed_refs(&ref_cache);
 
-	/* Remove refname from the cache: */
-	if (remove_entry(packed, refname) == -1) {
+	/* Remove refnames from the cache: */
+	for (i = 0; i < n; ++i)
+		if (remove_entry(packed, refnames[i]) != -1)
+			removed = 1;
+	if (!removed) {
 		/*
-		 * The packed entry disappeared while we were
+		 * All packed entries disappeared while we were
 		 * acquiring the lock.
 		 */
 		rollback_packed_refs();
@@ -2450,6 +2460,11 @@ static int repack_without_ref(const char *refname)
 	return commit_packed_refs();
 }
 
+static int repack_without_ref(const char *refname)
+{
+	return repack_without_refs(&refname, 1);
+}
+
 static int delete_ref_loose(struct ref_lock *lock, int flag)
 {
 	int err, i, ret = 0;
-- 
1.7.10.4

[PATCH v2 6/8] refs: add update_refs for multiple simultaneous updates

[Brad King]

Add 'struct ref_update' to encode the information needed to update or
delete a ref (name, new sha1, optional old sha1, no-deref flag).  Add
function 'update_refs' accepting an array of updates to perform.  First
sort the input array to order locks consistently everywhere and reject
multiple updates to the same ref.  Then acquire locks on all refs with
verified old values.  Then update or delete all refs accordingly.  Fail
if any one lock cannot be obtained or any one old value does not match.

Though the refs themeselves cannot be modified together in a single
atomic transaction, this function does enable some useful semantics.
For example, a caller may create a new branch starting from the head of
another branch and rewind the original branch at the same time.  This
transfers ownership of commits between branches without risk of losing
commits added to the original branch by a concurrent process, or risk of
a concurrent process creating the new branch first.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 refs.c |  121 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 refs.h |   14 ++++++++
 2 files changed, 135 insertions(+)

diff --git a/refs.c b/refs.c
index 3bcd26e..901a38a 100644
--- a/refs.c
+++ b/refs.c
@@ -3238,6 +3238,127 @@ int update_ref(const char *action, const char *refname,
 	return update_ref_write(action, refname, sha1, lock, onerr);
 }
 
+static int ref_update_compare(const void *r1, const void *r2)
+{
+	struct ref_update *u1 = (struct ref_update *)(r1);
+	struct ref_update *u2 = (struct ref_update *)(r2);
+	int ret;
+	ret = strcmp(u1->ref_name, u2->ref_name);
+	if (ret)
+		return ret;
+	ret = hashcmp(u1->new_sha1, u2->new_sha1);
+	if (ret)
+		return ret;
+	ret = hashcmp(u1->old_sha1, u2->old_sha1);
+	if (ret)
+		return ret;
+	ret = u1->flags - u2->flags;
+	if (ret)
+		return ret;
+	return u1->have_old - u2->have_old;
+}
+
+static int ref_update_reject_duplicates(struct ref_update *updates, int n,
+					enum action_on_err onerr)
+{
+	int i;
+	for (i = 1; i < n; ++i)
+		if (!strcmp(updates[i - 1].ref_name, updates[i].ref_name))
+			break;
+	if (i < n) {
+		const char *str = "Multiple updates for ref '%s' not allowed.";
+		switch (onerr) {
+		case MSG_ON_ERR: error(str, updates[i].ref_name); break;
+		case DIE_ON_ERR: die(str, updates[i].ref_name); break;
+		case QUIET_ON_ERR: break;
+		}
+		return 1;
+	}
+	return 0;
+}
+
+int update_refs(const char *action, const struct ref_update *updates_orig,
+		int n, enum action_on_err onerr)
+{
+	int ret = 0, delnum = 0, i;
+	struct ref_update *updates;
+	int *types;
+	struct ref_lock **locks;
+	const char **delnames;
+
+	if (!updates_orig || !n)
+		return 0;
+
+	/* Allocate work space: */
+	updates = xmalloc(sizeof(struct ref_update) * n);
+	types = xmalloc(sizeof(int) * n);
+	locks = xmalloc(sizeof(struct ref_lock *) * n);
+	delnames = xmalloc(sizeof(const char *) * n);
+
+	/* Copy, sort, and reject duplicate refs: */
+	memcpy(updates, updates_orig, sizeof(struct ref_update) * n);
+	qsort(updates, n, sizeof(struct ref_update), ref_update_compare);
+	if (ref_update_reject_duplicates(updates, n, onerr)) {
+		free(updates);
+		free(types);
+		free(locks);
+		free(delnames);
+		return 1;
+	}
+
+	/* Acquire all locks while verifying old values: */
+	for (i = 0; i < n; ++i) {
+		locks[i] = update_ref_lock(updates[i].ref_name,
+					   (updates[i].have_old ?
+					    updates[i].old_sha1 : NULL),
+					   updates[i].flags,
+					   &types[i], onerr);
+		if (!locks[i])
+			break;
+	}
+
+	/* Abort if we did not get all locks: */
+	if (i < n) {
+		while (--i >= 0)
+			unlock_ref(locks[i]);
+		free(updates);
+		free(types);
+		free(locks);
+		free(delnames);
+		return 1;
+	}
+
+	/* Perform updates first so live commits remain referenced: */
+	for (i = 0; i < n; ++i)
+		if (!is_null_sha1(updates[i].new_sha1)) {
+			ret |= update_ref_write(action,
+						updates[i].ref_name,
+						updates[i].new_sha1,
+						locks[i], onerr);
+			locks[i] = 0; /* freed by update_ref_write */
+		}
+
+	/* Perform deletes now that updates are safely completed: */
+	for (i = 0; i < n; ++i)
+		if (locks[i]) {
+			delnames[delnum++] = locks[i]->ref_name;
+			ret |= delete_ref_loose(locks[i], types[i]);
+		}
+	ret |= repack_without_refs(delnames, delnum);
+	for (i = 0; i < delnum; ++i)
+		unlink_or_warn(git_path("logs/%s", delnames[i]));
+	clear_loose_ref_cache(&ref_cache);
+	for (i = 0; i < n; ++i)
+		if (locks[i])
+			unlock_ref(locks[i]);
+
+	free(updates);
+	free(types);
+	free(locks);
+	free(delnames);
+	return ret;
+}
+
 struct ref *find_ref_by_name(const struct ref *list, const char *name)
 {
 	for ( ; list; list = list->next)
diff --git a/refs.h b/refs.h
index 2cd307a..a8a7cc6 100644
--- a/refs.h
+++ b/refs.h
@@ -214,6 +214,20 @@ int update_ref(const char *action, const char *refname,
 		const unsigned char *sha1, const unsigned char *oldval,
 		int flags, enum action_on_err onerr);
 
+struct ref_update {
+	const char *ref_name;
+	unsigned char new_sha1[20];
+	unsigned char old_sha1[20];
+	int flags;
+	int have_old;
+};
+
+/**
+ * Lock all refs and then perform all modifications.
+ */
+int update_refs(const char *action, const struct ref_update *updates,
+		int n, enum action_on_err onerr);
+
 extern int parse_hide_refs_config(const char *var, const char *value, const char *);
 extern int ref_is_hidden(const char *);
 
-- 
1.7.10.4

Re: [PATCH v2 6/8] refs: add update_refs for multiple simultaneous updates

[Michael Haggerty]

On 08/30/2013 08:12 PM, Brad King wrote:
> Add 'struct ref_update' to encode the information needed to update or
> delete a ref (name, new sha1, optional old sha1, no-deref flag).  Add
> function 'update_refs' accepting an array of updates to perform.  First
> sort the input array to order locks consistently everywhere and reject
> multiple updates to the same ref.  Then acquire locks on all refs with
> verified old values.  Then update or delete all refs accordingly.  Fail
> if any one lock cannot be obtained or any one old value does not match.
> 
> Though the refs themeselves cannot be modified together in a single

s/themeselves/themselves/

> atomic transaction, this function does enable some useful semantics.
> For example, a caller may create a new branch starting from the head of
> another branch and rewind the original branch at the same time.  This
> transfers ownership of commits between branches without risk of losing
> commits added to the original branch by a concurrent process, or risk of
> a concurrent process creating the new branch first.
> 
> Signed-off-by: Brad King <brad.king@kitware.com>
> ---
>  refs.c |  121 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>  refs.h |   14 ++++++++
>  2 files changed, 135 insertions(+)
> 
> diff --git a/refs.c b/refs.c
> index 3bcd26e..901a38a 100644
> --- a/refs.c
> +++ b/refs.c
> @@ -3238,6 +3238,127 @@ int update_ref(const char *action, const char *refname,
>  	return update_ref_write(action, refname, sha1, lock, onerr);
>  }
>  
> +static int ref_update_compare(const void *r1, const void *r2)
> +{
> +	struct ref_update *u1 = (struct ref_update *)(r1);
> +	struct ref_update *u2 = (struct ref_update *)(r2);

If you declare u1 and u2 to be "const struct ref_update *" (i.e., add
"const"), then you have const correctness and don't need the explicit
casts.  (And the parentheses around r1 and r2 are superfluous in any case.)

> +	int ret;

Style: we usually put a blank line between variable declarations and the
first line of code.

> +	ret = strcmp(u1->ref_name, u2->ref_name);
> +	if (ret)
> +		return ret;
> +	ret = hashcmp(u1->new_sha1, u2->new_sha1);
> +	if (ret)
> +		return ret;
> +	ret = hashcmp(u1->old_sha1, u2->old_sha1);
> +	if (ret)
> +		return ret;
> +	ret = u1->flags - u2->flags;
> +	if (ret)
> +		return ret;
> +	return u1->have_old - u2->have_old;
> +}

Is there a need to compare more than ref_name?  If two entries are found
with the same name, then ref_update_reject_duplicates() will error out
anyway.  So the relative order among entries with the same name is
irrelevant.  I think it would be OK to return 0 for any entries with the
same ref_name, even if they differ in other fields.

> +
> +static int ref_update_reject_duplicates(struct ref_update *updates, int n,
> +					enum action_on_err onerr)
> +{
> +	int i;
> +	for (i = 1; i < n; ++i)
> +		if (!strcmp(updates[i - 1].ref_name, updates[i].ref_name))
> +			break;

The error handling code could be right here instead of the "break"
statement, removing the need for the "if" conditional.

> +	if (i < n) {
> +		const char *str = "Multiple updates for ref '%s' not allowed.";
> +		switch (onerr) {
> +		case MSG_ON_ERR: error(str, updates[i].ref_name); break;
> +		case DIE_ON_ERR: die(str, updates[i].ref_name); break;
> +		case QUIET_ON_ERR: break;
> +		}
> +		return 1;
> +	}
> +	return 0;
> +}
> +
> +int update_refs(const char *action, const struct ref_update *updates_orig,
> +		int n, enum action_on_err onerr)
> +{
> +	int ret = 0, delnum = 0, i;
> +	struct ref_update *updates;
> +	int *types;
> +	struct ref_lock **locks;
> +	const char **delnames;
> +
> +	if (!updates_orig || !n)
> +		return 0;
> +
> +	/* Allocate work space: */
> +	updates = xmalloc(sizeof(struct ref_update) * n);

It seems preferred here to write

	updates = xmalloc(sizeof(*updates) * n);

as this will continue to work if the type of updates is ever changed.
Similarly for the next lines.

> +	types = xmalloc(sizeof(int) * n);
> +	locks = xmalloc(sizeof(struct ref_lock *) * n);
> +	delnames = xmalloc(sizeof(const char *) * n);

An alternative to managing separate arrays to hold types and locks would
be to include the scratch space in struct ref_update and document it
"for internal use only; need not be initialized by caller".  On the one
hand it's ugly to cruft up the "interface" with internal implementation
details; on the other hand there is precedent for this sort of thing
(e.g., ref_lock::force_write or lock_file::on_list) and it would
simplify the code.

> +
> +	/* Copy, sort, and reject duplicate refs: */
> +	memcpy(updates, updates_orig, sizeof(struct ref_update) * n);
> +	qsort(updates, n, sizeof(struct ref_update), ref_update_compare);

You could save some space and memory shuffling (during memcpy() and
qsort()) if you would declare "updates" to be an array of pointers to
"struct ref_update" rather than an array of structs.  Sorting could then
be done by moving pointers around instead of moving the structs.  This
would also make it easier for update_refs() to pass information about
the references back to its caller, should that ever be needed.

But I suppose that n will usually be small, so this suggestion can be
considered optional.

> +	if (ref_update_reject_duplicates(updates, n, onerr)) {
> +		free(updates);
> +		free(types);
> +		free(locks);
> +		free(delnames);
> +		return 1;
> +	}
> +
> +	/* Acquire all locks while verifying old values: */
> +	for (i = 0; i < n; ++i) {
> +		locks[i] = update_ref_lock(updates[i].ref_name,
> +					   (updates[i].have_old ?
> +					    updates[i].old_sha1 : NULL),
> +					   updates[i].flags,
> +					   &types[i], onerr);
> +		if (!locks[i])
> +			break;

The error handling code could go here in place of the "break",
especially if it's only "ret = 1; goto label;" as suggested below.

> +	}
> +
> +	/* Abort if we did not get all locks: */
> +	if (i < n) {
> +		while (--i >= 0)
> +			unlock_ref(locks[i]);
> +		free(updates);
> +		free(types);
> +		free(locks);
> +		free(delnames);
> +		return 1;
> +	}
> +
> +	/* Perform updates first so live commits remain referenced: */
> +	for (i = 0; i < n; ++i)
> +		if (!is_null_sha1(updates[i].new_sha1)) {
> +			ret |= update_ref_write(action,
> +						updates[i].ref_name,
> +						updates[i].new_sha1,
> +						locks[i], onerr);
> +			locks[i] = 0; /* freed by update_ref_write */
> +		}
> +

Hmmm, if one of the calls to update_ref_write() fails, would it be safer
to abort the rest of the work (especially the reference deletions)?

> +	/* Perform deletes now that updates are safely completed: */
> +	for (i = 0; i < n; ++i)
> +		if (locks[i]) {
> +			delnames[delnum++] = locks[i]->ref_name;
> +			ret |= delete_ref_loose(locks[i], types[i]);
> +		}
> +	ret |= repack_without_refs(delnames, delnum);
> +	for (i = 0; i < delnum; ++i)
> +		unlink_or_warn(git_path("logs/%s", delnames[i]));
> +	clear_loose_ref_cache(&ref_cache);
> +	for (i = 0; i < n; ++i)
> +		if (locks[i])
> +			unlock_ref(locks[i]);
> +
> +	free(updates);
> +	free(types);
> +	free(locks);
> +	free(delnames);
> +	return ret;
> +}

There's a lot of duplicated cleanup code in the function.  If you put a
label before the final for loop, and if you initialize the locks array
to zeros (e.g., by using xcalloc()), then the three exits could all
share the same code "ret = 1; goto cleanup;".

> +
>  struct ref *find_ref_by_name(const struct ref *list, const char *name)
>  {
>  	for ( ; list; list = list->next)
> diff --git a/refs.h b/refs.h
> index 2cd307a..a8a7cc6 100644
> --- a/refs.h
> +++ b/refs.h
> @@ -214,6 +214,20 @@ int update_ref(const char *action, const char *refname,
>  		const unsigned char *sha1, const unsigned char *oldval,
>  		int flags, enum action_on_err onerr);
>  
> +struct ref_update {
> +	const char *ref_name;
> +	unsigned char new_sha1[20];
> +	unsigned char old_sha1[20];
> +	int flags;
> +	int have_old;
> +};

Please document this structure, especially the relationship between
have_old and old_sha1.

> +
> +/**
> + * Lock all refs and then perform all modifications.
> + */
> +int update_refs(const char *action, const struct ref_update *updates,
> +		int n, enum action_on_err onerr);
> +
>  extern int parse_hide_refs_config(const char *var, const char *value, const char *);
>  extern int ref_is_hidden(const char *);
>  
> 

Overall, thanks; it looks good.  I think this change is useful by itself
and is also a good start towards implementing true reference
transactions (which I think will be necessary pretty soon, at least for
some applications).  I will write another email discussing how your
changes are related to some changes that I have been working on lately.

Michael

-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/

Re: [PATCH v2 6/8] refs: add update_refs for multiple simultaneous updates

[Brad King]

On 08/31/2013 02:19 PM, Michael Haggerty wrote:
> s/themeselves/themselves/

Fixed.

>> +	struct ref_update *u1 = (struct ref_update *)(r1);
>> +	struct ref_update *u2 = (struct ref_update *)(r2);
> 
> If you declare u1 and u2 to be "const struct ref_update *" (i.e., add
> "const"), then you have const correctness and don't need the explicit
> casts.  (And the parentheses around r1 and r2 are superfluous in any case.)

Fixed.

>> +	ret = strcmp(u1->ref_name, u2->ref_name);
> 
> Is there a need to compare more than ref_name?  If two entries are found
> with the same name, then ref_update_reject_duplicates() will error out

Junio mentioned possibility of auto-combining identical entries which would
need full ordering.  I think that can be added later so for now we can sort
only by ref name.  Thanks.

>> +		if (!strcmp(updates[i - 1].ref_name, updates[i].ref_name))
>> +			break;
> 
> The error handling code could be right here instead of the "break"
> statement, removing the need for the "if" conditional.

Fixed.

>> +	/* Allocate work space: */
>> +	updates = xmalloc(sizeof(struct ref_update) * n);
> 
> It seems preferred here to write
> 
> 	updates = xmalloc(sizeof(*updates) * n);
> 
> as this will continue to work if the type of updates is ever changed.

Yes, thanks.

> Similarly for the next lines.
> 
>> +	types = xmalloc(sizeof(int) * n);
>> +	locks = xmalloc(sizeof(struct ref_lock *) * n);
>> +	delnames = xmalloc(sizeof(const char *) * n);
> 
> An alternative to managing separate arrays to hold types and locks would
> be to include the scratch space in struct ref_update and document it
> "for internal use only; need not be initialized by caller".  On the one
> hand it's ugly to cruft up the "interface" with internal implementation
> details; on the other hand there is precedent for this sort of thing
> (e.g., ref_lock::force_write or lock_file::on_list) and it would
> simplify the code.

I think the "goto cleanup" reorganization simplifies the code enough
to not need this.  After changing "updates" to an array of pointers
it needs to be separate so we can sort.  Also "delnames" needs to be
a separate array to pass to repack_without_refs.

>> +	/* Copy, sort, and reject duplicate refs: */
>> +	memcpy(updates, updates_orig, sizeof(struct ref_update) * n);
>> +	qsort(updates, n, sizeof(struct ref_update), ref_update_compare);
> 
> You could save some space and memory shuffling (during memcpy() and
> qsort()) if you would declare "updates" to be an array of pointers to
> "struct ref_update" rather than an array of structs.  Sorting could then
> be done by moving pointers around instead of moving the structs.  This
> would also make it easier for update_refs() to pass information about
> the references back to its caller, should that ever be needed.

Good idea.  Changed in next revision.

>> +			ret |= update_ref_write(action,
>> +						updates[i].ref_name,
>> +						updates[i].new_sha1,
>> +						locks[i], onerr);
>> +			locks[i] = 0; /* freed by update_ref_write */
>> +		}
>> +
> 
> Hmmm, if one of the calls to update_ref_write() fails, would it be safer
> to abort the rest of the work (especially the reference deletions)?

Yes.  Since we already have the lock at this point something must be
going pretty wrong if this fails so it is best to abort altogether.

>> +	free(updates);
>> +	free(types);
>> +	free(locks);
>> +	free(delnames);
>> +	return ret;
>> +}
> 
> There's a lot of duplicated cleanup code in the function.  If you put a
> label before the final for loop, and if you initialize the locks array
> to zeros (e.g., by using xcalloc()), then the three exits could all
> share the same code "ret = 1; goto cleanup;".

Done, thanks.

>> +struct ref_update {
> 
> Please document this structure, especially the relationship between
> have_old and old_sha1.

Done.  I also moved it to the top of the header just under ref_lock
so it can be used by other APIs later.

Thanks,
-Brad

Re: [PATCH v2 6/8] refs: add update_refs for multiple simultaneous updates

[Junio C Hamano]

Brad King <brad.king@kitware.com> writes:

> Add 'struct ref_update' to encode the information needed to update or
> delete a ref (name, new sha1, optional old sha1, no-deref flag).  Add
> function 'update_refs' accepting an array of updates to perform.  First
> sort the input array to order locks consistently everywhere and reject
> multiple updates to the same ref.  Then acquire locks on all refs with
> verified old values.  Then update or delete all refs accordingly.  Fail
> if any one lock cannot be obtained or any one old value does not match.

OK.  The code releases the locks it acquired so far when it fails,
which is good.

> Though the refs themeselves cannot be modified together in a single

"themselves".

> atomic transaction, this function does enable some useful semantics.
> For example, a caller may create a new branch starting from the head of
> another branch and rewind the original branch at the same time.  This
> transfers ownership of commits between branches without risk of losing
> commits added to the original branch by a concurrent process, or risk of
> a concurrent process creating the new branch first.

> +static int ref_update_compare(const void *r1, const void *r2)
> +{
> +	struct ref_update *u1 = (struct ref_update *)(r1);
> +	struct ref_update *u2 = (struct ref_update *)(r2);
> +	int ret;

Let's have a blank line between the end of decls and the beginning
of the body here.

> +	ret = strcmp(u1->ref_name, u2->ref_name);
> +	if (ret)
> +		return ret;
> +	ret = hashcmp(u1->new_sha1, u2->new_sha1);
> +	if (ret)
> +		return ret;
> +	ret = hashcmp(u1->old_sha1, u2->old_sha1);
> +	if (ret)
> +		return ret;
> +	ret = u1->flags - u2->flags;
> +	if (ret)
> +		return ret;
> +	return u1->have_old - u2->have_old;
> +}

I notice that we are using an array of structures and letting qsort
swap 50~64 bytes of data, instead of sorting an array of pointers,
each element of which points at a structure.  This may not matter
unless we are asked to update thousands at once, so I think it is OK
for now.

> +static int ref_update_reject_duplicates(struct ref_update *updates, int n,
> +					enum action_on_err onerr)
> +{
> +	int i;
> +	for (i = 1; i < n; ++i)
> +		if (!strcmp(updates[i - 1].ref_name, updates[i].ref_name))
> +			break;

Optionally we could silently dedup multiple identical updates and
not fail it in ref-update-reject-duplicates.  But that does not have
to be done until we find people's script would benefit from such a
nicety.

By the way, unless there is a strong reason not to do so,
post-increment "i++" (and pre-decrement "--i", if you use it) is the
norm around here.  Especially in places like the third part of a
for(;;) loop where people are used to see "i++", breaking the idiom
makes readers wonder if there is something else going on.

> +	/* Perform updates first so live commits remain referenced: */
> +	for (i = 0; i < n; ++i)
> +		if (!is_null_sha1(updates[i].new_sha1)) {
> +			ret |= update_ref_write(action,
> +						updates[i].ref_name,
> +						updates[i].new_sha1,
> +						locks[i], onerr);
> +			locks[i] = 0; /* freed by update_ref_write */

I think what is assigned here is a NULL pointer.

Will locally tweak while queuing.  Thanks.

Re: [PATCH v2 6/8] refs: add update_refs for multiple simultaneous updates

[Brad King]

On 09/01/2013 02:08 AM, Junio C Hamano wrote:
>> Though the refs themeselves cannot be modified together in a single
> 
> "themselves".

Fixed.

> I notice that we are using an array of structures and letting qsort
> swap 50~64 bytes of data

Michael suggested this too, so fixed.

> Optionally we could silently dedup multiple identical updates and
> not fail it in ref-update-reject-duplicates.  But that does not have
> to be done until we find people's script would benefit from such a
> nicety.

We can always be less strict about input later, so I'd like to keep
the implementation simpler for now.

> By the way, unless there is a strong reason not to do so,
> post-increment "i++" (and pre-decrement "--i", if you use it) is the
> norm around here.

Okay, fixed in entire series.

>> +			locks[i] = 0; /* freed by update_ref_write */
> 
> I think what is assigned here is a NULL pointer.

Yes, fixed.

Thanks,
-Brad

Re: [PATCH v2 6/8] refs: add update_refs for multiple simultaneous updates

[Michael Haggerty]

On 09/02/2013 07:20 PM, Brad King wrote:
> On 09/01/2013 02:08 AM, Junio C Hamano wrote:
>>> Though the refs themeselves cannot be modified together in a single
>>
>> "themselves".
> 
> Fixed.
> 
>> I notice that we are using an array of structures and letting qsort
>> swap 50~64 bytes of data
> 
> Michael suggested this too, so fixed.

Hmmm, I see that you changed the signature of update_refs() to take an
array of pointers.  My suggestion was unclear, but I didn't mean that
the function signature had to be changed.  Rather, I meant that *within*
the function, you could have created an array of pointers to the
structures in the input array and thereafter accessed it via the pointers:

int update_refs(const char *action, const struct ref_update *updates_orig,
		int n, enum action_on_err onerr)
{
	[...]
	struct ref_update **updates;

	[...]
	updates = xcalloc(n, sizeof(*updates));
	for (i = 0; i < n; i++)
		updates[i] = &updates_orig[i];
	[...]
}

However, your approach is also fine.  It will typically involve more
malloc()s but smaller memcpy()s (i.e., via ALLOC_GROW()) at the caller,
and since usually the number of ref_updates being done at one time will
be limited anyway, I don't see a reason to prefer one version over the
other.

Thanks for making the change.

Michael

-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/

Re: [PATCH v2 6/8] refs: add update_refs for multiple simultaneous updates

[Brad King]

On 09/03/2013 12:43 AM, Michael Haggerty wrote:
> Hmmm, I see that you changed the signature of update_refs() to take an
> array of pointers.  My suggestion was unclear, but I didn't mean that
> the function signature had to be changed.
[snip]
> However, your approach is also fine.

Okay.  Thanks for reviewing!

-Brad

[PATCH v2 7/8] update-ref: support multiple simultaneous updates

[Brad King]

Add a --stdin signature to read update instructions from standard input
and apply multiple ref updates together.  Use an input format that
supports any update that could be specified via the command-line,
including object names like 'branch:path with space'.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 Documentation/git-update-ref.txt |   21 ++++++-
 builtin/update-ref.c             |  121 +++++++++++++++++++++++++++++++++++++-
 2 files changed, 140 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
index 0df13ff..295d0bb 100644
--- a/Documentation/git-update-ref.txt
+++ b/Documentation/git-update-ref.txt
@@ -8,7 +8,7 @@ git-update-ref - Update the object name stored in a ref safely
 SYNOPSIS
 --------
 [verse]
-'git update-ref' [-m <reason>] (-d <ref> [<oldvalue>] | [--no-deref] <ref> <newvalue> [<oldvalue>])
+'git update-ref' [-m <reason>] (-d <ref> [<oldvalue>] | [--no-deref] <ref> <newvalue> [<oldvalue>] | --stdin)
 
 DESCRIPTION
 -----------
@@ -58,6 +58,25 @@ archive by creating a symlink tree).
 With `-d` flag, it deletes the named <ref> after verifying it
 still contains <oldvalue>.
 
+With `--stdin`, update-ref reads instructions from standard input and
+performs all modifications together.  Empty lines are ignored.
+Each non-empty line is parsed as whitespace-separated arguments.
+Use single-quotes to enclose whitespace and backslashes and an
+unquoted backslash to escape a single quote.  Specify updates with
+lines of the form:
+
+	[--no-deref] [--] <ref> <newvalue> [<oldvalue>]
+
+Lines of any other format or a repeated <ref> produce an error.
+Specify a zero <newvalue> to delete a ref and/or a zero <oldvalue>
+to make sure that a ref not exist.  Use either 40 "0" or the
+empty string (written as '') to specify a zero value.
+
+If all <ref>s can be locked with matching <oldvalue>s
+simultaneously all modifications are performed.  Otherwise, no
+modifications are performed.  Note that while each individual
+<ref> is updated or deleted atomically, a concurrent reader may
+still see a subset of the modifications.
 
 Logging Updates
 ---------------
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 51d2684..eb8db85 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -6,19 +6,129 @@
 static const char * const git_update_ref_usage[] = {
 	N_("git update-ref [options] -d <refname> [<oldval>]"),
 	N_("git update-ref [options]    <refname> <newval> [<oldval>]"),
+	N_("git update-ref [options] --stdin"),
 	NULL
 };
 
+static const char blank[] = " \t\r\n";
+
+static int updates_size;
+static int updates_count;
+static struct ref_update *updates;
+
+static const char* update_refs_stdin_next_arg(const char* next,
+					      struct strbuf *arg)
+{
+	/* Skip leading whitespace: */
+	while (isspace(*next))
+		++next;
+
+	/* Return NULL when no argument is found: */
+	if (!*next)
+		return NULL;
+
+	/* Parse the argument: */
+	strbuf_reset(arg);
+	for (;;) {
+		char c = *next;
+		if (!c || isspace(c))
+			break;
+		++next;
+		if (c == '\'') {
+			size_t len = strcspn(next, "'");
+			if (!next[len])
+				die("unterminated single-quote: '%s", next);
+			strbuf_add(arg, next, len);
+			next += len + 1;
+			continue;
+		}
+		if (c == '\\') {
+			if (*next == '\'')
+				c = *next++;
+			else
+				die("unquoted backslash not escaping "
+				    "single-quote: \\%s", next);
+		}
+		strbuf_addch(arg, c);
+	}
+	return next;
+}
+
+static void update_refs_stdin(const char *line)
+{
+	int options = 1, flags = 0, argc = 0;
+	char *argv[3] = {0, 0, 0};
+	struct strbuf arg = STRBUF_INIT;
+	struct ref_update *update;
+	const char *next = line;
+
+	/* Skip blank lines: */
+	if (!line[0])
+		return;
+
+	/* Parse arguments on this line: */
+	while ((next = update_refs_stdin_next_arg(next, &arg)) != NULL) {
+		if (options && arg.buf[0] == '-')
+			if (!strcmp(arg.buf, "--no-deref"))
+				flags |= REF_NODEREF;
+			else if (!strcmp(arg.buf, "--"))
+				options = 0;
+			else
+				die("unknown option %s", arg.buf);
+		else if (argc >= 3)
+			die("too many arguments on line: %s", line);
+		else {
+			argv[argc++] = xstrdup(arg.buf);
+			options = 0;
+		}
+	}
+	strbuf_release(&arg);
+
+	/* Allocate and zero-init a struct ref_update: */
+	if (updates_count == updates_size) {
+		updates_size = updates_size ? (updates_size * 2) : 16;
+		updates = xrealloc(updates, sizeof(*updates) * updates_size);
+		memset(updates + updates_count, 0,
+		       sizeof(*updates) * (updates_size - updates_count));
+	}
+	update = &updates[updates_count++];
+	update->flags = flags;
+
+	/* Set the update ref_name: */
+	if (!argv[0])
+		die("no ref on line: %s", line);
+	if (check_refname_format(argv[0], REFNAME_ALLOW_ONELEVEL))
+		die("invalid ref format on line: %s", line);
+	update->ref_name = argv[0];
+	argv[0] = 0;
+
+	/* Set the update new_sha1 and, if specified, old_sha1: */
+	if (!argv[1])
+		die("missing new value on line: %s", line);
+	if (*argv[1] && get_sha1(argv[1], update->new_sha1))
+		die("invalid new value on line: %s", line);
+	if (argv[2]) {
+		update->have_old = 1;
+		if (*argv[2] && get_sha1(argv[2], update->old_sha1))
+			die("invalid old value on line: %s", line);
+	}
+
+	while (argc > 0)
+		free(argv[--argc]);
+}
+
 int cmd_update_ref(int argc, const char **argv, const char *prefix)
 {
 	const char *refname, *oldval, *msg = NULL;
 	unsigned char sha1[20], oldsha1[20];
-	int delete = 0, no_deref = 0, flags = 0;
+	int delete = 0, no_deref = 0, read_stdin = 0, flags = 0;
+	struct strbuf line = STRBUF_INIT;
 	struct option options[] = {
 		OPT_STRING( 'm', NULL, &msg, N_("reason"), N_("reason of the update")),
 		OPT_BOOLEAN('d', NULL, &delete, N_("delete the reference")),
 		OPT_BOOLEAN( 0 , "no-deref", &no_deref,
 					N_("update <refname> not the one it points to")),
+		OPT_BOOLEAN( 0 , "stdin", &read_stdin, N_("read updates from stdin")),
 		OPT_END(),
 	};
 
@@ -28,6 +138,15 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)
 	if (msg && !*msg)
 		die("Refusing to perform update with empty message.");
 
+	if (read_stdin) {
+		if (delete || no_deref || argc > 0)
+			usage_with_options(git_update_ref_usage, options);
+		while (strbuf_getline(&line, stdin, '\n') != EOF)
+			update_refs_stdin(line.buf);
+		strbuf_release(&line);
+		return update_refs(msg, updates, updates_count, DIE_ON_ERR);
+	}
+
 	if (delete) {
 		if (argc < 1 || argc > 2)
 			usage_with_options(git_update_ref_usage, options);
-- 
1.7.10.4

Re: [PATCH v2 7/8] update-ref: support multiple simultaneous updates

[Junio C Hamano]

Brad King <brad.king@kitware.com> writes:

> Add a --stdin signature to read update instructions from standard input
> and apply multiple ref updates together.  Use an input format that
> supports any update that could be specified via the command-line,
> including object names like 'branch:path with space'.
>
> Signed-off-by: Brad King <brad.king@kitware.com>
> ---
>  Documentation/git-update-ref.txt |   21 ++++++-
>  builtin/update-ref.c             |  121 +++++++++++++++++++++++++++++++++++++-
>  2 files changed, 140 insertions(+), 2 deletions(-)
>
> diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
> index 0df13ff..295d0bb 100644
> --- a/Documentation/git-update-ref.txt
> +++ b/Documentation/git-update-ref.txt
> @@ -8,7 +8,7 @@ git-update-ref - Update the object name stored in a ref safely
>  SYNOPSIS
>  --------
>  [verse]
> -'git update-ref' [-m <reason>] (-d <ref> [<oldvalue>] | [--no-deref] <ref> <newvalue> [<oldvalue>])
> +'git update-ref' [-m <reason>] (-d <ref> [<oldvalue>] | [--no-deref] <ref> <newvalue> [<oldvalue>] | --stdin)
>  
>  DESCRIPTION
>  -----------
> @@ -58,6 +58,25 @@ archive by creating a symlink tree).
>  With `-d` flag, it deletes the named <ref> after verifying it
>  still contains <oldvalue>.
>  
> +With `--stdin`, update-ref reads instructions from standard input and
> +performs all modifications together.  Empty lines are ignored.
> +Each non-empty line is parsed as whitespace-separated arguments.
> +Use single-quotes to enclose whitespace and backslashes and an
> +unquoted backslash to escape a single quote.

That is somewhat unusual.

When we need to deal with arbitrary strings (like pathnames), other
parts of the system usually give the user two interfaces, --stdin
with and without -z, and the strings are C-quoted when run without
the -z option, and terminated with NUL when run with the -z option.

> +Specify updates with
> +lines of the form:
> +
> +	[--no-deref] [--] <ref> <newvalue> [<oldvalue>]

What is -- doing here?  refs given to update-ref begin with refs/
(otherwise it is HEAD), no?

Re: [PATCH v2 7/8] update-ref: support multiple simultaneous updates

[Brad King]

On 08/30/2013 06:51 PM, Junio C Hamano wrote:
> Brad King <brad.king@kitware.com> writes:
>> +With `--stdin`, update-ref reads instructions from standard input and
>> +performs all modifications together.  Empty lines are ignored.
>> +Each non-empty line is parsed as whitespace-separated arguments.
>> +Use single-quotes to enclose whitespace and backslashes and an
>> +unquoted backslash to escape a single quote.
> 
> That is somewhat unusual.
> 
> When we need to deal with arbitrary strings (like pathnames), other
> parts of the system usually give the user two interfaces, --stdin
> with and without -z, and the strings are C-quoted when run without
> the -z option, and terminated with NUL when run with the -z option.

Great, this was the kind of suggestion I was looking for in the original
PATCH/RFC cover letter.  Thanks.  I'll start with the C-quoted version
and think about adding -z once we've agreed on that format.

>> +Specify updates with
>> +lines of the form:
>> +
>> +	[--no-deref] [--] <ref> <newvalue> [<oldvalue>]
> 
> What is -- doing here?  refs given to update-ref begin with refs/
> (otherwise it is HEAD), no?

The existing update-ref command line can be used to create all kinds
of refs, even starting in "-".  I didn't want to add any restriction
in the stdin format.  I'm not opposed to it though.

Thanks,
-Brad

Re: [PATCH v2 7/8] update-ref: support multiple simultaneous updates

[Michael Haggerty]

On 08/30/2013 08:12 PM, Brad King wrote:
> Add a --stdin signature to read update instructions from standard input
> and apply multiple ref updates together.  Use an input format that
> supports any update that could be specified via the command-line,
> including object names like 'branch:path with space'.
> 
> Signed-off-by: Brad King <brad.king@kitware.com>
> ---
>  Documentation/git-update-ref.txt |   21 ++++++-
>  builtin/update-ref.c             |  121 +++++++++++++++++++++++++++++++++++++-
>  2 files changed, 140 insertions(+), 2 deletions(-)
> 
> diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
> index 0df13ff..295d0bb 100644
> --- a/Documentation/git-update-ref.txt
> +++ b/Documentation/git-update-ref.txt
> @@ -8,7 +8,7 @@ git-update-ref - Update the object name stored in a ref safely
>  SYNOPSIS
>  --------
>  [verse]
> -'git update-ref' [-m <reason>] (-d <ref> [<oldvalue>] | [--no-deref] <ref> <newvalue> [<oldvalue>])
> +'git update-ref' [-m <reason>] (-d <ref> [<oldvalue>] | [--no-deref] <ref> <newvalue> [<oldvalue>] | --stdin)
>  
>  DESCRIPTION
>  -----------
> @@ -58,6 +58,25 @@ archive by creating a symlink tree).
>  With `-d` flag, it deletes the named <ref> after verifying it
>  still contains <oldvalue>.
>  
> +With `--stdin`, update-ref reads instructions from standard input and
> +performs all modifications together.  Empty lines are ignored.
> +Each non-empty line is parsed as whitespace-separated arguments.
> +Use single-quotes to enclose whitespace and backslashes and an
> +unquoted backslash to escape a single quote.  Specify updates with
> +lines of the form:
> +
> +	[--no-deref] [--] <ref> <newvalue> [<oldvalue>]
> +
> +Lines of any other format or a repeated <ref> produce an error.
> +Specify a zero <newvalue> to delete a ref and/or a zero <oldvalue>
> +to make sure that a ref not exist.  Use either 40 "0" or the
> +empty string (written as '') to specify a zero value.
> +
> +If all <ref>s can be locked with matching <oldvalue>s
> +simultaneously all modifications are performed.  Otherwise, no

Comma after "simultaneously".

> +modifications are performed.  Note that while each individual
> +<ref> is updated or deleted atomically, a concurrent reader may
> +still see a subset of the modifications.
>  
>  Logging Updates
>  ---------------
> diff --git a/builtin/update-ref.c b/builtin/update-ref.c
> index 51d2684..eb8db85 100644
> --- a/builtin/update-ref.c
> +++ b/builtin/update-ref.c
> @@ -6,19 +6,129 @@
>  static const char * const git_update_ref_usage[] = {
>  	N_("git update-ref [options] -d <refname> [<oldval>]"),
>  	N_("git update-ref [options]    <refname> <newval> [<oldval>]"),
> +	N_("git update-ref [options] --stdin"),
>  	NULL
>  };
>  
> +static const char blank[] = " \t\r\n";
> +
> +static int updates_size;
> +static int updates_count;
> +static struct ref_update *updates;
> +
> +static const char* update_refs_stdin_next_arg(const char* next,
> +					      struct strbuf *arg)
> +{
> +	/* Skip leading whitespace: */
> +	while (isspace(*next))
> +		++next;
> +
> +	/* Return NULL when no argument is found: */
> +	if (!*next)
> +		return NULL;
> +
> +	/* Parse the argument: */
> +	strbuf_reset(arg);
> +	for (;;) {
> +		char c = *next;
> +		if (!c || isspace(c))
> +			break;
> +		++next;
> +		if (c == '\'') {
> +			size_t len = strcspn(next, "'");

I agree with Junio that your quoting rules are peculiar.

> +			if (!next[len])
> +				die("unterminated single-quote: '%s", next);
> +			strbuf_add(arg, next, len);
> +			next += len + 1;
> +			continue;
> +		}
> +		if (c == '\\') {
> +			if (*next == '\'')
> +				c = *next++;
> +			else
> +				die("unquoted backslash not escaping "
> +				    "single-quote: \\%s", next);
> +		}
> +		strbuf_addch(arg, c);
> +	}
> +	return next;
> +}
> +
> +static void update_refs_stdin(const char *line)
> +{
> +	int options = 1, flags = 0, argc = 0;
> +	char *argv[3] = {0, 0, 0};
> +	struct strbuf arg = STRBUF_INIT;
> +	struct ref_update *update;
> +	const char *next = line;
> +
> +	/* Skip blank lines: */
> +	if (!line[0])
> +		return;
> +
> +	/* Parse arguments on this line: */
> +	while ((next = update_refs_stdin_next_arg(next, &arg)) != NULL) {
> +		if (options && arg.buf[0] == '-')
> +			if (!strcmp(arg.buf, "--no-deref"))
> +				flags |= REF_NODEREF;
> +			else if (!strcmp(arg.buf, "--"))
> +				options = 0;
> +			else
> +				die("unknown option %s", arg.buf);
> +		else if (argc >= 3)
> +			die("too many arguments on line: %s", line);
> +		else {
> +			argv[argc++] = xstrdup(arg.buf);
> +			options = 0;
> +		}
> +	}
> +	strbuf_release(&arg);
> +
> +	/* Allocate and zero-init a struct ref_update: */
> +	if (updates_count == updates_size) {
> +		updates_size = updates_size ? (updates_size * 2) : 16;
> +		updates = xrealloc(updates, sizeof(*updates) * updates_size);
> +		memset(updates + updates_count, 0,
> +		       sizeof(*updates) * (updates_size - updates_count));
> +	}

Here you can use ARRAY_GROW().  See

    Documentation/technical/api-allocation-growing.txt

> +	update = &updates[updates_count++];
> +	update->flags = flags;
> +
> +	/* Set the update ref_name: */
> +	if (!argv[0])
> +		die("no ref on line: %s", line);
> +	if (check_refname_format(argv[0], REFNAME_ALLOW_ONELEVEL))
> +		die("invalid ref format on line: %s", line);
> +	update->ref_name = argv[0];
> +	argv[0] = 0;
> +
> +	/* Set the update new_sha1 and, if specified, old_sha1: */
> +	if (!argv[1])
> +		die("missing new value on line: %s", line);
> +	if (*argv[1] && get_sha1(argv[1], update->new_sha1))
> +		die("invalid new value on line: %s", line);
> +	if (argv[2]) {
> +		update->have_old = 1;
> +		if (*argv[2] && get_sha1(argv[2], update->old_sha1))
> +			die("invalid old value on line: %s", line);
> +	}
> +
> +	while (argc > 0)
> +		free(argv[--argc]);
> +}
> +
>  int cmd_update_ref(int argc, const char **argv, const char *prefix)
>  {
>  	const char *refname, *oldval, *msg = NULL;
>  	unsigned char sha1[20], oldsha1[20];
> -	int delete = 0, no_deref = 0, flags = 0;
> +	int delete = 0, no_deref = 0, read_stdin = 0, flags = 0;
> +	struct strbuf line = STRBUF_INIT;
>  	struct option options[] = {
>  		OPT_STRING( 'm', NULL, &msg, N_("reason"), N_("reason of the update")),
>  		OPT_BOOLEAN('d', NULL, &delete, N_("delete the reference")),
>  		OPT_BOOLEAN( 0 , "no-deref", &no_deref,
>  					N_("update <refname> not the one it points to")),
> +		OPT_BOOLEAN( 0 , "stdin", &read_stdin, N_("read updates from stdin")),
>  		OPT_END(),
>  	};
>  
> @@ -28,6 +138,15 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)
>  	if (msg && !*msg)
>  		die("Refusing to perform update with empty message.");
>  
> +	if (read_stdin) {
> +		if (delete || no_deref || argc > 0)
> +			usage_with_options(git_update_ref_usage, options);
> +		while (strbuf_getline(&line, stdin, '\n') != EOF)
> +			update_refs_stdin(line.buf);
> +		strbuf_release(&line);
> +		return update_refs(msg, updates, updates_count, DIE_ON_ERR);
> +	}
> +
>  	if (delete) {
>  		if (argc < 1 || argc > 2)
>  			usage_with_options(git_update_ref_usage, options);
> 


-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/

Re: [PATCH v2 7/8] update-ref: support multiple simultaneous updates

[Brad King]

On 08/31/2013 02:42 PM, Michael Haggerty wrote:
> On 08/30/2013 08:12 PM, Brad King wrote:
>> +If all <ref>s can be locked with matching <oldvalue>s
>> +simultaneously all modifications are performed.  Otherwise, no
> 
> Comma after "simultaneously".

Fixed.

> I agree with Junio that your quoting rules are peculiar.

I won't disagree.  That's why I asked for suggestions in the
original PATCH/RFC cover letter ;)

>> +	/* Allocate and zero-init a struct ref_update: */
> 
> Here you can use ARRAY_GROW().  See
> 
>     Documentation/technical/api-allocation-growing.txt

Fixed.

Thanks,
-Brad

[PATCH v2 8/8] update-ref: add test cases covering --stdin signature

[Brad King]

Extend t/t1400-update-ref.sh to cover cases using the --stdin option.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 t/t1400-update-ref.sh |  206 +++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 206 insertions(+)

diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index e415ee0..9fd03fc 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -302,4 +302,210 @@ test_expect_success \
 	'git cat-file blob master@{2005-05-26 23:42}:F (expect OTHER)' \
 	'test OTHER = $(git cat-file blob "master@{2005-05-26 23:42}:F")'
 
+a=refs/heads/a
+b=refs/heads/b
+c=refs/heads/c
+z=0000000000000000000000000000000000000000
+e="''"
+
+test_expect_success 'stdin works with no input' '
+	rm -f stdin &&
+	touch stdin &&
+	git update-ref --stdin < stdin &&
+	git rev-parse --verify -q $m
+'
+
+test_expect_success 'stdin fails with bad line lines' '
+	echo " " > stdin &&
+	test_must_fail git update-ref --stdin < stdin 2> err &&
+	grep "fatal: no ref on line:  " err &&
+	echo "--" > stdin &&
+	test_must_fail git update-ref --stdin < stdin 2> err &&
+	grep "fatal: no ref on line: --" err &&
+	echo "--bad-option" > stdin &&
+	test_must_fail git update-ref --stdin < stdin 2> err &&
+	grep "fatal: unknown option --bad-option" err &&
+	echo "-\'"'"' $a $m" > stdin &&
+	test_must_fail git update-ref --stdin < stdin 2> err &&
+	grep "fatal: unknown option -'"'"'" err &&
+	echo "~a $m" > stdin &&
+	test_must_fail git update-ref --stdin < stdin 2> err &&
+	grep "fatal: invalid ref format on line: ~a $m" err &&
+	echo "$a '"'"'master" > stdin &&
+	test_must_fail git update-ref --stdin < stdin 2> err &&
+	grep "fatal: unterminated single-quote: '"'"'master" err &&
+	echo "$a \master" > stdin &&
+	test_must_fail git update-ref --stdin < stdin 2> err &&
+	grep "fatal: unquoted backslash not escaping single-quote: \\\\master" err &&
+	echo "$a $m $m $m" > stdin &&
+	test_must_fail git update-ref --stdin < stdin 2> err &&
+	grep "fatal: too many arguments on line: $a $m $m $m" err &&
+	echo "$a" > stdin &&
+	test_must_fail git update-ref --stdin < stdin 2> err &&
+	grep "fatal: missing new value on line: $a" err
+'
+
+test_expect_success 'stdin fails with duplicate refs' '
+	echo "$a $m" > stdin &&
+	echo "$b $m" >> stdin &&
+	echo "$a $m" >> stdin &&
+	test_must_fail git update-ref --stdin < stdin 2> err &&
+	grep "fatal: Multiple updates for ref '"'"'$a'"'"' not allowed." err
+'
+
+test_expect_success 'stdin create ref works with no old value' '
+	echo "$a $m" > stdin &&
+	git update-ref --stdin < stdin &&
+	git rev-parse $m > expect &&
+	git rev-parse $a > actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin create ref works with zero old value' '
+	echo "$b $m $z" > stdin &&
+	git update-ref --stdin < stdin &&
+	git rev-parse $m > expect &&
+	git rev-parse $b > actual &&
+	test_cmp expect actual &&
+	git update-ref -d $b &&
+	echo "$b $m $e" > stdin &&
+	git update-ref --stdin < stdin &&
+	git rev-parse $m > expect &&
+	git rev-parse $b > actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin create ref fails with wrong old value' '
+	echo "$c $m $m~1" > stdin &&
+	test_must_fail git update-ref --stdin < stdin 2> err &&
+	grep "fatal: Cannot lock the ref '"'"'$c'"'"'" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin create ref fails with bad old value' '
+	echo "$c $m does-not-exist" > stdin &&
+	test_must_fail git update-ref --stdin < stdin 2> err &&
+	grep "fatal: invalid old value on line: $c $m does-not-exist" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin create ref fails with bad new value' '
+	echo "$c does-not-exist" > stdin &&
+	test_must_fail git update-ref --stdin < stdin 2> err &&
+	grep "fatal: invalid new value on line: $c does-not-exist" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin update ref works with right old value' '
+	echo "$b $m~1 $m" > stdin &&
+	git update-ref --stdin < stdin &&
+	git rev-parse $m~1 > expect &&
+	git rev-parse $b > actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin update ref fails with wrong old value' '
+	echo "$b $m~1 $m" > stdin &&
+	test_must_fail git update-ref --stdin < stdin 2> err &&
+	grep "fatal: Cannot lock the ref '"'"'$b'"'"'" err &&
+	git rev-parse $m~1 > expect &&
+	git rev-parse $b > actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin delete ref fails with wrong old value' '
+	echo "$a $e $m~1" > stdin &&
+	test_must_fail git update-ref --stdin < stdin 2> err &&
+	grep "fatal: Cannot lock the ref '"'"'$a'"'"'" err &&
+	git rev-parse $m > expect &&
+	git rev-parse $a > actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin update symref works with --no-deref' '
+	git symbolic-ref TESTSYMREF $b &&
+	echo "--no-deref TESTSYMREF $a $b" > stdin &&
+	git update-ref --stdin < stdin &&
+	git rev-parse TESTSYMREF > expect &&
+	git rev-parse $a > actual &&
+	test_cmp expect actual &&
+	git rev-parse $m~1 > expect &&
+	git rev-parse $b > actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin delete symref works with --no-deref' '
+	git symbolic-ref TESTSYMREF $b &&
+	echo "--no-deref TESTSYMREF $e $b" > stdin &&
+	git update-ref --stdin < stdin &&
+	test_must_fail git rev-parse --verify -q TESTSYMREF &&
+	git rev-parse $m~1 > expect &&
+	git rev-parse $b > actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin delete ref works with right old value' '
+	echo "$b $e $m~1" > stdin &&
+	git update-ref --stdin < stdin &&
+	test_must_fail git rev-parse --verify -q $b
+'
+
+test_expect_success 'stdin create refs works with some old values' '
+	echo "$a $m" > stdin &&
+	echo "$b $m $z" >> stdin &&
+	echo "$c $z $z" >> stdin &&
+	git update-ref --stdin < stdin &&
+	git rev-parse $m > expect &&
+	git rev-parse $a > actual &&
+	test_cmp expect actual &&
+	git rev-parse $b > actual &&
+	test_cmp expect actual &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin update refs works with identity updates' '
+	echo "" > stdin && # also test blank lines
+	echo "$a $m $m" >> stdin &&
+	echo "" >> stdin &&
+	echo " '"'"'$b'"'"'  $m $m " >> stdin &&
+	echo "" >> stdin &&
+	echo "-- $c  $z  $e  " >> stdin &&
+	echo "" >> stdin &&
+	git update-ref --stdin < stdin &&
+	git rev-parse $m > expect &&
+	git rev-parse $a > actual &&
+	test_cmp expect actual &&
+	git rev-parse $b > actual &&
+	test_cmp expect actual &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin update refs fails with wrong old value' '
+	git update-ref $c $m &&
+	echo "$a $m $m" > stdin &&
+	echo "$b $m $m" >> stdin &&
+	echo "$c $e $e" >> stdin &&
+	test_must_fail git update-ref --stdin < stdin 2> err &&
+	grep "fatal: Cannot lock the ref '"'"'$c'"'"'" err &&
+	git rev-parse $m > expect &&
+	git rev-parse $a > actual &&
+	test_cmp expect actual &&
+	git rev-parse $b > actual &&
+	test_cmp expect actual &&
+	git rev-parse $c > actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin delete refs works with packed and loose refs' '
+	git pack-refs --all &&
+	git update-ref $c $m~1 &&
+	echo "$a $z $m" > stdin &&
+	echo "$b $z $m" >> stdin &&
+	echo "$c $e $m~1" >> stdin &&
+	git update-ref --stdin < stdin &&
+	test_must_fail git rev-parse --verify -q $a &&
+	test_must_fail git rev-parse --verify -q $b &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
 test_done
-- 
1.7.10.4

Re: [PATCH v2 8/8] update-ref: add test cases covering --stdin signature

[Eric Sunshine]

On Fri, Aug 30, 2013 at 2:12 PM, Brad King <brad.king@kitware.com> wrote:
> Extend t/t1400-update-ref.sh to cover cases using the --stdin option.
>
> Signed-off-by: Brad King <brad.king@kitware.com>
> ---
>  t/t1400-update-ref.sh |  206 +++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 206 insertions(+)
>
> diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
> index e415ee0..9fd03fc 100755
> --- a/t/t1400-update-ref.sh
> +++ b/t/t1400-update-ref.sh
> @@ -302,4 +302,210 @@ test_expect_success \
>         'git cat-file blob master@{2005-05-26 23:42}:F (expect OTHER)' \
>         'test OTHER = $(git cat-file blob "master@{2005-05-26 23:42}:F")'
>
> +a=refs/heads/a
> +b=refs/heads/b
> +c=refs/heads/c
> +z=0000000000000000000000000000000000000000
> +e="''"
> +
> +test_expect_success 'stdin works with no input' '
> +       rm -f stdin &&
> +       touch stdin &&

Unless the timestamp of 'stdin' has particular significance, modern
git tests avoid 'touch' in favor of creating the empty file like this

    >stdin &&

> +       git update-ref --stdin < stdin &&

Style: Git test scripts omit whitespace following <, >, <<, and >>.

> +       git rev-parse --verify -q $m
> +'
> +
> +test_expect_success 'stdin fails with bad line lines' '
> +       echo " " > stdin &&
> +       test_must_fail git update-ref --stdin < stdin 2> err &&
> +       grep "fatal: no ref on line:  " err &&
> +       echo "--" > stdin &&
> +       test_must_fail git update-ref --stdin < stdin 2> err &&
> +       grep "fatal: no ref on line: --" err &&
> +       echo "--bad-option" > stdin &&
> +       test_must_fail git update-ref --stdin < stdin 2> err &&
> +       grep "fatal: unknown option --bad-option" err &&
> +       echo "-\'"'"' $a $m" > stdin &&
> +       test_must_fail git update-ref --stdin < stdin 2> err &&
> +       grep "fatal: unknown option -'"'"'" err &&
> +       echo "~a $m" > stdin &&
> +       test_must_fail git update-ref --stdin < stdin 2> err &&
> +       grep "fatal: invalid ref format on line: ~a $m" err &&
> +       echo "$a '"'"'master" > stdin &&
> +       test_must_fail git update-ref --stdin < stdin 2> err &&
> +       grep "fatal: unterminated single-quote: '"'"'master" err &&
> +       echo "$a \master" > stdin &&
> +       test_must_fail git update-ref --stdin < stdin 2> err &&
> +       grep "fatal: unquoted backslash not escaping single-quote: \\\\master" err &&
> +       echo "$a $m $m $m" > stdin &&
> +       test_must_fail git update-ref --stdin < stdin 2> err &&
> +       grep "fatal: too many arguments on line: $a $m $m $m" err &&
> +       echo "$a" > stdin &&
> +       test_must_fail git update-ref --stdin < stdin 2> err &&
> +       grep "fatal: missing new value on line: $a" err
> +'

Despite the semantic relationship between all these cases, if there is
a regression in one case, the person reading the verbose output has to
study it carefully to determine the offending case. If you decompose
this monolith so that each case is in its own test_expect_success,
then the regressed case becomes immediately obvious.

> +test_expect_success 'stdin fails with duplicate refs' '
> +       echo "$a $m" > stdin &&
> +       echo "$b $m" >> stdin &&
> +       echo "$a $m" >> stdin &&

These multi-line preparations of 'stdin' might be more readable with a heredoc:

    cat >stdin <<-EOF &&
    $a $m
    $b $m
    $a $m
    EOF

> +       test_must_fail git update-ref --stdin < stdin 2> err &&
> +       grep "fatal: Multiple updates for ref '"'"'$a'"'"' not allowed." err
> +'

Re: [PATCH v2 8/8] update-ref: add test cases covering --stdin signature

[Brad King]

On 08/31/2013 11:41 PM, Eric Sunshine wrote:
>> +       rm -f stdin &&
>> +       touch stdin &&
> 
> Unless the timestamp of 'stdin' has particular significance, modern
> git tests avoid 'touch' in favor of creating the empty file like this
> 
>     >stdin &&

Fixed.

>> +       git update-ref --stdin < stdin &&
> 
> Style: Git test scripts omit whitespace following <, >, <<, and >>.

Fixed.

>> +test_expect_success 'stdin fails with bad line lines' '
> 
> Despite the semantic relationship between all these cases, if there is
> a regression in one case, the person reading the verbose output has to
> study it carefully to determine the offending case. If you decompose
> this monolith so that each case is in its own test_expect_success,
> then the regressed case becomes immediately obvious.

Yes, of course.  Fixed.

> multi-line preparations of 'stdin' might be more readable with a heredoc:
> 
>     cat >stdin <<-EOF &&
>     $a $m
>     $b $m
>     $a $m
>     EOF

Fixed.

Thanks,
-Brad

Re: [PATCH v2 0/8] Multiple simultaneously locked ref updates

[Michael Haggerty]

On 08/30/2013 08:11 PM, Brad King wrote:
> Here is the second revision of a series to support locking multiple
> refs at the same time to update all of them consistently.  The first
> series can be found at $gmane/233260.  This revision is ready to
> consider for integration.

I'm very interested in this area and I regret that I have been so
invisible lately.  I definitely like the way your changes are going.

I have been doing some work in the same neighborhood and our work
overlaps somewhat.  Namely, it is incorrect that delete_ref() currently
deletes the loose ref before rewriting packed-refs, because it can cause
a simultaneous reader to see the packed value for a moment in time, and
the packed value might not even point to a valid object anymore.  In
fact, the current version is even worse: it deletes the loose reference
before even locking the packed-refs file, so if the packed refs file
cannot be rewritten (e.g., because it is locked by another process) then
the reference is permanently left in a corrupt state.

On the other hand, writing the packed-refs file first would also be
incorrect, because a pack-refs process could jump in before the loose
refs file was deleted, read the loose value, and write it to the new
packed-refs file.  The result would be that the first process would
think that it had deleted the reference but it would still exist (not
such a catastrophe, but incorrect nevertheless).

The solution that I have been working on is to first lock *both* the
loose and packed refs files, then rewrite the packed-refs file *but
retain a lock on it*, then rewrite the loose-ref file and release its
lock, then release the lock on the packed-refs file.  By retaining the
lock on the packed-refs file during the whole "transaction", we prevent
another process from trying to pack the refs before our reference is
completely deleted.  This requires the file-locking API to be enhanced
to allow a file to be replaced by its new version while still retaining
a lock on the file.

Your code has the same bug as the original (it's not your fault!) so I
think it will eventually have to be fixed to look something like

    acquire lock on packed-refs

    for reference in ref_updates:
        lock reference
        if old sha1 known:
            verify old sha1 is still current

    for reference in ref_updates:
        if reference should be created/modified:
            modify reference
            release lock on reference

    delete references from packed-refs file and activate new
            version of the file *but retain a lock on the
            packed-refs file*
    for reference in ref_updates:
        if reference should be deleted:
            delete loose version of reference
            release lock on reference

    release lock on packed-refs file

This is really all just for your information; there is certainly no
obligation for you to fix this pre-existing problem.  And I'm working on
it anyway; if you happen to be interested you can view my current
work-in-progress on GitHub (though it still doesn't work!):

    https://github.com/mhagger/git/tree/WIP-delete-ref-locking

Feedback would of course be welcome.

Michael

-- 
Michael Haggerty
mhagger@alum.mit.edu
http://softwareswirl.blogspot.com/

[PATCH v3 0/8] Multiple simultaneously locked ref updates

[Brad King]

Hi Folks,

Here is the third revision of a series to support locking multiple
refs at the same time to update all of them consistently.  The
previous revisions of the series can be found at $gmane/233260 and
$gmane/233458.

Updates since the previous revision of the series:

* Incorporated style fixes suggested in patches 6-8.

* In patch 3, the local "lock" variables in update_ref_lock and
  update_ref now drop the existing "static" declaration.

* In patch 4, delete_ref_loose internals have been cleaned up
  as Michael suggested.

* In patch 6:

  - struct ref_update has been documented
  - update_refs now takes an array of pointers to struct ref_update
    as Michael and Junio both suggested
  - update_refs return cases were simplified with a label and goto
  - update_refs now stops immediately if any ref write fails
  - ref_update_compare now compares only the ref name

* In patch 7, another new input format is proposed.  It now uses
  quoting based on unquote_c_style.

* In patch 8, more new test cases have been added.  Failure cases
  are now covered in separate steps to simplify diagnosis.

-Brad

Brad King (8):
  reset: rename update_refs to reset_refs
  refs: report ref type from lock_any_ref_for_update
  refs: factor update_ref steps into helpers
  refs: factor delete_ref loose ref step into a helper
  refs: add function to repack without multiple refs
  refs: add update_refs for multiple simultaneous updates
  update-ref: support multiple simultaneous updates
  update-ref: add test cases covering --stdin signature

 Documentation/git-update-ref.txt |   20 ++-
 branch.c                         |    2 +-
 builtin/commit.c                 |    2 +-
 builtin/fetch.c                  |    3 +-
 builtin/receive-pack.c           |    3 +-
 builtin/reflog.c                 |    2 +-
 builtin/replace.c                |    2 +-
 builtin/reset.c                  |    4 +-
 builtin/tag.c                    |    2 +-
 builtin/update-ref.c             |  103 ++++++++++++++-
 fast-import.c                    |    2 +-
 refs.c                           |  191 ++++++++++++++++++++++++----
 refs.h                           |   22 +++-
 sequencer.c                      |    3 +-
 t/t1400-update-ref.sh            |  256 ++++++++++++++++++++++++++++++++++++++
 15 files changed, 578 insertions(+), 39 deletions(-)

-- 
1.7.10.4

[PATCH v3 1/8] reset: rename update_refs to reset_refs

[Brad King]

The function resets refs rather than doing arbitrary updates.
Rename it to allow a future general-purpose update_refs function
to be added.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 builtin/reset.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/builtin/reset.c b/builtin/reset.c
index afa6e02..789ee48 100644
--- a/builtin/reset.c
+++ b/builtin/reset.c
@@ -219,7 +219,7 @@ static const char **parse_args(const char **argv, const char *prefix, const char
 	return argv[0] ? get_pathspec(prefix, argv) : NULL;
 }
 
-static int update_refs(const char *rev, const unsigned char *sha1)
+static int reset_refs(const char *rev, const unsigned char *sha1)
 {
 	int update_ref_status;
 	struct strbuf msg = STRBUF_INIT;
@@ -350,7 +350,7 @@ int cmd_reset(int argc, const char **argv, const char *prefix)
 	if (!pathspec && !unborn) {
 		/* Any resets without paths update HEAD to the head being
 		 * switched to, saving the previous head in ORIG_HEAD before. */
-		update_ref_status = update_refs(rev, sha1);
+		update_ref_status = reset_refs(rev, sha1);
 
 		if (reset_type == HARD && !update_ref_status && !quiet)
 			print_new_head_line(lookup_commit_reference(sha1));
-- 
1.7.10.4

[PATCH v3 2/8] refs: report ref type from lock_any_ref_for_update

[Brad King]

Expose lock_ref_sha1_basic's type_p argument to callers of
lock_any_ref_for_update.  Update all call sites to ignore it by passing
NULL for now.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 branch.c               |    2 +-
 builtin/commit.c       |    2 +-
 builtin/fetch.c        |    3 ++-
 builtin/receive-pack.c |    3 ++-
 builtin/reflog.c       |    2 +-
 builtin/replace.c      |    2 +-
 builtin/tag.c          |    2 +-
 fast-import.c          |    2 +-
 refs.c                 |    7 ++++---
 refs.h                 |    2 +-
 sequencer.c            |    3 ++-
 11 files changed, 17 insertions(+), 13 deletions(-)

diff --git a/branch.c b/branch.c
index c5c6984..f2d383f 100644
--- a/branch.c
+++ b/branch.c
@@ -291,7 +291,7 @@ void create_branch(const char *head,
 	hashcpy(sha1, commit->object.sha1);
 
 	if (!dont_change_ref) {
-		lock = lock_any_ref_for_update(ref.buf, NULL, 0);
+		lock = lock_any_ref_for_update(ref.buf, NULL, 0, NULL);
 		if (!lock)
 			die_errno(_("Failed to lock ref for update"));
 	}
diff --git a/builtin/commit.c b/builtin/commit.c
index 10acc53..be08f41 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1618,7 +1618,7 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 					   !current_head
 					   ? NULL
 					   : current_head->object.sha1,
-					   0);
+					   0, NULL);
 
 	nl = strchr(sb.buf, '\n');
 	if (nl)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index d784b2e..28e4025 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -246,7 +246,8 @@ static int s_update_ref(const char *action,
 		rla = default_rla.buf;
 	snprintf(msg, sizeof(msg), "%s: %s", rla, action);
 	lock = lock_any_ref_for_update(ref->name,
-				       check_old ? ref->old_sha1 : NULL, 0);
+				       check_old ? ref->old_sha1 : NULL,
+				       0, NULL);
 	if (!lock)
 		return errno == ENOTDIR ? STORE_REF_ERROR_DF_CONFLICT :
 					  STORE_REF_ERROR_OTHER;
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index e3eb5fc..a323070 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -524,7 +524,8 @@ static const char *update(struct command *cmd)
 		return NULL; /* good */
 	}
 	else {
-		lock = lock_any_ref_for_update(namespaced_name, old_sha1, 0);
+		lock = lock_any_ref_for_update(namespaced_name, old_sha1,
+					       0, NULL);
 		if (!lock) {
 			rp_error("failed to lock %s", name);
 			return "failed to lock";
diff --git a/builtin/reflog.c b/builtin/reflog.c
index 54184b3..28d756a 100644
--- a/builtin/reflog.c
+++ b/builtin/reflog.c
@@ -366,7 +366,7 @@ static int expire_reflog(const char *ref, const unsigned char *sha1, int unused,
 	 * we take the lock for the ref itself to prevent it from
 	 * getting updated.
 	 */
-	lock = lock_any_ref_for_update(ref, sha1, 0);
+	lock = lock_any_ref_for_update(ref, sha1, 0, NULL);
 	if (!lock)
 		return error("cannot lock ref '%s'", ref);
 	log_file = git_pathdup("logs/%s", ref);
diff --git a/builtin/replace.c b/builtin/replace.c
index 59d3115..1ecae8d 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -105,7 +105,7 @@ static int replace_object(const char *object_ref, const char *replace_ref,
 	else if (!force)
 		die("replace ref '%s' already exists", ref);
 
-	lock = lock_any_ref_for_update(ref, prev, 0);
+	lock = lock_any_ref_for_update(ref, prev, 0, NULL);
 	if (!lock)
 		die("%s: cannot lock the ref", ref);
 	if (write_ref_sha1(lock, repl, NULL) < 0)
diff --git a/builtin/tag.c b/builtin/tag.c
index af3af3f..2c867d2 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -577,7 +577,7 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
 	if (annotate)
 		create_tag(object, tag, &buf, &opt, prev, object);
 
-	lock = lock_any_ref_for_update(ref.buf, prev, 0);
+	lock = lock_any_ref_for_update(ref.buf, prev, 0, NULL);
 	if (!lock)
 		die(_("%s: cannot lock the ref"), ref.buf);
 	if (write_ref_sha1(lock, object, NULL) < 0)
diff --git a/fast-import.c b/fast-import.c
index 23f625f..5c329f6 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -1678,7 +1678,7 @@ static int update_branch(struct branch *b)
 		return 0;
 	if (read_ref(b->name, old_sha1))
 		hashclr(old_sha1);
-	lock = lock_any_ref_for_update(b->name, old_sha1, 0);
+	lock = lock_any_ref_for_update(b->name, old_sha1, 0, NULL);
 	if (!lock)
 		return error("Unable to lock %s", b->name);
 	if (!force_update && !is_null_sha1(old_sha1)) {
diff --git a/refs.c b/refs.c
index 7922261..c69fd68 100644
--- a/refs.c
+++ b/refs.c
@@ -2121,11 +2121,12 @@ struct ref_lock *lock_ref_sha1(const char *refname, const unsigned char *old_sha
 }
 
 struct ref_lock *lock_any_ref_for_update(const char *refname,
-					 const unsigned char *old_sha1, int flags)
+					 const unsigned char *old_sha1,
+					 int flags, int *type_p)
 {
 	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL))
 		return NULL;
-	return lock_ref_sha1_basic(refname, old_sha1, flags, NULL);
+	return lock_ref_sha1_basic(refname, old_sha1, flags, type_p);
 }
 
 /*
@@ -3174,7 +3175,7 @@ int update_ref(const char *action, const char *refname,
 		int flags, enum action_on_err onerr)
 {
 	static struct ref_lock *lock;
-	lock = lock_any_ref_for_update(refname, oldval, flags);
+	lock = lock_any_ref_for_update(refname, oldval, flags, NULL);
 	if (!lock) {
 		const char *str = "Cannot lock the ref '%s'.";
 		switch (onerr) {
diff --git a/refs.h b/refs.h
index 9e5db3a..2cd307a 100644
--- a/refs.h
+++ b/refs.h
@@ -137,7 +137,7 @@ extern struct ref_lock *lock_ref_sha1(const char *refname, const unsigned char *
 #define REF_NODEREF	0x01
 extern struct ref_lock *lock_any_ref_for_update(const char *refname,
 						const unsigned char *old_sha1,
-						int flags);
+						int flags, int *type_p);
 
 /** Close the file descriptor owned by a lock and return the status */
 extern int close_ref(struct ref_lock *lock);
diff --git a/sequencer.c b/sequencer.c
index 351548f..06e52b4 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -279,7 +279,8 @@ static int fast_forward_to(const unsigned char *to, const unsigned char *from,
 	read_cache();
 	if (checkout_fast_forward(from, to, 1))
 		exit(1); /* the callee should have complained already */
-	ref_lock = lock_any_ref_for_update("HEAD", unborn ? null_sha1 : from, 0);
+	ref_lock = lock_any_ref_for_update("HEAD", unborn ? null_sha1 : from,
+					   0, NULL);
 	strbuf_addf(&sb, "%s: fast-forward", action_name(opts));
 	ret = write_ref_sha1(ref_lock, to, sb.buf);
 	strbuf_release(&sb);
-- 
1.7.10.4

[PATCH v3 3/8] refs: factor update_ref steps into helpers

[Brad King]

Factor the lock and write steps and error handling into helper functions
update_ref_lock and update_ref_write to allow later use elsewhere.
Expose lock_any_ref_for_update's type_p to update_ref_lock callers.

While at it, drop "static" from the local "lock" variable as it is not
necessary to keep across invocations.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 refs.c |   30 ++++++++++++++++++++++++------
 1 file changed, 24 insertions(+), 6 deletions(-)

diff --git a/refs.c b/refs.c
index c69fd68..4347826 100644
--- a/refs.c
+++ b/refs.c
@@ -3170,12 +3170,13 @@ int for_each_reflog(each_ref_fn fn, void *cb_data)
 	return retval;
 }
 
-int update_ref(const char *action, const char *refname,
-		const unsigned char *sha1, const unsigned char *oldval,
-		int flags, enum action_on_err onerr)
+static struct ref_lock *update_ref_lock(const char *refname,
+					const unsigned char *oldval,
+					int flags, int *type_p,
+					enum action_on_err onerr)
 {
-	static struct ref_lock *lock;
-	lock = lock_any_ref_for_update(refname, oldval, flags, NULL);
+	struct ref_lock *lock;
+	lock = lock_any_ref_for_update(refname, oldval, flags, type_p);
 	if (!lock) {
 		const char *str = "Cannot lock the ref '%s'.";
 		switch (onerr) {
@@ -3183,8 +3184,14 @@ int update_ref(const char *action, const char *refname,
 		case DIE_ON_ERR: die(str, refname); break;
 		case QUIET_ON_ERR: break;
 		}
-		return 1;
 	}
+	return lock;
+}
+
+static int update_ref_write(const char *action, const char *refname,
+			    const unsigned char *sha1, struct ref_lock *lock,
+			    enum action_on_err onerr)
+{
 	if (write_ref_sha1(lock, sha1, action) < 0) {
 		const char *str = "Cannot update the ref '%s'.";
 		switch (onerr) {
@@ -3197,6 +3204,17 @@ int update_ref(const char *action, const char *refname,
 	return 0;
 }
 
+int update_ref(const char *action, const char *refname,
+	       const unsigned char *sha1, const unsigned char *oldval,
+	       int flags, enum action_on_err onerr)
+{
+	struct ref_lock *lock;
+	lock = update_ref_lock(refname, oldval, flags, 0, onerr);
+	if (!lock)
+		return 1;
+	return update_ref_write(action, refname, sha1, lock, onerr);
+}
+
 struct ref *find_ref_by_name(const struct ref *list, const char *name)
 {
 	for ( ; list; list = list->next)
-- 
1.7.10.4

[PATCH v3 4/8] refs: factor delete_ref loose ref step into a helper

[Brad King]

Factor loose ref deletion into helper function delete_ref_loose to allow
later use elsewhere.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 refs.c |   27 +++++++++++++++++----------
 1 file changed, 17 insertions(+), 10 deletions(-)

diff --git a/refs.c b/refs.c
index 4347826..ab9d22e 100644
--- a/refs.c
+++ b/refs.c
@@ -2450,24 +2450,31 @@ static int repack_without_ref(const char *refname)
 	return commit_packed_refs();
 }
 
+static int delete_ref_loose(struct ref_lock *lock, int flag)
+{
+	if (!(flag & REF_ISPACKED) || flag & REF_ISSYMREF) {
+		/* loose */
+		int err, i = strlen(lock->lk->filename) - 5; /* .lock */
+
+		lock->lk->filename[i] = 0;
+		err = unlink_or_warn(lock->lk->filename);
+		lock->lk->filename[i] = '.';
+		if (err && errno != ENOENT)
+			return 1;
+	}
+	return 0;
+}
+
 int delete_ref(const char *refname, const unsigned char *sha1, int delopt)
 {
 	struct ref_lock *lock;
-	int err, i = 0, ret = 0, flag = 0;
+	int ret = 0, flag = 0;
 
 	lock = lock_ref_sha1_basic(refname, sha1, delopt, &flag);
 	if (!lock)
 		return 1;
-	if (!(flag & REF_ISPACKED) || flag & REF_ISSYMREF) {
-		/* loose */
-		i = strlen(lock->lk->filename) - 5; /* .lock */
-		lock->lk->filename[i] = 0;
-		err = unlink_or_warn(lock->lk->filename);
-		if (err && errno != ENOENT)
-			ret = 1;
+	ret |= delete_ref_loose(lock, flag);
 
-		lock->lk->filename[i] = '.';
-	}
 	/* removing the loose one could have resurrected an earlier
 	 * packed one.  Also, if it was not loose we need to repack
 	 * without it.
-- 
1.7.10.4

[PATCH v3 5/8] refs: add function to repack without multiple refs

[Brad King]

Generalize repack_without_ref as repack_without_refs to support a list
of refs and implement the former in terms of the latter.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 refs.c |   29 ++++++++++++++++++++++-------
 1 file changed, 22 insertions(+), 7 deletions(-)

diff --git a/refs.c b/refs.c
index ab9d22e..599504b 100644
--- a/refs.c
+++ b/refs.c
@@ -2414,25 +2414,35 @@ static int curate_packed_ref_fn(struct ref_entry *entry, void *cb_data)
 	return 0;
 }
 
-static int repack_without_ref(const char *refname)
+static int repack_without_refs(const char **refnames, int n)
 {
 	struct ref_dir *packed;
 	struct string_list refs_to_delete = STRING_LIST_INIT_DUP;
 	struct string_list_item *ref_to_delete;
+	int i, removed = 0;
+
+	/* Look for a packed ref: */
+	for (i = 0; i < n; i++)
+		if (get_packed_ref(refnames[i]))
+			break;
 
-	if (!get_packed_ref(refname))
-		return 0; /* refname does not exist in packed refs */
+	/* Avoid locking if we have nothing to do: */
+	if (i == n)
+		return 0; /* no refname exists in packed refs */
 
 	if (lock_packed_refs(0)) {
 		unable_to_lock_error(git_path("packed-refs"), errno);
-		return error("cannot delete '%s' from packed refs", refname);
+		return error("cannot delete '%s' from packed refs", refnames[i]);
 	}
 	packed = get_packed_refs(&ref_cache);
 
-	/* Remove refname from the cache: */
-	if (remove_entry(packed, refname) == -1) {
+	/* Remove refnames from the cache: */
+	for (i = 0; i < n; i++)
+		if (remove_entry(packed, refnames[i]) != -1)
+			removed = 1;
+	if (!removed) {
 		/*
-		 * The packed entry disappeared while we were
+		 * All packed entries disappeared while we were
 		 * acquiring the lock.
 		 */
 		rollback_packed_refs();
@@ -2450,6 +2460,11 @@ static int repack_without_ref(const char *refname)
 	return commit_packed_refs();
 }
 
+static int repack_without_ref(const char *refname)
+{
+	return repack_without_refs(&refname, 1);
+}
+
 static int delete_ref_loose(struct ref_lock *lock, int flag)
 {
 	if (!(flag & REF_ISPACKED) || flag & REF_ISSYMREF) {
-- 
1.7.10.4

[PATCH v3 6/8] refs: add update_refs for multiple simultaneous updates

[Brad King]

Add 'struct ref_update' to encode the information needed to update or
delete a ref (name, new sha1, optional old sha1, no-deref flag).  Add
function 'update_refs' accepting an array of updates to perform.  First
sort the input array to order locks consistently everywhere and reject
multiple updates to the same ref.  Then acquire locks on all refs with
verified old values.  Then update or delete all refs accordingly.  Fail
if any one lock cannot be obtained or any one old value does not match.

Though the refs themselves cannot be modified together in a single
atomic transaction, this function does enable some useful semantics.
For example, a caller may create a new branch starting from the head of
another branch and rewind the original branch at the same time.  This
transfers ownership of commits between branches without risk of losing
commits added to the original branch by a concurrent process, or risk of
a concurrent process creating the new branch first.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 refs.c |  100 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 refs.h |   20 +++++++++++++
 2 files changed, 120 insertions(+)

diff --git a/refs.c b/refs.c
index 599504b..53e8774 100644
--- a/refs.c
+++ b/refs.c
@@ -3237,6 +3237,106 @@ int update_ref(const char *action, const char *refname,
 	return update_ref_write(action, refname, sha1, lock, onerr);
 }
 
+static int ref_update_compare(const void *r1, const void *r2)
+{
+	const struct ref_update * const *u1 = r1;
+	const struct ref_update * const *u2 = r2;
+	return strcmp((*u1)->ref_name, (*u2)->ref_name);
+}
+
+static int ref_update_reject_duplicates(struct ref_update **updates, int n,
+					enum action_on_err onerr)
+{
+	int i;
+	for (i = 1; i < n; i++)
+		if (!strcmp(updates[i - 1]->ref_name, updates[i]->ref_name)) {
+			const char *str =
+				"Multiple updates for ref '%s' not allowed.";
+			switch (onerr) {
+			case MSG_ON_ERR:
+				error(str, updates[i]->ref_name); break;
+			case DIE_ON_ERR:
+				die(str, updates[i]->ref_name); break;
+			case QUIET_ON_ERR:
+				break;
+			}
+			return 1;
+		}
+	return 0;
+}
+
+int update_refs(const char *action, const struct ref_update **updates_orig,
+		int n, enum action_on_err onerr)
+{
+	int ret = 0, delnum = 0, i;
+	struct ref_update **updates;
+	int *types;
+	struct ref_lock **locks;
+	const char **delnames;
+
+	if (!updates_orig || !n)
+		return 0;
+
+	/* Allocate work space: */
+	updates = xmalloc(sizeof(*updates) * n);
+	types = xmalloc(sizeof(*types) * n);
+	locks = xcalloc(n, sizeof(*locks));
+	delnames = xmalloc(sizeof(*delnames) * n);
+
+	/* Copy, sort, and reject duplicate refs: */
+	memcpy(updates, updates_orig, sizeof(*updates) * n);
+	qsort(updates, n, sizeof(*updates), ref_update_compare);
+	ret = ref_update_reject_duplicates(updates, n, onerr);
+	if (ret)
+		goto cleanup;
+
+	/* Acquire all locks while verifying old values: */
+	for (i = 0; i < n; i++) {
+		locks[i] = update_ref_lock(updates[i]->ref_name,
+					   (updates[i]->have_old ?
+					    updates[i]->old_sha1 : NULL),
+					   updates[i]->flags,
+					   &types[i], onerr);
+		if (!locks[i]) {
+			ret = 1;
+			goto cleanup;
+		}
+	}
+
+	/* Perform updates first so live commits remain referenced: */
+	for (i = 0; i < n; i++)
+		if (!is_null_sha1(updates[i]->new_sha1)) {
+			ret = update_ref_write(action,
+					       updates[i]->ref_name,
+					       updates[i]->new_sha1,
+					       locks[i], onerr);
+			locks[i] = NULL; /* freed by update_ref_write */
+			if (ret)
+				goto cleanup;
+		}
+
+	/* Perform deletes now that updates are safely completed: */
+	for (i = 0; i < n; i++)
+		if (locks[i]) {
+			delnames[delnum++] = locks[i]->ref_name;
+			ret |= delete_ref_loose(locks[i], types[i]);
+		}
+	ret |= repack_without_refs(delnames, delnum);
+	for (i = 0; i < delnum; i++)
+		unlink_or_warn(git_path("logs/%s", delnames[i]));
+	clear_loose_ref_cache(&ref_cache);
+
+cleanup:
+	for (i = 0; i < n; i++)
+		if (locks[i])
+			unlock_ref(locks[i]);
+	free(updates);
+	free(types);
+	free(locks);
+	free(delnames);
+	return ret;
+}
+
 struct ref *find_ref_by_name(const struct ref *list, const char *name)
 {
 	for ( ; list; list = list->next)
diff --git a/refs.h b/refs.h
index 2cd307a..b113377 100644
--- a/refs.h
+++ b/refs.h
@@ -10,6 +10,20 @@ struct ref_lock {
 	int force_write;
 };
 
+/**
+ * Information needed for a single ref update.  Set new_sha1 to the
+ * new value or to zero to delete the ref.  To check the old value
+ * while locking the ref, set have_old to 1 and set old_sha1 to the
+ * value or to zero to ensure the ref does not exist before update.
+ */
+struct ref_update {
+	const char *ref_name;
+	unsigned char new_sha1[20];
+	unsigned char old_sha1[20];
+	int flags; /* REF_NODEREF? */
+	int have_old; /* 1 if old_sha1 is valid, 0 otherwise */
+};
+
 /*
  * Bit values set in the flags argument passed to each_ref_fn():
  */
@@ -214,6 +228,12 @@ int update_ref(const char *action, const char *refname,
 		const unsigned char *sha1, const unsigned char *oldval,
 		int flags, enum action_on_err onerr);
 
+/**
+ * Lock all refs and then perform all modifications.
+ */
+int update_refs(const char *action, const struct ref_update **updates,
+		int n, enum action_on_err onerr);
+
 extern int parse_hide_refs_config(const char *var, const char *value, const char *);
 extern int ref_is_hidden(const char *);
 
-- 
1.7.10.4

[PATCH v3 7/8] update-ref: support multiple simultaneous updates

[Brad King]

Add a --stdin signature to read update instructions from standard input
and apply multiple ref updates together.  Use an input format that
supports any update that could be specified via the command-line,
including object names like "branch:path with space".

Signed-off-by: Brad King <brad.king@kitware.com>
---
 Documentation/git-update-ref.txt |   20 +++++++-
 builtin/update-ref.c             |  103 +++++++++++++++++++++++++++++++++++++-
 2 files changed, 121 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
index 0df13ff..01019f1 100644
--- a/Documentation/git-update-ref.txt
+++ b/Documentation/git-update-ref.txt
@@ -8,7 +8,7 @@ git-update-ref - Update the object name stored in a ref safely
 SYNOPSIS
 --------
 [verse]
-'git update-ref' [-m <reason>] (-d <ref> [<oldvalue>] | [--no-deref] <ref> <newvalue> [<oldvalue>])
+'git update-ref' [-m <reason>] (-d <ref> [<oldvalue>] | [--no-deref] <ref> <newvalue> [<oldvalue>] | --stdin)
 
 DESCRIPTION
 -----------
@@ -58,6 +58,24 @@ archive by creating a symlink tree).
 With `-d` flag, it deletes the named <ref> after verifying it
 still contains <oldvalue>.
 
+With `--stdin`, update-ref reads instructions from standard input and
+performs all modifications together.  Empty lines are ignored.
+Each non-empty line is parsed as whitespace-separated arguments.
+Quote arguments containing whitespace as if in C source code.
+Specify updates with lines of the form:
+
+	[--no-deref] [--] <ref> <newvalue> [<oldvalue>]
+
+Lines of any other format or a repeated <ref> produce an error.
+Specify a zero <newvalue> to delete a ref and/or a zero <oldvalue>
+to make sure that a ref not exist.  Use either 40 "0" or the
+empty string (written as "") to specify a zero value.
+
+If all <ref>s can be locked with matching <oldvalue>s
+simultaneously, all modifications are performed.  Otherwise, no
+modifications are performed.  Note that while each individual
+<ref> is updated or deleted atomically, a concurrent reader may
+still see a subset of the modifications.
 
 Logging Updates
 ---------------
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 51d2684..12a3c76 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -2,23 +2,115 @@
 #include "refs.h"
 #include "builtin.h"
 #include "parse-options.h"
+#include "quote.h"
 
 static const char * const git_update_ref_usage[] = {
 	N_("git update-ref [options] -d <refname> [<oldval>]"),
 	N_("git update-ref [options]    <refname> <newval> [<oldval>]"),
+	N_("git update-ref [options] --stdin"),
 	NULL
 };
 
+static int updates_alloc;
+static int updates_count;
+static const struct ref_update **updates;
+
+static const char* update_refs_stdin_next_arg(const char* next,
+					      struct strbuf *arg)
+{
+	/* Skip leading whitespace: */
+	while (isspace(*next))
+		++next;
+
+	/* Return NULL when no argument is found: */
+	if (!*next)
+		return NULL;
+
+	/* Parse the argument: */
+	strbuf_reset(arg);
+	if (*next == '"') {
+		if (unquote_c_style(arg, next, &next))
+			die("badly quoted argument: %s", next);
+		return next;
+	}
+	while (*next && !isspace(*next))
+		strbuf_addch(arg, *next++);
+	return next;
+}
+
+static void update_refs_stdin(const char *line)
+{
+	int options = 1, flags = 0, argc = 0;
+	char *argv[3] = {0, 0, 0};
+	struct strbuf arg = STRBUF_INIT;
+	struct ref_update *update;
+	const char *next = line;
+
+	/* Skip blank lines: */
+	if (!line[0])
+		return;
+
+	/* Parse arguments on this line: */
+	while ((next = update_refs_stdin_next_arg(next, &arg)) != NULL) {
+		if (options && arg.buf[0] == '-')
+			if (!strcmp(arg.buf, "--no-deref"))
+				flags |= REF_NODEREF;
+			else if (!strcmp(arg.buf, "--"))
+				options = 0;
+			else
+				die("unknown option %s", arg.buf);
+		else if (argc >= 3)
+			die("too many arguments on line: %s", line);
+		else {
+			argv[argc++] = xstrdup(arg.buf);
+			options = 0;
+		}
+	}
+	strbuf_release(&arg);
+
+	/* Allocate and zero-init a struct ref_update: */
+	update = xcalloc(1, sizeof(*update));
+	ALLOC_GROW(updates, updates_count+1, updates_alloc);
+	updates[updates_count++] = update;
+
+	/* Set the update ref_name: */
+	if (!argv[0])
+		die("no ref on line: %s", line);
+	if (check_refname_format(argv[0], REFNAME_ALLOW_ONELEVEL))
+		die("invalid ref format on line: %s", line);
+	update->ref_name = argv[0];
+	argv[0] = 0;
+
+	/* Set the update new_sha1 and, if specified, old_sha1: */
+	if (!argv[1])
+		die("missing new value on line: %s", line);
+	if (*argv[1] && get_sha1(argv[1], update->new_sha1))
+		die("invalid new value on line: %s", line);
+	if (argv[2]) {
+		update->have_old = 1;
+		if (*argv[2] && get_sha1(argv[2], update->old_sha1))
+			die("invalid old value on line: %s", line);
+	}
+
+	/* Set the update flags: */
+	update->flags = flags;
+
+	while (argc > 0)
+		free(argv[--argc]);
+}
+
 int cmd_update_ref(int argc, const char **argv, const char *prefix)
 {
 	const char *refname, *oldval, *msg = NULL;
 	unsigned char sha1[20], oldsha1[20];
-	int delete = 0, no_deref = 0, flags = 0;
+	int delete = 0, no_deref = 0, read_stdin = 0, flags = 0;
+	struct strbuf line = STRBUF_INIT;
 	struct option options[] = {
 		OPT_STRING( 'm', NULL, &msg, N_("reason"), N_("reason of the update")),
 		OPT_BOOLEAN('d', NULL, &delete, N_("delete the reference")),
 		OPT_BOOLEAN( 0 , "no-deref", &no_deref,
 					N_("update <refname> not the one it points to")),
+		OPT_BOOLEAN( 0 , "stdin", &read_stdin, N_("read updates from stdin")),
 		OPT_END(),
 	};
 
@@ -28,6 +120,15 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)
 	if (msg && !*msg)
 		die("Refusing to perform update with empty message.");
 
+	if (read_stdin) {
+		if (delete || no_deref || argc > 0)
+			usage_with_options(git_update_ref_usage, options);
+		while (strbuf_getline(&line, stdin, '\n') != EOF)
+			update_refs_stdin(line.buf);
+		strbuf_release(&line);
+		return update_refs(msg, updates, updates_count, DIE_ON_ERR);
+	}
+
 	if (delete) {
 		if (argc < 1 || argc > 2)
 			usage_with_options(git_update_ref_usage, options);
-- 
1.7.10.4

Re: [PATCH v3 7/8] update-ref: support multiple simultaneous updates

[Brad King]

On 09/02/2013 01:48 PM, Brad King wrote:
> +	/* Parse the argument: */
> +	strbuf_reset(arg);
> +	if (*next == '"') {
> +		if (unquote_c_style(arg, next, &next))
> +			die("badly quoted argument: %s", next);
> +		return next;
> +	}
> +	while (*next && !isspace(*next))
> +		strbuf_addch(arg, *next++);
> +	return next;

This quoting proposal was written in response to $gmane/233479:

On 08/30/2013 06:51 PM, Junio C Hamano wrote:
> When we need to deal with arbitrary strings (like pathnames), other
> parts of the system usually give the user two interfaces, --stdin
> with and without -z, and the strings are C-quoted when run without
> the -z option, and terminated with NUL when run with the -z option.

1. Do we want to allow arbitrary non-space characters in unquoted
arguments (while loop above) or reserve some syntax for future use?

2. Thinking about how the -z variation might work, I ran:

$ git grep '\[0\] == '"'"'"' -- '*.c'
builtin/check-attr.c:           if (line_termination && buf.buf[0] == '"') {
builtin/check-ignore.c:         if (line_termination && buf.buf[0] == '"') {
builtin/checkout-index.c:                       if (line_termination && buf.buf[0] == '"') {
builtin/hash-object.c:          if (buf.buf[0] == '"') {
builtin/mktree.c:       if (line_termination && path[0] == '"') {
builtin/update-index.c:         if (line_termination && path_name[0] == '"') {
builtin/update-index.c:                 if (line_termination && buf.buf[0] == '"') {

All of these support quoting only in the non-z mode (the hash-object.c
line follows a getline using hard-coded '\n').  However, they are
all in cases looking for one value on a line or at the end of a line
so their -z option allows NUL-terminated lines containing LF.

What distinguishes the "update-ref --stdin" case is that we want to
represent multiple arguments on one line, each allowing arbitrary
characters or an empty string.  From a brief search a couple places
I found that do something related are:

* apply: Read multiple paths from a diff header, using unquote_c_style
  for quoted paths and separated by spaces.  There is no -z input mode.

* config: Output keyword=value\n becomes keyword\nvalue\0 in -z mode.
  This works because the first piece (keyword) cannot have a LF
  and there is at most one value so all LFs belong to it.

* quote.c: sq_dequote_to_argv handles single quotes like a shell
  would but allows only one space between arguments.  No -z mode.
  This is similar to my v2 proposal.

If we use unquote_c_style and spaces to divide LF-terminated lines,
how shall we divide arguments on NUL-terminated lines?

Thanks,
-Brad

[PATCH v3 8/8] update-ref: add test cases covering --stdin signature

[Brad King]

Extend t/t1400-update-ref.sh to cover cases using the --stdin option.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 t/t1400-update-ref.sh |  256 +++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 256 insertions(+)

diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index e415ee0..b6d7dfa 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -302,4 +302,260 @@ test_expect_success \
 	'git cat-file blob master@{2005-05-26 23:42}:F (expect OTHER)' \
 	'test OTHER = $(git cat-file blob "master@{2005-05-26 23:42}:F")'
 
+a=refs/heads/a
+b=refs/heads/b
+c=refs/heads/c
+z=0000000000000000000000000000000000000000
+e='""'
+pws='path with space'
+
+test_expect_success 'stdin test setup' '
+	echo "$pws" >"$pws" &&
+	git add -- "$pws" &&
+	git commit -m "$pws"
+'
+
+test_expect_success 'stdin works with no input' '
+	>stdin &&
+	git update-ref --stdin <stdin &&
+	git rev-parse --verify -q $m
+'
+
+test_expect_success 'stdin fails on bad input line with only whitespace' '
+	echo " " >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: no ref on line:  " err
+'
+
+test_expect_success 'stdin fails on bad input line with only --' '
+	echo "--" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: no ref on line: --" err
+'
+
+test_expect_success 'stdin fails on bad input line with only --bad-option' '
+	echo "--bad-option" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: unknown option --bad-option" err
+'
+
+test_expect_success 'stdin fails on bad ref name' '
+	echo "~a $m" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: invalid ref format on line: ~a $m" err
+'
+
+test_expect_success 'stdin fails on badly quoted input' '
+	echo "$a \"master" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: badly quoted argument: \\\"master" err
+'
+
+test_expect_success 'stdin fails on bad input line with too many arguments' '
+	echo "$a $m $m $m" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: too many arguments on line: $a $m $m $m" err
+'
+
+test_expect_success 'stdin fails on bad input line with too few arguments' '
+	echo "$a" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: missing new value on line: $a" err
+'
+
+test_expect_success 'stdin fails with duplicate refs' '
+	cat >stdin <<-EOF &&
+$a $m
+$b $m
+$a $m
+EOF
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: Multiple updates for ref '"'"'$a'"'"' not allowed." err
+'
+
+test_expect_success 'stdin create ref works with no old value' '
+	echo "$a $m" >stdin &&
+	git update-ref --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin create ref works with zero old value' '
+	echo "$b $m $z" >stdin &&
+	git update-ref --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	git update-ref -d $b &&
+	echo "$b $m $e" >stdin &&
+	git update-ref --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin create ref works with path with space to blob' '
+	echo "refs/blobs/pws \"$m:$pws\"" >stdin &&
+	git update-ref --stdin <stdin &&
+	git rev-parse "$m:$pws" >expect &&
+	git rev-parse refs/blobs/pws >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin create ref fails with wrong old value' '
+	echo "$c $m $m~1" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: Cannot lock the ref '"'"'$c'"'"'" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin create ref fails with bad old value' '
+	echo "$c $m does-not-exist" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: invalid old value on line: $c $m does-not-exist" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin create ref fails with bad new value' '
+	echo "$c does-not-exist" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: invalid new value on line: $c does-not-exist" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin update ref works with right old value' '
+	echo "$b $m~1 $m" >stdin &&
+	git update-ref --stdin <stdin &&
+	git rev-parse $m~1 >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin update ref fails with wrong old value' '
+	echo "$b $m~1 $m" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: Cannot lock the ref '"'"'$b'"'"'" err &&
+	git rev-parse $m~1 >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin delete ref fails with wrong old value' '
+	echo "$a $e $m~1" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: Cannot lock the ref '"'"'$a'"'"'" err &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin update symref works with --no-deref' '
+	git symbolic-ref TESTSYMREF $b &&
+	echo "--no-deref TESTSYMREF $a $b" >stdin &&
+	git update-ref --stdin <stdin &&
+	git rev-parse TESTSYMREF >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $m~1 >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin delete symref works with --no-deref' '
+	git symbolic-ref TESTSYMREF $b &&
+	echo "--no-deref TESTSYMREF $e $b" >stdin &&
+	git update-ref --stdin <stdin &&
+	test_must_fail git rev-parse --verify -q TESTSYMREF &&
+	git rev-parse $m~1 >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin delete ref works with right old value' '
+	echo "$b $e $m~1" >stdin &&
+	git update-ref --stdin <stdin &&
+	test_must_fail git rev-parse --verify -q $b
+'
+
+test_expect_success 'stdin create refs works with some old values' '
+	cat >stdin <<-EOF &&
+$a $m
+$b $m $z
+$c $z $z
+EOF
+	git update-ref --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin update refs works with identity updates' '
+	cat >stdin <<-EOF &&
+$a $m $m
+$b $m $m
+$c $z $e
+EOF
+	git update-ref --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin update refs works with extra whitespace' '
+	cat >stdin <<-EOF &&
+
+$a $m $m
+
+ "$b"  $m $m ''
+
+-- $c  $z  $e  ''
+EOF
+	git update-ref --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin update refs fails with wrong old value' '
+	git update-ref $c $m &&
+	cat >stdin <<-EOF &&
+$a $m $m
+$b $m $m
+$c $e $e
+EOF
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: Cannot lock the ref '"'"'$c'"'"'" err &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	git rev-parse $c >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin delete refs works with packed and loose refs' '
+	git pack-refs --all &&
+	git update-ref $c $m~1 &&
+	cat >stdin <<-EOF &&
+$a $z $m
+$b $z $m
+$c $e $m~1
+EOF
+	git update-ref --stdin <stdin &&
+	test_must_fail git rev-parse --verify -q $a &&
+	test_must_fail git rev-parse --verify -q $b &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
 test_done
-- 
1.7.10.4

Re: [PATCH v3 8/8] update-ref: add test cases covering --stdin signature

[Eric Sunshine]

On Mon, Sep 2, 2013 at 1:48 PM, Brad King <brad.king@kitware.com> wrote:
> Extend t/t1400-update-ref.sh to cover cases using the --stdin option.
>
> Signed-off-by: Brad King <brad.king@kitware.com>
> ---
>  t/t1400-update-ref.sh |  256 +++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 256 insertions(+)
>
> diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
> index e415ee0..b6d7dfa 100755
> --- a/t/t1400-update-ref.sh
> +++ b/t/t1400-update-ref.sh
> @@ -302,4 +302,260 @@ test_expect_success \
>         'git cat-file blob master@{2005-05-26 23:42}:F (expect OTHER)' \
>         'test OTHER = $(git cat-file blob "master@{2005-05-26 23:42}:F")'
>
> +a=refs/heads/a
> +b=refs/heads/b
> +c=refs/heads/c
> +z=0000000000000000000000000000000000000000
> +e='""'
> +pws='path with space'
> +
> +test_expect_success 'stdin fails on bad input line with only whitespace' '
> +       echo " " >stdin &&
> +       test_must_fail git update-ref --stdin <stdin 2>err &&
> +       grep "fatal: no ref on line:  " err
> +'
> +
> +test_expect_success 'stdin fails on bad input line with only --' '
> +       echo "--" >stdin &&
> +       test_must_fail git update-ref --stdin <stdin 2>err &&
> +       grep "fatal: no ref on line: --" err
> +'
> +
> +test_expect_success 'stdin fails on bad input line with only --bad-option' '
> +       echo "--bad-option" >stdin &&
> +       test_must_fail git update-ref --stdin <stdin 2>err &&
> +       grep "fatal: unknown option --bad-option" err
> +'

When you decomposed the monolithic test from v1 into individual tests,
you dropped a couple cases ("fatal: unknown option'" and "fatal:
unterminated single-quote"). Was this intentional?

> +test_expect_success 'stdin fails with duplicate refs' '
> +       cat >stdin <<-EOF &&
> +$a $m
> +$b $m
> +$a $m
> +EOF
> +       test_must_fail git update-ref --stdin <stdin 2>err &&
> +       grep "fatal: Multiple updates for ref '"'"'$a'"'"' not allowed." err
> +'

The leading '-' on '-EOF' allows you to indent the content of the
heredoc and the terminating EOF, which makes the test read nicely:

    test_expect_success 'stdin fails with duplicate refs' '
        cat >stdin <<-EOF &&
        $a $m
        $b $m
        $a $m
        EOF
        test_must_fail git update-ref ...
    '

> +
> +test_expect_success 'stdin create ref works with no old value' '
> +       echo "$a $m" >stdin &&
> +       git update-ref --stdin <stdin &&
> +       git rev-parse $m >expect &&
> +       git rev-parse $a >actual &&
> +       test_cmp expect actual
> +'

Re: [PATCH v3 8/8] update-ref: add test cases covering --stdin signature

[Brad King]

On 09/03/2013 04:16 AM, Eric Sunshine wrote:
> When you decomposed the monolithic test from v1 into individual tests,
> you dropped a couple cases ("fatal: unknown option'" and "fatal:
> unterminated single-quote"). Was this intentional?

Yes.  The v3 patch 7 changed the set of error messages to be covered.

> The leading '-' on '-EOF' allows you to indent the content of the
> heredoc and the terminating EOF, which makes the test read nicely:

Very nice.  Fixed for next iteration.

Thanks,
-Brad

[PATCH v4 0/8] Multiple simultaneously locked ref updates

[Brad King]

Hi Folks,

Here is the fourth revision of a series to support locking multiple
refs at the same time to update all of them consistently.  The
previous revisions of the series can be found at $gmane/233260,
$gmane/233458, and $gmane/233647.

Updates since the previous revision of the series:

* Patches 1-4 are identical

* Patches 5-7 no longer have ":" at the end of comments, a style I
  "learned" from the context of patch 5 but that I saw Junio squash
  out of patch 5 v2 when he queued it as 53237ae4.

* Patch 7 has a re-organized parser and now defines a -z format for
  stdin that terminates arguments with NUL and lines with LF NUL:

   ... <ref> NUL <newvalue> NUL [ <oldvalue> NUL ] LF NUL

* Patch 8 now has test cases for -z mode and updated error cases for
  the re-organized parser.

Note to maintainer:

* Patch 4 needs to be re-queued to replace c7c80f49 due to the
  tweak suggested in $gmane/233521 and made in v3 and kept in v4.

Thanks,
-Brad

Brad King (8):
  reset: rename update_refs to reset_refs
  refs: report ref type from lock_any_ref_for_update
  refs: factor update_ref steps into helpers
  refs: factor delete_ref loose ref step into a helper
  refs: add function to repack without multiple refs
  refs: add update_refs for multiple simultaneous updates
  update-ref: support multiple simultaneous updates
  update-ref: add test cases covering --stdin signature

 Documentation/git-update-ref.txt |  22 +-
 branch.c                         |   2 +-
 builtin/commit.c                 |   2 +-
 builtin/fetch.c                  |   3 +-
 builtin/receive-pack.c           |   3 +-
 builtin/reflog.c                 |   2 +-
 builtin/replace.c                |   2 +-
 builtin/reset.c                  |   4 +-
 builtin/tag.c                    |   2 +-
 builtin/update-ref.c             | 144 ++++++++++++-
 fast-import.c                    |   2 +-
 refs.c                           | 195 ++++++++++++++---
 refs.h                           |  22 +-
 sequencer.c                      |   3 +-
 t/t1400-update-ref.sh            | 445 +++++++++++++++++++++++++++++++++++++++
 15 files changed, 812 insertions(+), 41 deletions(-)

-- 
1.8.4.rc3

[PATCH v4 1/8] reset: rename update_refs to reset_refs

[Brad King]

The function resets refs rather than doing arbitrary updates.
Rename it to allow a future general-purpose update_refs function
to be added.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 builtin/reset.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/builtin/reset.c b/builtin/reset.c
index afa6e02..789ee48 100644
--- a/builtin/reset.c
+++ b/builtin/reset.c
@@ -219,7 +219,7 @@ static const char **parse_args(const char **argv, const char *prefix, const char
 	return argv[0] ? get_pathspec(prefix, argv) : NULL;
 }
 
-static int update_refs(const char *rev, const unsigned char *sha1)
+static int reset_refs(const char *rev, const unsigned char *sha1)
 {
 	int update_ref_status;
 	struct strbuf msg = STRBUF_INIT;
@@ -350,7 +350,7 @@ int cmd_reset(int argc, const char **argv, const char *prefix)
 	if (!pathspec && !unborn) {
 		/* Any resets without paths update HEAD to the head being
 		 * switched to, saving the previous head in ORIG_HEAD before. */
-		update_ref_status = update_refs(rev, sha1);
+		update_ref_status = reset_refs(rev, sha1);
 
 		if (reset_type == HARD && !update_ref_status && !quiet)
 			print_new_head_line(lookup_commit_reference(sha1));
-- 
1.8.4.rc3

[PATCH v4 2/8] refs: report ref type from lock_any_ref_for_update

[Brad King]

Expose lock_ref_sha1_basic's type_p argument to callers of
lock_any_ref_for_update.  Update all call sites to ignore it by passing
NULL for now.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 branch.c               | 2 +-
 builtin/commit.c       | 2 +-
 builtin/fetch.c        | 3 ++-
 builtin/receive-pack.c | 3 ++-
 builtin/reflog.c       | 2 +-
 builtin/replace.c      | 2 +-
 builtin/tag.c          | 2 +-
 fast-import.c          | 2 +-
 refs.c                 | 7 ++++---
 refs.h                 | 2 +-
 sequencer.c            | 3 ++-
 11 files changed, 17 insertions(+), 13 deletions(-)

diff --git a/branch.c b/branch.c
index c5c6984..f2d383f 100644
--- a/branch.c
+++ b/branch.c
@@ -291,7 +291,7 @@ void create_branch(const char *head,
 	hashcpy(sha1, commit->object.sha1);
 
 	if (!dont_change_ref) {
-		lock = lock_any_ref_for_update(ref.buf, NULL, 0);
+		lock = lock_any_ref_for_update(ref.buf, NULL, 0, NULL);
 		if (!lock)
 			die_errno(_("Failed to lock ref for update"));
 	}
diff --git a/builtin/commit.c b/builtin/commit.c
index 10acc53..be08f41 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1618,7 +1618,7 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 					   !current_head
 					   ? NULL
 					   : current_head->object.sha1,
-					   0);
+					   0, NULL);
 
 	nl = strchr(sb.buf, '\n');
 	if (nl)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index d784b2e..28e4025 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -246,7 +246,8 @@ static int s_update_ref(const char *action,
 		rla = default_rla.buf;
 	snprintf(msg, sizeof(msg), "%s: %s", rla, action);
 	lock = lock_any_ref_for_update(ref->name,
-				       check_old ? ref->old_sha1 : NULL, 0);
+				       check_old ? ref->old_sha1 : NULL,
+				       0, NULL);
 	if (!lock)
 		return errno == ENOTDIR ? STORE_REF_ERROR_DF_CONFLICT :
 					  STORE_REF_ERROR_OTHER;
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index e3eb5fc..a323070 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -524,7 +524,8 @@ static const char *update(struct command *cmd)
 		return NULL; /* good */
 	}
 	else {
-		lock = lock_any_ref_for_update(namespaced_name, old_sha1, 0);
+		lock = lock_any_ref_for_update(namespaced_name, old_sha1,
+					       0, NULL);
 		if (!lock) {
 			rp_error("failed to lock %s", name);
 			return "failed to lock";
diff --git a/builtin/reflog.c b/builtin/reflog.c
index 54184b3..28d756a 100644
--- a/builtin/reflog.c
+++ b/builtin/reflog.c
@@ -366,7 +366,7 @@ static int expire_reflog(const char *ref, const unsigned char *sha1, int unused,
 	 * we take the lock for the ref itself to prevent it from
 	 * getting updated.
 	 */
-	lock = lock_any_ref_for_update(ref, sha1, 0);
+	lock = lock_any_ref_for_update(ref, sha1, 0, NULL);
 	if (!lock)
 		return error("cannot lock ref '%s'", ref);
 	log_file = git_pathdup("logs/%s", ref);
diff --git a/builtin/replace.c b/builtin/replace.c
index 59d3115..1ecae8d 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -105,7 +105,7 @@ static int replace_object(const char *object_ref, const char *replace_ref,
 	else if (!force)
 		die("replace ref '%s' already exists", ref);
 
-	lock = lock_any_ref_for_update(ref, prev, 0);
+	lock = lock_any_ref_for_update(ref, prev, 0, NULL);
 	if (!lock)
 		die("%s: cannot lock the ref", ref);
 	if (write_ref_sha1(lock, repl, NULL) < 0)
diff --git a/builtin/tag.c b/builtin/tag.c
index af3af3f..2c867d2 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -577,7 +577,7 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
 	if (annotate)
 		create_tag(object, tag, &buf, &opt, prev, object);
 
-	lock = lock_any_ref_for_update(ref.buf, prev, 0);
+	lock = lock_any_ref_for_update(ref.buf, prev, 0, NULL);
 	if (!lock)
 		die(_("%s: cannot lock the ref"), ref.buf);
 	if (write_ref_sha1(lock, object, NULL) < 0)
diff --git a/fast-import.c b/fast-import.c
index 23f625f..5c329f6 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -1678,7 +1678,7 @@ static int update_branch(struct branch *b)
 		return 0;
 	if (read_ref(b->name, old_sha1))
 		hashclr(old_sha1);
-	lock = lock_any_ref_for_update(b->name, old_sha1, 0);
+	lock = lock_any_ref_for_update(b->name, old_sha1, 0, NULL);
 	if (!lock)
 		return error("Unable to lock %s", b->name);
 	if (!force_update && !is_null_sha1(old_sha1)) {
diff --git a/refs.c b/refs.c
index 7922261..c69fd68 100644
--- a/refs.c
+++ b/refs.c
@@ -2121,11 +2121,12 @@ struct ref_lock *lock_ref_sha1(const char *refname, const unsigned char *old_sha
 }
 
 struct ref_lock *lock_any_ref_for_update(const char *refname,
-					 const unsigned char *old_sha1, int flags)
+					 const unsigned char *old_sha1,
+					 int flags, int *type_p)
 {
 	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL))
 		return NULL;
-	return lock_ref_sha1_basic(refname, old_sha1, flags, NULL);
+	return lock_ref_sha1_basic(refname, old_sha1, flags, type_p);
 }
 
 /*
@@ -3174,7 +3175,7 @@ int update_ref(const char *action, const char *refname,
 		int flags, enum action_on_err onerr)
 {
 	static struct ref_lock *lock;
-	lock = lock_any_ref_for_update(refname, oldval, flags);
+	lock = lock_any_ref_for_update(refname, oldval, flags, NULL);
 	if (!lock) {
 		const char *str = "Cannot lock the ref '%s'.";
 		switch (onerr) {
diff --git a/refs.h b/refs.h
index 9e5db3a..2cd307a 100644
--- a/refs.h
+++ b/refs.h
@@ -137,7 +137,7 @@ extern struct ref_lock *lock_ref_sha1(const char *refname, const unsigned char *
 #define REF_NODEREF	0x01
 extern struct ref_lock *lock_any_ref_for_update(const char *refname,
 						const unsigned char *old_sha1,
-						int flags);
+						int flags, int *type_p);
 
 /** Close the file descriptor owned by a lock and return the status */
 extern int close_ref(struct ref_lock *lock);
diff --git a/sequencer.c b/sequencer.c
index 351548f..06e52b4 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -279,7 +279,8 @@ static int fast_forward_to(const unsigned char *to, const unsigned char *from,
 	read_cache();
 	if (checkout_fast_forward(from, to, 1))
 		exit(1); /* the callee should have complained already */
-	ref_lock = lock_any_ref_for_update("HEAD", unborn ? null_sha1 : from, 0);
+	ref_lock = lock_any_ref_for_update("HEAD", unborn ? null_sha1 : from,
+					   0, NULL);
 	strbuf_addf(&sb, "%s: fast-forward", action_name(opts));
 	ret = write_ref_sha1(ref_lock, to, sb.buf);
 	strbuf_release(&sb);
-- 
1.8.4.rc3

[PATCH v4 3/8] refs: factor update_ref steps into helpers

[Brad King]

Factor the lock and write steps and error handling into helper functions
update_ref_lock and update_ref_write to allow later use elsewhere.
Expose lock_any_ref_for_update's type_p to update_ref_lock callers.

While at it, drop "static" from the local "lock" variable as it is not
necessary to keep across invocations.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 refs.c | 30 ++++++++++++++++++++++++------
 1 file changed, 24 insertions(+), 6 deletions(-)

diff --git a/refs.c b/refs.c
index c69fd68..4347826 100644
--- a/refs.c
+++ b/refs.c
@@ -3170,12 +3170,13 @@ int for_each_reflog(each_ref_fn fn, void *cb_data)
 	return retval;
 }
 
-int update_ref(const char *action, const char *refname,
-		const unsigned char *sha1, const unsigned char *oldval,
-		int flags, enum action_on_err onerr)
+static struct ref_lock *update_ref_lock(const char *refname,
+					const unsigned char *oldval,
+					int flags, int *type_p,
+					enum action_on_err onerr)
 {
-	static struct ref_lock *lock;
-	lock = lock_any_ref_for_update(refname, oldval, flags, NULL);
+	struct ref_lock *lock;
+	lock = lock_any_ref_for_update(refname, oldval, flags, type_p);
 	if (!lock) {
 		const char *str = "Cannot lock the ref '%s'.";
 		switch (onerr) {
@@ -3183,8 +3184,14 @@ int update_ref(const char *action, const char *refname,
 		case DIE_ON_ERR: die(str, refname); break;
 		case QUIET_ON_ERR: break;
 		}
-		return 1;
 	}
+	return lock;
+}
+
+static int update_ref_write(const char *action, const char *refname,
+			    const unsigned char *sha1, struct ref_lock *lock,
+			    enum action_on_err onerr)
+{
 	if (write_ref_sha1(lock, sha1, action) < 0) {
 		const char *str = "Cannot update the ref '%s'.";
 		switch (onerr) {
@@ -3197,6 +3204,17 @@ int update_ref(const char *action, const char *refname,
 	return 0;
 }
 
+int update_ref(const char *action, const char *refname,
+	       const unsigned char *sha1, const unsigned char *oldval,
+	       int flags, enum action_on_err onerr)
+{
+	struct ref_lock *lock;
+	lock = update_ref_lock(refname, oldval, flags, 0, onerr);
+	if (!lock)
+		return 1;
+	return update_ref_write(action, refname, sha1, lock, onerr);
+}
+
 struct ref *find_ref_by_name(const struct ref *list, const char *name)
 {
 	for ( ; list; list = list->next)
-- 
1.8.4.rc3

[PATCH v4 4/8] refs: factor delete_ref loose ref step into a helper

[Brad King]

Factor loose ref deletion into helper function delete_ref_loose to allow
later use elsewhere.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 refs.c | 27 +++++++++++++++++----------
 1 file changed, 17 insertions(+), 10 deletions(-)

diff --git a/refs.c b/refs.c
index 4347826..ab9d22e 100644
--- a/refs.c
+++ b/refs.c
@@ -2450,24 +2450,31 @@ static int repack_without_ref(const char *refname)
 	return commit_packed_refs();
 }
 
+static int delete_ref_loose(struct ref_lock *lock, int flag)
+{
+	if (!(flag & REF_ISPACKED) || flag & REF_ISSYMREF) {
+		/* loose */
+		int err, i = strlen(lock->lk->filename) - 5; /* .lock */
+
+		lock->lk->filename[i] = 0;
+		err = unlink_or_warn(lock->lk->filename);
+		lock->lk->filename[i] = '.';
+		if (err && errno != ENOENT)
+			return 1;
+	}
+	return 0;
+}
+
 int delete_ref(const char *refname, const unsigned char *sha1, int delopt)
 {
 	struct ref_lock *lock;
-	int err, i = 0, ret = 0, flag = 0;
+	int ret = 0, flag = 0;
 
 	lock = lock_ref_sha1_basic(refname, sha1, delopt, &flag);
 	if (!lock)
 		return 1;
-	if (!(flag & REF_ISPACKED) || flag & REF_ISSYMREF) {
-		/* loose */
-		i = strlen(lock->lk->filename) - 5; /* .lock */
-		lock->lk->filename[i] = 0;
-		err = unlink_or_warn(lock->lk->filename);
-		if (err && errno != ENOENT)
-			ret = 1;
+	ret |= delete_ref_loose(lock, flag);
 
-		lock->lk->filename[i] = '.';
-	}
 	/* removing the loose one could have resurrected an earlier
 	 * packed one.  Also, if it was not loose we need to repack
 	 * without it.
-- 
1.8.4.rc3

[PATCH v4 5/8] refs: add function to repack without multiple refs

[Brad King]

Generalize repack_without_ref as repack_without_refs to support a list
of refs and implement the former in terms of the latter.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 refs.c | 33 ++++++++++++++++++++++++---------
 1 file changed, 24 insertions(+), 9 deletions(-)

diff --git a/refs.c b/refs.c
index ab9d22e..92d801c 100644
--- a/refs.c
+++ b/refs.c
@@ -2414,42 +2414,57 @@ static int curate_packed_ref_fn(struct ref_entry *entry, void *cb_data)
 	return 0;
 }
 
-static int repack_without_ref(const char *refname)
+static int repack_without_refs(const char **refnames, int n)
 {
 	struct ref_dir *packed;
 	struct string_list refs_to_delete = STRING_LIST_INIT_DUP;
 	struct string_list_item *ref_to_delete;
+	int i, removed = 0;
+
+	/* Look for a packed ref */
+	for (i = 0; i < n; i++)
+		if (get_packed_ref(refnames[i]))
+			break;
 
-	if (!get_packed_ref(refname))
-		return 0; /* refname does not exist in packed refs */
+	/* Avoid locking if we have nothing to do */
+	if (i == n)
+		return 0; /* no refname exists in packed refs */
 
 	if (lock_packed_refs(0)) {
 		unable_to_lock_error(git_path("packed-refs"), errno);
-		return error("cannot delete '%s' from packed refs", refname);
+		return error("cannot delete '%s' from packed refs", refnames[i]);
 	}
 	packed = get_packed_refs(&ref_cache);
 
-	/* Remove refname from the cache: */
-	if (remove_entry(packed, refname) == -1) {
+	/* Remove refnames from the cache */
+	for (i = 0; i < n; i++)
+		if (remove_entry(packed, refnames[i]) != -1)
+			removed = 1;
+	if (!removed) {
 		/*
-		 * The packed entry disappeared while we were
+		 * All packed entries disappeared while we were
 		 * acquiring the lock.
 		 */
 		rollback_packed_refs();
 		return 0;
 	}
 
-	/* Remove any other accumulated cruft: */
+	/* Remove any other accumulated cruft */
 	do_for_each_entry_in_dir(packed, 0, curate_packed_ref_fn, &refs_to_delete);
 	for_each_string_list_item(ref_to_delete, &refs_to_delete) {
 		if (remove_entry(packed, ref_to_delete->string) == -1)
 			die("internal error");
 	}
 
-	/* Write what remains: */
+	/* Write what remains */
 	return commit_packed_refs();
 }
 
+static int repack_without_ref(const char *refname)
+{
+	return repack_without_refs(&refname, 1);
+}
+
 static int delete_ref_loose(struct ref_lock *lock, int flag)
 {
 	if (!(flag & REF_ISPACKED) || flag & REF_ISSYMREF) {
-- 
1.8.4.rc3

[PATCH v4 6/8] refs: add update_refs for multiple simultaneous updates

[Brad King]

Add 'struct ref_update' to encode the information needed to update or
delete a ref (name, new sha1, optional old sha1, no-deref flag).  Add
function 'update_refs' accepting an array of updates to perform.  First
sort the input array to order locks consistently everywhere and reject
multiple updates to the same ref.  Then acquire locks on all refs with
verified old values.  Then update or delete all refs accordingly.  Fail
if any one lock cannot be obtained or any one old value does not match.

Though the refs themselves cannot be modified together in a single
atomic transaction, this function does enable some useful semantics.
For example, a caller may create a new branch starting from the head of
another branch and rewind the original branch at the same time.  This
transfers ownership of commits between branches without risk of losing
commits added to the original branch by a concurrent process, or risk of
a concurrent process creating the new branch first.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 refs.c | 100 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 refs.h |  20 +++++++++++++
 2 files changed, 120 insertions(+)

diff --git a/refs.c b/refs.c
index 92d801c..46177ad 100644
--- a/refs.c
+++ b/refs.c
@@ -3237,6 +3237,106 @@ int update_ref(const char *action, const char *refname,
 	return update_ref_write(action, refname, sha1, lock, onerr);
 }
 
+static int ref_update_compare(const void *r1, const void *r2)
+{
+	const struct ref_update * const *u1 = r1;
+	const struct ref_update * const *u2 = r2;
+	return strcmp((*u1)->ref_name, (*u2)->ref_name);
+}
+
+static int ref_update_reject_duplicates(struct ref_update **updates, int n,
+					enum action_on_err onerr)
+{
+	int i;
+	for (i = 1; i < n; i++)
+		if (!strcmp(updates[i - 1]->ref_name, updates[i]->ref_name)) {
+			const char *str =
+				"Multiple updates for ref '%s' not allowed.";
+			switch (onerr) {
+			case MSG_ON_ERR:
+				error(str, updates[i]->ref_name); break;
+			case DIE_ON_ERR:
+				die(str, updates[i]->ref_name); break;
+			case QUIET_ON_ERR:
+				break;
+			}
+			return 1;
+		}
+	return 0;
+}
+
+int update_refs(const char *action, const struct ref_update **updates_orig,
+		int n, enum action_on_err onerr)
+{
+	int ret = 0, delnum = 0, i;
+	struct ref_update **updates;
+	int *types;
+	struct ref_lock **locks;
+	const char **delnames;
+
+	if (!updates_orig || !n)
+		return 0;
+
+	/* Allocate work space */
+	updates = xmalloc(sizeof(*updates) * n);
+	types = xmalloc(sizeof(*types) * n);
+	locks = xcalloc(n, sizeof(*locks));
+	delnames = xmalloc(sizeof(*delnames) * n);
+
+	/* Copy, sort, and reject duplicate refs */
+	memcpy(updates, updates_orig, sizeof(*updates) * n);
+	qsort(updates, n, sizeof(*updates), ref_update_compare);
+	ret = ref_update_reject_duplicates(updates, n, onerr);
+	if (ret)
+		goto cleanup;
+
+	/* Acquire all locks while verifying old values */
+	for (i = 0; i < n; i++) {
+		locks[i] = update_ref_lock(updates[i]->ref_name,
+					   (updates[i]->have_old ?
+					    updates[i]->old_sha1 : NULL),
+					   updates[i]->flags,
+					   &types[i], onerr);
+		if (!locks[i]) {
+			ret = 1;
+			goto cleanup;
+		}
+	}
+
+	/* Perform updates first so live commits remain referenced */
+	for (i = 0; i < n; i++)
+		if (!is_null_sha1(updates[i]->new_sha1)) {
+			ret = update_ref_write(action,
+					       updates[i]->ref_name,
+					       updates[i]->new_sha1,
+					       locks[i], onerr);
+			locks[i] = NULL; /* freed by update_ref_write */
+			if (ret)
+				goto cleanup;
+		}
+
+	/* Perform deletes now that updates are safely completed */
+	for (i = 0; i < n; i++)
+		if (locks[i]) {
+			delnames[delnum++] = locks[i]->ref_name;
+			ret |= delete_ref_loose(locks[i], types[i]);
+		}
+	ret |= repack_without_refs(delnames, delnum);
+	for (i = 0; i < delnum; i++)
+		unlink_or_warn(git_path("logs/%s", delnames[i]));
+	clear_loose_ref_cache(&ref_cache);
+
+cleanup:
+	for (i = 0; i < n; i++)
+		if (locks[i])
+			unlock_ref(locks[i]);
+	free(updates);
+	free(types);
+	free(locks);
+	free(delnames);
+	return ret;
+}
+
 struct ref *find_ref_by_name(const struct ref *list, const char *name)
 {
 	for ( ; list; list = list->next)
diff --git a/refs.h b/refs.h
index 2cd307a..b113377 100644
--- a/refs.h
+++ b/refs.h
@@ -10,6 +10,20 @@ struct ref_lock {
 	int force_write;
 };
 
+/**
+ * Information needed for a single ref update.  Set new_sha1 to the
+ * new value or to zero to delete the ref.  To check the old value
+ * while locking the ref, set have_old to 1 and set old_sha1 to the
+ * value or to zero to ensure the ref does not exist before update.
+ */
+struct ref_update {
+	const char *ref_name;
+	unsigned char new_sha1[20];
+	unsigned char old_sha1[20];
+	int flags; /* REF_NODEREF? */
+	int have_old; /* 1 if old_sha1 is valid, 0 otherwise */
+};
+
 /*
  * Bit values set in the flags argument passed to each_ref_fn():
  */
@@ -214,6 +228,12 @@ int update_ref(const char *action, const char *refname,
 		const unsigned char *sha1, const unsigned char *oldval,
 		int flags, enum action_on_err onerr);
 
+/**
+ * Lock all refs and then perform all modifications.
+ */
+int update_refs(const char *action, const struct ref_update **updates,
+		int n, enum action_on_err onerr);
+
 extern int parse_hide_refs_config(const char *var, const char *value, const char *);
 extern int ref_is_hidden(const char *);
 
-- 
1.8.4.rc3

[PATCH v4 7/8] update-ref: support multiple simultaneous updates

[Brad King]

Add a --stdin signature to read update instructions from standard input
and apply multiple ref updates together.  Use an input format that
supports any update that could be specified via the command-line,
including object names like "branch:path with space".

Signed-off-by: Brad King <brad.king@kitware.com>
---
 Documentation/git-update-ref.txt |  22 +++++-
 builtin/update-ref.c             | 144 ++++++++++++++++++++++++++++++++++++++-
 2 files changed, 164 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
index 0df13ff..11dd9d3 100644
--- a/Documentation/git-update-ref.txt
+++ b/Documentation/git-update-ref.txt
@@ -8,7 +8,7 @@ git-update-ref - Update the object name stored in a ref safely
 SYNOPSIS
 --------
 [verse]
-'git update-ref' [-m <reason>] (-d <ref> [<oldvalue>] | [--no-deref] <ref> <newvalue> [<oldvalue>])
+'git update-ref' [-m <reason>] (-d <ref> [<oldvalue>] | [--no-deref] <ref> <newvalue> [<oldvalue>] | --stdin)
 
 DESCRIPTION
 -----------
@@ -58,6 +58,26 @@ archive by creating a symlink tree).
 With `-d` flag, it deletes the named <ref> after verifying it
 still contains <oldvalue>.
 
+With `--stdin`, update-ref reads instructions from standard input and
+performs all modifications together.  Empty lines are ignored.
+Each non-empty line is parsed as whitespace-separated arguments.
+Quote arguments containing whitespace as if in C source code.
+Specify updates with lines of the form:
+
+	[--no-deref] [--] <ref> <newvalue> [<oldvalue>]
+
+Lines of any other format or a repeated <ref> produce an error.
+Specify a zero <newvalue> to delete a ref and/or a zero <oldvalue>
+to make sure that a ref not exist.  Use either 40 "0" or the
+empty string (written as "") to specify a zero value.  Use `-z`
+to specify input with no whitespace, quoting, or escaping, and
+terminate each argument by NUL and each line by LF NUL.
+
+If all <ref>s can be locked with matching <oldvalue>s
+simultaneously, all modifications are performed.  Otherwise, no
+modifications are performed.  Note that while each individual
+<ref> is updated or deleted atomically, a concurrent reader may
+still see a subset of the modifications.
 
 Logging Updates
 ---------------
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 51d2684..9c348fb 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -2,23 +2,152 @@
 #include "refs.h"
 #include "builtin.h"
 #include "parse-options.h"
+#include "quote.h"
+#include "argv-array.h"
 
 static const char * const git_update_ref_usage[] = {
 	N_("git update-ref [options] -d <refname> [<oldval>]"),
 	N_("git update-ref [options]    <refname> <newval> [<oldval>]"),
+	N_("git update-ref [options] --stdin [-z]"),
 	NULL
 };
 
+static int updates_alloc;
+static int updates_count;
+static const struct ref_update **updates;
+
+static void update_refs_stdin(int argc, const char **argv)
+{
+	struct ref_update *update;
+
+	/* Skip blank lines */
+	if (!argc)
+		return;
+
+	/* Allocate and zero-init a struct ref_update */
+	update = xcalloc(1, sizeof(*update));
+	ALLOC_GROW(updates, updates_count+1, updates_alloc);
+	updates[updates_count++] = update;
+
+	/* Process options */
+	while (argc > 0 && argv[0][0] == '-') {
+		const char *arg = argv[0];
+		--argc;
+		++argv;
+		if (!strcmp(arg, "--no-deref"))
+			update->flags |= REF_NODEREF;
+		else if (!strcmp(arg, "--"))
+			break;
+		else
+			die("unknown option %s", arg);
+	}
+
+	/* Set the update ref_name */
+	if (argc < 1)
+		die("input line with no ref!");
+	if (check_refname_format(argv[0], REFNAME_ALLOW_ONELEVEL))
+		die("invalid ref format: %s", argv[0]);
+	update->ref_name = xstrdup(argv[0]);
+
+	/* Set the update new_sha1 and, if specified, old_sha1 */
+	if (argc < 2)
+		die("missing new value for ref %s", update->ref_name);
+	if (*argv[1] && get_sha1(argv[1], update->new_sha1))
+		die("invalid new value for ref %s: %s",
+		    update->ref_name, argv[1]);
+	if (argc >= 3) {
+		update->have_old = 1;
+		if (*argv[2] && get_sha1(argv[2], update->old_sha1))
+			die("invalid old value for ref %s: %s",
+			    update->ref_name, argv[2]);
+	}
+
+	if (argc > 3)
+		die("too many arguments for ref %s", update->ref_name);
+}
+
+static const char *update_refs_stdin_parse_arg(const char *next,
+					       struct strbuf *arg)
+{
+	/* Skip leading whitespace */
+	while (isspace(*next))
+		++next;
+
+	/* Return NULL when no argument is found */
+	if (!*next)
+		return NULL;
+
+	/* Parse the argument */
+	strbuf_reset(arg);
+	if (*next == '"') {
+		if (unquote_c_style(arg, next, &next))
+			die("badly quoted argument: %s", next);
+		return next;
+	}
+	while (*next && !isspace(*next))
+		strbuf_addch(arg, *next++);
+	return next;
+}
+
+static void update_refs_stdin_parse_line(const char *next)
+{
+	struct strbuf arg = STRBUF_INIT;
+	static struct argv_array args = ARGV_ARRAY_INIT;
+
+	/* Parse arguments on this line */
+	while ((next = update_refs_stdin_parse_arg(next, &arg)) != NULL)
+		argv_array_push(&args, arg.buf);
+
+	/* Process this command */
+	update_refs_stdin(args.argc, args.argv);
+
+	argv_array_clear(&args);
+	strbuf_release(&arg);
+}
+
+static void update_refs_stdin_read_n()
+{
+	struct strbuf line = STRBUF_INIT;
+
+	while (strbuf_getline(&line, stdin, '\n') != EOF)
+		update_refs_stdin_parse_line(line.buf);
+
+	strbuf_release(&line);
+}
+
+static void update_refs_stdin_read_z()
+{
+	struct strbuf arg = STRBUF_INIT;
+	static struct argv_array args = ARGV_ARRAY_INIT;
+
+	/* Process NUL-terminated arguments with commands ending in LF */
+	while (strbuf_getline(&arg, stdin, '\0') != EOF) {
+		if (!strcmp(arg.buf, "\n")) {
+			update_refs_stdin(args.argc, args.argv);
+			argv_array_clear(&args);
+		} else {
+			argv_array_push(&args, arg.buf);
+		}
+	}
+
+	if (args.argc > 0)
+		die("unterminated -z input sequence");
+
+	strbuf_release(&arg);
+}
+
 int cmd_update_ref(int argc, const char **argv, const char *prefix)
 {
 	const char *refname, *oldval, *msg = NULL;
 	unsigned char sha1[20], oldsha1[20];
-	int delete = 0, no_deref = 0, flags = 0;
+	int delete = 0, no_deref = 0, read_stdin = 0, end_null = 0, flags = 0;
 	struct option options[] = {
 		OPT_STRING( 'm', NULL, &msg, N_("reason"), N_("reason of the update")),
 		OPT_BOOLEAN('d', NULL, &delete, N_("delete the reference")),
+		OPT_BOOLEAN('z', NULL, &end_null, N_("stdin has NUL-terminated arguments")),
 		OPT_BOOLEAN( 0 , "no-deref", &no_deref,
 					N_("update <refname> not the one it points to")),
+		OPT_BOOLEAN( 0 , "stdin", &read_stdin, N_("read updates from stdin")),
 		OPT_END(),
 	};
 
@@ -28,6 +157,19 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)
 	if (msg && !*msg)
 		die("Refusing to perform update with empty message.");
 
+	if (read_stdin) {
+		if (delete || no_deref || argc > 0)
+			usage_with_options(git_update_ref_usage, options);
+		if (end_null)
+			update_refs_stdin_read_z();
+		else
+			update_refs_stdin_read_n();
+		return update_refs(msg, updates, updates_count, DIE_ON_ERR);
+	}
+
+	if (end_null)
+		usage_with_options(git_update_ref_usage, options);
+
 	if (delete) {
 		if (argc < 1 || argc > 2)
 			usage_with_options(git_update_ref_usage, options);
-- 
1.8.4.rc3

Re: [PATCH v4 7/8] update-ref: support multiple simultaneous updates

[Junio C Hamano]

Brad King <brad.king@kitware.com> writes:

> +With `--stdin`, update-ref reads instructions from standard input and
> +performs all modifications together.  Empty lines are ignored.
> +Each non-empty line is parsed as whitespace-separated arguments.

"whitespace-separated" implies that we may allow fields separated
with not a single SP, but with double SPs or even HTs between them.
I personally do not think we should be so loose; it is not like we
are allowing the user to type these from the terminal, so limiting
"separated by a single SP" may be perfectly adequate, and make it
less error prone (e.g. there will not be a confusion "Is 'A SP SP B'
A followed by B, or A followed by an empty string followed by B?").

> +Quote arguments containing whitespace as if in C source code.

Probably "as if they were strings in C source code".

> +Specify updates with lines of the form:
> +
> +	[--no-deref] [--] <ref> <newvalue> [<oldvalue>]
> +
> +Lines of any other format or a repeated <ref> produce an error.
> +Specify a zero <newvalue> to delete a ref and/or a zero <oldvalue>
> +to make sure that a ref not exist.  Use either 40 "0" or the
> +empty string (written as "") to specify a zero value.  Use `-z`
> +to specify input with no whitespace, quoting, or escaping, and
> +terminate each argument by NUL and each line by LF NUL.

This is a somewhat interesting choice of the record terminator. Do
we have a precedent to use LF NUL elsewhere?  If this is the first
such case where we need to express variable number of NUL-separated
fields in a record, I think I am fine with LF NUL (but I am sure
other people would give us a better convention if we ask them
politely ;-)), but I just want to make sure we do it the same way as
other codepaths, if exist, that have to handle this kind of thing.

Re: [PATCH v4 7/8] update-ref: support multiple simultaneous updates

[Brad King]

On 09/04/2013 02:23 PM, Junio C Hamano wrote:
> "whitespace-separated" implies that we may allow fields separated with not a single SP, but with double SPs or even HTs between them. I personally do not think we should be so loose

Okay, I will look at making it more strict.  See proposed format
below.

>> +Quote arguments containing whitespace as if in C source code.
> 
> Probably "as if they were strings in C source code".

Fixed.

>> +terminate each argument by NUL and each line by LF NUL.
> 
> This is a somewhat interesting choice of the record terminator. Do we have a precedent to use LF NUL elsewhere?  If this is the first such case where we need to express variable number of NUL-separated fields in a record, I think I am fine with LF NUL (but I am sure other people would give us a
> better convention if we ask them politely ;-)), but I just want to make sure we do it the same way as other codepaths, if exist, that have to handle this kind of thing.

Nothing else uses LF NUL.  I chose it as a starting point for
this very discussion, which I asked about in $gmane/233653.
In this particular use case we know the last field will never
be LF but that may not be so for future cases.  There is no way
to represent sentinel-terminated arbitrary variable-width records
of NUL-terminated fields without some kind of escaping for the
sentinel value, but the whole point of -z is to avoid escaping.

Below is a survey of all mentions of NUL and \0 in documented
formats as of v1.8.4.  The summary is that most are fixed-width
records but a few have variable width allowing n or n+1 fields.
In all variable-width cases there is structured information in
the first field that indicates the number of NUL-terminated
fields to expect.

In the motivating case here, we could use a --no-old or
--have-old option to indicate in one field how many more to
expect in the record, but that will be quite verbose.

Side note: I'd like to reserve room for the leading options to
include things like "-m NUL <reason> NUL" so we cannot keep them
all in in a single NUL-terminated, SP-separated field.

Another approach is to introduce a way to represent "not here"
for the <oldvalue> argument that is not an otherwise valid value.
This would make the non-option part of the record have a fixed
width of 3 fields.  For example, we could use SP in -z mode:

 [-<opt> NUL]... <ref> NUL <new> NUL (<old>|SP) NUL

and the last field can be optional in non-z mode anyway:

 [-<opt> SP]... <ref> SP <new> [SP <old>] LF

Or we could use a character like "~" (other ideas?):

 [-<opt> NUL]... <ref> NUL <new> NUL (<old>|~) NUL

and make it available in non-z mode too:

 [-<opt> SP]... <ref> SP <new> [SP (<old>|~)] LF

Thoughts?
-Brad


Survey of NUL in documented formats:
------------------------------------------------------------------------
* Documentation/diff-format.txt: The -z mode for --numstat prints
  NUL-terminated lines but there is exactly one path at the end of
  each entry and the earlier fields are separated by TAB because
  they are structured.

* Documentation/diff-options.txt: The -z mode for diff-tree output
  prints structured SP/TAB-separated fields in a NUL-terminated
  field followed by either one or two NUL-terminated paths.  This
  is variable width but the first field tells us how wide.

* Documentation/git-apply.txt: The -z mode forwards to --numstat
  diff options.

* Documentation/git-check-attr.txt: The -z mode for stdin reads
  NUL-terminated paths.

* Documentation/git-check-ignore.txt: The -z mode for stdin reads
  NUL-terminated paths.  The -z mode for output prints a fixed-width
  table with every group of four NUL-terminated fields forming a row.

* Documentation/git-checkout-index.txt: The -z mode reads
  NUL-terminated paths.

* Documentation/git-commit.txt: The -z mode forwards to git-status.

* Documentation/git-grep.txt: The -z mode separates file names from
  the matched line by a NUL.  Therefore NUL divides LF-terminated
  lines into two pieces.

* Documentation/git-ls-files.txt: The -z mode prints NUL-terminated
  lines but there is exactly one path at the end of each entry and the
  earlier fields are separated by SP and TAB because they are
  structured.

* Documentation/git-ls-tree.txt: The -z mode prints NUL-terminated
  lines but there is exactly one path at the end of each entry and the
  earlier fields are separated by SP and TAB because they are
  structured.

* Documentation/git-mktree.txt: The -z mode reads NUL-terminated lines
  as output by ls-tree -z.

* Documentation/git-status.txt: The -z mode of --porcelain separates
  a variable number of entries by NUL.  The beginning of each entry
  allows one to know the number of NUL-terminated fields to expect
  (A = 1 total NUL, R = 2 total NULs, etc.).

* Documentation/git-update-index.txt: The -z mode of --stdin separates
  paths by NUL.  The -z mode of --index-info separates entries by NUL
  but there is exactly one path at the end of each entry and the earlier
  fields are separated by SP and TAB because they are structured.

* Documentation/rev-list-options.txt: The --header option prints
  commits separated by NUL but they are never empty.
------------------------------------------------------------------------

Re: [PATCH v4 7/8] update-ref: support multiple simultaneous updates

[Junio C Hamano]

Brad King <brad.king@kitware.com> writes:

> Nothing else uses LF NUL.  I chose it as a starting point for
> this very discussion, which I asked about in $gmane/233653.

The primary reason why LF raised my eyebrow was because the reason
why many subcommands use "-z" (and NUL) is often because the payload
may have LF in a record and LF cannot be used as a record separator
without escaping.  And they use NUL knowing that the payload data in
fields cannot contain a NUL.  If we used LF as a signal to define
the structure of the record, it pretty much defeats the whole point
of defining "-z" format.  The -m reason string will be made into a
single liner deep in the codepath but it _can_ contain LF.

I would have been more receptive to, say, double-NUL as a record
terminator, while using a NUL as a field terminator, or something,
but then we would need to have a way to express an empty field.

> In this particular use case we know the last field will never
> be LF but that may not be so for future cases.  There is no way
> to represent sentinel-terminated arbitrary variable-width records
> of NUL-terminated fields without some kind of escaping for the
> sentinel value, but the whole point of -z is to avoid escaping.

Indeed, but one escape hatch we have is that payload will not
contain NUL anywhere, so whenever we see a NUL, we can trust that it
defines the structure of the record, and is not a part of the
payload.

Stepping back a bit, here are some observations on the arguments
update-ref can take:

 * "-m <reason>" is a reason given for this entire update. As the
   point of this new feature is to give an all-or-none update to one
   or more refs, I think we should not have to accept more than one
   reason (more specifically, the -m option does _not_ belong to a
   specific record that describes what happens to a single ref).

 * "-d <ref> <oldvalue>" is a way to delete a ref. <oldvalue> may be
   missing.

 * "--no-deref <ref> <newvalue> <oldvalue>" and "<ref> <newvalue>
   <oldvalue>" are ways to update or create a ref. Again <oldvalue>
   may be missing.

So it looks to me that one possible format that is easy to generate
by machine without ambiguity may be:

    * The first record could be

      m NUL <reason strong> NUL

      but it is optional. The reason string may contain LF but just
      like invocation from the command line, LF will eventually
      cleaned up into a SP.

    * Then a series of records of different kinds follow.

      - A delete record looks like this:

        d NUL <ref> NUL <oldvalue> NUL

        If you want to delete the ref without "oldvalue" protection,
        just say

        d NUL <ref> NUL NUL

      - A create/update record looks like one of these:

        u NUL <ref> NUL <newvalue> NUL <oldvalue> NUL
        n NUL <ref> NUL <newvalue> NUL <oldvalue> NUL

        Again, if you want to delete the ref without "oldvalue"
        protection, just say

        u NUL <ref> NUL <newvalue> NUL NUL
        n NUL <ref> NUL <newvalue> NUL NUL

     * EOF signals the end of the request.

I am not saying the above is the best format, but the point is that
the mode of the operation defines the structure, so unlike parsing
xml or json where you first parse the structure and then interpret
what each element means, you can define a simple format where the
kind of element comes upfront to allow the parser/interpreter know
what is expected to follow it.

Re: [PATCH v4 7/8] update-ref: support multiple simultaneous updates

[Brad King]

On 09/04/2013 05:27 PM, Junio C Hamano wrote:
> I am not saying the above is the best format, but the point is that
> the mode of the operation defines the structure

Great, thanks for your comments.  Based on that I've prototyped a
new format.  Rather than jumping straight to the patch, here is my
proposed format documentation for review:

-------------------------------------------------------------------------
With `--stdin`, update-ref reads instructions from standard input and
performs all modifications together.  Specify commands of the form:

	create SP <ref> SP <newvalue> LF
	update SP <ref> SP <newvalue> [SP <oldvalue>] LF
	delete SP <ref> [SP <oldvalue>] LF
	verify SP <ref> [SP <oldvalue>] LF
	option SP <opt> LF

Quote fields containing whitespace as if they were strings in C source
code.  Alternatively, use `-z` to specify commands without quoting:

	create SP <ref> NUL <newvalue> NUL
	update SP <ref> NUL <newvalue> NUL [<oldvalue>] NUL
	delete SP <ref> NUL [<oldvalue>] NUL
	verify SP <ref> NUL [<oldvalue>] NUL
	option SP <opt> NUL

Lines of any other format or a repeated <ref> produce an error.
Command meanings are:

create::
	Create <ref> with <newvalue> only if it does not exist.

update::
	Update <ref> to be <newvalue>, verifying <oldvalue> if given.
	Specify a zero <newvalue> to delete a ref and/or a zero
	<oldvalue> to make sure that a ref does not exist.

delete::
	Delete <ref>, verifying <oldvalue> if given.

verify::
	Verify <ref> against <oldvalue> but do not change it.  If
	<oldvalue> zero or missing, the ref must not exist.

option::
	Specify an option to take effect for following commands.
	Valid options are `deref` and `no-deref` to specify whether
	to dereference symbolic refs.

Use 40 "0" or the empty string to specify a zero value, except that
with `-z` an empty <oldvalue> is considered missing.

If all <ref>s can be locked with matching <oldvalue>s
simultaneously, all modifications are performed.  Otherwise, no
modifications are performed.  Note that while each individual
<ref> is updated or deleted atomically, a concurrent reader may
still see a subset of the modifications.
-------------------------------------------------------------------------

Thanks,
-Brad

Re: [PATCH v4 7/8] update-ref: support multiple simultaneous updates

[Junio C Hamano]

Brad King <brad.king@kitware.com> writes:

> On 09/04/2013 05:27 PM, Junio C Hamano wrote:
>> I am not saying the above is the best format, but the point is that
>> the mode of the operation defines the structure
>
> Great, thanks for your comments.  Based on that I've prototyped a
> new format.  Rather than jumping straight to the patch, here is my
> proposed format documentation for review:
>
> -------------------------------------------------------------------------
> With `--stdin`, update-ref reads instructions from standard input and
> performs all modifications together.  Specify commands of the form:
>
> 	create SP <ref> SP <newvalue> LF
> 	update SP <ref> SP <newvalue> [SP <oldvalue>] LF
> 	delete SP <ref> [SP <oldvalue>] LF
> 	verify SP <ref> [SP <oldvalue>] LF
> 	option SP <opt> LF
>
> Quote fields containing whitespace as if they were strings in C source
> code.  Alternatively, use `-z` to specify commands without quoting:
>
> 	create SP <ref> NUL <newvalue> NUL
> 	update SP <ref> NUL <newvalue> NUL [<oldvalue>] NUL
> 	delete SP <ref> NUL [<oldvalue>] NUL
> 	verify SP <ref> NUL [<oldvalue>] NUL
> 	option SP <opt> NUL

That SP in '-z' format looks strange.  Was there a reason why NUL
was inappropriate?

> Lines of any other format or a repeated <ref> produce an error.
> Command meanings are:
>
> create::
> 	Create <ref> with <newvalue> only if it does not exist.
>
> update::
> 	Update <ref> to be <newvalue>, verifying <oldvalue> if given.
> 	Specify a zero <newvalue> to delete a ref and/or a zero
> 	<oldvalue> to make sure that a ref does not exist.
>
> delete::
> 	Delete <ref>, verifying <oldvalue> if given.
>
> verify::
> 	Verify <ref> against <oldvalue> but do not change it.  If
> 	<oldvalue> zero or missing, the ref must not exist.
>
> option::
> 	Specify an option to take effect for following commands.
> 	Valid options are `deref` and `no-deref` to specify whether
> 	to dereference symbolic refs.

This last one is somewhat peculiar, especially because it says
"following command*s*".

How would I request to update refs HEAD and next in an all-or-none
fashion, while applying 'no-deref' only to HEAD but not next?

	update refs/heads/next <newvalue>
	option --no-deref
        update HEAD <newvalue>

sounds somewhat convoluted.

When I said "create or update" in the message you are responding to,
I did not mean that we should have two separate commands.  The
regular command line does create-or-update; if it exists already, it
is an update, and if it does not, it is a create.

If we were to have two, I would say we should have:

	create-or-update <ref> <newvalue> [<oldvalue>]
	create-or-update-no-deref <ref> <newvalue> [<oldvalue>]
        
An old value of 0{40} would mean "I should be the one creating; if
somebody else created it while I was preparing this request, please
fail".

Similarly for delete:

	delete <ref> [<oldvalue>]
        delete-no-deref <ref> [<oldvalue>]

Also how would one set the reflog message for the proposed update?

Re: [PATCH v4 7/8] update-ref: support multiple simultaneous updates

[Brad King]

On 9/5/2013 5:23 PM, Junio C Hamano wrote:
> Brad King <brad.king@kitware.com> writes:
>> 	create SP <ref> NUL <newvalue> NUL
>> 	update SP <ref> NUL <newvalue> NUL [<oldvalue>] NUL
> 
> That SP in '-z' format looks strange.  Was there a reason why NUL
> was inappropriate?

The precedent I saw in the -z survey I posted is that NUL is used
to terminate fields that can contain arbitrary text but otherwise
SP or TAB is still used to terminate simple fields.  I recall no
cases that use NUL after fields that only contain simple values.

>> option::
>> 	Specify an option to take effect for following commands.
> 
> This last one is somewhat peculiar, especially because it says
> "following command*s*".
> 
> How would I request to update refs HEAD and next in an all-or-none
> fashion, while applying 'no-deref' only to HEAD but not next?

You're right.  It will be simpler for clients to generate each
sequence of options followed by a <ref> command without worrying
about options possibly generated for preceding <ref>s.  So:

option::
	Modify behavior of the next command naming a <ref>.
	The only valid option is `no-deref` to avoid dereferencing
	a symbolic ref.

> When I said "create or update" in the message you are responding to,
> I did not mean that we should have two separate commands.

The nice thing about "create" is that it implies oldvalue=zero, just
as "delete" implies newvalue=zero.  The symmetry feels clean.  Also,
in -z mode we need to treat an empty string <oldvalue> as missing
rather than zero.  The only way to specify create with the "update"
command is with literal 40 "0" as <oldvalue>.

> The regular command line does create-or-update; if it exists already,
> it is an update, and if it does not, it is a create.

The proposed update command can be used for create, update, delete,
or verify with proper arguments and use of 40 "0".  The other <ref>
commands are for convenience, shorter input, and statement of intent.

> 	create-or-update-no-deref <ref> <newvalue> [<oldvalue>]
>         delete-no-deref <ref> [<oldvalue>]
> 
> Also how would one set the reflog message for the proposed update?

This is why I proposed a separate option command.  It can be used
both for no-deref and for other options we want to add later e.g.

  option SP message SP <reason> LF

and with -z:

  option SP message SP <reason> NUL

For now I think it is sufficient for the -m <reason> command-line
option to set the same message for all updates.  The option command
leaves room to add a per-ref message later.

BTW, the reasons I propose message as an option rather than its own
command are:

* It is simpler to document the reach of the one option command
  (affects next <ref>) in one place rather than separately for
  each option-like command.

* It reduces the number of commands that do not take a <ref>.

-Brad

Re: [PATCH v4 7/8] update-ref: support multiple simultaneous updates

[Junio C Hamano]

Brad King <brad.king@kitware.com> writes:

> +static void update_refs_stdin_read_n()
> +{
> +	struct strbuf line = STRBUF_INIT;
> +
> +	while (strbuf_getline(&line, stdin, '\n') != EOF)
> +		update_refs_stdin_parse_line(line.buf);
> +
> +	strbuf_release(&line);
> +}
> +
> +static void update_refs_stdin_read_z()
> +{

These need to be defined with explicit (void) argument list.

	static void foo(void)
        {
        	...

Re: [PATCH v4 7/8] update-ref: support multiple simultaneous updates

[Brad King]

On 09/04/2013 03:17 PM, Junio C Hamano wrote:
> Brad King <brad.king@kitware.com> writes:
>> +static void update_refs_stdin_read_n()
>> +static void update_refs_stdin_read_z()
> 
> These need to be defined with explicit (void) argument list.

Oops, fixed.

Thanks,
-Brad

[PATCH v4 8/8] update-ref: add test cases covering --stdin signature

[Brad King]

Extend t/t1400-update-ref.sh to cover cases using the --stdin option.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 t/t1400-update-ref.sh | 445 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 445 insertions(+)

diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index e415ee0..e8ba0d2 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -302,4 +302,449 @@ test_expect_success \
 	'git cat-file blob master@{2005-05-26 23:42}:F (expect OTHER)' \
 	'test OTHER = $(git cat-file blob "master@{2005-05-26 23:42}:F")'
 
+a=refs/heads/a
+b=refs/heads/b
+c=refs/heads/c
+E='""'
+pws='path with space'
+
+print_nul() {
+	while test $# -gt 0; do
+		printf -- "$1" &&
+		printf -- "Q" | q_to_nul &&
+		shift || return
+	done
+}
+
+test_expect_success '-z fails without --stdin' '
+	test_must_fail git update-ref -z $m $m $m 2>err &&
+	grep "usage: git update-ref" err
+'
+
+test_expect_success 'stdin test setup' '
+	echo "$pws" >"$pws" &&
+	git add -- "$pws" &&
+	git commit -m "$pws"
+'
+
+test_expect_success 'stdin works with no input' '
+	>stdin &&
+	git update-ref --stdin <stdin &&
+	git rev-parse --verify -q $m
+'
+
+test_expect_success 'stdin works with whitespace-only input' '
+	echo " " >stdin &&
+	git update-ref --stdin <stdin 2>err &&
+	git rev-parse --verify -q $m
+'
+
+test_expect_success 'stdin fails on bad input line with only --' '
+	echo "--" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: input line with no ref" err
+'
+
+test_expect_success 'stdin fails on bad input line with only --bad-option' '
+	echo "--bad-option" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: unknown option --bad-option" err
+'
+
+test_expect_success 'stdin fails on bad ref name' '
+	echo "~a $m" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: invalid ref format: ~a" err
+'
+
+test_expect_success 'stdin fails on badly quoted input' '
+	echo "$a \"master" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: badly quoted argument: \\\"master" err
+'
+
+test_expect_success 'stdin fails on bad input line with too many arguments' '
+	echo "$a $m $m $m" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: too many arguments for ref $a" err
+'
+
+test_expect_success 'stdin fails on bad input line with too few arguments' '
+	echo "$a" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: missing new value for ref $a" err
+'
+
+test_expect_success 'stdin fails with duplicate refs' '
+	cat >stdin <<-EOF &&
+	$a $m
+	$b $m
+	$a $m
+	EOF
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: Multiple updates for ref '"'"'$a'"'"' not allowed." err
+'
+
+test_expect_success 'stdin create ref works with no old value' '
+	echo "$a $m" >stdin &&
+	git update-ref --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin create ref works with zero old value' '
+	echo "$b $m $Z" >stdin &&
+	git update-ref --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	git update-ref -d $b
+'
+
+test_expect_success 'stdin create ref works with empty old value' '
+	echo "$b $m $E" >stdin &&
+	git update-ref --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin create ref works with path with space to blob' '
+	echo "refs/blobs/pws \"$m:$pws\"" >stdin &&
+	git update-ref --stdin <stdin &&
+	git rev-parse "$m:$pws" >expect &&
+	git rev-parse refs/blobs/pws >actual &&
+	test_cmp expect actual &&
+	git update-ref -d refs/blobs/pws
+'
+
+test_expect_success 'stdin create ref fails with wrong old value' '
+	echo "$c $m $m~1" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: Cannot lock the ref '"'"'$c'"'"'" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin create ref fails with bad old value' '
+	echo "$c $m does-not-exist" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: invalid old value for ref $c: does-not-exist" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin create ref fails with bad new value' '
+	echo "$c does-not-exist" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: invalid new value for ref $c: does-not-exist" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin update ref works with right old value' '
+	echo "$b $m~1 $m" >stdin &&
+	git update-ref --stdin <stdin &&
+	git rev-parse $m~1 >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin update ref fails with wrong old value' '
+	echo "$b $m~1 $m" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: Cannot lock the ref '"'"'$b'"'"'" err &&
+	git rev-parse $m~1 >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin delete ref fails with wrong old value' '
+	echo "$a $E $m~1" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: Cannot lock the ref '"'"'$a'"'"'" err &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin update symref works with --no-deref' '
+	git symbolic-ref TESTSYMREF $b &&
+	echo "--no-deref TESTSYMREF $a $b" >stdin &&
+	git update-ref --stdin <stdin &&
+	git rev-parse TESTSYMREF >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $m~1 >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin delete symref works with --no-deref' '
+	git symbolic-ref TESTSYMREF $b &&
+	echo "--no-deref TESTSYMREF $E $b" >stdin &&
+	git update-ref --stdin <stdin &&
+	test_must_fail git rev-parse --verify -q TESTSYMREF &&
+	git rev-parse $m~1 >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin delete ref works with right old value' '
+	echo "$b $E $m~1" >stdin &&
+	git update-ref --stdin <stdin &&
+	test_must_fail git rev-parse --verify -q $b
+'
+
+test_expect_success 'stdin create refs works with some old values' '
+	cat >stdin <<-EOF &&
+	$a $m
+	$b $m $Z
+	$c $Z $Z
+	EOF
+	git update-ref --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin update refs works with identity updates' '
+	cat >stdin <<-EOF &&
+	$a $m $m
+	$b $m $m
+	$c $Z $E
+	EOF
+	git update-ref --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin update refs works with extra whitespace' '
+	cat >stdin <<-EOF &&
+	''
+	$a $m $m
+	''
+	 "$b"  $m $m ''
+	''
+	-- $c  $Z  $E  ''
+	EOF
+	git update-ref --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin update refs fails with wrong old value' '
+	git update-ref $c $m &&
+	cat >stdin <<-EOF &&
+	$a $m $m
+	$b $m $m
+	$c $E $E
+	EOF
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: Cannot lock the ref '"'"'$c'"'"'" err &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	git rev-parse $c >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin delete refs works with packed and loose refs' '
+	git pack-refs --all &&
+	git update-ref $c $m~1 &&
+	cat >stdin <<-EOF &&
+	$a $Z $m
+	$b $Z $m
+	$c $E $m~1
+	EOF
+	git update-ref --stdin <stdin &&
+	test_must_fail git rev-parse --verify -q $a &&
+	test_must_fail git rev-parse --verify -q $b &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z works on empty input' '
+	>stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse --verify -q $m
+'
+
+test_expect_success 'stdin -z works on empty input sequence' '
+	print_nul "\n" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse --verify -q $m
+'
+
+test_expect_success 'stdin -z fails on unterminated input sequence' '
+	print_nul "$a" "$m" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: unterminated -z input sequence" err
+'
+
+test_expect_success 'stdin -z create ref works with no old value' '
+	print_nul "$a" "$m" "\n" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin -z create ref works with zero old value' '
+	print_nul "$b" "$m" "$Z" "\n" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	git update-ref -d $b
+'
+
+test_expect_success 'stdin -z create ref works with empty old value' '
+	print_nul "$b" "$m" "" "\n" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin -z create ref works with path with space to blob' '
+	print_nul "refs/blobs/pws" "$m:$pws" "\n" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse "$m:$pws" >expect &&
+	git rev-parse refs/blobs/pws >actual &&
+	test_cmp expect actual &&
+	git update-ref -d refs/blobs/pws
+'
+
+test_expect_success 'stdin -z create ref fails with wrong old value' '
+	print_nul "$c" "$m" "$m~1" "\n" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: Cannot lock the ref '"'"'$c'"'"'" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z create ref fails with bad old value' '
+	print_nul "$c" "$m" "does-not-exist" "\n" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: invalid old value for ref $c: does-not-exist" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z create ref fails with bad new value' '
+	print_nul "$c" "does-not-exist" "\n" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: invalid new value for ref $c: does-not-exist" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z update ref works with right old value' '
+	print_nul "$b" "$m~1" "$m" "\n" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse $m~1 >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin -z update ref fails with wrong old value' '
+	print_nul "$b" "$m~1" "$m" "\n" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: Cannot lock the ref '"'"'$b'"'"'" err &&
+	git rev-parse $m~1 >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin -z delete ref fails with wrong old value' '
+	print_nul "$a" "" "$m~1" "\n" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: Cannot lock the ref '"'"'$a'"'"'" err &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin -z update symref works with --no-deref' '
+	git symbolic-ref TESTSYMREF $b &&
+	print_nul "--no-deref" "TESTSYMREF" "$a" "$b" "\n" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse TESTSYMREF >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $m~1 >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin -z delete symref works with --no-deref' '
+	git symbolic-ref TESTSYMREF $b &&
+	print_nul "--no-deref" "TESTSYMREF" "" "$b" "\n" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	test_must_fail git rev-parse --verify -q TESTSYMREF &&
+	git rev-parse $m~1 >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin -z delete ref works with right old value' '
+	print_nul "$b" "" "$m~1" "\n" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	test_must_fail git rev-parse --verify -q $b
+'
+
+test_expect_success 'stdin -z create refs works with some old values' '
+	print_nul "$a" "$m" "\n" "$b" "$m" "$Z" "\n" "$c" "$Z" "$Z" "\n" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z update refs works with identity updates' '
+	print_nul "$a" "$m" "$m" "\n" "$b" "$m" "$m" "\n" "$c" "$Z" "" "\n" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z update refs fails with wrong old value' '
+	git update-ref $c $m &&
+	print_nul "$a" "$m" "$m" "\n" "$b" "$m" "$m" "\n" "$c" "" "" "\n" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: Cannot lock the ref '"'"'$c'"'"'" err &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	git rev-parse $c >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin -z delete refs works with packed and loose refs' '
+	git pack-refs --all &&
+	git update-ref $c $m~1 &&
+	print_nul "$a" "$Z" "$m" "\n" "$b" "$Z" "$m" "\n" "$c" "" "$m~1" "\n" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	test_must_fail git rev-parse --verify -q $a &&
+	test_must_fail git rev-parse --verify -q $b &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
 test_done
-- 
1.8.4.rc3

[PATCH v5 0/8] Multiple simultaneously locked ref updates

[Brad King]

Hi Folks,

Here is the fifth revision of a series to support locking multiple
refs at the same time to update all of them consistently.  The
previous revisions of the series can be found at $gmane/233260,
$gmane/233458, $gmane/233647, and $gmane/233840.

Updates since the previous revision of the series:

* Patches 1-6 are identical to v4 so are not re-sent here.

* Patch 7 and 8 now implement and test the input format proposed and
  discussed at $gmane/233990.

-Brad

Brad King (2):
  update-ref: support multiple simultaneous updates
  update-ref: add test cases covering --stdin signature

 Documentation/git-update-ref.txt |  54 +++-
 builtin/update-ref.c             | 252 ++++++++++++++-
 t/t1400-update-ref.sh            | 639 +++++++++++++++++++++++++++++++++++++++
 3 files changed, 943 insertions(+), 2 deletions(-)

-- 
1.8.4.rc3

[PATCH v5 7/8] update-ref: support multiple simultaneous updates

[Brad King]

Add a --stdin signature to read update instructions from standard input
and apply multiple ref updates together.  Use an input format that
supports any update that could be specified via the command-line,
including object names like "branch:path with space".

Signed-off-by: Brad King <brad.king@kitware.com>
---
 Documentation/git-update-ref.txt |  54 ++++++++-
 builtin/update-ref.c             | 252 ++++++++++++++++++++++++++++++++++++++-
 2 files changed, 304 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
index 0df13ff..0a0a551 100644
--- a/Documentation/git-update-ref.txt
+++ b/Documentation/git-update-ref.txt
@@ -8,7 +8,7 @@ git-update-ref - Update the object name stored in a ref safely
 SYNOPSIS
 --------
 [verse]
-'git update-ref' [-m <reason>] (-d <ref> [<oldvalue>] | [--no-deref] <ref> <newvalue> [<oldvalue>])
+'git update-ref' [-m <reason>] (-d <ref> [<oldvalue>] | [--no-deref] <ref> <newvalue> [<oldvalue>] | --stdin [-z])
 
 DESCRIPTION
 -----------
@@ -58,6 +58,58 @@ archive by creating a symlink tree).
 With `-d` flag, it deletes the named <ref> after verifying it
 still contains <oldvalue>.
 
+With `--stdin`, update-ref reads instructions from standard input and
+performs all modifications together.  Specify commands of the form:
+
+	update SP <ref> SP <newvalue> [SP <oldvalue>] LF
+	create SP <ref> SP <newvalue> LF
+	delete SP <ref> [SP <oldvalue>] LF
+	verify SP <ref> [SP <oldvalue>] LF
+	option SP <opt> LF
+
+Quote fields containing whitespace as if they were strings in C source
+code.  Alternatively, use `-z` to specify commands without quoting:
+
+	update SP <ref> NUL <newvalue> NUL [<oldvalue>] NUL
+	create SP <ref> NUL <newvalue> NUL
+	delete SP <ref> NUL [<oldvalue>] NUL
+	verify SP <ref> NUL [<oldvalue>] NUL
+	option SP <opt> NUL
+
+Lines of any other format or a repeated <ref> produce an error.
+Command meanings are:
+
+update::
+	Set <ref> to <newvalue> after verifying <oldvalue>, if given.
+	Specify a zero <newvalue> to ensure the ref does not exist
+	after the update and/or a zero <oldvalue> to make sure the
+	ref does not exist before the update.
+
+create::
+	Create <ref> with <newvalue> after verifying it does not
+	exist.  The given <newvalue> may not be zero.
+
+delete::
+	Delete <ref> after verifying it exists with <oldvalue>, if
+	given.  If given, <oldvalue> may not be zero.
+
+verify::
+	Verify <ref> against <oldvalue> but do not change it.  If
+	<oldvalue> zero or missing, the ref must not exist.
+
+option::
+	Modify behavior of the next command naming a <ref>.
+	The only valid option is `no-deref` to avoid dereferencing
+	a symbolic ref.
+
+Use 40 "0" or the empty string to specify a zero value, except that
+with `-z` an empty <oldvalue> is considered missing.
+
+If all <ref>s can be locked with matching <oldvalue>s
+simultaneously, all modifications are performed.  Otherwise, no
+modifications are performed.  Note that while each individual
+<ref> is updated or deleted atomically, a concurrent reader may
+still see a subset of the modifications.
 
 Logging Updates
 ---------------
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 51d2684..894f16b 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -2,23 +2,261 @@
 #include "refs.h"
 #include "builtin.h"
 #include "parse-options.h"
+#include "quote.h"
+#include "argv-array.h"
 
 static const char * const git_update_ref_usage[] = {
 	N_("git update-ref [options] -d <refname> [<oldval>]"),
 	N_("git update-ref [options]    <refname> <newval> [<oldval>]"),
+	N_("git update-ref [options] --stdin [-z]"),
 	NULL
 };
 
+static int updates_alloc;
+static int updates_count;
+static const struct ref_update **updates;
+
+static char line_termination = '\n';
+static int update_flags;
+
+static struct ref_update *update_alloc(void)
+{
+	struct ref_update *update;
+
+	/* Allocate and zero-init a struct ref_update */
+	update = xcalloc(1, sizeof(*update));
+	ALLOC_GROW(updates, updates_count + 1, updates_alloc);
+	updates[updates_count++] = update;
+
+	/* Store and reset accumulated options */
+	update->flags = update_flags;
+	update_flags = 0;
+
+	return update;
+}
+
+static void update_store_ref_name(struct ref_update *update,
+				  const char *ref_name)
+{
+	if (check_refname_format(ref_name, REFNAME_ALLOW_ONELEVEL))
+		die("invalid ref format: %s", ref_name);
+	update->ref_name = xstrdup(ref_name);
+}
+
+static void update_store_new_sha1(struct ref_update *update,
+				  const char *newvalue)
+{
+	if (*newvalue && get_sha1(newvalue, update->new_sha1))
+		die("invalid new value for ref %s: %s",
+		    update->ref_name, newvalue);
+}
+
+static void update_store_old_sha1(struct ref_update *update,
+				  const char *oldvalue)
+{
+	if (*oldvalue && get_sha1(oldvalue, update->old_sha1))
+		die("invalid old value for ref %s: %s",
+		    update->ref_name, oldvalue);
+
+	/* We have an old value if non-empty, or if empty without -z */
+	update->have_old = *oldvalue || line_termination;
+}
+
+static const char *parse_arg(const char *next, struct strbuf *arg)
+{
+	/* Parse SP-terminated, possibly C-quoted argument */
+	if (*next != '"')
+		while (*next && !isspace(*next))
+			strbuf_addch(arg, *next++);
+	else if (unquote_c_style(arg, next, &next))
+		die("badly quoted argument: %s", next);
+
+	/* Return position after the argument */
+	return next;
+}
+
+static const char *parse_first_arg(const char *next, struct strbuf *arg)
+{
+	/* Parse argument immediately after "command SP" */
+	strbuf_reset(arg);
+	if (line_termination) {
+		/* Without -z, use the next argument */
+		next = parse_arg(next, arg);
+	} else {
+		/* With -z, use rest of first NUL-terminated line */
+		strbuf_addstr(arg, next);
+		next = next + arg->len;
+	}
+	return next;
+}
+
+static const char *parse_next_arg(const char *next, struct strbuf *arg)
+{
+	/* Parse next SP-terminated or NUL-terminated argument, if any */
+	strbuf_reset(arg);
+	if (line_termination) {
+		/* Without -z, consume SP and use next argument */
+		if (!*next)
+			return NULL;
+		if (*next != ' ')
+			die("expected SP but got: %s", next);
+		next = parse_arg(next + 1, arg);
+	} else {
+		/* With -z, read the next NUL-terminated line */
+		if (*next)
+			die("expected NUL but got: %s", next);
+		if (strbuf_getline(arg, stdin, '\0') == EOF)
+			return NULL;
+		next = arg->buf + arg->len;
+	}
+	return next;
+}
+
+static void parse_cmd_update(const char *next)
+{
+	struct strbuf ref = STRBUF_INIT;
+	struct strbuf newvalue = STRBUF_INIT;
+	struct strbuf oldvalue = STRBUF_INIT;
+	struct ref_update *update;
+
+	update = update_alloc();
+
+	if ((next = parse_first_arg(next, &ref)) != NULL && ref.buf[0])
+		update_store_ref_name(update, ref.buf);
+	else
+		die("update line missing <ref>");
+
+	if ((next = parse_next_arg(next, &newvalue)) != NULL)
+		update_store_new_sha1(update, newvalue.buf);
+	else
+		die("update %s missing <newvalue>", ref.buf);
+
+	if ((next = parse_next_arg(next, &oldvalue)) != NULL)
+		update_store_old_sha1(update, oldvalue.buf);
+	else if(!line_termination)
+		die("update %s missing [<oldvalue>] NUL", ref.buf);
+
+	if (next && *next)
+		die("update %s has extra input: %s", ref.buf, next);
+}
+
+static void parse_cmd_create(const char *next)
+{
+	struct strbuf ref = STRBUF_INIT;
+	struct strbuf newvalue = STRBUF_INIT;
+	struct ref_update *update;
+
+	update = update_alloc();
+
+	if ((next = parse_first_arg(next, &ref)) != NULL && ref.buf[0])
+		update_store_ref_name(update, ref.buf);
+	else
+		die("create line missing <ref>");
+
+	if ((next = parse_next_arg(next, &newvalue)) != NULL)
+		update_store_new_sha1(update, newvalue.buf);
+	else
+		die("create %s missing <newvalue>", ref.buf);
+	if (is_null_sha1(update->new_sha1))
+		die("create %s given zero new value", ref.buf);
+
+	if (next && *next)
+		die("create %s has extra input: %s", ref.buf, next);
+}
+
+static void parse_cmd_delete(const char *next)
+{
+	struct strbuf ref = STRBUF_INIT;
+	struct strbuf oldvalue = STRBUF_INIT;
+	struct ref_update *update;
+
+	update = update_alloc();
+
+	if ((next = parse_first_arg(next, &ref)) != NULL && ref.buf[0])
+		update_store_ref_name(update, ref.buf);
+	else
+		die("delete line missing <ref>");
+
+	if ((next = parse_next_arg(next, &oldvalue)) != NULL)
+		update_store_old_sha1(update, oldvalue.buf);
+	else if(!line_termination)
+		die("delete %s missing [<oldvalue>] NUL", ref.buf);
+	if (update->have_old && is_null_sha1(update->old_sha1))
+		die("delete %s given zero old value", ref.buf);
+
+	if (next && *next)
+		die("delete %s has extra input: %s", ref.buf, next);
+}
+
+static void parse_cmd_verify(const char *next)
+{
+	struct strbuf ref = STRBUF_INIT;
+	struct strbuf value = STRBUF_INIT;
+	struct ref_update *update;
+
+	update = update_alloc();
+
+	if ((next = parse_first_arg(next, &ref)) != NULL && ref.buf[0])
+		update_store_ref_name(update, ref.buf);
+	else
+		die("verify line missing <ref>");
+
+	if ((next = parse_next_arg(next, &value)) != NULL) {
+		update_store_old_sha1(update, value.buf);
+		update_store_new_sha1(update, value.buf);
+	} else if(!line_termination)
+		die("verify %s missing [<oldvalue>] NUL", ref.buf);
+
+	if (next && *next)
+		die("verify %s has extra input: %s", ref.buf, next);
+}
+
+static void parse_cmd_option(const char *next)
+{
+	if (!strcmp(next, "no-deref"))
+		update_flags |= REF_NODEREF;
+	else
+		die("option unknown: %s", next);
+}
+
+static void update_refs_stdin(void)
+{
+	struct strbuf cmd = STRBUF_INIT;
+
+	/* Read each line dispatch its command */
+	while (strbuf_getline(&cmd, stdin, line_termination) != EOF)
+		if (!cmd.buf[0])
+			die("empty command in input");
+		else if (isspace(*cmd.buf))
+			die("whitespace before command: %s", cmd.buf);
+		else if (!prefixcmp(cmd.buf, "update "))
+			parse_cmd_update(cmd.buf + 7);
+		else if (!prefixcmp(cmd.buf, "create "))
+			parse_cmd_create(cmd.buf + 7);
+		else if (!prefixcmp(cmd.buf, "delete "))
+			parse_cmd_delete(cmd.buf + 7);
+		else if (!prefixcmp(cmd.buf, "verify "))
+			parse_cmd_verify(cmd.buf + 7);
+		else if (!prefixcmp(cmd.buf, "option "))
+			parse_cmd_option(cmd.buf + 7);
+		else
+			die("unknown command: %s", cmd.buf);
+
+	strbuf_release(&cmd);
+}
+
 int cmd_update_ref(int argc, const char **argv, const char *prefix)
 {
 	const char *refname, *oldval, *msg = NULL;
 	unsigned char sha1[20], oldsha1[20];
-	int delete = 0, no_deref = 0, flags = 0;
+	int delete = 0, no_deref = 0, read_stdin = 0, end_null = 0, flags = 0;
 	struct option options[] = {
 		OPT_STRING( 'm', NULL, &msg, N_("reason"), N_("reason of the update")),
 		OPT_BOOLEAN('d', NULL, &delete, N_("delete the reference")),
+		OPT_BOOLEAN('z', NULL, &end_null, N_("stdin has NUL-terminated arguments")),
 		OPT_BOOLEAN( 0 , "no-deref", &no_deref,
 					N_("update <refname> not the one it points to")),
+		OPT_BOOLEAN( 0 , "stdin", &read_stdin, N_("read updates from stdin")),
 		OPT_END(),
 	};
 
@@ -28,6 +266,18 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)
 	if (msg && !*msg)
 		die("Refusing to perform update with empty message.");
 
+	if (read_stdin) {
+		if (delete || no_deref || argc > 0)
+			usage_with_options(git_update_ref_usage, options);
+		if (end_null)
+			line_termination = '\0';
+		update_refs_stdin();
+		return update_refs(msg, updates, updates_count, DIE_ON_ERR);
+	}
+
+	if (end_null)
+		usage_with_options(git_update_ref_usage, options);
+
 	if (delete) {
 		if (argc < 1 || argc > 2)
 			usage_with_options(git_update_ref_usage, options);
-- 
1.8.4.rc3

[PATCH v5 8/8] update-ref: add test cases covering --stdin signature

[Brad King]

Extend t/t1400-update-ref.sh to cover cases using the --stdin option.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 t/t1400-update-ref.sh | 639 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 639 insertions(+)

diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index e415ee0..a510500 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -302,4 +302,643 @@ test_expect_success \
 	'git cat-file blob master@{2005-05-26 23:42}:F (expect OTHER)' \
 	'test OTHER = $(git cat-file blob "master@{2005-05-26 23:42}:F")'
 
+a=refs/heads/a
+b=refs/heads/b
+c=refs/heads/c
+E='""'
+pws='path with space'
+
+print_nul() {
+	while test $# -gt 0; do
+		printf -- "$1" &&
+		printf -- "Q" | q_to_nul &&
+		shift || return
+	done
+}
+
+test_expect_success 'stdin test setup' '
+	echo "$pws" >"$pws" &&
+	git add -- "$pws" &&
+	git commit -m "$pws"
+'
+
+test_expect_success '-z fails without --stdin' '
+	test_must_fail git update-ref -z $m $m $m 2>err &&
+	grep "usage: git update-ref" err
+'
+
+test_expect_success 'stdin works with no input' '
+	>stdin &&
+	git update-ref --stdin <stdin &&
+	git rev-parse --verify -q $m
+'
+
+test_expect_success 'stdin fails on empty line' '
+	echo "" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: empty command in input" err
+'
+
+test_expect_success 'stdin fails on only whitespace' '
+	echo " " >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: whitespace before command:  " err
+'
+
+test_expect_success 'stdin fails on leading whitespace' '
+	echo " create $a $m" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: whitespace before command:  create $a $m" err
+'
+
+test_expect_success 'stdin fails on unknown command' '
+	echo "unknown $a" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: unknown command: unknown $a" err
+'
+
+test_expect_success 'stdin fails on badly quoted input' '
+	echo "create $a \"master" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: badly quoted argument: \\\"master" err
+'
+
+test_expect_success 'stdin fails on arguments not separated by space' '
+	echo "create \"$a\"master" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: expected SP but got: master" err
+'
+
+test_expect_success 'stdin fails create with no ref' '
+	echo "create " >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: create line missing <ref>" err
+'
+
+test_expect_success 'stdin fails create with bad ref name' '
+	echo "create ~a $m" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: invalid ref format: ~a" err
+'
+
+test_expect_success 'stdin fails create with no new value' '
+	echo "create $a" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: create $a missing <newvalue>" err
+'
+
+test_expect_success 'stdin fails create with too many arguments' '
+	echo "create $a $m $m" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: create $a has extra input:  $m" err
+'
+
+test_expect_success 'stdin fails update with no ref' '
+	echo "update " >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: update line missing <ref>" err
+'
+
+test_expect_success 'stdin fails update with bad ref name' '
+	echo "update ~a $m" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: invalid ref format: ~a" err
+'
+
+test_expect_success 'stdin fails update with no new value' '
+	echo "update $a" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: update $a missing <newvalue>" err
+'
+
+test_expect_success 'stdin fails update with too many arguments' '
+	echo "update $a $m $m $m" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: update $a has extra input:  $m" err
+'
+
+test_expect_success 'stdin fails delete with no ref' '
+	echo "delete " >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: delete line missing <ref>" err
+'
+
+test_expect_success 'stdin fails delete with bad ref name' '
+	echo "delete ~a $m" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: invalid ref format: ~a" err
+'
+
+test_expect_success 'stdin fails delete with too many arguments' '
+	echo "delete $a $m $m" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: delete $a has extra input:  $m" err
+'
+
+test_expect_success 'stdin fails verify with too many arguments' '
+	echo "verify $a $m $m" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: verify $a has extra input:  $m" err
+'
+
+test_expect_success 'stdin fails option with unknown name' '
+	echo "option unknown" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: option unknown: unknown" err
+'
+
+test_expect_success 'stdin fails with duplicate refs' '
+	cat >stdin <<-EOF &&
+	create $a $m
+	create $b $m
+	create $a $m
+	EOF
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: Multiple updates for ref '"'"'$a'"'"' not allowed." err
+'
+
+test_expect_success 'stdin create ref works' '
+	echo "create $a $m" >stdin &&
+	git update-ref --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin update ref creates with zero old value' '
+	echo "update $b $m $Z" >stdin &&
+	git update-ref --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	git update-ref -d $b
+'
+
+test_expect_success 'stdin update ref creates with empty old value' '
+	echo "update $b $m $E" >stdin &&
+	git update-ref --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin create ref works with path with space to blob' '
+	echo "create refs/blobs/pws \"$m:$pws\"" >stdin &&
+	git update-ref --stdin <stdin &&
+	git rev-parse "$m:$pws" >expect &&
+	git rev-parse refs/blobs/pws >actual &&
+	test_cmp expect actual &&
+	git update-ref -d refs/blobs/pws
+'
+
+test_expect_success 'stdin update ref fails with wrong old value' '
+	echo "update $c $m $m~1" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: Cannot lock the ref '"'"'$c'"'"'" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin update ref fails with bad old value' '
+	echo "update $c $m does-not-exist" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: invalid old value for ref $c: does-not-exist" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin create ref fails with bad new value' '
+	echo "create $c does-not-exist" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: invalid new value for ref $c: does-not-exist" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin create ref fails with zero new value' '
+	echo "create $c " >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: create $c given zero new value" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin update ref works with right old value' '
+	echo "update $b $m~1 $m" >stdin &&
+	git update-ref --stdin <stdin &&
+	git rev-parse $m~1 >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin delete ref fails with wrong old value' '
+	echo "delete $a $m~1" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: Cannot lock the ref '"'"'$a'"'"'" err &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin delete ref fails with zero old value' '
+	echo "delete $a " >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: delete $a given zero old value" err &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin update symref works option no-deref' '
+	git symbolic-ref TESTSYMREF $b &&
+	cat >stdin <<-EOF &&
+	option no-deref
+	update TESTSYMREF $a $b
+	EOF
+	git update-ref --stdin <stdin &&
+	git rev-parse TESTSYMREF >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $m~1 >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin delete symref works option no-deref' '
+	git symbolic-ref TESTSYMREF $b &&
+	cat >stdin <<-EOF &&
+	option no-deref
+	delete TESTSYMREF $b
+	EOF
+	git update-ref --stdin <stdin &&
+	test_must_fail git rev-parse --verify -q TESTSYMREF &&
+	git rev-parse $m~1 >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin delete ref works with right old value' '
+	echo "delete $b $m~1" >stdin &&
+	git update-ref --stdin <stdin &&
+	test_must_fail git rev-parse --verify -q $b
+'
+
+test_expect_success 'stdin update/create/verify combination works' '
+	cat >stdin <<-EOF &&
+	update $a $m
+	create $b $m
+	verify $c
+	EOF
+	git update-ref --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin update refs works with identity updates' '
+	cat >stdin <<-EOF &&
+	update $a $m $m
+	update $b $m $m
+	update $c $Z $E
+	EOF
+	git update-ref --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin update refs fails with wrong old value' '
+	git update-ref $c $m &&
+	cat >stdin <<-EOF &&
+	update $a $m $m
+	update $b $m $m
+	update $c  ''
+	EOF
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: Cannot lock the ref '"'"'$c'"'"'" err &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	git rev-parse $c >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin delete refs works with packed and loose refs' '
+	git pack-refs --all &&
+	git update-ref $c $m~1 &&
+	cat >stdin <<-EOF &&
+	delete $a $m
+	update $b $Z $m
+	update $c $E $m~1
+	EOF
+	git update-ref --stdin <stdin &&
+	test_must_fail git rev-parse --verify -q $a &&
+	test_must_fail git rev-parse --verify -q $b &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z works on empty input' '
+	>stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse --verify -q $m
+'
+
+test_expect_success 'stdin -z fails on empty line' '
+	echo "" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: whitespace before command: " err
+'
+
+test_expect_success 'stdin -z fails on empty command' '
+	print_nul "" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: empty command in input" err
+'
+
+test_expect_success 'stdin -z fails on only whitespace' '
+	print_nul " " >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: whitespace before command:  " err
+'
+
+test_expect_success 'stdin -z fails on leading whitespace' '
+	print_nul " create $a" "$m" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: whitespace before command:  create $a" err
+'
+
+test_expect_success 'stdin -z fails on unknown command' '
+	print_nul "unknown $a" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: unknown command: unknown $a" err
+'
+
+test_expect_success 'stdin -z fails create with no ref' '
+	print_nul "create " >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: create line missing <ref>" err
+'
+
+test_expect_success 'stdin -z fails create with bad ref name' '
+	print_nul "create ~a " "$m" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: invalid ref format: ~a " err
+'
+
+test_expect_success 'stdin -z fails create with no new value' '
+	print_nul "create $a" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: create $a missing <newvalue>" err
+'
+
+test_expect_success 'stdin -z fails create with too many arguments' '
+	print_nul "create $a" "$m" "$m" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: unknown command: $m" err
+'
+
+test_expect_success 'stdin -z fails update with no ref' '
+	print_nul "update " >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: update line missing <ref>" err
+'
+
+test_expect_success 'stdin -z fails update with bad ref name' '
+	print_nul "update ~a" "$m" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: invalid ref format: ~a" err
+'
+
+test_expect_success 'stdin -z fails update with no new value' '
+	print_nul "update $a" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: update $a missing <newvalue>" err
+'
+
+test_expect_success 'stdin -z fails update with no old value' '
+	print_nul "update $a" "$m" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: update $a missing \\[<oldvalue>\\] NUL" err
+'
+
+test_expect_success 'stdin -z fails update with too many arguments' '
+	print_nul "update $a" "$m" "$m" "$m" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: unknown command: $m" err
+'
+
+test_expect_success 'stdin -z fails delete with no ref' '
+	print_nul "delete " >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: delete line missing <ref>" err
+'
+
+test_expect_success 'stdin -z fails delete with bad ref name' '
+	print_nul "delete ~a" "$m" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: invalid ref format: ~a" err
+'
+
+test_expect_success 'stdin -z fails delete with no old value' '
+	print_nul "delete $a" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: delete $a missing \\[<oldvalue>\\] NUL" err
+'
+
+test_expect_success 'stdin -z fails delete with too many arguments' '
+	print_nul "delete $a" "$m" "$m" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: unknown command: $m" err
+'
+
+test_expect_success 'stdin -z fails verify with too many arguments' '
+	print_nul "verify $a" "$m" "$m" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: unknown command: $m" err
+'
+
+test_expect_success 'stdin -z fails verify with no old value' '
+	print_nul "verify $a" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: verify $a missing \\[<oldvalue>\\] NUL" err
+'
+
+test_expect_success 'stdin -z fails option with unknown name' '
+	print_nul "option unknown" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: option unknown: unknown" err
+'
+
+test_expect_success 'stdin -z fails with duplicate refs' '
+	print_nul "create $a" "$m" "create $b" "$m" "create $a" "$m" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: Multiple updates for ref '"'"'$a'"'"' not allowed." err
+'
+
+test_expect_success 'stdin -z create ref works' '
+	print_nul "create $a" "$m" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin -z update ref creates with zero old value' '
+	print_nul "update $b" "$m" "$Z" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	git update-ref -d $b
+'
+
+test_expect_success 'stdin -z update ref creates with empty old value' '
+	print_nul "update $b" "$m" "" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin -z create ref works with path with space to blob' '
+	print_nul "create refs/blobs/pws" "$m:$pws" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse "$m:$pws" >expect &&
+	git rev-parse refs/blobs/pws >actual &&
+	test_cmp expect actual &&
+	git update-ref -d refs/blobs/pws
+'
+
+test_expect_success 'stdin -z update ref fails with wrong old value' '
+	print_nul "update $c" "$m" "$m~1" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: Cannot lock the ref '"'"'$c'"'"'" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z update ref fails with bad old value' '
+	print_nul "update $c" "$m" "does-not-exist" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: invalid old value for ref $c: does-not-exist" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z create ref fails with bad new value' '
+	print_nul "create $c" "does-not-exist" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: invalid new value for ref $c: does-not-exist" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z create ref fails with zero new value' '
+	print_nul "create $c" "" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: create $c given zero new value" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z update ref works with right old value' '
+	print_nul "update $b" "$m~1" "$m" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse $m~1 >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin -z delete ref fails with wrong old value' '
+	print_nul "delete $a" "$m~1" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: Cannot lock the ref '"'"'$a'"'"'" err &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin -z delete ref fails with zero old value' '
+	print_nul "delete $a" "$Z" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: delete $a given zero old value" err &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin -z update symref works option no-deref' '
+	git symbolic-ref TESTSYMREF $b &&
+	print_nul "option no-deref" "update TESTSYMREF" "$a" "$b" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse TESTSYMREF >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $m~1 >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin -z delete symref works option no-deref' '
+	git symbolic-ref TESTSYMREF $b &&
+	print_nul "option no-deref" "delete TESTSYMREF" "$b" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	test_must_fail git rev-parse --verify -q TESTSYMREF &&
+	git rev-parse $m~1 >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin -z delete ref works with right old value' '
+	print_nul "delete $b" "$m~1" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	test_must_fail git rev-parse --verify -q $b
+'
+
+test_expect_success 'stdin -z update/create/verify combination works' '
+	print_nul "update $a" "$m" "" "create $b" "$m" "verify $c" "" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z update refs works with identity updates' '
+	print_nul "update $a" "$m" "$m" "update $b" "$m" "$m" "update $c" "$Z" "" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z update refs fails with wrong old value' '
+	git update-ref $c $m &&
+	print_nul "update $a" "$m" "$m" "update $b" "$m" "$m" "update $c" "" "$Z" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: Cannot lock the ref '"'"'$c'"'"'" err &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	git rev-parse $c >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin -z delete refs works with packed and loose refs' '
+	git pack-refs --all &&
+	git update-ref $c $m~1 &&
+	print_nul "delete $a" "$m" "update $b" "$Z" "$m" "update $c" "" "$m~1" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	test_must_fail git rev-parse --verify -q $a &&
+	test_must_fail git rev-parse --verify -q $b &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
 test_done
-- 
1.8.4.rc3

[PATCH v6 0/8] Multiple simultaneously locked ref updates

[Brad King]

Hi Folks,

Here is the sixth revision of a series to support locking multiple
refs at the same time to update all of them consistently.  The
previous revisions of the series can be found at $gmane/233260,
$gmane/233458, $gmane/233647, $gmane/233840, and $gmane/234324.

Updates since the previous revision of the series:

* The entire series was rebased on master at bb80ee09; it was
  previously based on v1.8.4.

* A conflict in refs.c with 47a59185 was resolved by preserving
  the elimination of find_ref_by_name while adding our new content.

* A conflict in builtin/update-ref.c with d5d09d47 (Replace deprecated
  OPT_BOOLEAN by OPT_BOOL, 2013-08-03) was resolved by integrating
  both changes.  The new options added in patch 7 now use OPT_BOOL.

-Brad

Brad King (8):
  reset: rename update_refs to reset_refs
  refs: report ref type from lock_any_ref_for_update
  refs: factor update_ref steps into helpers
  refs: factor delete_ref loose ref step into a helper
  refs: add function to repack without multiple refs
  refs: add update_refs for multiple simultaneous updates
  update-ref: support multiple simultaneous updates
  update-ref: add test cases covering --stdin signature

 Documentation/git-update-ref.txt |  54 +++-
 branch.c                         |   2 +-
 builtin/commit.c                 |   2 +-
 builtin/fetch.c                  |   3 +-
 builtin/receive-pack.c           |   3 +-
 builtin/reflog.c                 |   2 +-
 builtin/replace.c                |   2 +-
 builtin/reset.c                  |   4 +-
 builtin/tag.c                    |   2 +-
 builtin/update-ref.c             | 252 ++++++++++++++-
 fast-import.c                    |   2 +-
 refs.c                           | 195 ++++++++++--
 refs.h                           |  22 +-
 sequencer.c                      |   3 +-
 t/t1400-update-ref.sh            | 639 +++++++++++++++++++++++++++++++++++++++
 15 files changed, 1146 insertions(+), 41 deletions(-)

-- 
1.8.4.rc3

[PATCH v6 1/8] reset: rename update_refs to reset_refs

[Brad King]

The function resets refs rather than doing arbitrary updates.
Rename it to allow a future general-purpose update_refs function
to be added.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 builtin/reset.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/builtin/reset.c b/builtin/reset.c
index 5e4c551..88b4fde 100644
--- a/builtin/reset.c
+++ b/builtin/reset.c
@@ -226,7 +226,7 @@ static void parse_args(struct pathspec *pathspec,
 		       prefix, argv);
 }
 
-static int update_refs(const char *rev, const unsigned char *sha1)
+static int reset_refs(const char *rev, const unsigned char *sha1)
 {
 	int update_ref_status;
 	struct strbuf msg = STRBUF_INIT;
@@ -357,7 +357,7 @@ int cmd_reset(int argc, const char **argv, const char *prefix)
 	if (!pathspec.nr && !unborn) {
 		/* Any resets without paths update HEAD to the head being
 		 * switched to, saving the previous head in ORIG_HEAD before. */
-		update_ref_status = update_refs(rev, sha1);
+		update_ref_status = reset_refs(rev, sha1);
 
 		if (reset_type == HARD && !update_ref_status && !quiet)
 			print_new_head_line(lookup_commit_reference(sha1));
-- 
1.8.4.rc3

Re: [PATCH v6 1/8] reset: rename update_refs to reset_refs

[Ramkumar Ramachandra]

Brad King wrote:
> The function resets refs rather than doing arbitrary updates.
> Rename it to allow a future general-purpose update_refs function
> to be added.

Makes sense; thanks.

[PATCH v6 2/8] refs: report ref type from lock_any_ref_for_update

[Brad King]

Expose lock_ref_sha1_basic's type_p argument to callers of
lock_any_ref_for_update.  Update all call sites to ignore it by passing
NULL for now.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 branch.c               | 2 +-
 builtin/commit.c       | 2 +-
 builtin/fetch.c        | 3 ++-
 builtin/receive-pack.c | 3 ++-
 builtin/reflog.c       | 2 +-
 builtin/replace.c      | 2 +-
 builtin/tag.c          | 2 +-
 fast-import.c          | 2 +-
 refs.c                 | 7 ++++---
 refs.h                 | 2 +-
 sequencer.c            | 3 ++-
 11 files changed, 17 insertions(+), 13 deletions(-)

diff --git a/branch.c b/branch.c
index c5c6984..f2d383f 100644
--- a/branch.c
+++ b/branch.c
@@ -291,7 +291,7 @@ void create_branch(const char *head,
 	hashcpy(sha1, commit->object.sha1);
 
 	if (!dont_change_ref) {
-		lock = lock_any_ref_for_update(ref.buf, NULL, 0);
+		lock = lock_any_ref_for_update(ref.buf, NULL, 0, NULL);
 		if (!lock)
 			die_errno(_("Failed to lock ref for update"));
 	}
diff --git a/builtin/commit.c b/builtin/commit.c
index 80d886a..33d6f5a 100644
--- a/builtin/commit.c
+++ b/builtin/commit.c
@@ -1615,7 +1615,7 @@ int cmd_commit(int argc, const char **argv, const char *prefix)
 					   !current_head
 					   ? NULL
 					   : current_head->object.sha1,
-					   0);
+					   0, NULL);
 
 	nl = strchr(sb.buf, '\n');
 	if (nl)
diff --git a/builtin/fetch.c b/builtin/fetch.c
index 9e654ef..bd7a101 100644
--- a/builtin/fetch.c
+++ b/builtin/fetch.c
@@ -262,7 +262,8 @@ static int s_update_ref(const char *action,
 		rla = default_rla.buf;
 	snprintf(msg, sizeof(msg), "%s: %s", rla, action);
 	lock = lock_any_ref_for_update(ref->name,
-				       check_old ? ref->old_sha1 : NULL, 0);
+				       check_old ? ref->old_sha1 : NULL,
+				       0, NULL);
 	if (!lock)
 		return errno == ENOTDIR ? STORE_REF_ERROR_DF_CONFLICT :
 					  STORE_REF_ERROR_OTHER;
diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 7434d9b..d8b074f 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -525,7 +525,8 @@ static const char *update(struct command *cmd)
 		return NULL; /* good */
 	}
 	else {
-		lock = lock_any_ref_for_update(namespaced_name, old_sha1, 0);
+		lock = lock_any_ref_for_update(namespaced_name, old_sha1,
+					       0, NULL);
 		if (!lock) {
 			rp_error("failed to lock %s", name);
 			return "failed to lock";
diff --git a/builtin/reflog.c b/builtin/reflog.c
index 54184b3..28d756a 100644
--- a/builtin/reflog.c
+++ b/builtin/reflog.c
@@ -366,7 +366,7 @@ static int expire_reflog(const char *ref, const unsigned char *sha1, int unused,
 	 * we take the lock for the ref itself to prevent it from
 	 * getting updated.
 	 */
-	lock = lock_any_ref_for_update(ref, sha1, 0);
+	lock = lock_any_ref_for_update(ref, sha1, 0, NULL);
 	if (!lock)
 		return error("cannot lock ref '%s'", ref);
 	log_file = git_pathdup("logs/%s", ref);
diff --git a/builtin/replace.c b/builtin/replace.c
index 11b0a55..301b45c 100644
--- a/builtin/replace.c
+++ b/builtin/replace.c
@@ -105,7 +105,7 @@ static int replace_object(const char *object_ref, const char *replace_ref,
 	else if (!force)
 		die("replace ref '%s' already exists", ref);
 
-	lock = lock_any_ref_for_update(ref, prev, 0);
+	lock = lock_any_ref_for_update(ref, prev, 0, NULL);
 	if (!lock)
 		die("%s: cannot lock the ref", ref);
 	if (write_ref_sha1(lock, repl, NULL) < 0)
diff --git a/builtin/tag.c b/builtin/tag.c
index b577af5..ea55f1d 100644
--- a/builtin/tag.c
+++ b/builtin/tag.c
@@ -574,7 +574,7 @@ int cmd_tag(int argc, const char **argv, const char *prefix)
 	if (annotate)
 		create_tag(object, tag, &buf, &opt, prev, object);
 
-	lock = lock_any_ref_for_update(ref.buf, prev, 0);
+	lock = lock_any_ref_for_update(ref.buf, prev, 0, NULL);
 	if (!lock)
 		die(_("%s: cannot lock the ref"), ref.buf);
 	if (write_ref_sha1(lock, object, NULL) < 0)
diff --git a/fast-import.c b/fast-import.c
index 21db3fc..8036cc4 100644
--- a/fast-import.c
+++ b/fast-import.c
@@ -1694,7 +1694,7 @@ static int update_branch(struct branch *b)
 		return 0;
 	if (read_ref(b->name, old_sha1))
 		hashclr(old_sha1);
-	lock = lock_any_ref_for_update(b->name, old_sha1, 0);
+	lock = lock_any_ref_for_update(b->name, old_sha1, 0, NULL);
 	if (!lock)
 		return error("Unable to lock %s", b->name);
 	if (!force_update && !is_null_sha1(old_sha1)) {
diff --git a/refs.c b/refs.c
index d78860c..5832a8f 100644
--- a/refs.c
+++ b/refs.c
@@ -2121,11 +2121,12 @@ struct ref_lock *lock_ref_sha1(const char *refname, const unsigned char *old_sha
 }
 
 struct ref_lock *lock_any_ref_for_update(const char *refname,
-					 const unsigned char *old_sha1, int flags)
+					 const unsigned char *old_sha1,
+					 int flags, int *type_p)
 {
 	if (check_refname_format(refname, REFNAME_ALLOW_ONELEVEL))
 		return NULL;
-	return lock_ref_sha1_basic(refname, old_sha1, flags, NULL);
+	return lock_ref_sha1_basic(refname, old_sha1, flags, type_p);
 }
 
 /*
@@ -3174,7 +3175,7 @@ int update_ref(const char *action, const char *refname,
 		int flags, enum action_on_err onerr)
 {
 	static struct ref_lock *lock;
-	lock = lock_any_ref_for_update(refname, oldval, flags);
+	lock = lock_any_ref_for_update(refname, oldval, flags, NULL);
 	if (!lock) {
 		const char *str = "Cannot lock the ref '%s'.";
 		switch (onerr) {
diff --git a/refs.h b/refs.h
index 9e5db3a..2cd307a 100644
--- a/refs.h
+++ b/refs.h
@@ -137,7 +137,7 @@ extern struct ref_lock *lock_ref_sha1(const char *refname, const unsigned char *
 #define REF_NODEREF	0x01
 extern struct ref_lock *lock_any_ref_for_update(const char *refname,
 						const unsigned char *old_sha1,
-						int flags);
+						int flags, int *type_p);
 
 /** Close the file descriptor owned by a lock and return the status */
 extern int close_ref(struct ref_lock *lock);
diff --git a/sequencer.c b/sequencer.c
index 351548f..06e52b4 100644
--- a/sequencer.c
+++ b/sequencer.c
@@ -279,7 +279,8 @@ static int fast_forward_to(const unsigned char *to, const unsigned char *from,
 	read_cache();
 	if (checkout_fast_forward(from, to, 1))
 		exit(1); /* the callee should have complained already */
-	ref_lock = lock_any_ref_for_update("HEAD", unborn ? null_sha1 : from, 0);
+	ref_lock = lock_any_ref_for_update("HEAD", unborn ? null_sha1 : from,
+					   0, NULL);
 	strbuf_addf(&sb, "%s: fast-forward", action_name(opts));
 	ret = write_ref_sha1(ref_lock, to, sb.buf);
 	strbuf_release(&sb);
-- 
1.8.4.rc3

[PATCH v6 3/8] refs: factor update_ref steps into helpers

[Brad King]

Factor the lock and write steps and error handling into helper functions
update_ref_lock and update_ref_write to allow later use elsewhere.
Expose lock_any_ref_for_update's type_p to update_ref_lock callers.

While at it, drop "static" from the local "lock" variable as it is not
necessary to keep across invocations.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 refs.c | 30 ++++++++++++++++++++++++------
 1 file changed, 24 insertions(+), 6 deletions(-)

diff --git a/refs.c b/refs.c
index 5832a8f..f7d3c09 100644
--- a/refs.c
+++ b/refs.c
@@ -3170,12 +3170,13 @@ int for_each_reflog(each_ref_fn fn, void *cb_data)
 	return retval;
 }
 
-int update_ref(const char *action, const char *refname,
-		const unsigned char *sha1, const unsigned char *oldval,
-		int flags, enum action_on_err onerr)
+static struct ref_lock *update_ref_lock(const char *refname,
+					const unsigned char *oldval,
+					int flags, int *type_p,
+					enum action_on_err onerr)
 {
-	static struct ref_lock *lock;
-	lock = lock_any_ref_for_update(refname, oldval, flags, NULL);
+	struct ref_lock *lock;
+	lock = lock_any_ref_for_update(refname, oldval, flags, type_p);
 	if (!lock) {
 		const char *str = "Cannot lock the ref '%s'.";
 		switch (onerr) {
@@ -3183,8 +3184,14 @@ int update_ref(const char *action, const char *refname,
 		case DIE_ON_ERR: die(str, refname); break;
 		case QUIET_ON_ERR: break;
 		}
-		return 1;
 	}
+	return lock;
+}
+
+static int update_ref_write(const char *action, const char *refname,
+			    const unsigned char *sha1, struct ref_lock *lock,
+			    enum action_on_err onerr)
+{
 	if (write_ref_sha1(lock, sha1, action) < 0) {
 		const char *str = "Cannot update the ref '%s'.";
 		switch (onerr) {
@@ -3197,6 +3204,17 @@ int update_ref(const char *action, const char *refname,
 	return 0;
 }
 
+int update_ref(const char *action, const char *refname,
+	       const unsigned char *sha1, const unsigned char *oldval,
+	       int flags, enum action_on_err onerr)
+{
+	struct ref_lock *lock;
+	lock = update_ref_lock(refname, oldval, flags, 0, onerr);
+	if (!lock)
+		return 1;
+	return update_ref_write(action, refname, sha1, lock, onerr);
+}
+
 /*
  * generate a format suitable for scanf from a ref_rev_parse_rules
  * rule, that is replace the "%.*s" spec with a "%s" spec
-- 
1.8.4.rc3

[PATCH v6 4/8] refs: factor delete_ref loose ref step into a helper

[Brad King]

Factor loose ref deletion into helper function delete_ref_loose to allow
later use elsewhere.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 refs.c | 27 +++++++++++++++++----------
 1 file changed, 17 insertions(+), 10 deletions(-)

diff --git a/refs.c b/refs.c
index f7d3c09..b14f59b 100644
--- a/refs.c
+++ b/refs.c
@@ -2450,24 +2450,31 @@ static int repack_without_ref(const char *refname)
 	return commit_packed_refs();
 }
 
+static int delete_ref_loose(struct ref_lock *lock, int flag)
+{
+	if (!(flag & REF_ISPACKED) || flag & REF_ISSYMREF) {
+		/* loose */
+		int err, i = strlen(lock->lk->filename) - 5; /* .lock */
+
+		lock->lk->filename[i] = 0;
+		err = unlink_or_warn(lock->lk->filename);
+		lock->lk->filename[i] = '.';
+		if (err && errno != ENOENT)
+			return 1;
+	}
+	return 0;
+}
+
 int delete_ref(const char *refname, const unsigned char *sha1, int delopt)
 {
 	struct ref_lock *lock;
-	int err, i = 0, ret = 0, flag = 0;
+	int ret = 0, flag = 0;
 
 	lock = lock_ref_sha1_basic(refname, sha1, delopt, &flag);
 	if (!lock)
 		return 1;
-	if (!(flag & REF_ISPACKED) || flag & REF_ISSYMREF) {
-		/* loose */
-		i = strlen(lock->lk->filename) - 5; /* .lock */
-		lock->lk->filename[i] = 0;
-		err = unlink_or_warn(lock->lk->filename);
-		if (err && errno != ENOENT)
-			ret = 1;
+	ret |= delete_ref_loose(lock, flag);
 
-		lock->lk->filename[i] = '.';
-	}
 	/* removing the loose one could have resurrected an earlier
 	 * packed one.  Also, if it was not loose we need to repack
 	 * without it.
-- 
1.8.4.rc3

[PATCH v6 5/8] refs: add function to repack without multiple refs

[Brad King]

Generalize repack_without_ref as repack_without_refs to support a list
of refs and implement the former in terms of the latter.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 refs.c | 33 ++++++++++++++++++++++++---------
 1 file changed, 24 insertions(+), 9 deletions(-)

diff --git a/refs.c b/refs.c
index b14f59b..4a63404 100644
--- a/refs.c
+++ b/refs.c
@@ -2414,42 +2414,57 @@ static int curate_packed_ref_fn(struct ref_entry *entry, void *cb_data)
 	return 0;
 }
 
-static int repack_without_ref(const char *refname)
+static int repack_without_refs(const char **refnames, int n)
 {
 	struct ref_dir *packed;
 	struct string_list refs_to_delete = STRING_LIST_INIT_DUP;
 	struct string_list_item *ref_to_delete;
+	int i, removed = 0;
+
+	/* Look for a packed ref */
+	for (i = 0; i < n; i++)
+		if (get_packed_ref(refnames[i]))
+			break;
 
-	if (!get_packed_ref(refname))
-		return 0; /* refname does not exist in packed refs */
+	/* Avoid locking if we have nothing to do */
+	if (i == n)
+		return 0; /* no refname exists in packed refs */
 
 	if (lock_packed_refs(0)) {
 		unable_to_lock_error(git_path("packed-refs"), errno);
-		return error("cannot delete '%s' from packed refs", refname);
+		return error("cannot delete '%s' from packed refs", refnames[i]);
 	}
 	packed = get_packed_refs(&ref_cache);
 
-	/* Remove refname from the cache: */
-	if (remove_entry(packed, refname) == -1) {
+	/* Remove refnames from the cache */
+	for (i = 0; i < n; i++)
+		if (remove_entry(packed, refnames[i]) != -1)
+			removed = 1;
+	if (!removed) {
 		/*
-		 * The packed entry disappeared while we were
+		 * All packed entries disappeared while we were
 		 * acquiring the lock.
 		 */
 		rollback_packed_refs();
 		return 0;
 	}
 
-	/* Remove any other accumulated cruft: */
+	/* Remove any other accumulated cruft */
 	do_for_each_entry_in_dir(packed, 0, curate_packed_ref_fn, &refs_to_delete);
 	for_each_string_list_item(ref_to_delete, &refs_to_delete) {
 		if (remove_entry(packed, ref_to_delete->string) == -1)
 			die("internal error");
 	}
 
-	/* Write what remains: */
+	/* Write what remains */
 	return commit_packed_refs();
 }
 
+static int repack_without_ref(const char *refname)
+{
+	return repack_without_refs(&refname, 1);
+}
+
 static int delete_ref_loose(struct ref_lock *lock, int flag)
 {
 	if (!(flag & REF_ISPACKED) || flag & REF_ISSYMREF) {
-- 
1.8.4.rc3

[PATCH v6 6/8] refs: add update_refs for multiple simultaneous updates

[Brad King]

Add 'struct ref_update' to encode the information needed to update or
delete a ref (name, new sha1, optional old sha1, no-deref flag).  Add
function 'update_refs' accepting an array of updates to perform.  First
sort the input array to order locks consistently everywhere and reject
multiple updates to the same ref.  Then acquire locks on all refs with
verified old values.  Then update or delete all refs accordingly.  Fail
if any one lock cannot be obtained or any one old value does not match.

Though the refs themselves cannot be modified together in a single
atomic transaction, this function does enable some useful semantics.
For example, a caller may create a new branch starting from the head of
another branch and rewind the original branch at the same time.  This
transfers ownership of commits between branches without risk of losing
commits added to the original branch by a concurrent process, or risk of
a concurrent process creating the new branch first.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 refs.c | 100 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 refs.h |  20 +++++++++++++
 2 files changed, 120 insertions(+)

diff --git a/refs.c b/refs.c
index 4a63404..6383813 100644
--- a/refs.c
+++ b/refs.c
@@ -3237,6 +3237,106 @@ int update_ref(const char *action, const char *refname,
 	return update_ref_write(action, refname, sha1, lock, onerr);
 }
 
+static int ref_update_compare(const void *r1, const void *r2)
+{
+	const struct ref_update * const *u1 = r1;
+	const struct ref_update * const *u2 = r2;
+	return strcmp((*u1)->ref_name, (*u2)->ref_name);
+}
+
+static int ref_update_reject_duplicates(struct ref_update **updates, int n,
+					enum action_on_err onerr)
+{
+	int i;
+	for (i = 1; i < n; i++)
+		if (!strcmp(updates[i - 1]->ref_name, updates[i]->ref_name)) {
+			const char *str =
+				"Multiple updates for ref '%s' not allowed.";
+			switch (onerr) {
+			case MSG_ON_ERR:
+				error(str, updates[i]->ref_name); break;
+			case DIE_ON_ERR:
+				die(str, updates[i]->ref_name); break;
+			case QUIET_ON_ERR:
+				break;
+			}
+			return 1;
+		}
+	return 0;
+}
+
+int update_refs(const char *action, const struct ref_update **updates_orig,
+		int n, enum action_on_err onerr)
+{
+	int ret = 0, delnum = 0, i;
+	struct ref_update **updates;
+	int *types;
+	struct ref_lock **locks;
+	const char **delnames;
+
+	if (!updates_orig || !n)
+		return 0;
+
+	/* Allocate work space */
+	updates = xmalloc(sizeof(*updates) * n);
+	types = xmalloc(sizeof(*types) * n);
+	locks = xcalloc(n, sizeof(*locks));
+	delnames = xmalloc(sizeof(*delnames) * n);
+
+	/* Copy, sort, and reject duplicate refs */
+	memcpy(updates, updates_orig, sizeof(*updates) * n);
+	qsort(updates, n, sizeof(*updates), ref_update_compare);
+	ret = ref_update_reject_duplicates(updates, n, onerr);
+	if (ret)
+		goto cleanup;
+
+	/* Acquire all locks while verifying old values */
+	for (i = 0; i < n; i++) {
+		locks[i] = update_ref_lock(updates[i]->ref_name,
+					   (updates[i]->have_old ?
+					    updates[i]->old_sha1 : NULL),
+					   updates[i]->flags,
+					   &types[i], onerr);
+		if (!locks[i]) {
+			ret = 1;
+			goto cleanup;
+		}
+	}
+
+	/* Perform updates first so live commits remain referenced */
+	for (i = 0; i < n; i++)
+		if (!is_null_sha1(updates[i]->new_sha1)) {
+			ret = update_ref_write(action,
+					       updates[i]->ref_name,
+					       updates[i]->new_sha1,
+					       locks[i], onerr);
+			locks[i] = NULL; /* freed by update_ref_write */
+			if (ret)
+				goto cleanup;
+		}
+
+	/* Perform deletes now that updates are safely completed */
+	for (i = 0; i < n; i++)
+		if (locks[i]) {
+			delnames[delnum++] = locks[i]->ref_name;
+			ret |= delete_ref_loose(locks[i], types[i]);
+		}
+	ret |= repack_without_refs(delnames, delnum);
+	for (i = 0; i < delnum; i++)
+		unlink_or_warn(git_path("logs/%s", delnames[i]));
+	clear_loose_ref_cache(&ref_cache);
+
+cleanup:
+	for (i = 0; i < n; i++)
+		if (locks[i])
+			unlock_ref(locks[i]);
+	free(updates);
+	free(types);
+	free(locks);
+	free(delnames);
+	return ret;
+}
+
 /*
  * generate a format suitable for scanf from a ref_rev_parse_rules
  * rule, that is replace the "%.*s" spec with a "%s" spec
diff --git a/refs.h b/refs.h
index 2cd307a..b113377 100644
--- a/refs.h
+++ b/refs.h
@@ -10,6 +10,20 @@ struct ref_lock {
 	int force_write;
 };
 
+/**
+ * Information needed for a single ref update.  Set new_sha1 to the
+ * new value or to zero to delete the ref.  To check the old value
+ * while locking the ref, set have_old to 1 and set old_sha1 to the
+ * value or to zero to ensure the ref does not exist before update.
+ */
+struct ref_update {
+	const char *ref_name;
+	unsigned char new_sha1[20];
+	unsigned char old_sha1[20];
+	int flags; /* REF_NODEREF? */
+	int have_old; /* 1 if old_sha1 is valid, 0 otherwise */
+};
+
 /*
  * Bit values set in the flags argument passed to each_ref_fn():
  */
@@ -214,6 +228,12 @@ int update_ref(const char *action, const char *refname,
 		const unsigned char *sha1, const unsigned char *oldval,
 		int flags, enum action_on_err onerr);
 
+/**
+ * Lock all refs and then perform all modifications.
+ */
+int update_refs(const char *action, const struct ref_update **updates,
+		int n, enum action_on_err onerr);
+
 extern int parse_hide_refs_config(const char *var, const char *value, const char *);
 extern int ref_is_hidden(const char *);
 
-- 
1.8.4.rc3

[PATCH v6 7/8] update-ref: support multiple simultaneous updates

[Brad King]

Add a --stdin signature to read update instructions from standard input
and apply multiple ref updates together.  Use an input format that
supports any update that could be specified via the command-line,
including object names like "branch:path with space".

Signed-off-by: Brad King <brad.king@kitware.com>
---
 Documentation/git-update-ref.txt |  54 ++++++++-
 builtin/update-ref.c             | 252 ++++++++++++++++++++++++++++++++++++++-
 2 files changed, 304 insertions(+), 2 deletions(-)

diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
index 0df13ff..0a0a551 100644
--- a/Documentation/git-update-ref.txt
+++ b/Documentation/git-update-ref.txt
@@ -8,7 +8,7 @@ git-update-ref - Update the object name stored in a ref safely
 SYNOPSIS
 --------
 [verse]
-'git update-ref' [-m <reason>] (-d <ref> [<oldvalue>] | [--no-deref] <ref> <newvalue> [<oldvalue>])
+'git update-ref' [-m <reason>] (-d <ref> [<oldvalue>] | [--no-deref] <ref> <newvalue> [<oldvalue>] | --stdin [-z])
 
 DESCRIPTION
 -----------
@@ -58,6 +58,58 @@ archive by creating a symlink tree).
 With `-d` flag, it deletes the named <ref> after verifying it
 still contains <oldvalue>.
 
+With `--stdin`, update-ref reads instructions from standard input and
+performs all modifications together.  Specify commands of the form:
+
+	update SP <ref> SP <newvalue> [SP <oldvalue>] LF
+	create SP <ref> SP <newvalue> LF
+	delete SP <ref> [SP <oldvalue>] LF
+	verify SP <ref> [SP <oldvalue>] LF
+	option SP <opt> LF
+
+Quote fields containing whitespace as if they were strings in C source
+code.  Alternatively, use `-z` to specify commands without quoting:
+
+	update SP <ref> NUL <newvalue> NUL [<oldvalue>] NUL
+	create SP <ref> NUL <newvalue> NUL
+	delete SP <ref> NUL [<oldvalue>] NUL
+	verify SP <ref> NUL [<oldvalue>] NUL
+	option SP <opt> NUL
+
+Lines of any other format or a repeated <ref> produce an error.
+Command meanings are:
+
+update::
+	Set <ref> to <newvalue> after verifying <oldvalue>, if given.
+	Specify a zero <newvalue> to ensure the ref does not exist
+	after the update and/or a zero <oldvalue> to make sure the
+	ref does not exist before the update.
+
+create::
+	Create <ref> with <newvalue> after verifying it does not
+	exist.  The given <newvalue> may not be zero.
+
+delete::
+	Delete <ref> after verifying it exists with <oldvalue>, if
+	given.  If given, <oldvalue> may not be zero.
+
+verify::
+	Verify <ref> against <oldvalue> but do not change it.  If
+	<oldvalue> zero or missing, the ref must not exist.
+
+option::
+	Modify behavior of the next command naming a <ref>.
+	The only valid option is `no-deref` to avoid dereferencing
+	a symbolic ref.
+
+Use 40 "0" or the empty string to specify a zero value, except that
+with `-z` an empty <oldvalue> is considered missing.
+
+If all <ref>s can be locked with matching <oldvalue>s
+simultaneously, all modifications are performed.  Otherwise, no
+modifications are performed.  Note that while each individual
+<ref> is updated or deleted atomically, a concurrent reader may
+still see a subset of the modifications.
 
 Logging Updates
 ---------------
diff --git a/builtin/update-ref.c b/builtin/update-ref.c
index 7484d36..b5479e2 100644
--- a/builtin/update-ref.c
+++ b/builtin/update-ref.c
@@ -2,23 +2,261 @@
 #include "refs.h"
 #include "builtin.h"
 #include "parse-options.h"
+#include "quote.h"
+#include "argv-array.h"
 
 static const char * const git_update_ref_usage[] = {
 	N_("git update-ref [options] -d <refname> [<oldval>]"),
 	N_("git update-ref [options]    <refname> <newval> [<oldval>]"),
+	N_("git update-ref [options] --stdin [-z]"),
 	NULL
 };
 
+static int updates_alloc;
+static int updates_count;
+static const struct ref_update **updates;
+
+static char line_termination = '\n';
+static int update_flags;
+
+static struct ref_update *update_alloc(void)
+{
+	struct ref_update *update;
+
+	/* Allocate and zero-init a struct ref_update */
+	update = xcalloc(1, sizeof(*update));
+	ALLOC_GROW(updates, updates_count + 1, updates_alloc);
+	updates[updates_count++] = update;
+
+	/* Store and reset accumulated options */
+	update->flags = update_flags;
+	update_flags = 0;
+
+	return update;
+}
+
+static void update_store_ref_name(struct ref_update *update,
+				  const char *ref_name)
+{
+	if (check_refname_format(ref_name, REFNAME_ALLOW_ONELEVEL))
+		die("invalid ref format: %s", ref_name);
+	update->ref_name = xstrdup(ref_name);
+}
+
+static void update_store_new_sha1(struct ref_update *update,
+				  const char *newvalue)
+{
+	if (*newvalue && get_sha1(newvalue, update->new_sha1))
+		die("invalid new value for ref %s: %s",
+		    update->ref_name, newvalue);
+}
+
+static void update_store_old_sha1(struct ref_update *update,
+				  const char *oldvalue)
+{
+	if (*oldvalue && get_sha1(oldvalue, update->old_sha1))
+		die("invalid old value for ref %s: %s",
+		    update->ref_name, oldvalue);
+
+	/* We have an old value if non-empty, or if empty without -z */
+	update->have_old = *oldvalue || line_termination;
+}
+
+static const char *parse_arg(const char *next, struct strbuf *arg)
+{
+	/* Parse SP-terminated, possibly C-quoted argument */
+	if (*next != '"')
+		while (*next && !isspace(*next))
+			strbuf_addch(arg, *next++);
+	else if (unquote_c_style(arg, next, &next))
+		die("badly quoted argument: %s", next);
+
+	/* Return position after the argument */
+	return next;
+}
+
+static const char *parse_first_arg(const char *next, struct strbuf *arg)
+{
+	/* Parse argument immediately after "command SP" */
+	strbuf_reset(arg);
+	if (line_termination) {
+		/* Without -z, use the next argument */
+		next = parse_arg(next, arg);
+	} else {
+		/* With -z, use rest of first NUL-terminated line */
+		strbuf_addstr(arg, next);
+		next = next + arg->len;
+	}
+	return next;
+}
+
+static const char *parse_next_arg(const char *next, struct strbuf *arg)
+{
+	/* Parse next SP-terminated or NUL-terminated argument, if any */
+	strbuf_reset(arg);
+	if (line_termination) {
+		/* Without -z, consume SP and use next argument */
+		if (!*next)
+			return NULL;
+		if (*next != ' ')
+			die("expected SP but got: %s", next);
+		next = parse_arg(next + 1, arg);
+	} else {
+		/* With -z, read the next NUL-terminated line */
+		if (*next)
+			die("expected NUL but got: %s", next);
+		if (strbuf_getline(arg, stdin, '\0') == EOF)
+			return NULL;
+		next = arg->buf + arg->len;
+	}
+	return next;
+}
+
+static void parse_cmd_update(const char *next)
+{
+	struct strbuf ref = STRBUF_INIT;
+	struct strbuf newvalue = STRBUF_INIT;
+	struct strbuf oldvalue = STRBUF_INIT;
+	struct ref_update *update;
+
+	update = update_alloc();
+
+	if ((next = parse_first_arg(next, &ref)) != NULL && ref.buf[0])
+		update_store_ref_name(update, ref.buf);
+	else
+		die("update line missing <ref>");
+
+	if ((next = parse_next_arg(next, &newvalue)) != NULL)
+		update_store_new_sha1(update, newvalue.buf);
+	else
+		die("update %s missing <newvalue>", ref.buf);
+
+	if ((next = parse_next_arg(next, &oldvalue)) != NULL)
+		update_store_old_sha1(update, oldvalue.buf);
+	else if(!line_termination)
+		die("update %s missing [<oldvalue>] NUL", ref.buf);
+
+	if (next && *next)
+		die("update %s has extra input: %s", ref.buf, next);
+}
+
+static void parse_cmd_create(const char *next)
+{
+	struct strbuf ref = STRBUF_INIT;
+	struct strbuf newvalue = STRBUF_INIT;
+	struct ref_update *update;
+
+	update = update_alloc();
+
+	if ((next = parse_first_arg(next, &ref)) != NULL && ref.buf[0])
+		update_store_ref_name(update, ref.buf);
+	else
+		die("create line missing <ref>");
+
+	if ((next = parse_next_arg(next, &newvalue)) != NULL)
+		update_store_new_sha1(update, newvalue.buf);
+	else
+		die("create %s missing <newvalue>", ref.buf);
+	if (is_null_sha1(update->new_sha1))
+		die("create %s given zero new value", ref.buf);
+
+	if (next && *next)
+		die("create %s has extra input: %s", ref.buf, next);
+}
+
+static void parse_cmd_delete(const char *next)
+{
+	struct strbuf ref = STRBUF_INIT;
+	struct strbuf oldvalue = STRBUF_INIT;
+	struct ref_update *update;
+
+	update = update_alloc();
+
+	if ((next = parse_first_arg(next, &ref)) != NULL && ref.buf[0])
+		update_store_ref_name(update, ref.buf);
+	else
+		die("delete line missing <ref>");
+
+	if ((next = parse_next_arg(next, &oldvalue)) != NULL)
+		update_store_old_sha1(update, oldvalue.buf);
+	else if(!line_termination)
+		die("delete %s missing [<oldvalue>] NUL", ref.buf);
+	if (update->have_old && is_null_sha1(update->old_sha1))
+		die("delete %s given zero old value", ref.buf);
+
+	if (next && *next)
+		die("delete %s has extra input: %s", ref.buf, next);
+}
+
+static void parse_cmd_verify(const char *next)
+{
+	struct strbuf ref = STRBUF_INIT;
+	struct strbuf value = STRBUF_INIT;
+	struct ref_update *update;
+
+	update = update_alloc();
+
+	if ((next = parse_first_arg(next, &ref)) != NULL && ref.buf[0])
+		update_store_ref_name(update, ref.buf);
+	else
+		die("verify line missing <ref>");
+
+	if ((next = parse_next_arg(next, &value)) != NULL) {
+		update_store_old_sha1(update, value.buf);
+		update_store_new_sha1(update, value.buf);
+	} else if(!line_termination)
+		die("verify %s missing [<oldvalue>] NUL", ref.buf);
+
+	if (next && *next)
+		die("verify %s has extra input: %s", ref.buf, next);
+}
+
+static void parse_cmd_option(const char *next)
+{
+	if (!strcmp(next, "no-deref"))
+		update_flags |= REF_NODEREF;
+	else
+		die("option unknown: %s", next);
+}
+
+static void update_refs_stdin(void)
+{
+	struct strbuf cmd = STRBUF_INIT;
+
+	/* Read each line dispatch its command */
+	while (strbuf_getline(&cmd, stdin, line_termination) != EOF)
+		if (!cmd.buf[0])
+			die("empty command in input");
+		else if (isspace(*cmd.buf))
+			die("whitespace before command: %s", cmd.buf);
+		else if (!prefixcmp(cmd.buf, "update "))
+			parse_cmd_update(cmd.buf + 7);
+		else if (!prefixcmp(cmd.buf, "create "))
+			parse_cmd_create(cmd.buf + 7);
+		else if (!prefixcmp(cmd.buf, "delete "))
+			parse_cmd_delete(cmd.buf + 7);
+		else if (!prefixcmp(cmd.buf, "verify "))
+			parse_cmd_verify(cmd.buf + 7);
+		else if (!prefixcmp(cmd.buf, "option "))
+			parse_cmd_option(cmd.buf + 7);
+		else
+			die("unknown command: %s", cmd.buf);
+
+	strbuf_release(&cmd);
+}
+
 int cmd_update_ref(int argc, const char **argv, const char *prefix)
 {
 	const char *refname, *oldval, *msg = NULL;
 	unsigned char sha1[20], oldsha1[20];
-	int delete = 0, no_deref = 0, flags = 0;
+	int delete = 0, no_deref = 0, read_stdin = 0, end_null = 0, flags = 0;
 	struct option options[] = {
 		OPT_STRING( 'm', NULL, &msg, N_("reason"), N_("reason of the update")),
 		OPT_BOOL('d', NULL, &delete, N_("delete the reference")),
+		OPT_BOOL('z', NULL, &end_null, N_("stdin has NUL-terminated arguments")),
 		OPT_BOOL( 0 , "no-deref", &no_deref,
 					N_("update <refname> not the one it points to")),
+		OPT_BOOL( 0 , "stdin", &read_stdin, N_("read updates from stdin")),
 		OPT_END(),
 	};
 
@@ -28,6 +266,18 @@ int cmd_update_ref(int argc, const char **argv, const char *prefix)
 	if (msg && !*msg)
 		die("Refusing to perform update with empty message.");
 
+	if (read_stdin) {
+		if (delete || no_deref || argc > 0)
+			usage_with_options(git_update_ref_usage, options);
+		if (end_null)
+			line_termination = '\0';
+		update_refs_stdin();
+		return update_refs(msg, updates, updates_count, DIE_ON_ERR);
+	}
+
+	if (end_null)
+		usage_with_options(git_update_ref_usage, options);
+
 	if (delete) {
 		if (argc < 1 || argc > 2)
 			usage_with_options(git_update_ref_usage, options);
-- 
1.8.4.rc3

Re: [PATCH v6 7/8] update-ref: support multiple simultaneous updates

[Eric Sunshine]

On Mon, Sep 9, 2013 at 8:57 PM, Brad King <brad.king@kitware.com> wrote:
> diff --git a/Documentation/git-update-ref.txt b/Documentation/git-update-ref.txt
> index 0df13ff..0a0a551 100644
> --- a/Documentation/git-update-ref.txt
> +++ b/Documentation/git-update-ref.txt
> @@ -58,6 +58,58 @@ archive by creating a symlink tree).
> +option::
> +       Modify behavior of the next command naming a <ref>.
> +       The only valid option is `no-deref` to avoid dereferencing
> +       a symbolic ref.
> +
> +Use 40 "0" or the empty string to specify a zero value, except that

Did you want an 's' after the "0"?

0's
"0"s
"0"'s
zeros
zeroes

> +with `-z` an empty <oldvalue> is considered missing.
> +
> +If all <ref>s can be locked with matching <oldvalue>s
> +simultaneously, all modifications are performed.  Otherwise, no
> +modifications are performed.  Note that while each individual
> +<ref> is updated or deleted atomically, a concurrent reader may
> +still see a subset of the modifications.

Re: [PATCH v6 7/8] update-ref: support multiple simultaneous updates

[Brad King]

On 09/10/2013 06:51 PM, Eric Sunshine wrote:
> On Mon, Sep 9, 2013 at 8:57 PM, Brad King <brad.king@kitware.com> wrote:
>> +Use 40 "0" or the empty string to specify a zero value, except that
> 
> Did you want an 's' after the "0"?

The same description without 's' already appears in git-update-ref.txt
above this location in the existing documentation of the command-line
option behavior.  I see 0{40} in git-receive-pack.txt and also in
howto/update-hook-example.txt.  Perhaps a follow-up change can be made
to choose a consistent way to describe 40 0s.

-Brad

Re: [PATCH v6 7/8] update-ref: support multiple simultaneous updates

[Eric Sunshine]

On Wed, Sep 11, 2013 at 8:36 AM, Brad King <brad.king@kitware.com> wrote:
> On 09/10/2013 06:51 PM, Eric Sunshine wrote:
>> On Mon, Sep 9, 2013 at 8:57 PM, Brad King <brad.king@kitware.com> wrote:
>>> +Use 40 "0" or the empty string to specify a zero value, except that
>>
>> Did you want an 's' after the "0"?
>
> The same description without 's' already appears in git-update-ref.txt
> above this location in the existing documentation of the command-line

Thanks for the explanation. (I could have checked the surrounding text
but didn't think to do so.)

> option behavior.  I see 0{40} in git-receive-pack.txt and also in
> howto/update-hook-example.txt.  Perhaps a follow-up change can be made
> to choose a consistent way to describe 40 0s.
>
> -Brad

[PATCH v6 8/8] update-ref: add test cases covering --stdin signature

[Brad King]

Extend t/t1400-update-ref.sh to cover cases using the --stdin option.

Signed-off-by: Brad King <brad.king@kitware.com>
---
 t/t1400-update-ref.sh | 639 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 639 insertions(+)

diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index e415ee0..a510500 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -302,4 +302,643 @@ test_expect_success \
 	'git cat-file blob master@{2005-05-26 23:42}:F (expect OTHER)' \
 	'test OTHER = $(git cat-file blob "master@{2005-05-26 23:42}:F")'
 
+a=refs/heads/a
+b=refs/heads/b
+c=refs/heads/c
+E='""'
+pws='path with space'
+
+print_nul() {
+	while test $# -gt 0; do
+		printf -- "$1" &&
+		printf -- "Q" | q_to_nul &&
+		shift || return
+	done
+}
+
+test_expect_success 'stdin test setup' '
+	echo "$pws" >"$pws" &&
+	git add -- "$pws" &&
+	git commit -m "$pws"
+'
+
+test_expect_success '-z fails without --stdin' '
+	test_must_fail git update-ref -z $m $m $m 2>err &&
+	grep "usage: git update-ref" err
+'
+
+test_expect_success 'stdin works with no input' '
+	>stdin &&
+	git update-ref --stdin <stdin &&
+	git rev-parse --verify -q $m
+'
+
+test_expect_success 'stdin fails on empty line' '
+	echo "" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: empty command in input" err
+'
+
+test_expect_success 'stdin fails on only whitespace' '
+	echo " " >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: whitespace before command:  " err
+'
+
+test_expect_success 'stdin fails on leading whitespace' '
+	echo " create $a $m" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: whitespace before command:  create $a $m" err
+'
+
+test_expect_success 'stdin fails on unknown command' '
+	echo "unknown $a" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: unknown command: unknown $a" err
+'
+
+test_expect_success 'stdin fails on badly quoted input' '
+	echo "create $a \"master" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: badly quoted argument: \\\"master" err
+'
+
+test_expect_success 'stdin fails on arguments not separated by space' '
+	echo "create \"$a\"master" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: expected SP but got: master" err
+'
+
+test_expect_success 'stdin fails create with no ref' '
+	echo "create " >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: create line missing <ref>" err
+'
+
+test_expect_success 'stdin fails create with bad ref name' '
+	echo "create ~a $m" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: invalid ref format: ~a" err
+'
+
+test_expect_success 'stdin fails create with no new value' '
+	echo "create $a" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: create $a missing <newvalue>" err
+'
+
+test_expect_success 'stdin fails create with too many arguments' '
+	echo "create $a $m $m" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: create $a has extra input:  $m" err
+'
+
+test_expect_success 'stdin fails update with no ref' '
+	echo "update " >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: update line missing <ref>" err
+'
+
+test_expect_success 'stdin fails update with bad ref name' '
+	echo "update ~a $m" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: invalid ref format: ~a" err
+'
+
+test_expect_success 'stdin fails update with no new value' '
+	echo "update $a" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: update $a missing <newvalue>" err
+'
+
+test_expect_success 'stdin fails update with too many arguments' '
+	echo "update $a $m $m $m" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: update $a has extra input:  $m" err
+'
+
+test_expect_success 'stdin fails delete with no ref' '
+	echo "delete " >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: delete line missing <ref>" err
+'
+
+test_expect_success 'stdin fails delete with bad ref name' '
+	echo "delete ~a $m" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: invalid ref format: ~a" err
+'
+
+test_expect_success 'stdin fails delete with too many arguments' '
+	echo "delete $a $m $m" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: delete $a has extra input:  $m" err
+'
+
+test_expect_success 'stdin fails verify with too many arguments' '
+	echo "verify $a $m $m" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: verify $a has extra input:  $m" err
+'
+
+test_expect_success 'stdin fails option with unknown name' '
+	echo "option unknown" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: option unknown: unknown" err
+'
+
+test_expect_success 'stdin fails with duplicate refs' '
+	cat >stdin <<-EOF &&
+	create $a $m
+	create $b $m
+	create $a $m
+	EOF
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: Multiple updates for ref '"'"'$a'"'"' not allowed." err
+'
+
+test_expect_success 'stdin create ref works' '
+	echo "create $a $m" >stdin &&
+	git update-ref --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin update ref creates with zero old value' '
+	echo "update $b $m $Z" >stdin &&
+	git update-ref --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	git update-ref -d $b
+'
+
+test_expect_success 'stdin update ref creates with empty old value' '
+	echo "update $b $m $E" >stdin &&
+	git update-ref --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin create ref works with path with space to blob' '
+	echo "create refs/blobs/pws \"$m:$pws\"" >stdin &&
+	git update-ref --stdin <stdin &&
+	git rev-parse "$m:$pws" >expect &&
+	git rev-parse refs/blobs/pws >actual &&
+	test_cmp expect actual &&
+	git update-ref -d refs/blobs/pws
+'
+
+test_expect_success 'stdin update ref fails with wrong old value' '
+	echo "update $c $m $m~1" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: Cannot lock the ref '"'"'$c'"'"'" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin update ref fails with bad old value' '
+	echo "update $c $m does-not-exist" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: invalid old value for ref $c: does-not-exist" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin create ref fails with bad new value' '
+	echo "create $c does-not-exist" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: invalid new value for ref $c: does-not-exist" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin create ref fails with zero new value' '
+	echo "create $c " >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: create $c given zero new value" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin update ref works with right old value' '
+	echo "update $b $m~1 $m" >stdin &&
+	git update-ref --stdin <stdin &&
+	git rev-parse $m~1 >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin delete ref fails with wrong old value' '
+	echo "delete $a $m~1" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: Cannot lock the ref '"'"'$a'"'"'" err &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin delete ref fails with zero old value' '
+	echo "delete $a " >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: delete $a given zero old value" err &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin update symref works option no-deref' '
+	git symbolic-ref TESTSYMREF $b &&
+	cat >stdin <<-EOF &&
+	option no-deref
+	update TESTSYMREF $a $b
+	EOF
+	git update-ref --stdin <stdin &&
+	git rev-parse TESTSYMREF >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $m~1 >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin delete symref works option no-deref' '
+	git symbolic-ref TESTSYMREF $b &&
+	cat >stdin <<-EOF &&
+	option no-deref
+	delete TESTSYMREF $b
+	EOF
+	git update-ref --stdin <stdin &&
+	test_must_fail git rev-parse --verify -q TESTSYMREF &&
+	git rev-parse $m~1 >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin delete ref works with right old value' '
+	echo "delete $b $m~1" >stdin &&
+	git update-ref --stdin <stdin &&
+	test_must_fail git rev-parse --verify -q $b
+'
+
+test_expect_success 'stdin update/create/verify combination works' '
+	cat >stdin <<-EOF &&
+	update $a $m
+	create $b $m
+	verify $c
+	EOF
+	git update-ref --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin update refs works with identity updates' '
+	cat >stdin <<-EOF &&
+	update $a $m $m
+	update $b $m $m
+	update $c $Z $E
+	EOF
+	git update-ref --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin update refs fails with wrong old value' '
+	git update-ref $c $m &&
+	cat >stdin <<-EOF &&
+	update $a $m $m
+	update $b $m $m
+	update $c  ''
+	EOF
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: Cannot lock the ref '"'"'$c'"'"'" err &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	git rev-parse $c >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin delete refs works with packed and loose refs' '
+	git pack-refs --all &&
+	git update-ref $c $m~1 &&
+	cat >stdin <<-EOF &&
+	delete $a $m
+	update $b $Z $m
+	update $c $E $m~1
+	EOF
+	git update-ref --stdin <stdin &&
+	test_must_fail git rev-parse --verify -q $a &&
+	test_must_fail git rev-parse --verify -q $b &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z works on empty input' '
+	>stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse --verify -q $m
+'
+
+test_expect_success 'stdin -z fails on empty line' '
+	echo "" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: whitespace before command: " err
+'
+
+test_expect_success 'stdin -z fails on empty command' '
+	print_nul "" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: empty command in input" err
+'
+
+test_expect_success 'stdin -z fails on only whitespace' '
+	print_nul " " >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: whitespace before command:  " err
+'
+
+test_expect_success 'stdin -z fails on leading whitespace' '
+	print_nul " create $a" "$m" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: whitespace before command:  create $a" err
+'
+
+test_expect_success 'stdin -z fails on unknown command' '
+	print_nul "unknown $a" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: unknown command: unknown $a" err
+'
+
+test_expect_success 'stdin -z fails create with no ref' '
+	print_nul "create " >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: create line missing <ref>" err
+'
+
+test_expect_success 'stdin -z fails create with bad ref name' '
+	print_nul "create ~a " "$m" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: invalid ref format: ~a " err
+'
+
+test_expect_success 'stdin -z fails create with no new value' '
+	print_nul "create $a" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: create $a missing <newvalue>" err
+'
+
+test_expect_success 'stdin -z fails create with too many arguments' '
+	print_nul "create $a" "$m" "$m" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: unknown command: $m" err
+'
+
+test_expect_success 'stdin -z fails update with no ref' '
+	print_nul "update " >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: update line missing <ref>" err
+'
+
+test_expect_success 'stdin -z fails update with bad ref name' '
+	print_nul "update ~a" "$m" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: invalid ref format: ~a" err
+'
+
+test_expect_success 'stdin -z fails update with no new value' '
+	print_nul "update $a" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: update $a missing <newvalue>" err
+'
+
+test_expect_success 'stdin -z fails update with no old value' '
+	print_nul "update $a" "$m" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: update $a missing \\[<oldvalue>\\] NUL" err
+'
+
+test_expect_success 'stdin -z fails update with too many arguments' '
+	print_nul "update $a" "$m" "$m" "$m" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: unknown command: $m" err
+'
+
+test_expect_success 'stdin -z fails delete with no ref' '
+	print_nul "delete " >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: delete line missing <ref>" err
+'
+
+test_expect_success 'stdin -z fails delete with bad ref name' '
+	print_nul "delete ~a" "$m" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: invalid ref format: ~a" err
+'
+
+test_expect_success 'stdin -z fails delete with no old value' '
+	print_nul "delete $a" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: delete $a missing \\[<oldvalue>\\] NUL" err
+'
+
+test_expect_success 'stdin -z fails delete with too many arguments' '
+	print_nul "delete $a" "$m" "$m" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: unknown command: $m" err
+'
+
+test_expect_success 'stdin -z fails verify with too many arguments' '
+	print_nul "verify $a" "$m" "$m" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: unknown command: $m" err
+'
+
+test_expect_success 'stdin -z fails verify with no old value' '
+	print_nul "verify $a" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: verify $a missing \\[<oldvalue>\\] NUL" err
+'
+
+test_expect_success 'stdin -z fails option with unknown name' '
+	print_nul "option unknown" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: option unknown: unknown" err
+'
+
+test_expect_success 'stdin -z fails with duplicate refs' '
+	print_nul "create $a" "$m" "create $b" "$m" "create $a" "$m" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: Multiple updates for ref '"'"'$a'"'"' not allowed." err
+'
+
+test_expect_success 'stdin -z create ref works' '
+	print_nul "create $a" "$m" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin -z update ref creates with zero old value' '
+	print_nul "update $b" "$m" "$Z" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	git update-ref -d $b
+'
+
+test_expect_success 'stdin -z update ref creates with empty old value' '
+	print_nul "update $b" "$m" "" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin -z create ref works with path with space to blob' '
+	print_nul "create refs/blobs/pws" "$m:$pws" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse "$m:$pws" >expect &&
+	git rev-parse refs/blobs/pws >actual &&
+	test_cmp expect actual &&
+	git update-ref -d refs/blobs/pws
+'
+
+test_expect_success 'stdin -z update ref fails with wrong old value' '
+	print_nul "update $c" "$m" "$m~1" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: Cannot lock the ref '"'"'$c'"'"'" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z update ref fails with bad old value' '
+	print_nul "update $c" "$m" "does-not-exist" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: invalid old value for ref $c: does-not-exist" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z create ref fails with bad new value' '
+	print_nul "create $c" "does-not-exist" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: invalid new value for ref $c: does-not-exist" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z create ref fails with zero new value' '
+	print_nul "create $c" "" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: create $c given zero new value" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z update ref works with right old value' '
+	print_nul "update $b" "$m~1" "$m" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse $m~1 >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin -z delete ref fails with wrong old value' '
+	print_nul "delete $a" "$m~1" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: Cannot lock the ref '"'"'$a'"'"'" err &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin -z delete ref fails with zero old value' '
+	print_nul "delete $a" "$Z" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: delete $a given zero old value" err &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin -z update symref works option no-deref' '
+	git symbolic-ref TESTSYMREF $b &&
+	print_nul "option no-deref" "update TESTSYMREF" "$a" "$b" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse TESTSYMREF >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $m~1 >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin -z delete symref works option no-deref' '
+	git symbolic-ref TESTSYMREF $b &&
+	print_nul "option no-deref" "delete TESTSYMREF" "$b" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	test_must_fail git rev-parse --verify -q TESTSYMREF &&
+	git rev-parse $m~1 >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin -z delete ref works with right old value' '
+	print_nul "delete $b" "$m~1" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	test_must_fail git rev-parse --verify -q $b
+'
+
+test_expect_success 'stdin -z update/create/verify combination works' '
+	print_nul "update $a" "$m" "" "create $b" "$m" "verify $c" "" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z update refs works with identity updates' '
+	print_nul "update $a" "$m" "$m" "update $b" "$m" "$m" "update $c" "$Z" "" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z update refs fails with wrong old value' '
+	git update-ref $c $m &&
+	print_nul "update $a" "$m" "$m" "update $b" "$m" "$m" "update $c" "" "$Z" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: Cannot lock the ref '"'"'$c'"'"'" err &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	git rev-parse $c >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin -z delete refs works with packed and loose refs' '
+	git pack-refs --all &&
+	git update-ref $c $m~1 &&
+	print_nul "delete $a" "$m" "update $b" "$Z" "$m" "update $c" "" "$m~1" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	test_must_fail git rev-parse --verify -q $a &&
+	test_must_fail git rev-parse --verify -q $b &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
 test_done
-- 
1.8.4.rc3

Re: [PATCH v6 8/8] update-ref: add test cases covering --stdin signature

[Eric Sunshine]

On Mon, Sep 9, 2013 at 8:57 PM, Brad King <brad.king@kitware.com> wrote:
> Extend t/t1400-update-ref.sh to cover cases using the --stdin option.
>
> Signed-off-by: Brad King <brad.king@kitware.com>
> ---
>  t/t1400-update-ref.sh | 639 ++++++++++++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 639 insertions(+)
>
> diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
> index e415ee0..a510500 100755
> --- a/t/t1400-update-ref.sh
> +++ b/t/t1400-update-ref.sh
> @@ -302,4 +302,643 @@ test_expect_success \
>         'git cat-file blob master@{2005-05-26 23:42}:F (expect OTHER)' \
>         'test OTHER = $(git cat-file blob "master@{2005-05-26 23:42}:F")'
>
> +a=refs/heads/a
> +b=refs/heads/b
> +c=refs/heads/c
> +E='""'
> +pws='path with space'
> +
> +print_nul() {
> +       while test $# -gt 0; do
> +               printf -- "$1" &&
> +               printf -- "Q" | q_to_nul &&
> +               shift || return
> +       done
> +}

I believe that current fashion in git test scripts is to add a space
before () in the function declaration. Likewise, the 'do' should be on
the line following 'while' and aligned with 'while' (and drop the
semicolon).

The '--' option to printf is not likely portable. POSIX [1] certainly
does not mention it.

You can get printf to emit a NUL more naturally via \0, so q_to_nul is
unnecessary.

Finally, printf reuses its 'format' argument as many times as needed
to output all arguments, so the while loop is unneeded.

Thus printf provides all the functionality you require, and
print_nul() function can be dropped. So:

    printf '%s\0' foo bar baz

is equivalent to:

    print_null foo bar baz

[1]: http://pubs.opengroup.org/onlinepubs/9699919799/utilities/printf.html

Re: [PATCH v6 8/8] update-ref: add test cases covering --stdin signature

[Junio C Hamano]

Eric Sunshine <sunshine@sunshineco.com> writes:

> Thus printf provides all the functionality you require, and
> print_nul() function can be dropped. So:
>
>     printf '%s\0' foo bar baz
>
> is equivalent to:
>
>     print_null foo bar baz

Good eyes.  Thanks, I missed them when I looked at the patches.

[PATCH v6.1 8/8] update-ref: add test cases covering --stdin signature

[Brad King]

Extend t/t1400-update-ref.sh to cover cases using the --stdin option.

Signed-off-by: Brad King <brad.king@kitware.com>
---

On 09/10/2013 06:46 PM, Eric Sunshine wrote:
> Thus printf provides all the functionality you require, and
> print_nul() function can be dropped. So:
> 
>     printf '%s\0' foo bar baz

Wonderful, thanks!  The single-quotes do not fit easily inside
test code blocks that are themselves single-quoted, so I packaged
the format up in a variable.  Here is a revised patch.

-Brad

 t/t1400-update-ref.sh | 632 ++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 632 insertions(+)

diff --git a/t/t1400-update-ref.sh b/t/t1400-update-ref.sh
index e415ee0..6ffd82f 100755
--- a/t/t1400-update-ref.sh
+++ b/t/t1400-update-ref.sh
@@ -302,4 +302,636 @@ test_expect_success \
 	'git cat-file blob master@{2005-05-26 23:42}:F (expect OTHER)' \
 	'test OTHER = $(git cat-file blob "master@{2005-05-26 23:42}:F")'
 
+a=refs/heads/a
+b=refs/heads/b
+c=refs/heads/c
+E='""'
+F='%s\0'
+pws='path with space'
+
+test_expect_success 'stdin test setup' '
+	echo "$pws" >"$pws" &&
+	git add -- "$pws" &&
+	git commit -m "$pws"
+'
+
+test_expect_success '-z fails without --stdin' '
+	test_must_fail git update-ref -z $m $m $m 2>err &&
+	grep "usage: git update-ref" err
+'
+
+test_expect_success 'stdin works with no input' '
+	>stdin &&
+	git update-ref --stdin <stdin &&
+	git rev-parse --verify -q $m
+'
+
+test_expect_success 'stdin fails on empty line' '
+	echo "" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: empty command in input" err
+'
+
+test_expect_success 'stdin fails on only whitespace' '
+	echo " " >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: whitespace before command:  " err
+'
+
+test_expect_success 'stdin fails on leading whitespace' '
+	echo " create $a $m" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: whitespace before command:  create $a $m" err
+'
+
+test_expect_success 'stdin fails on unknown command' '
+	echo "unknown $a" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: unknown command: unknown $a" err
+'
+
+test_expect_success 'stdin fails on badly quoted input' '
+	echo "create $a \"master" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: badly quoted argument: \\\"master" err
+'
+
+test_expect_success 'stdin fails on arguments not separated by space' '
+	echo "create \"$a\"master" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: expected SP but got: master" err
+'
+
+test_expect_success 'stdin fails create with no ref' '
+	echo "create " >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: create line missing <ref>" err
+'
+
+test_expect_success 'stdin fails create with bad ref name' '
+	echo "create ~a $m" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: invalid ref format: ~a" err
+'
+
+test_expect_success 'stdin fails create with no new value' '
+	echo "create $a" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: create $a missing <newvalue>" err
+'
+
+test_expect_success 'stdin fails create with too many arguments' '
+	echo "create $a $m $m" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: create $a has extra input:  $m" err
+'
+
+test_expect_success 'stdin fails update with no ref' '
+	echo "update " >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: update line missing <ref>" err
+'
+
+test_expect_success 'stdin fails update with bad ref name' '
+	echo "update ~a $m" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: invalid ref format: ~a" err
+'
+
+test_expect_success 'stdin fails update with no new value' '
+	echo "update $a" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: update $a missing <newvalue>" err
+'
+
+test_expect_success 'stdin fails update with too many arguments' '
+	echo "update $a $m $m $m" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: update $a has extra input:  $m" err
+'
+
+test_expect_success 'stdin fails delete with no ref' '
+	echo "delete " >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: delete line missing <ref>" err
+'
+
+test_expect_success 'stdin fails delete with bad ref name' '
+	echo "delete ~a $m" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: invalid ref format: ~a" err
+'
+
+test_expect_success 'stdin fails delete with too many arguments' '
+	echo "delete $a $m $m" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: delete $a has extra input:  $m" err
+'
+
+test_expect_success 'stdin fails verify with too many arguments' '
+	echo "verify $a $m $m" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: verify $a has extra input:  $m" err
+'
+
+test_expect_success 'stdin fails option with unknown name' '
+	echo "option unknown" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: option unknown: unknown" err
+'
+
+test_expect_success 'stdin fails with duplicate refs' '
+	cat >stdin <<-EOF &&
+	create $a $m
+	create $b $m
+	create $a $m
+	EOF
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: Multiple updates for ref '"'"'$a'"'"' not allowed." err
+'
+
+test_expect_success 'stdin create ref works' '
+	echo "create $a $m" >stdin &&
+	git update-ref --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin update ref creates with zero old value' '
+	echo "update $b $m $Z" >stdin &&
+	git update-ref --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	git update-ref -d $b
+'
+
+test_expect_success 'stdin update ref creates with empty old value' '
+	echo "update $b $m $E" >stdin &&
+	git update-ref --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin create ref works with path with space to blob' '
+	echo "create refs/blobs/pws \"$m:$pws\"" >stdin &&
+	git update-ref --stdin <stdin &&
+	git rev-parse "$m:$pws" >expect &&
+	git rev-parse refs/blobs/pws >actual &&
+	test_cmp expect actual &&
+	git update-ref -d refs/blobs/pws
+'
+
+test_expect_success 'stdin update ref fails with wrong old value' '
+	echo "update $c $m $m~1" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: Cannot lock the ref '"'"'$c'"'"'" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin update ref fails with bad old value' '
+	echo "update $c $m does-not-exist" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: invalid old value for ref $c: does-not-exist" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin create ref fails with bad new value' '
+	echo "create $c does-not-exist" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: invalid new value for ref $c: does-not-exist" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin create ref fails with zero new value' '
+	echo "create $c " >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: create $c given zero new value" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin update ref works with right old value' '
+	echo "update $b $m~1 $m" >stdin &&
+	git update-ref --stdin <stdin &&
+	git rev-parse $m~1 >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin delete ref fails with wrong old value' '
+	echo "delete $a $m~1" >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: Cannot lock the ref '"'"'$a'"'"'" err &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin delete ref fails with zero old value' '
+	echo "delete $a " >stdin &&
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: delete $a given zero old value" err &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin update symref works option no-deref' '
+	git symbolic-ref TESTSYMREF $b &&
+	cat >stdin <<-EOF &&
+	option no-deref
+	update TESTSYMREF $a $b
+	EOF
+	git update-ref --stdin <stdin &&
+	git rev-parse TESTSYMREF >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $m~1 >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin delete symref works option no-deref' '
+	git symbolic-ref TESTSYMREF $b &&
+	cat >stdin <<-EOF &&
+	option no-deref
+	delete TESTSYMREF $b
+	EOF
+	git update-ref --stdin <stdin &&
+	test_must_fail git rev-parse --verify -q TESTSYMREF &&
+	git rev-parse $m~1 >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin delete ref works with right old value' '
+	echo "delete $b $m~1" >stdin &&
+	git update-ref --stdin <stdin &&
+	test_must_fail git rev-parse --verify -q $b
+'
+
+test_expect_success 'stdin update/create/verify combination works' '
+	cat >stdin <<-EOF &&
+	update $a $m
+	create $b $m
+	verify $c
+	EOF
+	git update-ref --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin update refs works with identity updates' '
+	cat >stdin <<-EOF &&
+	update $a $m $m
+	update $b $m $m
+	update $c $Z $E
+	EOF
+	git update-ref --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin update refs fails with wrong old value' '
+	git update-ref $c $m &&
+	cat >stdin <<-EOF &&
+	update $a $m $m
+	update $b $m $m
+	update $c  ''
+	EOF
+	test_must_fail git update-ref --stdin <stdin 2>err &&
+	grep "fatal: Cannot lock the ref '"'"'$c'"'"'" err &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	git rev-parse $c >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin delete refs works with packed and loose refs' '
+	git pack-refs --all &&
+	git update-ref $c $m~1 &&
+	cat >stdin <<-EOF &&
+	delete $a $m
+	update $b $Z $m
+	update $c $E $m~1
+	EOF
+	git update-ref --stdin <stdin &&
+	test_must_fail git rev-parse --verify -q $a &&
+	test_must_fail git rev-parse --verify -q $b &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z works on empty input' '
+	>stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse --verify -q $m
+'
+
+test_expect_success 'stdin -z fails on empty line' '
+	echo "" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: whitespace before command: " err
+'
+
+test_expect_success 'stdin -z fails on empty command' '
+	printf $F "" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: empty command in input" err
+'
+
+test_expect_success 'stdin -z fails on only whitespace' '
+	printf $F " " >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: whitespace before command:  " err
+'
+
+test_expect_success 'stdin -z fails on leading whitespace' '
+	printf $F " create $a" "$m" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: whitespace before command:  create $a" err
+'
+
+test_expect_success 'stdin -z fails on unknown command' '
+	printf $F "unknown $a" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: unknown command: unknown $a" err
+'
+
+test_expect_success 'stdin -z fails create with no ref' '
+	printf $F "create " >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: create line missing <ref>" err
+'
+
+test_expect_success 'stdin -z fails create with bad ref name' '
+	printf $F "create ~a " "$m" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: invalid ref format: ~a " err
+'
+
+test_expect_success 'stdin -z fails create with no new value' '
+	printf $F "create $a" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: create $a missing <newvalue>" err
+'
+
+test_expect_success 'stdin -z fails create with too many arguments' '
+	printf $F "create $a" "$m" "$m" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: unknown command: $m" err
+'
+
+test_expect_success 'stdin -z fails update with no ref' '
+	printf $F "update " >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: update line missing <ref>" err
+'
+
+test_expect_success 'stdin -z fails update with bad ref name' '
+	printf $F "update ~a" "$m" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: invalid ref format: ~a" err
+'
+
+test_expect_success 'stdin -z fails update with no new value' '
+	printf $F "update $a" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: update $a missing <newvalue>" err
+'
+
+test_expect_success 'stdin -z fails update with no old value' '
+	printf $F "update $a" "$m" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: update $a missing \\[<oldvalue>\\] NUL" err
+'
+
+test_expect_success 'stdin -z fails update with too many arguments' '
+	printf $F "update $a" "$m" "$m" "$m" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: unknown command: $m" err
+'
+
+test_expect_success 'stdin -z fails delete with no ref' '
+	printf $F "delete " >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: delete line missing <ref>" err
+'
+
+test_expect_success 'stdin -z fails delete with bad ref name' '
+	printf $F "delete ~a" "$m" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: invalid ref format: ~a" err
+'
+
+test_expect_success 'stdin -z fails delete with no old value' '
+	printf $F "delete $a" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: delete $a missing \\[<oldvalue>\\] NUL" err
+'
+
+test_expect_success 'stdin -z fails delete with too many arguments' '
+	printf $F "delete $a" "$m" "$m" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: unknown command: $m" err
+'
+
+test_expect_success 'stdin -z fails verify with too many arguments' '
+	printf $F "verify $a" "$m" "$m" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: unknown command: $m" err
+'
+
+test_expect_success 'stdin -z fails verify with no old value' '
+	printf $F "verify $a" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: verify $a missing \\[<oldvalue>\\] NUL" err
+'
+
+test_expect_success 'stdin -z fails option with unknown name' '
+	printf $F "option unknown" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: option unknown: unknown" err
+'
+
+test_expect_success 'stdin -z fails with duplicate refs' '
+	printf $F "create $a" "$m" "create $b" "$m" "create $a" "$m" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: Multiple updates for ref '"'"'$a'"'"' not allowed." err
+'
+
+test_expect_success 'stdin -z create ref works' '
+	printf $F "create $a" "$m" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin -z update ref creates with zero old value' '
+	printf $F "update $b" "$m" "$Z" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	git update-ref -d $b
+'
+
+test_expect_success 'stdin -z update ref creates with empty old value' '
+	printf $F "update $b" "$m" "" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin -z create ref works with path with space to blob' '
+	printf $F "create refs/blobs/pws" "$m:$pws" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse "$m:$pws" >expect &&
+	git rev-parse refs/blobs/pws >actual &&
+	test_cmp expect actual &&
+	git update-ref -d refs/blobs/pws
+'
+
+test_expect_success 'stdin -z update ref fails with wrong old value' '
+	printf $F "update $c" "$m" "$m~1" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: Cannot lock the ref '"'"'$c'"'"'" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z update ref fails with bad old value' '
+	printf $F "update $c" "$m" "does-not-exist" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: invalid old value for ref $c: does-not-exist" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z create ref fails with bad new value' '
+	printf $F "create $c" "does-not-exist" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: invalid new value for ref $c: does-not-exist" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z create ref fails with zero new value' '
+	printf $F "create $c" "" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: create $c given zero new value" err &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z update ref works with right old value' '
+	printf $F "update $b" "$m~1" "$m" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse $m~1 >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin -z delete ref fails with wrong old value' '
+	printf $F "delete $a" "$m~1" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: Cannot lock the ref '"'"'$a'"'"'" err &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin -z delete ref fails with zero old value' '
+	printf $F "delete $a" "$Z" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: delete $a given zero old value" err &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin -z update symref works option no-deref' '
+	git symbolic-ref TESTSYMREF $b &&
+	printf $F "option no-deref" "update TESTSYMREF" "$a" "$b" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse TESTSYMREF >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $m~1 >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin -z delete symref works option no-deref' '
+	git symbolic-ref TESTSYMREF $b &&
+	printf $F "option no-deref" "delete TESTSYMREF" "$b" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	test_must_fail git rev-parse --verify -q TESTSYMREF &&
+	git rev-parse $m~1 >expect &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin -z delete ref works with right old value' '
+	printf $F "delete $b" "$m~1" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	test_must_fail git rev-parse --verify -q $b
+'
+
+test_expect_success 'stdin -z update/create/verify combination works' '
+	printf $F "update $a" "$m" "" "create $b" "$m" "verify $c" "" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z update refs works with identity updates' '
+	printf $F "update $a" "$m" "$m" "update $b" "$m" "$m" "update $c" "$Z" "" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
+test_expect_success 'stdin -z update refs fails with wrong old value' '
+	git update-ref $c $m &&
+	printf $F "update $a" "$m" "$m" "update $b" "$m" "$m" "update $c" "" "$Z" >stdin &&
+	test_must_fail git update-ref -z --stdin <stdin 2>err &&
+	grep "fatal: Cannot lock the ref '"'"'$c'"'"'" err &&
+	git rev-parse $m >expect &&
+	git rev-parse $a >actual &&
+	test_cmp expect actual &&
+	git rev-parse $b >actual &&
+	test_cmp expect actual &&
+	git rev-parse $c >actual &&
+	test_cmp expect actual
+'
+
+test_expect_success 'stdin -z delete refs works with packed and loose refs' '
+	git pack-refs --all &&
+	git update-ref $c $m~1 &&
+	printf $F "delete $a" "$m" "update $b" "$Z" "$m" "update $c" "" "$m~1" >stdin &&
+	git update-ref -z --stdin <stdin &&
+	test_must_fail git rev-parse --verify -q $a &&
+	test_must_fail git rev-parse --verify -q $b &&
+	test_must_fail git rev-parse --verify -q $c
+'
+
 test_done
-- 
1.8.4.rc3

Re: [PATCH v6 0/8] Multiple simultaneously locked ref updates

[Junio C Hamano]

Brad King <brad.king@kitware.com> writes:

> Updates since the previous revision of the series:
>
> * The entire series was rebased on master at bb80ee09; it was
>   previously based on v1.8.4.
>
> * A conflict in refs.c with 47a59185 was resolved by preserving
>   the elimination of find_ref_by_name while adding our new content.
>
> * A conflict in builtin/update-ref.c with d5d09d47 (Replace deprecated
>   OPT_BOOLEAN by OPT_BOOL, 2013-08-03) was resolved by integrating
>   both changes.  The new options added in patch 7 now use OPT_BOOL.

I just test-applied these on top of bb80ee09 (Update draft release
notes to 1.8.5 for the second batch of topics, 2013-09-09), and
compared the result with the result of merging the tip of the
previous round 511910e1 (update-ref: add test cases covering --stdin
signature, 2013-09-09) with bb80ee09, and they more-or-less match
(the order of options[] array elements may differ in
update-index.c), which validates that existing merge conflict
resolution matches your expectation.

Thanks.  I am not sure if I should rewind and rebuild the series
with these patches, though.  This is a new feature and does not have
to be merged to 'maint', so rebasing is perfectly fine, but it is
not strictly necessary, either.

Re: [PATCH v6 0/8] Multiple simultaneously locked ref updates

[Brad King]

On 09/10/2013 12:30 PM, Junio C Hamano wrote:
> Thanks.  I am not sure if I should rewind and rebuild the series
> with these patches, though.  This is a new feature and does not have
> to be merged to 'maint', so rebasing is perfectly fine, but it is
> not strictly necessary, either.

I just thought I'd help out with the conflict resolution.  If you're
happy with resolving the conflicts in v5 then there is no reason to
use v6.

Thanks,
-Brad

Re: [PATCH v6 0/8] Multiple simultaneously locked ref updates

[Junio C Hamano]

Brad King <brad.king@kitware.com> writes:

> On 09/10/2013 12:30 PM, Junio C Hamano wrote:
>> Thanks.  I am not sure if I should rewind and rebuild the series
>> with these patches, though.  This is a new feature and does not have
>> to be merged to 'maint', so rebasing is perfectly fine, but it is
>> not strictly necessary, either.
>
> I just thought I'd help out with the conflict resolution.

Yeah, such an independent confirmation of the conflict resolution is
very much appreciated.