git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Ronnie Sahlberg <sahlberg@google.com>
To: git@vger.kernel.org
Cc: mhagger@alum.mit.edu, Ronnie Sahlberg <sahlberg@google.com>
Subject: [PATCH v11 26/41] walker.c: use ref transaction for ref updates
Date: Tue, 27 May 2014 13:25:45 -0700	[thread overview]
Message-ID: <1401222360-21175-27-git-send-email-sahlberg@google.com> (raw)
In-Reply-To: <1401222360-21175-1-git-send-email-sahlberg@google.com>

Switch to using ref transactions in walker_fetch(). As part of the refactoring
to use ref transactions we also fix a potential memory leak where in the
original code if write_ref_sha1() would fail we would end up returning from
the function without free()ing the msg string.

This changes the locking slightly for walker_fetch. Previously the code would
lock all refs before writing them but now we do not lock the refs until the
commit stage. There is thus a very short window where changes could be done
locally during the fetch which would be overwritten when the fetch completes
and commits its transaction. But this window should be reasonably short.
Even if this race does trigger, since both the old code and the new code
just overwrites the refs to the new values without checking or comparing
them with the previous value, this is not too dissimilar to a similar scenario
where you first do a ref change locally and then later do a fetch that
overwrites the local change. With this in mind I do not see the change in
locking semantics to be critical.

Note that this function is only called when fetching from a remote HTTP
repository onto the local (most of the time single-user) repository which
likely means that the type of collissions that the previous locking would
protect against and cause the fetch to fail for to be even more rare.

Signed-off-by: Ronnie Sahlberg <sahlberg@google.com>
---
 walker.c | 56 +++++++++++++++++++++++++++++++++-----------------------
 1 file changed, 33 insertions(+), 23 deletions(-)

diff --git a/walker.c b/walker.c
index 1dd86b8..51ce1c6 100644
--- a/walker.c
+++ b/walker.c
@@ -251,39 +251,37 @@ void walker_targets_free(int targets, char **target, const char **write_ref)
 int walker_fetch(struct walker *walker, int targets, char **target,
 		 const char **write_ref, const char *write_ref_log_details)
 {
-	struct ref_lock **lock = xcalloc(targets, sizeof(struct ref_lock *));
+	struct strbuf ref_name = STRBUF_INIT;
+	struct strbuf err = STRBUF_INIT;
+	struct ref_transaction *transaction;
 	unsigned char *sha1 = xmalloc(targets * 20);
 	char *msg;
-	int ret;
 	int i;
 
 	save_commit_buffer = 0;
 
-	for (i = 0; i < targets; i++) {
-		if (!write_ref || !write_ref[i])
-			continue;
-
-		lock[i] = lock_ref_sha1(write_ref[i], NULL);
-		if (!lock[i]) {
-			error("Can't lock ref %s", write_ref[i]);
-			goto unlock_and_fail;
+	if (write_ref) {
+		transaction = ref_transaction_begin(&err);
+		if (!transaction) {
+			error("%s", err.buf);
+			strbuf_release(&err);
+			return -1;
 		}
 	}
-
 	if (!walker->get_recover)
 		for_each_ref(mark_complete, NULL);
 
 	for (i = 0; i < targets; i++) {
 		if (interpret_target(walker, target[i], &sha1[20 * i])) {
 			error("Could not interpret response from server '%s' as something to pull", target[i]);
-			goto unlock_and_fail;
+			goto rollback_and_fail;
 		}
 		if (process(walker, lookup_unknown_object(&sha1[20 * i])))
-			goto unlock_and_fail;
+			goto rollback_and_fail;
 	}
 
 	if (loop(walker))
-		goto unlock_and_fail;
+		goto rollback_and_fail;
 
 	if (write_ref_log_details) {
 		msg = xmalloc(strlen(write_ref_log_details) + 12);
@@ -294,19 +292,31 @@ int walker_fetch(struct walker *walker, int targets, char **target,
 	for (i = 0; i < targets; i++) {
 		if (!write_ref || !write_ref[i])
 			continue;
-		ret = write_ref_sha1(lock[i], &sha1[20 * i], msg ? msg : "fetch (unknown)");
-		lock[i] = NULL;
-		if (ret)
-			goto unlock_and_fail;
+		strbuf_reset(&ref_name);
+		strbuf_addf(&ref_name, "refs/%s", write_ref[i]);
+		if (ref_transaction_update(transaction, ref_name.buf,
+					   &sha1[20 * i], NULL, 0, 0,
+					   &err))
+			break;
+	}
+	if (write_ref) {
+		if (ref_transaction_commit(transaction,
+					   msg ? msg : "fetch (unknown)",
+					   &err)) {
+			error("%s", err.buf);
+			ref_transaction_free(transaction);
+			goto rollback_and_fail;
+		}
+		ref_transaction_free(transaction);
 	}
-	free(msg);
 
+	free(msg);
 	return 0;
 
-unlock_and_fail:
-	for (i = 0; i < targets; i++)
-		if (lock[i])
-			unlock_ref(lock[i]);
+rollback_and_fail:
+	free(msg);
+	strbuf_release(&err);
+	strbuf_release(&ref_name);
 
 	return -1;
 }
-- 
2.0.0.rc3.474.g0203784

  parent reply	other threads:[~2014-05-27 20:26 UTC|newest]

Thread overview: 89+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-05-27 20:25 [PATCH v11 00/41] Use ref transactions Ronnie Sahlberg
2014-05-27 20:25 ` [PATCH v11 01/41] refs.c: remove ref_transaction_rollback Ronnie Sahlberg
2014-05-27 20:25 ` [PATCH v11 02/41] refs.c: ref_transaction_commit should not free the transaction Ronnie Sahlberg
2014-05-27 23:20   ` Jonathan Nieder
2014-05-27 20:25 ` [PATCH v11 03/41] refs.c: constify the sha arguments for ref_transaction_create|delete|update Ronnie Sahlberg
2014-05-27 20:25 ` [PATCH v11 04/41] refs.c: allow passing NULL to ref_transaction_free Ronnie Sahlberg
2014-05-27 20:25 ` [PATCH v11 05/41] refs.c: add a strbuf argument to ref_transaction_commit for error logging Ronnie Sahlberg
2014-05-27 20:25 ` [PATCH v11 06/41] refs.c: add an err argument to repack_without_refs Ronnie Sahlberg
2014-05-28  0:11   ` Jonathan Nieder
2014-05-28 15:31     ` Ronnie Sahlberg
2014-05-28 18:30       ` Jonathan Nieder
2014-05-27 20:25 ` [PATCH v11 07/41] refs.c: make ref_update_reject_duplicates take a strbuf argument for errors Ronnie Sahlberg
2014-05-27 20:25 ` [PATCH v11 08/41] refs.c: add an err argument to delete_ref_loose Ronnie Sahlberg
2014-05-28  0:25   ` Jonathan Nieder
2014-05-28 14:43     ` Ronnie Sahlberg
2014-05-27 20:25 ` [PATCH v11 09/41] refs.c: make update_ref_write update a strbuf on failure Ronnie Sahlberg
2014-05-27 20:25 ` [PATCH v11 10/41] update-ref.c: log transaction error from the update_ref Ronnie Sahlberg
2014-05-28  0:27   ` Jonathan Nieder
2014-05-28 14:46     ` Ronnie Sahlberg
2014-05-28 16:49       ` Jonathan Nieder
2014-05-27 20:25 ` [PATCH v11 11/41] refs.c: remove the onerr argument to ref_transaction_commit Ronnie Sahlberg
2014-05-27 20:25 ` [PATCH v11 12/41] refs.c: change ref_transaction_update() to do error checking and return status Ronnie Sahlberg
2014-05-27 20:25 ` [PATCH v11 13/41] refs.c: change ref_transaction_create " Ronnie Sahlberg
2014-05-28  0:42   ` Jonathan Nieder
2014-05-28 14:55     ` Ronnie Sahlberg
2014-05-28 17:07       ` Jonathan Nieder
2014-05-28 17:15         ` Ronnie Sahlberg
2014-05-28 17:25           ` Jonathan Nieder
2014-05-28 18:01             ` Ronnie Sahlberg
2014-05-28 18:09               ` Jonathan Nieder
2014-05-27 20:25 ` [PATCH v11 14/41] refs.c: update ref_transaction_delete to check for error " Ronnie Sahlberg
2014-05-28 18:42   ` Jonathan Nieder
2014-05-27 20:25 ` [PATCH v11 15/41] refs.c: make ref_transaction_begin take an err argument Ronnie Sahlberg
2014-05-27 20:25 ` [PATCH v11 16/41] refs.c: add transaction.status and track OPEN/CLOSED/ERROR Ronnie Sahlberg
2014-05-28 18:51   ` Jonathan Nieder
2014-05-28 21:50     ` Ronnie Sahlberg
2014-05-27 20:25 ` [PATCH v11 17/41] tag.c: use ref transactions when doing updates Ronnie Sahlberg
2014-05-27 20:25 ` [PATCH v11 18/41] replace.c: use the ref transaction functions for updates Ronnie Sahlberg
2014-05-27 20:25 ` [PATCH v11 19/41] commit.c: use ref transactions " Ronnie Sahlberg
2014-05-27 20:25 ` [PATCH v11 20/41] sequencer.c: use ref transactions for all ref updates Ronnie Sahlberg
2014-05-28 18:53   ` Jonathan Nieder
2014-05-27 20:25 ` [PATCH v11 21/41] fast-import.c: change update_branch to use ref transactions Ronnie Sahlberg
2014-05-28 18:54   ` Jonathan Nieder
2014-05-27 20:25 ` [PATCH v11 22/41] branch.c: use ref transaction for all ref updates Ronnie Sahlberg
2014-05-27 20:25 ` [PATCH v11 23/41] refs.c: change update_ref to use a transaction Ronnie Sahlberg
2014-05-28 19:31   ` Jonathan Nieder
2014-05-28 21:14     ` Ronnie Sahlberg
2014-05-27 20:25 ` [PATCH v11 24/41] receive-pack.c: use a reference transaction for updating the refs Ronnie Sahlberg
2014-05-27 20:25 ` [PATCH v11 25/41] fast-import.c: use a ref transaction when dumping tags Ronnie Sahlberg
2014-05-28 19:47   ` Jonathan Nieder
2014-05-28 22:06     ` Ronnie Sahlberg
2014-05-28 22:17       ` Jonathan Nieder
2014-05-28 23:20         ` Ronnie Sahlberg
2014-05-28 23:39           ` Jonathan Nieder
2014-05-29 16:10             ` Ronnie Sahlberg
2014-05-29 17:41               ` Jonathan Nieder
2014-06-03 21:32                 ` Ronnie Sahlberg
2014-05-27 20:25 ` Ronnie Sahlberg [this message]
2014-05-28 19:56   ` [PATCH v11 26/41] walker.c: use ref transaction for ref updates Jonathan Nieder
2014-05-28 20:56     ` Ronnie Sahlberg
2014-05-27 20:25 ` [PATCH v11 27/41] refs.c: make lock_ref_sha1 static Ronnie Sahlberg
2014-05-27 20:25 ` [PATCH v11 28/41] refs.c: remove the update_ref_lock function Ronnie Sahlberg
2014-05-27 20:25 ` [PATCH v11 29/41] refs.c: remove the update_ref_write function Ronnie Sahlberg
2014-05-28 20:39   ` Jonathan Nieder
2014-05-27 20:25 ` [PATCH v11 30/41] refs.c: remove lock_ref_sha1 Ronnie Sahlberg
2014-05-27 20:25 ` [PATCH v11 31/41] refs.c: make prune_ref use a transaction to delete the ref Ronnie Sahlberg
2014-05-28 21:51   ` Jonathan Nieder
2014-05-28 21:56     ` Ronnie Sahlberg
2014-05-27 20:25 ` [PATCH v11 32/41] refs.c: make delete_ref use a transaction Ronnie Sahlberg
2014-05-30 17:28   ` Jonathan Nieder
2014-06-03 16:59     ` Ronnie Sahlberg
2014-05-27 20:25 ` [PATCH v11 33/41] refs.c: pass the ref log message to _create/delete/update instead of _commit Ronnie Sahlberg
2014-05-30 17:38   ` Jonathan Nieder
2014-06-03 17:01     ` Ronnie Sahlberg
2014-05-27 20:25 ` [PATCH v11 34/41] refs.c: pass NULL as *flags to read_ref_full Ronnie Sahlberg
2014-05-27 20:25 ` [PATCH v11 35/41] refs.c: pack all refs before we start to rename a ref Ronnie Sahlberg
2014-05-30 18:08   ` Jonathan Nieder
2014-06-03 16:39     ` Ronnie Sahlberg
2014-05-27 20:25 ` [PATCH v11 36/41] refs.c: move the check for valid refname to lock_ref_sha1_basic Ronnie Sahlberg
2014-05-30 18:12   ` Jonathan Nieder
2014-06-03 18:33     ` Ronnie Sahlberg
2014-05-27 20:25 ` [PATCH v11 37/41] refs.c: call lock_ref_sha1_basic directly from commit Ronnie Sahlberg
2014-05-27 20:25 ` [PATCH v11 38/41] refs.c: pass a skip list to name_conflict_fn Ronnie Sahlberg
2014-05-27 20:25 ` [PATCH v11 39/41] refs.c: propagate any errno==ENOTDIR from _commit back to the callers Ronnie Sahlberg
2014-05-31  0:22   ` Jonathan Nieder
2014-06-03 18:21     ` Ronnie Sahlberg
2014-05-27 20:25 ` [PATCH v11 40/41] fetch.c: change s_update_ref to use a ref transaction Ronnie Sahlberg
2014-05-27 20:26 ` [PATCH v11 41/41] refs.c: make write_ref_sha1 static Ronnie Sahlberg
2014-06-03 18:31 ` [PATCH v11 00/41] Use ref transactions Jonathan Nieder

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1401222360-21175-27-git-send-email-sahlberg@google.com \
    --to=sahlberg@google.com \
    --cc=git@vger.kernel.org \
    --cc=mhagger@alum.mit.edu \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).