git.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Michael Haggerty <mhagger@alum.mit.edu>
To: Junio C Hamano <gitster@pobox.com>, Jeff King <peff@peff.net>,
	Heiko Voigt <hvoigt@hvoigt.net>
Cc: git@vger.kernel.org, Michael Haggerty <mhagger@alum.mit.edu>
Subject: [PATCH 21/33] repack_without_ref(): write peeled refs in the rewritten file
Date: Sun, 14 Apr 2013 14:54:36 +0200	[thread overview]
Message-ID: <1365944088-10588-22-git-send-email-mhagger@alum.mit.edu> (raw)
In-Reply-To: <1365944088-10588-1-git-send-email-mhagger@alum.mit.edu>

When a reference that existed in the packed-refs file is deleted, the
packed-refs file must be rewritten.  Previously, the file was
rewritten without any peeled refs, even if the file contained peeled
refs when it was read.  This was not a bug, because the packed-refs
file header didn't claim that the file contained peeled values.  But
it had a performance cost, because the repository would lose the
benefit of having precomputed peeled references until pack-refs was
run again.

Teach repack_without_ref() to write peeled refs to the packed-refs
file (regardless of whether they were present in the old version of
the file).

This means that if the old version of the packed-refs file was not
fully peeled, then repack_without_ref() will have to peel references.
To avoid the expense of reading lots of loose references, we take two
shortcuts relative to pack-refs:

* If the peeled value of a reference is already known (i.e., because
  it was read from the old version of the packed-refs file), then
  output that peeled value again without any checks.  This is the
  usual code path and should avoid any noticeable overhead.  (This is
  different than pack-refs, which always re-peels references.)

* We don't verify that the packed ref is still current.  It could be
  that a packed references is overridden by a loose reference, in
  which case the packed ref is no longer needed and might even refer
  to an object that has been garbage collected.  But we don't check;
  instead, we just try to peel all references.  If peeling is
  successful, the peeled value is written out (even though it might
  not be needed any more); if not, then the reference is silently
  omitted from the output.

The extra overhead of peeling references in repack_without_ref()
should only be incurred the first time the packed-refs file is written
by a version of Git that knows about the "fully-peeled" attribute.

Signed-off-by: Michael Haggerty <mhagger@alum.mit.edu>
---

This change could cause a noticeable delay on deleting a reference if
it causes a non-fully-peeled packed-refs file to be fully peeled.  But
after the file has been fully peeled once, deleting a reference should
be as fast as before.  In exchange, reference advertising should be
faster after this change because peeled values will not be lost
whenever a packed reference is deleted.

 refs.c              | 23 +++++++++++++++++++++++
 t/t3211-peel-ref.sh |  2 +-
 2 files changed, 24 insertions(+), 1 deletion(-)

diff --git a/refs.c b/refs.c
index 3c20853..140360f 100644
--- a/refs.c
+++ b/refs.c
@@ -878,6 +878,13 @@ void invalidate_ref_cache(const char *submodule)
 #define PEELED_LINE_LENGTH 42
 
 /*
+ * The packed-refs header line that we write out.  Perhaps other
+ * traits will be added later.  The trailing space is required.
+ */
+static const char PACKED_REFS_HEADER[] =
+	"# pack-refs with: peeled fully-peeled \n";
+
+/*
  * Parse one line from a packed-refs file.  Write the SHA1 to sha1.
  * Return a pointer to the refname within the line (null-terminated),
  * or NULL if there was a problem.
@@ -1392,6 +1399,12 @@ static enum peel_status peel_object(const unsigned char *name, unsigned char *sh
 
 /*
  * Peel the entry (if possible) and return its new peel_status.
+ *
+ * It is OK to call this function with a packed reference entry that
+ * might be stale and might even refer to an object that has since
+ * been garbage-collected.  In such a case, if the entry has
+ * REF_KNOWS_PEELED then leave the status unchanged and return
+ * PEEL_PEELED or PEEL_NON_TAG; otherwise, return PEEL_INVALID.
  */
 static enum peel_status peel_entry(struct ref_entry *entry)
 {
@@ -1958,6 +1971,15 @@ static int repack_ref_fn(struct ref_entry *entry, void *cb_data)
 	if (len > sizeof(line))
 		die("too long a refname '%s'", entry->name);
 	write_or_die(*fd, line, len);
+	if (!peel_entry(entry)) {
+		/* This reference could be peeled; write the peeled value: */
+		if (snprintf(line, sizeof(line), "^%s\n",
+			     sha1_to_hex(entry->u.value.peeled)) !=
+		    PEELED_LINE_LENGTH)
+			die("internal error");
+		write_or_die(*fd, line, PEELED_LINE_LENGTH);
+	}
+
 	return 0;
 }
 
@@ -1988,6 +2010,7 @@ static int repack_without_ref(const char *refname)
 		rollback_lock_file(&packlock);
 		return 0;
 	}
+	write_or_die(fd, PACKED_REFS_HEADER, strlen(PACKED_REFS_HEADER));
 	do_for_each_entry_in_dir(packed, 0, repack_ref_fn, &fd);
 	return commit_lock_file(&packlock);
 }
diff --git a/t/t3211-peel-ref.sh b/t/t3211-peel-ref.sh
index cca1acb..3b7caca 100755
--- a/t/t3211-peel-ref.sh
+++ b/t/t3211-peel-ref.sh
@@ -61,7 +61,7 @@ test_expect_success 'refs are peeled outside of refs/tags (old packed)' '
 	test_cmp expect actual
 '
 
-test_expect_failure 'peeled refs survive deletion of packed ref' '
+test_expect_success 'peeled refs survive deletion of packed ref' '
 	git pack-refs --all &&
 	cp .git/packed-refs fully-peeled &&
 	git branch yadda &&
-- 
1.8.2.1

  parent reply	other threads:[~2013-04-14 12:56 UTC|newest]

Thread overview: 64+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2013-04-14 12:54 [PATCH 00/33] Various cleanups around reference packing and peeling Michael Haggerty
2013-04-14 12:54 ` [PATCH 01/33] refs: document flags constants REF_* Michael Haggerty
2013-04-14 12:54 ` [PATCH 02/33] refs: document the fields of struct ref_value Michael Haggerty
2013-04-14 12:54 ` [PATCH 03/33] refs: document do_for_each_ref() and do_one_ref() Michael Haggerty
2013-04-15 17:38   ` Junio C Hamano
2013-04-16  9:11     ` Michael Haggerty
2013-04-14 12:54 ` [PATCH 04/33] refs: document how current_ref is used Michael Haggerty
2013-04-14 12:54 ` [PATCH 05/33] refs: define constant PEELED_LINE_LENGTH Michael Haggerty
2013-04-14 12:54 ` [PATCH 06/33] do_for_each_ref_in_dirs(): remove dead code Michael Haggerty
2013-04-14 12:54 ` [PATCH 07/33] get_packed_ref(): return a ref_entry Michael Haggerty
2013-04-14 12:54 ` [PATCH 08/33] peel_ref(): use function get_packed_ref() Michael Haggerty
2013-04-14 12:54 ` [PATCH 09/33] repack_without_ref(): " Michael Haggerty
2013-04-14 12:54 ` [PATCH 10/33] refs: extract a function ref_resolves_to_object() Michael Haggerty
2013-04-15 16:51   ` Junio C Hamano
2013-04-16  9:27     ` Michael Haggerty
2013-04-16 18:08       ` Junio C Hamano
2013-04-14 12:54 ` [PATCH 11/33] refs: extract function peel_object() Michael Haggerty
2013-04-14 12:54 ` [PATCH 12/33] peel_object(): give more specific information in return value Michael Haggerty
2013-04-15 17:38   ` Junio C Hamano
2013-04-14 12:54 ` [PATCH 13/33] peel_ref(): fix return value for non-peelable, not-current reference Michael Haggerty
2013-04-15 17:38   ` Junio C Hamano
2013-04-16  9:38     ` Michael Haggerty
2013-04-14 12:54 ` [PATCH 14/33] refs: extract a function peel_entry() Michael Haggerty
2013-04-15 17:38   ` Junio C Hamano
2013-04-16 13:07     ` Michael Haggerty
2013-04-16 17:55       ` Junio C Hamano
2013-04-16 22:01         ` Michael Haggerty
2013-04-14 12:54 ` [PATCH 15/33] refs: change the internal reference-iteration API Michael Haggerty
2013-04-15 17:38   ` Junio C Hamano
2013-04-16 13:27     ` Michael Haggerty
2013-04-16 17:47       ` Junio C Hamano
2013-04-14 12:54 ` [PATCH 16/33] t3210: test for spurious error messages for dangling packed refs Michael Haggerty
2013-04-15 17:39   ` Junio C Hamano
2013-04-16 14:14     ` Michael Haggerty
2013-04-16 23:22       ` Junio C Hamano
2013-04-16 23:57         ` Jeff King
2013-04-17  4:42           ` Junio C Hamano
2013-04-17 22:38             ` Junio C Hamano
2013-04-18  7:46               ` [PATCH 0/2] Add documentation for new expiry option values Michael Haggerty
2013-04-18  7:46                 ` [PATCH 1/2] git-gc.txt, git-reflog.txt: document new expiry options Michael Haggerty
2013-04-18  7:46                 ` [PATCH 2/2] api-parse-options.txt: document "no-" for non-boolean options Michael Haggerty
2013-04-25 18:13                 ` [PATCH] prune: introduce OPT_EXPIRY_DATE() and use it Junio C Hamano
2013-04-17  8:11         ` [PATCH 16/33] t3210: test for spurious error messages for dangling packed refs Michael Haggerty
2013-04-14 12:54 ` [PATCH 17/33] repack_without_ref(): silence errors " Michael Haggerty
2013-04-15 17:39   ` Junio C Hamano
2013-04-17  8:41     ` Michael Haggerty
2013-04-14 12:54 ` [PATCH 18/33] search_ref_dir(): return an index rather than a pointer Michael Haggerty
2013-04-14 12:54 ` [PATCH 19/33] refs: change how packed refs are deleted Michael Haggerty
2013-04-14 12:54 ` [PATCH 20/33] t3211: demonstrate loss of peeled refs if a packed ref is deleted Michael Haggerty
2013-04-14 12:54 ` Michael Haggerty [this message]
2013-04-15 17:39   ` [PATCH 21/33] repack_without_ref(): write peeled refs in the rewritten file Junio C Hamano
2013-04-14 12:54 ` [PATCH 22/33] refs: extract a function write_packed_entry() Michael Haggerty
2013-04-14 12:54 ` [PATCH 23/33] pack-refs: rename handle_one_ref() to pack_one_ref() Michael Haggerty
2013-04-14 12:54 ` [PATCH 24/33] pack-refs: merge code from pack-refs.{c,h} into refs.{c,h} Michael Haggerty
2013-04-14 12:54 ` [PATCH 25/33] pack_one_ref(): rename "path" parameter to "refname" Michael Haggerty
2013-04-14 12:54 ` [PATCH 26/33] refs: use same lock_file object for both ref-packing functions Michael Haggerty
2013-04-14 12:54 ` [PATCH 27/33] pack_refs(): change to use do_for_each_entry() Michael Haggerty
2013-04-14 12:54 ` [PATCH 28/33] refs: inline function do_not_prune() Michael Haggerty
2013-04-14 12:54 ` [PATCH 29/33] pack_one_ref(): use function peel_entry() Michael Haggerty
2013-04-14 12:54 ` [PATCH 30/33] pack_one_ref(): use write_packed_entry() to do the writing Michael Haggerty
2013-04-14 12:54 ` [PATCH 31/33] pack_one_ref(): do some cheap tests before a more expensive one Michael Haggerty
2013-04-14 12:54 ` [PATCH 32/33] refs: change do_for_each_*() functions to take ref_cache arguments Michael Haggerty
2013-04-15 17:40   ` Junio C Hamano
2013-04-14 12:54 ` [PATCH 33/33] refs: handle the main ref_cache specially Michael Haggerty

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1365944088-10588-22-git-send-email-mhagger@alum.mit.edu \
    --to=mhagger@alum.mit.edu \
    --cc=git@vger.kernel.org \
    --cc=gitster@pobox.com \
    --cc=hvoigt@hvoigt.net \
    --cc=peff@peff.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).