[PATCH v2 3/3] Teach git-fetch to exploit server side automatic tag following

[PATCH v2 3/3] Teach git-fetch to exploit server side automatic tag following

[Shawn O. Pearce]

If the remote peer upload-pack process supports the auto-include-tag
protocol extension then we can avoid running a second fetch cycle
on the client side by letting the server send us the annotated tags
along with the objects it is packing for us.  In the following graph
we can now fetch both "tag1" and "tag2" on the same connection that
we fetched "master" from the remote when we only have L available
on the local side:

         T - tag1          S - tag2
        /                 /
   L - o ------ o ------ B
    \                     \
     \                     \
      origin/master         master

The objects for "tag1" are implicitly downloaded without our direct
knowledge.  The existing "quickfetch" optimization within git-fetch
discovers that tag1 is complete after the first connection and does
not open a second connection.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
---
 builtin-fetch.c      |    3 +++
 t/t5503-tagfollow.sh |   26 ++++++++++++++++++++++++++
 transport.c          |    5 +++++
 transport.h          |    3 +++
 4 files changed, 37 insertions(+), 0 deletions(-)

diff --git a/builtin-fetch.c b/builtin-fetch.c
index 26c3d74..55f611e 100644
--- a/builtin-fetch.c
+++ b/builtin-fetch.c
@@ -555,6 +555,8 @@ static int do_fetch(struct transport *transport,
 			read_ref(rm->peer_ref->name, rm->peer_ref->old_sha1);
 	}
 
+	if (tags == TAGS_DEFAULT && autotags)
+		transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, "1");
 	if (fetch_refs(transport, ref_map)) {
 		free_refs(ref_map);
 		return 1;
@@ -568,6 +570,7 @@ static int do_fetch(struct transport *transport,
 		ref_map = NULL;
 		find_non_local_tags(transport, &ref_map, &tail);
 		if (ref_map) {
+			transport_set_option(transport, TRANS_OPT_FOLLOWTAGS, NULL);
 			transport_set_option(transport, TRANS_OPT_DEPTH, "0");
 			fetch_refs(transport, ref_map);
 		}
diff --git a/t/t5503-tagfollow.sh b/t/t5503-tagfollow.sh
index 45ff982..86e5b9b 100755
--- a/t/t5503-tagfollow.sh
+++ b/t/t5503-tagfollow.sh
@@ -121,4 +121,30 @@ test_expect_success 'fetch B, S (commit and tag : 1 connection)' '
 	git diff expect actual
 '
 
+cat - <<EOF >expect
+#S
+want $B
+want $S
+#E
+EOF
+test_expect_success 'new clone fetch master and tags' '
+	git branch -D cat
+	rm -f $U
+	(
+		mkdir clone2 &&
+		cd clone2 &&
+		git init &&
+		git remote add origin .. &&
+		GIT_DEBUG_SEND_PACK=3 git fetch 3>../$U &&
+		test $B = $(git rev-parse --verify origin/master) &&
+		test $S = $(git rev-parse --verify tag2) &&
+		test $B = $(git rev-parse --verify tag2^0) &&
+		test $T = $(git rev-parse --verify tag1) &&
+		test $A = $(git rev-parse --verify tag1^0)
+	) &&
+	test -s $U &&
+	cut -d" " -f1,2 $U >actual &&
+	git diff expect actual
+'
+
 test_done
diff --git a/transport.c b/transport.c
index 166c1d1..12873db 100644
--- a/transport.c
+++ b/transport.c
@@ -560,6 +560,7 @@ static int close_bundle(struct transport *transport)
 struct git_transport_data {
 	unsigned thin : 1;
 	unsigned keep : 1;
+	unsigned followtags : 1;
 	int depth;
 	struct child_process *conn;
 	int fd[2];
@@ -580,6 +581,9 @@ static int set_git_option(struct transport *connection,
 	} else if (!strcmp(name, TRANS_OPT_THIN)) {
 		data->thin = !!value;
 		return 0;
+	} else if (!strcmp(name, TRANS_OPT_FOLLOWTAGS)) {
+		data->followtags = !!value;
+		return 0;
 	} else if (!strcmp(name, TRANS_OPT_KEEP)) {
 		data->keep = !!value;
 		return 0;
@@ -628,6 +632,7 @@ static int fetch_refs_via_pack(struct transport *transport,
 	args.keep_pack = data->keep;
 	args.lock_pack = 1;
 	args.use_thin_pack = data->thin;
+	args.auto_include_tag = data->followtags;
 	args.verbose = transport->verbose > 0;
 	args.depth = data->depth;
 
diff --git a/transport.h b/transport.h
index 6fb4526..8abfc0a 100644
--- a/transport.h
+++ b/transport.h
@@ -53,6 +53,9 @@ struct transport *transport_get(struct remote *, const char *);
 /* Limit the depth of the fetch if not null */
 #define TRANS_OPT_DEPTH "depth"
 
+/* Aggressively fetch annotated tags if possible */
+#define TRANS_OPT_FOLLOWTAGS "followtags"
+
 /**
  * Returns 0 if the option was used, non-zero otherwise. Prints a
  * message to stderr if the option is not used.
-- 
1.5.4.3.529.gb25fb

Re: [PATCH v2 3/3] Teach git-fetch to exploit server side automatic tag following

[Johannes Schindelin]

Hi,

just two comments from me for this patch series:

- it's a beaut, and

- should followtags not be the default?

Ciao,
Dscho

Re: [PATCH v2 3/3] Teach git-fetch to exploit server side automatic tag following

[Shawn O. Pearce]

Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> just two comments from me for this patch series:
> 
> - it's a beaut, and

Thanks.
 
> - should followtags not be the default?

No.  Absolutely not.

The client may not want tag objects from this remote.  It might not
want those tags for all sorts of reasons.  Maybe they are doing a
one-shot pull.  Maybe they don't trust this remote.  Maybe they trust
this remote but this remote is famous for 192M tags containing PDFs
of photo images of build logs printed out on paper, then photographed
on a wood table and finally scanned in at 600 dpi.

So it should only be enabled *if* we were willing to open a second
connection to this remote to followtags from it.  And that's not
a change from prior behavior either, its just faster.

-- 
Shawn.

Re: [PATCH v2 3/3] Teach git-fetch to exploit server side automatic tag following

[Johannes Schindelin]

Hi,

On Wed, 5 Mar 2008, Shawn O. Pearce wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
>  
> > - should followtags not be the default?
> 
> No.  Absolutely not.
> 
> The client may not want tag objects from this remote.  It might not want 
> those tags for all sorts of reasons.  Maybe they are doing a one-shot 
> pull.  Maybe they don't trust this remote.  Maybe they trust this remote 
> but this remote is famous for 192M tags containing PDFs of photo images 
> of build logs printed out on paper, then photographed on a wood table 
> and finally scanned in at 600 dpi.

Sorry, I meant to say: "should the followtags feature not be on by default 
in the circumstances where we would follow tags anyway"...

Maybe you do that, but I did not see it.

> So it should only be enabled *if* we were willing to open a second 
> connection to this remote to followtags from it.  And that's not a 
> change from prior behavior either, its just faster.

Thanks,
Dscho

Re: [PATCH v2 3/3] Teach git-fetch to exploit server side automatic tag following

[Shawn O. Pearce]

Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> On Wed, 5 Mar 2008, Shawn O. Pearce wrote:
> > Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> >  
> > > - should followtags not be the default?
> > 
> > No.  Absolutely not.
> > 
> > The client may not want tag objects from this remote.  It might not want 
> > those tags for all sorts of reasons.  Maybe they are doing a one-shot 
> > pull.  Maybe they don't trust this remote.  Maybe they trust this remote 
> > but this remote is famous for 192M tags containing PDFs of photo images 
> > of build logs printed out on paper, then photographed on a wood table 
> > and finally scanned in at 600 dpi.
> 
> Sorry, I meant to say: "should the followtags feature not be on by default 
> in the circumstances where we would follow tags anyway"...
> 
> Maybe you do that, but I did not see it.

Nope.  followtags should be on if the client wants tags, and off if
the client does not want tags.  Its that simple.  There's no reason
for the client to disable followtags if the server will actually
support it, as disabling it when you do have tags to follow will
only cost you extra net latency.

-- 
Shawn.