[PATCH/RFC] clone: inform the user we are checking out

[PATCH/RFC] clone: inform the user we are checking out

[Erik Faye-Lund]

When cloning a large repository over a local file-system, git
can use hard-links to the old repository files, making the
repository-cloning very fast. However, git also perform an
implicit checkout of the files, which can be a lengthy
operation.

Inform the user about this, and move the logic to notify the
user that we are done until we actually are.

Signed-off-by: Erik Faye-Lund <kusmabite@gmail.com>
---

I just tried to clone a very large (proprietary) repository over
a local filesystyem, and the output struck me as confusing:

---8<---
$ git clone some-repo.git some-other-repo
Cloning into 'some-other-repo'...    <happens instantly>
done.                                <hangs for minutes>
$
---8<---

Now, seems to be because repo gets hard-linked, so the cloning
part did not actually take much time. However, we perform a
check-out at the end, and this does take time for my large repo.
So the behavior probably makes sense from a git-internals point
of view.

But from an end-user point of view, it's confusing. I asked git
to clone, and it told me it finished, only to hang around for
several minutes while, judging by the output, doing nothing.

So, perhaps it could make sense to do something along these lines?

This gives me this output instead:

---8<---
$ git clone some-repo.git some-other-repo
Cloning into 'some-repo'...    <happens instantly>
Checking out HEAD...           <hangs for minutes>
done.
$
---8<---

...which seems much more informative to me.

 builtin/clone.c | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/builtin/clone.c b/builtin/clone.c
index bbd5c96..3f863a1 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -368,9 +368,6 @@ static void clone_local(const char *src_repo, const char *dest_repo)
 		strbuf_release(&src);
 		strbuf_release(&dest);
 	}
-
-	if (0 <= option_verbosity)
-		printf(_("done.\n"));
 }
 
 static const char *junk_work_tree;
@@ -544,6 +541,9 @@ static int checkout(void)
 	if (option_no_checkout)
 		return 0;
 
+	if (0 <= option_verbosity)
+		printf(_("Checking out HEAD...\n"));
+
 	head = resolve_refdup("HEAD", sha1, 1, NULL);
 	if (!head) {
 		warning(_("remote HEAD refers to nonexistent ref, "
@@ -870,5 +870,9 @@ int cmd_clone(int argc, const char **argv, const char *prefix)
 	strbuf_release(&key);
 	strbuf_release(&value);
 	junk_pid = 0;
+
+	if (is_local && 0 <= option_verbosity)
+		printf(_("done.\n"));
+
 	return err;
 }
-- 
1.7.10.1.457.g8275905

Re: [PATCH/RFC] clone: inform the user we are checking out

[Johannes Sixt]

Am 5/7/2012 11:09, schrieb Erik Faye-Lund:
> When cloning a large repository over a local file-system
...
> $ git clone some-repo.git some-other-repo
> Cloning into 'some-other-repo'...    <happens instantly>
> done.                                <hangs for minutes>
> $
...
> I asked git
> to clone, and it told me it finished, only to hang around for
> several minutes while, judging by the output, doing nothing.

We have a nice "Checking out files" progress indicator. I wonder why you
do not see it.

At any rate, it's better to write "done" only after we're really done.

-- Hannes

Re: [PATCH/RFC] clone: inform the user we are checking out

[Erik Faye-Lund]

On Mon, May 7, 2012 at 11:26 AM, Johannes Sixt <j.sixt@viscovery.net> wrote:
> Am 5/7/2012 11:09, schrieb Erik Faye-Lund:
>> When cloning a large repository over a local file-system
> ...
>> $ git clone some-repo.git some-other-repo
>> Cloning into 'some-other-repo'...    <happens instantly>
>> done.                                <hangs for minutes>
>> $
> ...
>> I asked git
>> to clone, and it told me it finished, only to hang around for
>> several minutes while, judging by the output, doing nothing.
>
> We have a nice "Checking out files" progress indicator. I wonder why you
> do not see it.
>

That's a very good point, I forgot about that one. I'll investigate
that part. Just to be clear, I'm seeing this problem on version
1.7.10.1.457.g8275905 (junio's 'master' as of right now, AFAICT) on
CentOS. I've built with:
NO_OPENSSL = YesPlease
NO_CURL = YesPlease

"git config -l" does not reveal that any progress related
config-options are set.

> At any rate, it's better to write "done" only after we're really done.

Well, perhaps, but at the same time perhaps not: We only write it for
local clones, because the remote-clones are quite chatty to begin
with, and the remote-stuff seems to inform about it's actions by
printing "<performing task x>...done!". Moving the final
"done."-printing to the very end might be a step in making these
outputs even less consistent. But it might not be important.

Re: [PATCH/RFC] clone: inform the user we are checking out

[Erik Faye-Lund]

On Mon, May 7, 2012 at 11:35 AM, Erik Faye-Lund <kusmabite@gmail.com> wrote:
> On Mon, May 7, 2012 at 11:26 AM, Johannes Sixt <j.sixt@viscovery.net> wrote:
>> Am 5/7/2012 11:09, schrieb Erik Faye-Lund:
>>> When cloning a large repository over a local file-system
>> ...
>>> $ git clone some-repo.git some-other-repo
>>> Cloning into 'some-other-repo'...    <happens instantly>
>>> done.                                <hangs for minutes>
>>> $
>> ...
>>> I asked git
>>> to clone, and it told me it finished, only to hang around for
>>> several minutes while, judging by the output, doing nothing.
>>
>> We have a nice "Checking out files" progress indicator. I wonder why you
>> do not see it.
>>
>
> That's a very good point, I forgot about that one. I'll investigate
> that part. Just to be clear, I'm seeing this problem on version
> 1.7.10.1.457.g8275905 (junio's 'master' as of right now, AFAICT) on
> CentOS. I've built with:
> NO_OPENSSL = YesPlease
> NO_CURL = YesPlease
>
> "git config -l" does not reveal that any progress related
> config-options are set.
>

It seems progress is only enabled when cloning with the verbose-flag.
This is because check_updates in unpack-trees.c only enabled progress
if both o->update and o->verbose_update is set, and checkout in
builtin/clone.c fills in opts.verbose_update with (option_verbosity >
0).

So that's that mystery. I still think the output without the
verbose-flag is confusing as-is.

Re: [PATCH/RFC] clone: inform the user we are checking out

[Erik Faye-Lund]

On Mon, May 7, 2012 at 11:46 AM, Erik Faye-Lund <kusmabite@gmail.com> wrote:
> On Mon, May 7, 2012 at 11:35 AM, Erik Faye-Lund <kusmabite@gmail.com> wrote:
>> On Mon, May 7, 2012 at 11:26 AM, Johannes Sixt <j.sixt@viscovery.net> wrote:
>>> Am 5/7/2012 11:09, schrieb Erik Faye-Lund:
>>>> I asked git
>>>> to clone, and it told me it finished, only to hang around for
>>>> several minutes while, judging by the output, doing nothing.
>>>
>>> We have a nice "Checking out files" progress indicator. I wonder why you
>>> do not see it.
>
> It seems progress is only enabled when cloning with the verbose-flag.
> This is because check_updates in unpack-trees.c only enabled progress
> if both o->update and o->verbose_update is set, and checkout in
> builtin/clone.c fills in opts.verbose_update with (option_verbosity >
> 0).
>
> So that's that mystery. I still think the output without the
> verbose-flag is confusing as-is.

OK, some blaming shows that this changed in 5bd631b3 ("clone: support
multiple levels of verbosity"), back in February 2010. Before this
patch, one would have to specify the "quiet"-flag to clone to suppress
progress-output, after the patch the default is progress being off.

This seems like the right thing to do if we want to resurrect the
progress-output's default-on behavior:
---8<---
diff --git a/builtin/clone.c b/builtin/clone.c
index 3f863a1..f48e603 100644
--- a/builtin/clone.c
+++ b/builtin/clone.c
@@ -569,7 +569,7 @@ static int checkout(void)
 	opts.update = 1;
 	opts.merge = 1;
 	opts.fn = oneway_merge;
-	opts.verbose_update = (option_verbosity > 0);
+	opts.verbose_update = (option_verbosity >= 0);
 	opts.src_index = &the_index;
 	opts.dst_index = &the_index;

---8<---

Re: [PATCH/RFC] clone: inform the user we are checking out

[Jeff King]

On Mon, May 07, 2012 at 12:02:46PM +0200, Erik Faye-Lund wrote:

> OK, some blaming shows that this changed in 5bd631b3 ("clone: support
> multiple levels of verbosity"), back in February 2010. Before this
> patch, one would have to specify the "quiet"-flag to clone to suppress
> progress-output, after the patch the default is progress being off.

Yeah, I think this is a bug. We already show progress indicators from
the transports without "-v", so there is no reason to require it for the
checkout progress meter. And reading fbd631b3, it looks like the change
is simply a mistake in the patch.

> This seems like the right thing to do if we want to resurrect the
> progress-output's default-on behavior:
> ---8<---
> diff --git a/builtin/clone.c b/builtin/clone.c
> index 3f863a1..f48e603 100644
> --- a/builtin/clone.c
> +++ b/builtin/clone.c
> @@ -569,7 +569,7 @@ static int checkout(void)
>  	opts.update = 1;
>  	opts.merge = 1;
>  	opts.fn = oneway_merge;
> -	opts.verbose_update = (option_verbosity > 0);
> +	opts.verbose_update = (option_verbosity >= 0);
>  	opts.src_index = &the_index;
>  	opts.dst_index = &the_index;
> 
> ---8<---

Yes, that looks correct to me. Thanks for looking into this.

-Peff

Re: [PATCH/RFC] clone: inform the user we are checking out

[Junio C Hamano]

Erik Faye-Lund <kusmabite@gmail.com> writes:

> OK, some blaming shows that this changed in 5bd631b3 ("clone: support
> multiple levels of verbosity"), back in February 2010. Before this
> patch, one would have to specify the "quiet"-flag to clone to suppress
> progress-output, after the patch the default is progress being off.
>
> This seems like the right thing to do if we want to resurrect the
> progress-output's default-on behavior:
> ---8<---
> diff --git a/builtin/clone.c b/builtin/clone.c
> index 3f863a1..f48e603 100644
> --- a/builtin/clone.c
> +++ b/builtin/clone.c
> @@ -569,7 +569,7 @@ static int checkout(void)
>  	opts.update = 1;
>  	opts.merge = 1;
>  	opts.fn = oneway_merge;
> -	opts.verbose_update = (option_verbosity > 0);
> +	opts.verbose_update = (option_verbosity >= 0);
>  	opts.src_index = &the_index;
>  	opts.dst_index = &the_index;
>
> ---8<---

Sounds sensible, as the original said "verbosely update unless we were
told to be quiet", so the normal case should be verbose_update which will
give progress only if it takes too long.

Care to roll a signed-off patch?