[PATCH 0/3] nicer receive-pack errors over http

[PATCH 0/3] nicer receive-pack errors over http

[Jeff King]

While we are on the subject of http user experience, I thought I'd
mention this patch to route more errors from index-pack back to the
user. We're not doing it yet at GitHub, but I plan to apply it soon.

The first patch is a cleanup and minor bug fix. The second is the
interesting one. The third is purely cosmetic, and doesn't need to be
tied to the others.

  [1/3]: receive-pack: redirect unpack-objects stdout to /dev/null
  [2/3]: receive-pack: send pack-processing stderr over sideband
  [3/3]: receive-pack: drop "n/a" on unpacker errors

-Peff

[PATCH 1/3] receive-pack: redirect unpack-objects stdout to /dev/null

[Jeff King]

The unpack-objects command should not generally produce any
output on stdout. However, if it's given extra input after
the packfile, it will spew the remainder to stdout. When
called by receive-pack, this means we will break protocol,
since our stdout is connected to the remote send-pack.

Signed-off-by: Jeff King <peff@peff.net>
---
I've never actually seen this bug in practice, but I noticed it while
auditing the outputs of receive-pack's children recently.

 builtin/receive-pack.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 9145f1a..5ba0c98 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -815,6 +815,7 @@ static const char *unpack(void)
 
 	if (ntohl(hdr.hdr_entries) < unpack_limit) {
 		int code, i = 0;
+		struct child_process child;
 		const char *unpacker[5];
 		unpacker[i++] = "unpack-objects";
 		if (quiet)
@@ -823,7 +824,11 @@ static const char *unpack(void)
 			unpacker[i++] = "--strict";
 		unpacker[i++] = hdr_arg;
 		unpacker[i++] = NULL;
-		code = run_command_v_opt(unpacker, RUN_GIT_CMD);
+		memset(&child, 0, sizeof(child));
+		child.argv = unpacker;
+		child.no_stdout = 1;
+		child.git_cmd = 1;
+		code = run_command(&child);
 		if (!code)
 			return NULL;
 		return "unpack-objects abnormal exit";
-- 
1.7.11.7.15.g085c6bd

[PATCH 2/3] receive-pack: send pack-processing stderr over sideband

[Jeff King]

Receive-pack invokes either unpack-objects or index-pack to
handle the incoming pack. However, we do not redirect the
stderr of the sub-processes at all, so it is never seen by
the client. From the initial thread adding sideband support,
which is here:

  http://thread.gmane.org/gmane.comp.version-control.git/139471

it is clear that some messages are specifically kept off the
sideband (with the assumption that they are of interest only
to an administrator, not the client). The stderr of the
subprocesses is mentioned in the thread, but it's unclear if
they are included in that group, or were simply forgotten.

However, there are a few good reasons to show them to the
client:

  1. In many cases, they are directly about the incoming
     packfile (e.g., fsck warnings with --strict, corruption
     in the packfile, etc). Without these messages, the
     client just gets "unpacker error" with no extra useful
     diagnosis.

  2. No matter what the cause, we are probably better off
     showing the errors to the client. If the client and the
     server admin are not the same entity, it is probably
     much easier for the client to cut-and-paste the errors
     they see than for the admin to try to dig them out of a
     log and correlate them with a particular session.

  3. Users of the ssh transport typically already see these
     stderr messages, as the remote's stderr is copied
     literally by ssh. This brings other transports (http,
     and push-over-git if you are crazy enough to enable it)
     more in line with ssh. As a bonus for ssh users,
     because the messages are now fed through the sideband
     and printed by the local git, they will have "remote:"
     prepended and be properly interleaved with any local
     output to stderr.

Signed-off-by: Jeff King <peff@peff.net>
---
This one is logically independent of the first patch, but relies
textually on the conversion of unpack-objects to use a separate
child_process.

 builtin/receive-pack.c | 26 ++++++++++++++++++++++++--
 1 file changed, 24 insertions(+), 2 deletions(-)

diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index 5ba0c98..ac679ab 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -795,7 +795,7 @@ static const char *pack_lockfile;
 
 static const char *pack_lockfile;
 
-static const char *unpack(void)
+static const char *unpack(int err_fd)
 {
 	struct pack_header hdr;
 	const char *hdr_err;
@@ -827,6 +827,7 @@ static const char *unpack(void)
 		memset(&child, 0, sizeof(child));
 		child.argv = unpacker;
 		child.no_stdout = 1;
+		child.err = err_fd;
 		child.git_cmd = 1;
 		code = run_command(&child);
 		if (!code)
@@ -853,6 +854,7 @@ static const char *unpack(void)
 		memset(&ip, 0, sizeof(ip));
 		ip.argv = keeper;
 		ip.out = -1;
+		ip.err = err_fd;
 		ip.git_cmd = 1;
 		status = start_command(&ip);
 		if (status) {
@@ -869,6 +871,26 @@ static const char *unpack(void)
 	}
 }
 
+static const char *unpack_with_sideband(void)
+{
+	struct async muxer;
+	const char *ret;
+
+	if (!use_sideband)
+		return unpack(0);
+
+	memset(&muxer, 0, sizeof(muxer));
+	muxer.proc = copy_to_sideband;
+	muxer.in = -1;
+	if (start_async(&muxer))
+		return NULL;
+
+	ret = unpack(muxer.in);
+
+	finish_async(&muxer);
+	return ret;
+}
+
 static void report(struct command *commands, const char *unpack_status)
 {
 	struct command *cmd;
@@ -966,7 +988,7 @@ int cmd_receive_pack(int argc, const char **argv, const char *prefix)
 		const char *unpack_status = NULL;
 
 		if (!delete_only(commands))
-			unpack_status = unpack();
+			unpack_status = unpack_with_sideband();
 		execute_commands(commands, unpack_status);
 		if (pack_lockfile)
 			unlink_or_warn(pack_lockfile);
-- 
1.7.11.7.15.g085c6bd

Re: [PATCH 2/3] receive-pack: send pack-processing stderr over sideband

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> Receive-pack invokes either unpack-objects or index-pack to
> handle the incoming pack. However, we do not redirect the
> stderr of the sub-processes at all, so it is never seen by
> the client. From the initial thread adding sideband support,
> which is here:
>
>   http://thread.gmane.org/gmane.comp.version-control.git/139471
>
> it is clear that some messages are specifically kept off the
> sideband (with the assumption that they are of interest only
> to an administrator, not the client). The stderr of the
> subprocesses is mentioned in the thread, but it's unclear if
> they are included in that group, or were simply forgotten.
>
> However, there are a few good reasons to show them to the
> client:
>
>   1. In many cases, they are directly about the incoming
>      packfile (e.g., fsck warnings with --strict, corruption
>      in the packfile, etc). Without these messages, the
>      client just gets "unpacker error" with no extra useful
>      diagnosis.
>
>   2. No matter what the cause, we are probably better off
>      showing the errors to the client. If the client and the
>      server admin are not the same entity, it is probably
>      much easier for the client to cut-and-paste the errors
>      they see than for the admin to try to dig them out of a
>      log and correlate them with a particular session.

I agree with the "probably" above (and also with points 1 and 3),
but it at the same time feel a bit iffy.  The server side would lose
log entries to check when the operator observes higher error rate
and starts suspecting something recently broke, and the lost clue
cannot be recovered without contacting the pushers, no?

>   3. Users of the ssh transport typically already see these
>      stderr messages, as the remote's stderr is copied
>      literally by ssh. This brings other transports (http,
>      and push-over-git if you are crazy enough to enable it)
>      more in line with ssh. As a bonus for ssh users,
>      because the messages are now fed through the sideband
>      and printed by the local git, they will have "remote:"
>      prepended and be properly interleaved with any local
>      output to stderr.
>
> Signed-off-by: Jeff King <peff@peff.net>
> ---

Will queue; thanks.

Re: [PATCH 2/3] receive-pack: send pack-processing stderr over sideband

[Jeff King]

On Fri, Sep 21, 2012 at 09:49:40AM -0700, Junio C Hamano wrote:

> >   2. No matter what the cause, we are probably better off
> >      showing the errors to the client. If the client and the
> >      server admin are not the same entity, it is probably
> >      much easier for the client to cut-and-paste the errors
> >      they see than for the admin to try to dig them out of a
> >      log and correlate them with a particular session.
> 
> I agree with the "probably" above (and also with points 1 and 3),
> but it at the same time feel a bit iffy.  The server side would lose
> log entries to check when the operator observes higher error rate
> and starts suspecting something recently broke, and the lost clue
> cannot be recovered without contacting the pushers, no?

Yeah, that is true, although that is already the case with ssh pushes.
Conversely, it also means that servers using the ssh transport have lost
the option of redirecting the server-side stderr (e.g., with a wrapper
around git-receive-pack) to a log if they were already doing so.

However, this does make things more consistent with upload-pack, which
connects the stderr of pack-objects to sideband (which it must to handle
progress). Furthermore, many of the messages from receive-pack are
handled by rp_error, which sends to the sideband. So if you were
monitoring your git purely by trying to capture stderr, you were already
only getting a fraction of the real data.

If a server side really did want to capture the error output, I think
the right way to do it would be:

  1. Modify send_sideband to send a copy of all band-2 data to stderr.

  2. Redirect stderr from all processes to a log processor (it's
     tempting to just send it somewhere besides stderr in (1) above, but
     you may also get clients which do not claim to support sidebands,
     in which case we will just spew to stderr).

  3. Do immediate culling on the output before storage, because some of
     what you get will be junk like progress reports (and they all come
     down the same fd, since it is typically stderr from a subprocess).

So I think this patch is not really making anything _worse_ if somebody
wanted to do that. It just moves the messages from being caught by step
(2) to being caught by step (1). But you have to do both either way.

-Peff

Re: [PATCH 2/3] receive-pack: send pack-processing stderr over sideband

[Junio C Hamano]

Jeff King <peff@peff.net> writes:

> On Fri, Sep 21, 2012 at 09:49:40AM -0700, Junio C Hamano wrote:
>
>> >   2. No matter what the cause, we are probably better off
>> >      showing the errors to the client. If the client and the
>> >      server admin are not the same entity, it is probably
>> >      much easier for the client to cut-and-paste the errors
>> >      they see than for the admin to try to dig them out of a
>> >      log and correlate them with a particular session.
>> 
>> I agree with the "probably" above (and also with points 1 and 3),
>> but it at the same time feel a bit iffy.  The server side would lose
>> log entries to check when the operator observes higher error rate
>> and starts suspecting something recently broke, and the lost clue
>> cannot be recovered without contacting the pushers, no?
>
> Yeah, that is true, although that is already the case with ssh pushes.
> Conversely, it also means that servers using the ssh transport have lost
> the option of redirecting the server-side stderr (e.g., with a wrapper
> around git-receive-pack) to a log if they were already doing so.

Yes.

> However, this does make things more consistent with upload-pack, which
> connects the stderr of pack-objects to sideband (which it must to handle
> progress). Furthermore, many of the messages from receive-pack are
> handled by rp_error, which sends to the sideband. So if you were
> monitoring your git purely by trying to capture stderr, you were already
> only getting a fraction of the real data.

The comments were not meant as a rejection notice ;-) Just to see if
some server operators have input on the matter.

I personally do not think tee-ing the error output is worth it; it
would be reasonably simple to arrange, and the server operators who
want it can ask later if that is need.

Thanks.

Re: [PATCH 2/3] receive-pack: send pack-processing stderr over sideband

[Jeff King]

On Fri, Sep 21, 2012 at 10:25:24AM -0700, Junio C Hamano wrote:

> > However, this does make things more consistent with upload-pack, which
> > connects the stderr of pack-objects to sideband (which it must to handle
> > progress). Furthermore, many of the messages from receive-pack are
> > handled by rp_error, which sends to the sideband. So if you were
> > monitoring your git purely by trying to capture stderr, you were already
> > only getting a fraction of the real data.
> 
> The comments were not meant as a rejection notice ;-) Just to see if
> some server operators have input on the matter.

I know.  But your comment made me second-guess a little whether
anybody would be inadvertently hurt, but thinking it through and writing
it out helped convince myself that it's the right thing to do.

One of the hardest parts of working on a mature software project is not
just thinking about what you want to do, but thinking about what
everyone else wants to do (or is doing). So it never hurts to
double-check your assumptions in such a case, and I don't mind working
through these "what ifs" even if they end in us doing the original
thing. I hope you do not mind reading them too much. :)

> I personally do not think tee-ing the error output is worth it; it
> would be reasonably simple to arrange, and the server operators who
> want it can ask later if that is need.

Agreed.

-Peff

[PATCH 3/3] receive-pack: drop "n/a" on unpacker errors

[Jeff King]

The output from git push currently looks like this:

  $ git push dest HEAD
  fatal: [some message from index-pack]
  error: unpack failed: index-pack abnormal exit
  To dest
   ! [remote rejected] HEAD -> master (n/a (unpacker error))

That n/a is meant to be "the per-ref status is not
available" but the nested parentheses just make it look
ugly. Let's turn the final line into just:

   ! [remote rejected] HEAD -> master (unpacker error)

Signed-off-by: Jeff King <peff@peff.net>
---
Maybe it is just me, but I have always found the "n/a" and extra
parentheses ugly and unnecessary. But obviously others may differ.
It doesn't really come up that often, since index-pack failing usually
implies a git bug. But with transfer.fsckObjects turn on, it is more
common.

I don't think there should be any backwards compatibility issues with
changing this. The "reason" field sent back by receive-pack has always
been a free-form human-readable string.

I also dislike the "index-pack abnormal exit" message. Again, when
index-pack really crashes, it's fine, but it can die due to bogus
objects, too, in which case it might be nice to have a more
human-readable message.

 builtin/receive-pack.c          | 2 +-
 t/t5504-fetch-receive-strict.sh | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/builtin/receive-pack.c b/builtin/receive-pack.c
index ac679ab..ff781fe 100644
--- a/builtin/receive-pack.c
+++ b/builtin/receive-pack.c
@@ -695,7 +695,7 @@ static void execute_commands(struct command *commands, const char *unpacker_erro
 
 	if (unpacker_error) {
 		for (cmd = commands; cmd; cmd = cmd->next)
-			cmd->error_string = "n/a (unpacker error)";
+			cmd->error_string = "unpacker error";
 		return;
 	}
 
diff --git a/t/t5504-fetch-receive-strict.sh b/t/t5504-fetch-receive-strict.sh
index 35ec294..69ee13c 100755
--- a/t/t5504-fetch-receive-strict.sh
+++ b/t/t5504-fetch-receive-strict.sh
@@ -89,7 +89,7 @@ To dst
 
 cat >exp <<EOF
 To dst
-!	refs/heads/master:refs/heads/test	[remote rejected] (n/a (unpacker error))
+!	refs/heads/master:refs/heads/test	[remote rejected] (unpacker error)
 EOF
 
 test_expect_success 'push with receive.fsckobjects' '
-- 
1.7.11.7.15.g085c6bd