[PATCH/RFC] notice error exit from pager

[PATCH/RFC] notice error exit from pager

[Clemens Buchacher]

If the pager fails to run, git produces no output, e.g.:

 $ GIT_PAGER=not-a-command git log

The error reporting fails for two reasons:

 (1) start_command: There is a mechanism that detects errors during
     execvp introduced in 2b541bf8 (start_command: detect execvp
     failures early). The child writes one byte to a pipe only if
     execvp fails.  The parent waits for either EOF, when the
     successful execvp automatically closes the pipe (see
     FD_CLOEXEC in fcntl(1)), or it reads a single byte, in which
     case it knows that the execvp failed. This mechanism is
     incompatible with the workaround introduced in 35ce8622
     (pager: Work around window resizing bug in 'less'), which
     waits for input from the parent before the exec. Since both
     the parent and the child are waiting for input from each
     other, that would result in a deadlock. In order to avoid
     that, the mechanism is disabled by closing the child_notifier
     file descriptor. That is my best guess, at least, from reading
     the code. Hannes, can you confirm if this analysis is correct?

 (2) finish_command: The parent correctly detects the 127 exit
     status from the child, but the error output goes nowhere,
     since by that time it is already being redirected to the
     child.

No simple solution for (1) comes to mind. I am not sure if this bug
is still relevant. I could not reproduce it here with less version
436. If this bug was fixed a long time ago, maybe we can remove the
workaround by now?

Number (2) can be solved by not sending error output to the pager.
But I am not completely satisfied with that solution.  Not
redirecting error output to the pager can result in the pager
overwriting error output with standard output. Right now, I can't
think of any common situations where this would be the case. But,
for example, a git log on a broken repo with some missing commits
will abort at some point with "error: could not read <sha1> [...]".
With this patch, if there is at least one page of standard output
before that error, the error message is not visible at all and the
output simply stops as if git log had reached the root commit.

The easiest solution is to produce the error output in the child.
Below is a patch for that. The previous code seems to go through
some pain to let the parent handle the error output. But I don't
see the point. Is there more to this than meets my eye?

Clemens

Signed-off-by: Clemens Buchacher <drizzd@aon.at>
---
 run-command.c |   11 ++++++-----
 1 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/run-command.c b/run-command.c
index 70e8a24..944a882 100644
--- a/run-command.c
+++ b/run-command.c
@@ -127,9 +127,6 @@ static int wait_or_whine(pid_t pid, const char *argv0, int silent_exec_failure)
 		if (code == 127) {
 			code = -1;
 			failed_errno = ENOENT;
-			if (!silent_exec_failure)
-				error("cannot run %s: %s", argv0,
-					strerror(ENOENT));
 		}
 	} else {
 		error("waitpid is confused (%s)", argv0);
@@ -287,10 +284,14 @@ fail_pipe:
 		 * Do not check for cmd->silent_exec_failure; the parent
 		 * process will check it when it sees this exit code.
 		 */
-		if (errno == ENOENT)
+		if (errno == ENOENT) {
+			if (!cmd->silent_exec_failure)
+				error("cannot run %s: %s", cmd->argv[0],
+					strerror(ENOENT));
 			exit(127);
-		else
+		} else {
 			die_errno("cannot exec '%s'", cmd->argv[0]);
+		}
 	}
 	if (cmd->pid < 0)
 		error("cannot fork() for %s: %s", cmd->argv[0],
-- 
1.7.3.1.105.g84915

Re: [PATCH/RFC] notice error exit from pager

[Linus Torvalds]

On Tue, Jul 26, 2011 at 2:04 PM, Clemens Buchacher <drizzd@aon.at> wrote:
>
> No simple solution for (1) comes to mind. I am not sure if this bug
> is still relevant. I could not reproduce it here with less version
> 436. If this bug was fixed a long time ago, maybe we can remove the
> workaround by now?

Hmm. I can confirm that it doesn't happen for me either any more,
although I don't know if that is an actual less bugfix, or it was
timing-dependent.

That said, the workaround has other nice properties too, like delaying
the screen cleaning that "less" often does. So I don't know if we
really want to get rid of it.

> Number (2) can be solved by not sending error output to the pager.

I think your patch looks reasonable.

                  Linus

Re: [PATCH/RFC] notice error exit from pager

[Johannes Sixt]

Am 26.07.2011 23:04, schrieb Clemens Buchacher:
> If the pager fails to run, git produces no output, e.g.:
> 
>  $ GIT_PAGER=not-a-command git log
> 
> The error reporting fails for two reasons:
> 
>  (1) start_command: There is a mechanism that detects errors during
>      execvp introduced in 2b541bf8 (start_command: detect execvp
>      failures early). ... This mechanism is
>      incompatible with the workaround introduced in 35ce8622
>      (pager: Work around window resizing bug in 'less')

You analysis is correct. I think the bug in less was fixed shortly after
this workaround was introduced. I would like to remove this workaround
for a different reason, but it seems there are people who are quite fond
of it ;)

> diff --git a/run-command.c b/run-command.c
> index 70e8a24..944a882 100644
> --- a/run-command.c
> +++ b/run-command.c
> @@ -127,9 +127,6 @@ static int wait_or_whine(pid_t pid, const char *argv0, int silent_exec_failure)
>  		if (code == 127) {
>  			code = -1;
>  			failed_errno = ENOENT;
> -			if (!silent_exec_failure)
> -				error("cannot run %s: %s", argv0,
> -					strerror(ENOENT));
>  		}
>  	} else {
>  		error("waitpid is confused (%s)", argv0);
> @@ -287,10 +284,14 @@ fail_pipe:
>  		 * Do not check for cmd->silent_exec_failure; the parent
>  		 * process will check it when it sees this exit code.
>  		 */
> -		if (errno == ENOENT)
> +		if (errno == ENOENT) {
> +			if (!cmd->silent_exec_failure)
> +				error("cannot run %s: %s", cmd->argv[0],
> +					strerror(ENOENT));

This change is not good enough: There is no guarantee that this message
goes to the right channel (stderr in the child can be redirected). Look
carefully: We set a special die routine in the forked child that ensures
that the die() messages are written to the right channel; we do not have
a similar facility for error() messages (or do we?).

>  			exit(127);
> -		else
> +		} else {
>  			die_errno("cannot exec '%s'", cmd->argv[0]);
> +		}
>  	}
>  	if (cmd->pid < 0)
>  		error("cannot fork() for %s: %s", cmd->argv[0],

-- Hannes

[PATCH] error_routine: use parent's stderr if exec fails

[Clemens Buchacher]

The new process's error output may be redirected elsewhere, but if
the exec fails, output should still go to the parent's stderr. This
has already been done for the die_routine. Do the same for
error_routine.

Signed-off-by: Clemens Buchacher <drizzd@aon.at>
---

On Wed, Jul 27, 2011 at 09:36:52PM +0200, Johannes Sixt wrote:
> 
> This change is not good enough: There is no guarantee that this message
> goes to the right channel (stderr in the child can be redirected).

Ah, good point. I skipped over that part thinking it was not
relevant.  This should take care of it.

Clemens

 git-compat-util.h |    2 ++
 run-command.c     |   15 +++++++--------
 usage.c           |   18 ++++++++++++++++++
 3 files changed, 27 insertions(+), 8 deletions(-)

diff --git a/git-compat-util.h b/git-compat-util.h
index a75530d..07cbfe9 100644
--- a/git-compat-util.h
+++ b/git-compat-util.h
@@ -240,6 +240,7 @@ extern char *gitbasename(char *);
 
 /* General helper functions */
 extern void vreportf(const char *prefix, const char *err, va_list params);
+extern void vwritef(int fd, const char *prefix, const char *err, va_list params);
 extern NORETURN void usage(const char *err);
 extern NORETURN void usagef(const char *err, ...) __attribute__((format (printf, 1, 2)));
 extern NORETURN void die(const char *err, ...) __attribute__((format (printf, 1, 2)));
@@ -248,6 +249,7 @@ extern int error(const char *err, ...) __attribute__((format (printf, 1, 2)));
 extern void warning(const char *err, ...) __attribute__((format (printf, 1, 2)));
 
 extern void set_die_routine(NORETURN_PTR void (*routine)(const char *err, va_list params));
+extern void set_error_routine(void (*routine)(const char *err, va_list params));
 
 extern int prefixcmp(const char *str, const char *prefix);
 extern int suffixcmp(const char *str, const char *suffix);
diff --git a/run-command.c b/run-command.c
index c752174..a2796c4 100644
--- a/run-command.c
+++ b/run-command.c
@@ -77,16 +77,14 @@ static void notify_parent(void)
 
 static NORETURN void die_child(const char *err, va_list params)
 {
-	char msg[4096];
-	int len = vsnprintf(msg, sizeof(msg), err, params);
-	if (len > sizeof(msg))
-		len = sizeof(msg);
-
-	write_in_full(child_err, "fatal: ", 7);
-	write_in_full(child_err, msg, len);
-	write_in_full(child_err, "\n", 1);
+	vwritef(child_err, "fatal: ", err, params);
 	exit(128);
 }
+
+static void error_child(const char *err, va_list params)
+{
+	vwritef(child_err, "error: ", err, params);
+}
 #endif
 
 static inline void set_cloexec(int fd)
@@ -214,6 +212,7 @@ fail_pipe:
 			set_cloexec(child_err);
 		}
 		set_die_routine(die_child);
+		set_error_routine(error_child);
 
 		close(notify_pipe[0]);
 		set_cloexec(notify_pipe[1]);
diff --git a/usage.c b/usage.c
index b5e67e3..a2a6678 100644
--- a/usage.c
+++ b/usage.c
@@ -4,6 +4,7 @@
  * Copyright (C) Linus Torvalds, 2005
  */
 #include "git-compat-util.h"
+#include "cache.h"
 
 void vreportf(const char *prefix, const char *err, va_list params)
 {
@@ -12,6 +13,18 @@ void vreportf(const char *prefix, const char *err, va_list params)
 	fprintf(stderr, "%s%s\n", prefix, msg);
 }
 
+void vwritef(int fd, const char *prefix, const char *err, va_list params)
+{
+	char msg[4096];
+	int len = vsnprintf(msg, sizeof(msg), err, params);
+	if (len > sizeof(msg))
+		len = sizeof(msg);
+
+	write_in_full(fd, prefix, strlen(prefix));
+	write_in_full(fd, msg, len);
+	write_in_full(fd, "\n", 1);
+}
+
 static NORETURN void usage_builtin(const char *err, va_list params)
 {
 	vreportf("usage: ", err, params);
@@ -46,6 +59,11 @@ void set_die_routine(NORETURN_PTR void (*routine)(const char *err, va_list param
 	die_routine = routine;
 }
 
+void set_error_routine(void (*routine)(const char *err, va_list params))
+{
+	error_routine = routine;
+}
+
 void NORETURN usagef(const char *err, ...)
 {
 	va_list params;
-- 
1.7.3.1.105.g84915

[PATCH] notice error exit from pager

[Clemens Buchacher]

If the pager fails to run, git produces no output, e.g.:

 $ GIT_PAGER=not-a-command git log

The error reporting fails for two reasons:

 (1) start_command: There is a mechanism that detects errors during
     execvp introduced in 2b541bf8 (start_command: detect execvp
     failures early). The child writes one byte to a pipe only if
     execvp fails.  The parent waits for either EOF, when the
     successful execvp automatically closes the pipe (see
     FD_CLOEXEC in fcntl(1)), or it reads a single byte, in which
     case it knows that the execvp failed. This mechanism is
     incompatible with the workaround introduced in 35ce8622
     (pager: Work around window resizing bug in 'less'), which
     waits for input from the parent before the exec. Since both
     the parent and the child are waiting for input from each
     other, that would result in a deadlock. In order to avoid
     that, the mechanism is disabled by closing the child_notifier
     file descriptor.

 (2) finish_command: The parent correctly detects the 127 exit
     status from the child, but the error output goes nowhere,
     since by that time it is already being redirected to the
     child.

No simple solution for (1) comes to mind.

Number (2) can be solved by not sending error output to the pager.
Not redirecting error output to the pager can result in the pager
overwriting error output with standard output, however.

Since there is no reliable way to handle error reporting in the
parent, produce the output in the child instead.

Signed-off-by: Clemens Buchacher <drizzd@aon.at>
---

Nothing new compared to the RFC, just a slightly trimmed commit
message.

 run-command.c |   15 ++++++---------
 1 files changed, 6 insertions(+), 9 deletions(-)

diff --git a/run-command.c b/run-command.c
index 5c91f37..a2796c4 100644
--- a/run-command.c
+++ b/run-command.c
@@ -125,9 +125,6 @@ static int wait_or_whine(pid_t pid, const char *argv0, int silent_exec_failure)
 		if (code == 127) {
 			code = -1;
 			failed_errno = ENOENT;
-			if (!silent_exec_failure)
-				error("cannot run %s: %s", argv0,
-					strerror(ENOENT));
 		}
 	} else {
 		error("waitpid is confused (%s)", argv0);
@@ -282,14 +279,14 @@ fail_pipe:
 		} else {
 			execvp(cmd->argv[0], (char *const*) cmd->argv);
 		}
-		/*
-		 * Do not check for cmd->silent_exec_failure; the parent
-		 * process will check it when it sees this exit code.
-		 */
-		if (errno == ENOENT)
+		if (errno == ENOENT) {
+			if (!cmd->silent_exec_failure)
+				error("cannot run %s: %s", cmd->argv[0],
+					strerror(ENOENT));
 			exit(127);
-		else
+		} else {
 			die_errno("cannot exec '%s'", cmd->argv[0]);
+		}
 	}
 	if (cmd->pid < 0)
 		error("cannot fork() for %s: %s", cmd->argv[0],
-- 
1.7.3.1.105.g84915

Re: [PATCH] notice error exit from pager

[Junio C Hamano]

Clemens Buchacher <drizzd@aon.at> writes:

> Since there is no reliable way to handle error reporting in the
> parent, produce the output in the child instead.

Hmm, how does this interact with your earlier "error_routine: use parent's
stderr if exec fails" patch?

> diff --git a/run-command.c b/run-command.c
> index 5c91f37..a2796c4 100644
> --- a/run-command.c
> +++ b/run-command.c
> @@ -125,9 +125,6 @@ static int wait_or_whine(pid_t pid, const char *argv0, int silent_exec_failure)
>  		if (code == 127) {
>  			code = -1;
>  			failed_errno = ENOENT;
> -			if (!silent_exec_failure)
> -				error("cannot run %s: %s", argv0,
> -					strerror(ENOENT));
>  		}
>  	} else {
>  		error("waitpid is confused (%s)", argv0);
> @@ -282,14 +279,14 @@ fail_pipe:
>  		} else {
>  			execvp(cmd->argv[0], (char *const*) cmd->argv);
>  		}
> -		/*
> -		 * Do not check for cmd->silent_exec_failure; the parent
> -		 * process will check it when it sees this exit code.
> -		 */
> -		if (errno == ENOENT)
> +		if (errno == ENOENT) {
> +			if (!cmd->silent_exec_failure)
> +				error("cannot run %s: %s", cmd->argv[0],
> +					strerror(ENOENT));
>  			exit(127);
> -		else
> +		} else {
>  			die_errno("cannot exec '%s'", cmd->argv[0]);
> +		}
>  	}
>  	if (cmd->pid < 0)
>  		error("cannot fork() for %s: %s", cmd->argv[0],

Re: [PATCH] notice error exit from pager

[Clemens Buchacher]

On Mon, Aug 01, 2011 at 01:17:28PM -0700, Junio C Hamano wrote:
> Clemens Buchacher <drizzd@aon.at> writes:
> 
> > Since there is no reliable way to handle error reporting in the
> > parent, produce the output in the child instead.
> 
> Hmm, how does this interact with your earlier "error_routine: use parent's
> stderr if exec fails" patch?

The use case addressed in this patch (i.e., the pager) does not
strictly dependent on the previous patch. But Johannes noted that
in general, error output of the child could be redirected, hiding
the execve failure again. The error_routine patch avoids that by
writing to the parent's stderr stream.

Clemens