[PATCH v2] unblock and unignore SIGPIPE

[PATCH v2] unblock and unignore SIGPIPE

[Patrick Reynolds]

Blocked and ignored signals -- but not caught signals -- are inherited
across exec.  Some callers with sloppy signal-handling behavior can call
git with SIGPIPE blocked or ignored, even non-deterministically.  When
SIGPIPE is blocked or ignored, several git commands can run indefinitely,
ignoring EPIPE returns from write() calls, even when the process that
called them has gone away.  Our specific case involved a pipe of git
diff-tree output to a script that reads a limited amount of diff data.

In an ideal world, git would never be called with SIGPIPE blocked or
ignored.  But in the real world, several real potential callers, including
Perl, Apache, and Unicorn, sometimes spawn subprocesses with SIGPIPE
ignored.  It is easier and more productive to harden git against this
mistake than to clean it up in every potential parent process.

Signed-off-by: Patrick Reynolds <patrick.reynolds@github.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
---
1. Merged Junio's work from pu: moved restore_sigpipe_to_default into
git.c and restyled the new tests.
2. Moved the new tests into t0005.  This meant switching back to `git
diff` as our data generator, as the sample repo in t0005 doesn't have any
files for `git ls-files` to output.
3. Squashed.

 git.c              | 22 ++++++++++++++++++++++
 t/t0005-signals.sh | 22 ++++++++++++++++++++++
 2 files changed, 44 insertions(+)

diff --git a/git.c b/git.c
index 210f1ae..0f03d56 100644
--- a/git.c
+++ b/git.c
@@ -593,6 +593,26 @@ static int run_argv(int *argcp, const char ***argv)
 	return done_alias;
 }
 
+/*
+ * Many parts of Git have subprograms communicate via pipe, expect the
+ * upstream of the pipe to die with SIGPIPE and the downstream process
+ * even knows to check and handle EPIPE correctly.  Some third-party
+ * programs that ignore or block SIGPIPE for their own reason forget
+ * to restore SIGPIPE handling to the default before spawning Git and
+ * break this carefully orchestrated machinery.
+ *
+ * Restore the way SIGPIPE is handled to default, which is what we
+ * expect.
+ */
+static void restore_sigpipe_to_default(void)
+{
+	sigset_t unblock;
+
+	sigemptyset(&unblock);
+	sigaddset(&unblock, SIGPIPE);
+	sigprocmask(SIG_UNBLOCK, &unblock, NULL);
+	signal(SIGPIPE, SIG_DFL);
+}
 
 int main(int argc, char **av)
 {
@@ -612,6 +632,8 @@ int main(int argc, char **av)
 	 */
 	sanitize_stdfds();
 
+	restore_sigpipe_to_default();
+
 	git_setup_gettext();
 
 	trace_command_performance(argv);
diff --git a/t/t0005-signals.sh b/t/t0005-signals.sh
index 981437b..638a355 100755
--- a/t/t0005-signals.sh
+++ b/t/t0005-signals.sh
@@ -27,4 +27,26 @@ test_expect_success !MINGW 'signals are propagated using shell convention' '
 	test_expect_code 143 git sigterm
 '
 
+large_git () {
+	for i in $(test_seq 1 100)
+	do
+		git diff --cached --binary || return
+	done
+}
+
+test_expect_success 'create blob' '
+	test-genrandom foo 16384 >file &&
+	git add file
+'
+
+test_expect_success 'a constipated git dies with SIGPIPE' '
+	OUT=$( ((large_git; echo $? 1>&3) | :) 3>&1 )
+	test "$OUT" -eq 141
+'
+
+test_expect_success 'a constipated git dies with SIGPIPE even if parent ignores it' '
+	OUT=$( ((trap "" PIPE; large_git; echo $? 1>&3) | :) 3>&1 )
+	test "$OUT" -eq 141
+'
+
 test_done
-- 
2.0.1

Re: [PATCH v2] unblock and unignore SIGPIPE

[Junio C Hamano]

Thanks!

Re: [PATCH v2] unblock and unignore SIGPIPE

[James Nylen]

Today I learned that software can be constipated.

On Thu, Sep 18, 2014 at 12:35 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Thanks!
> --
> To unsubscribe from this list: send the line "unsubscribe git" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2] unblock and unignore SIGPIPE

[Johannes Sixt]

Am 18.09.2014 um 18:57 schrieb Patrick Reynolds:
> Blocked and ignored signals -- but not caught signals -- are inherited
> across exec.  Some callers with sloppy signal-handling behavior can call
> git with SIGPIPE blocked or ignored, even non-deterministically.  When
> SIGPIPE is blocked or ignored, several git commands can run indefinitely,
> ignoring EPIPE returns from write() calls, even when the process that
> called them has gone away.  Our specific case involved a pipe of git
> diff-tree output to a script that reads a limited amount of diff data.
> 
> In an ideal world, git would never be called with SIGPIPE blocked or
> ignored.  But in the real world, several real potential callers, including
> Perl, Apache, and Unicorn, sometimes spawn subprocesses with SIGPIPE
> ignored.  It is easier and more productive to harden git against this
> mistake than to clean it up in every potential parent process.
> 
> Signed-off-by: Patrick Reynolds <patrick.reynolds@github.com>
> Signed-off-by: Junio C Hamano <gitster@pobox.com>
> ---
> 1. Merged Junio's work from pu: moved restore_sigpipe_to_default into
> git.c and restyled the new tests.
> 2. Moved the new tests into t0005.  This meant switching back to `git
> diff` as our data generator, as the sample repo in t0005 doesn't have any
> files for `git ls-files` to output.
> 3. Squashed.
> 
>  git.c              | 22 ++++++++++++++++++++++
>  t/t0005-signals.sh | 22 ++++++++++++++++++++++
>  2 files changed, 44 insertions(+)
> 
> diff --git a/git.c b/git.c
> index 210f1ae..0f03d56 100644
> --- a/git.c
> +++ b/git.c
> @@ -593,6 +593,26 @@ static int run_argv(int *argcp, const char ***argv)
>  	return done_alias;
>  }
>  
> +/*
> + * Many parts of Git have subprograms communicate via pipe, expect the
> + * upstream of the pipe to die with SIGPIPE and the downstream process
> + * even knows to check and handle EPIPE correctly.  Some third-party
> + * programs that ignore or block SIGPIPE for their own reason forget
> + * to restore SIGPIPE handling to the default before spawning Git and
> + * break this carefully orchestrated machinery.
> + *
> + * Restore the way SIGPIPE is handled to default, which is what we
> + * expect.
> + */
> +static void restore_sigpipe_to_default(void)
> +{
> +	sigset_t unblock;
> +
> +	sigemptyset(&unblock);
> +	sigaddset(&unblock, SIGPIPE);
> +	sigprocmask(SIG_UNBLOCK, &unblock, NULL);
> +	signal(SIGPIPE, SIG_DFL);
> +}

This does not build on MinGW due to missing sigaddset() and
sigprocmask(). I've a patch that adds dummies for them (but I ran out of
time to complete it for submission). But then the test cases ...

> +test_expect_success 'a constipated git dies with SIGPIPE' '
> +	OUT=$( ((large_git; echo $? 1>&3) | :) 3>&1 )
> +	test "$OUT" -eq 141
> +'
> +
> +test_expect_success 'a constipated git dies with SIGPIPE even if parent ignores it' '
> +	OUT=$( ((trap "" PIPE; large_git; echo $? 1>&3) | :) 3>&1 )
> +	test "$OUT" -eq 141
> +'

... fail always because we neither get SIGPIPE (we don't have it on
Windows) nor do we see a write error (e.g. EPIPE) when writing to the
pipe. Should I protect these tests with !MINGW or would it be an option
to drop these tests alltogether?

-- Hannes

Re: [PATCH v2] unblock and unignore SIGPIPE

[Junio C Hamano]

Johannes Sixt <j6t@kdbg.org> writes:

>> +static void restore_sigpipe_to_default(void)
>> +{
>> +	sigset_t unblock;
>> +
>> +	sigemptyset(&unblock);
>> +	sigaddset(&unblock, SIGPIPE);
>> +	sigprocmask(SIG_UNBLOCK, &unblock, NULL);
>> +	signal(SIGPIPE, SIG_DFL);
>> +}
>
> This does not build on MinGW due to missing sigaddset() and
> sigprocmask(). I've a patch that adds dummies for them (but I ran out of
> time to complete it for submission). But then the test cases ...
>
>> +test_expect_success 'a constipated git dies with SIGPIPE' '
>> +	OUT=$( ((large_git; echo $? 1>&3) | :) 3>&1 )
>> +	test "$OUT" -eq 141
>> +'
>> +
>> +test_expect_success 'a constipated git dies with SIGPIPE even if parent ignores it' '
>> +	OUT=$( ((trap "" PIPE; large_git; echo $? 1>&3) | :) 3>&1 )
>> +	test "$OUT" -eq 141
>> +'
>
> ... fail always because we neither get SIGPIPE (we don't have it on
> Windows) nor do we see a write error (e.g. EPIPE) when writing to the
> pipe. Should I protect these tests with !MINGW or would it be an option
> to drop these tests alltogether?

Let's do !MINGW for now, unless somebody can think of a reason why
this change and tests are a bad idea (e.g. "we are not in the
business of preventing users from shooting themselves; have the
users bug those who wrote the software that spawns us with SIGPIPE
ignored", to which I am sympathetic to some degree but not very much
because I am also a practical person).

Thanks.

[PATCH] mingw.h: add dummy functions for sigset_t operations

[Johannes Sixt]

Windows does not have POSIX-like signals, and so we ignore all
operations on the non-existent signal mask machinery.

Do not turn sigemptyset into a function, but leave it a macro that
erases the code in the argument because it is used to set sa_mask
of a struct sigaction, but our dummy in mingw.h does not have that
member.

Signed-off-by: Johannes Sixt <j6t@kdbg.org>
---
 compat/mingw.h     | 7 ++++++-
 t/t0005-signals.sh | 4 ++--
 2 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/compat/mingw.h b/compat/mingw.h
index 0b5f2fe..0e42653 100644
--- a/compat/mingw.h
+++ b/compat/mingw.h
@@ -69,7 +69,6 @@ struct sigaction {
 	sig_handler_t sa_handler;
 	unsigned sa_flags;
 };
-#define sigemptyset(x) (void)0
 #define SA_RESTART 0
 
 struct itimerval {
@@ -116,6 +115,12 @@ static inline int fcntl(int fd, int cmd, ...)
 }
 /* bash cannot reliably detect negative return codes as failure */
 #define exit(code) exit((code) & 0xff)
+#define sigemptyset(x) (void)0
+static inline int sigaddset(sigset_t *set, int signum)
+{ return 0; }
+#define SIG_UNBLOCK 0
+static inline int sigprocmask(int how, const sigset_t *set, sigset_t *oldset)
+{ return 0; }
 
 /*
  * simple adaptors
diff --git a/t/t0005-signals.sh b/t/t0005-signals.sh
index 638a355..aeea50c 100755
--- a/t/t0005-signals.sh
+++ b/t/t0005-signals.sh
@@ -39,12 +39,12 @@ test_expect_success 'create blob' '
 	git add file
 '
 
-test_expect_success 'a constipated git dies with SIGPIPE' '
+test_expect_success !MINGW 'a constipated git dies with SIGPIPE' '
 	OUT=$( ((large_git; echo $? 1>&3) | :) 3>&1 )
 	test "$OUT" -eq 141
 '
 
-test_expect_success 'a constipated git dies with SIGPIPE even if parent ignores it' '
+test_expect_success !MINGW 'a constipated git dies with SIGPIPE even if parent ignores it' '
 	OUT=$( ((trap "" PIPE; large_git; echo $? 1>&3) | :) 3>&1 )
 	test "$OUT" -eq 141
 '
-- 
2.0.0.12.gbcf935e