[PATCH] Ignore "yes: standard output: Broken pipe" errors

[PATCH] Ignore "yes: standard output: Broken pipe" errors

[Jan Kara]

Recently, some tests started failing because they had

yes: standard output: Broken pipe

in their output. Fix the problem by discarding errors from yes(1)
program.

Signed-off-by: Jan Kara <jack@suse.cz>
---
 common/rc         | 12 ++++++------
 tests/generic/081 |  2 +-
 tests/generic/108 |  2 +-
 3 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/common/rc b/common/rc
index bd989bb5f4dd..d3a0b5ea9023 100644
--- a/common/rc
+++ b/common/rc
@@ -736,7 +736,7 @@ _test_mkfs()
 	$MKFS_PROG -t $FSTYP -- -F $MKFS_OPTIONS $* $TEST_DEV
 	;;
     *)
-	yes | $MKFS_PROG -t $FSTYP -- $MKFS_OPTIONS $* $TEST_DEV
+	yes 2>/dev/null | $MKFS_PROG -t $FSTYP -- $MKFS_OPTIONS $* $TEST_DEV
 	;;
     esac
 }
@@ -765,7 +765,7 @@ _mkfs_dev()
 	;;
 
     *)
-	yes | $MKFS_PROG -t $FSTYP -- $MKFS_OPTIONS $* \
+	yes 2>/dev/null | $MKFS_PROG -t $FSTYP -- $MKFS_OPTIONS $* \
 		2>$tmp_dir.mkfserr 1>$tmp_dir.mkfsstd
 	;;
     esac
@@ -858,11 +858,11 @@ _scratch_mkfs()
 		mkfs_filter="cat"
 		;;
 	ocfs2)
-		mkfs_cmd="yes | $MKFS_PROG -t $FSTYP --"
+		mkfs_cmd="yes 2>/dev/null | $MKFS_PROG -t $FSTYP --"
 		mkfs_filter="grep -v -e ^mkfs\.ocfs2"
 		;;
 	*)
-		mkfs_cmd="yes | $MKFS_PROG -t $FSTYP --"
+		mkfs_cmd="yes 2>/dev/null | $MKFS_PROG -t $FSTYP --"
 		mkfs_filter="cat"
 		;;
 	esac
@@ -1061,7 +1061,7 @@ _scratch_mkfs_sized()
 	${MKFS_PROG}.$FSTYP -F $MKFS_OPTIONS -b $blocksize $SCRATCH_DEV $blocks
 	;;
     ocfs2)
-	yes | ${MKFS_PROG}.$FSTYP -F $MKFS_OPTIONS -b $blocksize $SCRATCH_DEV $blocks
+	yes 2>/dev/null | ${MKFS_PROG}.$FSTYP -F $MKFS_OPTIONS -b $blocksize $SCRATCH_DEV $blocks
 	;;
     udf)
 	$MKFS_UDF_PROG $MKFS_OPTIONS -b $blocksize $SCRATCH_DEV $blocks
@@ -1142,7 +1142,7 @@ _scratch_mkfs_blocksized()
 	${MKFS_PROG}.$FSTYP -F $MKFS_OPTIONS -b $blocksize $SCRATCH_DEV
 	;;
     ocfs2)
-	yes | ${MKFS_PROG}.$FSTYP -F $MKFS_OPTIONS -b $blocksize -C $blocksize $SCRATCH_DEV
+	yes 2>/dev/null | ${MKFS_PROG}.$FSTYP -F $MKFS_OPTIONS -b $blocksize -C $blocksize $SCRATCH_DEV
 	;;
     *)
 	_notrun "Filesystem $FSTYP not supported in _scratch_mkfs_blocksized"
diff --git a/tests/generic/081 b/tests/generic/081
index 426766d9bddb..56b41e2875e3 100755
--- a/tests/generic/081
+++ b/tests/generic/081
@@ -69,7 +69,7 @@ _scratch_mkfs_sized $((300 * 1024 * 1024)) >>$seqres.full 2>&1
 $LVM_PROG vgcreate -f $vgname $SCRATCH_DEV >>$seqres.full 2>&1
 # We use yes pipe instead of 'lvcreate --yes' because old version of lvm
 # (like 2.02.95 in RHEL6) don't support --yes option
-yes | $LVM_PROG lvcreate -L 256M -n $lvname $vgname >>$seqres.full 2>&1
+yes 2>/dev/null | $LVM_PROG lvcreate -L 256M -n $lvname $vgname >>$seqres.full 2>&1
 # wait for lvcreation to fully complete
 $UDEV_SETTLE_PROG >>$seqres.full 2>&1
 
diff --git a/tests/generic/108 b/tests/generic/108
index 1525dfb64a47..1de16c2cf3c9 100755
--- a/tests/generic/108
+++ b/tests/generic/108
@@ -74,7 +74,7 @@ $LVM_PROG pvcreate -f $SCSI_DEBUG_DEV $SCRATCH_DEV >>$seqres.full 2>&1
 $LVM_PROG vgcreate -f $vgname $SCSI_DEBUG_DEV $SCRATCH_DEV >>$seqres.full 2>&1
 # We use yes pipe instead of 'lvcreate --yes' because old version of lvm
 # (like 2.02.95 in RHEL6) don't support --yes option
-yes | $LVM_PROG lvcreate -i 2 -I 4m -l 100%FREE -n $lvname $vgname \
+yes 2>/dev/null | $LVM_PROG lvcreate -i 2 -I 4m -l 100%FREE -n $lvname $vgname \
 	>>$seqres.full 2>&1
 # wait for lv creation to fully complete
 $UDEV_SETTLE_PROG >>$seqres.full 2>&1
-- 
2.12.3

Re: [PATCH] Ignore "yes: standard output: Broken pipe" errors

[Eryu Guan]

On Tue, Jul 25, 2017 at 04:33:20PM +0200, Jan Kara wrote:
> Recently, some tests started failing because they had
> 
> yes: standard output: Broken pipe
> 
> in their output. Fix the problem by discarding errors from yes(1)
> program.
> 
> Signed-off-by: Jan Kara <jack@suse.cz>
> ---
>  common/rc         | 12 ++++++------
>  tests/generic/081 |  2 +-
>  tests/generic/108 |  2 +-
>  3 files changed, 8 insertions(+), 8 deletions(-)

I'm wondering which case failed for you and what's the diff output and
.full file? And what change caused the failure for you? Because I didn't
see any failure on my RHEL7 box nor Fedora rawhide box. I compiled
latest yes from coreutils git repo and didn't hit the error either.

So I suspect that it might be other commands in the pipeline that failed
and caused the broken pipe.

Thanks,
Eryu

Re: [PATCH] Ignore "yes: standard output: Broken pipe" errors

[Jan Kara]

On Wed 26-07-17 15:47:19, Eryu Guan wrote:
> On Tue, Jul 25, 2017 at 04:33:20PM +0200, Jan Kara wrote:
> > Recently, some tests started failing because they had
> > 
> > yes: standard output: Broken pipe
> > 
> > in their output. Fix the problem by discarding errors from yes(1)
> > program.
> > 
> > Signed-off-by: Jan Kara <jack@suse.cz>
> > ---
> >  common/rc         | 12 ++++++------
> >  tests/generic/081 |  2 +-
> >  tests/generic/108 |  2 +-
> >  3 files changed, 8 insertions(+), 8 deletions(-)
> 
> I'm wondering which case failed for you and what's the diff output and
> .full file? And what change caused the failure for you? Because I didn't
> see any failure on my RHEL7 box nor Fedora rawhide box. I compiled
> latest yes from coreutils git repo and didn't hit the error either.
> 
> So I suspect that it might be other commands in the pipeline that failed
> and caused the broken pipe.

I've started seeing failures after I've updated the testing VM to openSUSE
Leap 42.2. And yes(1) failing with this error is "normal" if SIGPIPE is
ignored - just try ( trap "" PIPE; yes | exit; ) and observe the error.
So it is just that in the new VM something makes yes(1) ignore SIGPIPE and
honestly I'm not sure what that is even though I've looked for a while. But
regardless of that, ignoring stderr from yes(1) seems like a safe thing to
do.

Regarding your question about failed tests: Tests generic/081, generic/347,
generic/361, shared/298, xfs/074 failed due to this and they all just had
this additional line in the output.

-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR

Re: [PATCH] Ignore "yes: standard output: Broken pipe" errors

[Eryu Guan]

On Wed, Jul 26, 2017 at 10:22:13AM +0200, Jan Kara wrote:
> On Wed 26-07-17 15:47:19, Eryu Guan wrote:
> > On Tue, Jul 25, 2017 at 04:33:20PM +0200, Jan Kara wrote:
> > > Recently, some tests started failing because they had
> > > 
> > > yes: standard output: Broken pipe
> > > 
> > > in their output. Fix the problem by discarding errors from yes(1)
> > > program.
> > > 
> > > Signed-off-by: Jan Kara <jack@suse.cz>
> > > ---
> > >  common/rc         | 12 ++++++------
> > >  tests/generic/081 |  2 +-
> > >  tests/generic/108 |  2 +-
> > >  3 files changed, 8 insertions(+), 8 deletions(-)
> > 
> > I'm wondering which case failed for you and what's the diff output and
> > .full file? And what change caused the failure for you? Because I didn't
> > see any failure on my RHEL7 box nor Fedora rawhide box. I compiled
> > latest yes from coreutils git repo and didn't hit the error either.
> > 
> > So I suspect that it might be other commands in the pipeline that failed
> > and caused the broken pipe.
> 
> I've started seeing failures after I've updated the testing VM to openSUSE
> Leap 42.2. And yes(1) failing with this error is "normal" if SIGPIPE is
> ignored - just try ( trap "" PIPE; yes | exit; ) and observe the error.
> So it is just that in the new VM something makes yes(1) ignore SIGPIPE and
> honestly I'm not sure what that is even though I've looked for a while. But
> regardless of that, ignoring stderr from yes(1) seems like a safe thing to
> do.

Ahh, I see the problem. Actually Eric Biggers has fixed a very similar
problem in commit 9bcb266cd778 ("generic/397: be compatible with ignored
SIGPIPE"), but at that time he only saw generic/397 failure, so he
decided to fix that test only.

How about print out warnings and exit if check finds SIGPIPE is ignored?
Because seems we can't un-ignore it in a sub-shell, if it's already
ignored on start. e.g.

if trap -p | grep -q "'' SIGPIPE"; then
	echo "SIGPIPE is ignored by current shell, this will cause unexpected"
	echo "broken pipe error in tests. Please unignore SIGPIPE and rerun."
	echo "For bash, just run 'trap - SIGPIPE'"
	exit
fi

Because I think ignoring SIGPIPE indicates bugs/issues somewhere, as the
python bug mentioned in comment 9bcb266cd778.

> 
> Regarding your question about failed tests: Tests generic/081, generic/347,
> generic/361, shared/298, xfs/074 failed due to this and they all just had
> this additional line in the output.

Thanks,
Eryu

> 
> -- 
> Jan Kara <jack@suse.com>
> SUSE Labs, CR
> --
> To unsubscribe from this list: send the line "unsubscribe fstests" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Ignore "yes: standard output: Broken pipe" errors

[Jan Kara]

On Wed 26-07-17 18:22:21, Eryu Guan wrote:
> On Wed, Jul 26, 2017 at 10:22:13AM +0200, Jan Kara wrote:
> > On Wed 26-07-17 15:47:19, Eryu Guan wrote:
> > > On Tue, Jul 25, 2017 at 04:33:20PM +0200, Jan Kara wrote:
> > > > Recently, some tests started failing because they had
> > > > 
> > > > yes: standard output: Broken pipe
> > > > 
> > > > in their output. Fix the problem by discarding errors from yes(1)
> > > > program.
> > > > 
> > > > Signed-off-by: Jan Kara <jack@suse.cz>
> > > > ---
> > > >  common/rc         | 12 ++++++------
> > > >  tests/generic/081 |  2 +-
> > > >  tests/generic/108 |  2 +-
> > > >  3 files changed, 8 insertions(+), 8 deletions(-)
> > > 
> > > I'm wondering which case failed for you and what's the diff output and
> > > .full file? And what change caused the failure for you? Because I didn't
> > > see any failure on my RHEL7 box nor Fedora rawhide box. I compiled
> > > latest yes from coreutils git repo and didn't hit the error either.
> > > 
> > > So I suspect that it might be other commands in the pipeline that failed
> > > and caused the broken pipe.
> > 
> > I've started seeing failures after I've updated the testing VM to openSUSE
> > Leap 42.2. And yes(1) failing with this error is "normal" if SIGPIPE is
> > ignored - just try ( trap "" PIPE; yes | exit; ) and observe the error.
> > So it is just that in the new VM something makes yes(1) ignore SIGPIPE and
> > honestly I'm not sure what that is even though I've looked for a while. But
> > regardless of that, ignoring stderr from yes(1) seems like a safe thing to
> > do.
> 
> Ahh, I see the problem. Actually Eric Biggers has fixed a very similar
> problem in commit 9bcb266cd778 ("generic/397: be compatible with ignored
> SIGPIPE"), but at that time he only saw generic/397 failure, so he
> decided to fix that test only.
> 
> How about print out warnings and exit if check finds SIGPIPE is ignored?
> Because seems we can't un-ignore it in a sub-shell, if it's already
> ignored on start. e.g.
> 
> if trap -p | grep -q "'' SIGPIPE"; then
> 	echo "SIGPIPE is ignored by current shell, this will cause unexpected"
> 	echo "broken pipe error in tests. Please unignore SIGPIPE and rerun."
> 	echo "For bash, just run 'trap - SIGPIPE'"
> 	exit
> fi
> 
> Because I think ignoring SIGPIPE indicates bugs/issues somewhere, as the
> python bug mentioned in comment 9bcb266cd778.

OK, fair enough.

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR