[PATCH] tests: allow non-inotify tailf to keep up

[PATCH] tests: allow non-inotify tailf to keep up

[Andreas Henriksson]

When tailf is not using inotify, it seems to sleep for 0.25s between
checks. Just giving a 0.1s time window between update and removal of the
input file thus gives little chance for tailf to succeed.
Similar between startup and append to file, make sure initial
content is read before appending additional data by bumping the time.

This should possibly be bumped much larger to make sure that
the tailf process actually gets a chance to run at all in the
given time window. Otherwise it might fail on really slow/overloaded
machines.
The drawback would then ofcourse be to increase the time it takes
to run the testsuite.

Signed-off-by: Andreas Henriksson <andreas@fatal.se>
---
 tests/ts/tailf/simple | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/ts/tailf/simple b/tests/ts/tailf/simple
index 955844d..d0a1c5c 100755
--- a/tests/ts/tailf/simple
+++ b/tests/ts/tailf/simple
@@ -27,9 +27,9 @@ echo {a..z} > $INPUT
 
 $TS_CMD_TAILF $INPUT > $TS_OUTPUT 2>&1 &
 
-sleep 0.1
+sleep 0.5
 echo {0..9} >> $INPUT
-sleep 0.1
+sleep 0.5
 
 rm -f $INPUT
 
-- 
2.0.1

Re: [PATCH] tests: allow non-inotify tailf to keep up

[Pádraig Brady]

On 07/26/2014 09:44 PM, Andreas Henriksson wrote:
> When tailf is not using inotify, it seems to sleep for 0.25s between
> checks. Just giving a 0.1s time window between update and removal of the
> input file thus gives little chance for tailf to succeed.
> Similar between startup and append to file, make sure initial
> content is read before appending additional data by bumping the time.
> 
> This should possibly be bumped much larger to make sure that
> the tailf process actually gets a chance to run at all in the
> given time window. Otherwise it might fail on really slow/overloaded
> machines.
> The drawback would then ofcourse be to increase the time it takes
> to run the testsuite.
> 
> Signed-off-by: Andreas Henriksson <andreas@fatal.se>
> ---
>  tests/ts/tailf/simple | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/tests/ts/tailf/simple b/tests/ts/tailf/simple
> index 955844d..d0a1c5c 100755
> --- a/tests/ts/tailf/simple
> +++ b/tests/ts/tailf/simple
> @@ -27,9 +27,9 @@ echo {a..z} > $INPUT
>  
>  $TS_CMD_TAILF $INPUT > $TS_OUTPUT 2>&1 &
>  
> -sleep 0.1
> +sleep 0.5
>  echo {0..9} >> $INPUT
> -sleep 0.1
> +sleep 0.5
>  
>  rm -f $INPUT

For such tests coreutils uses a helper function
to apply a truncated exponential backoff,
to run quickly in the common case, but also
delay longer if necessary. See retry_delay_() at:

http://git.sv.gnu.org/gitweb/?p=coreutils.git;a=blob;f=init.cfg;h=725ee121;hb=HEAD#l608

cheers,
Pádraig.

Re: [PATCH] tests: allow non-inotify tailf to keep up

[Karel Zak]

On Sun, Jul 27, 2014 at 01:36:01PM +0100, Pádraig Brady wrote:
> On 07/26/2014 09:44 PM, Andreas Henriksson wrote:
> > When tailf is not using inotify, it seems to sleep for 0.25s between
> > checks. Just giving a 0.1s time window between update and removal of the
> > input file thus gives little chance for tailf to succeed.
> > Similar between startup and append to file, make sure initial
> > content is read before appending additional data by bumping the time.
> > 
> > This should possibly be bumped much larger to make sure that
> > the tailf process actually gets a chance to run at all in the
> > given time window. Otherwise it might fail on really slow/overloaded
> > machines.
> > The drawback would then ofcourse be to increase the time it takes
> > to run the testsuite.
> > 
> > Signed-off-by: Andreas Henriksson <andreas@fatal.se>
> > ---
> >  tests/ts/tailf/simple | 4 ++--
> >  1 file changed, 2 insertions(+), 2 deletions(-)
> > 
> > diff --git a/tests/ts/tailf/simple b/tests/ts/tailf/simple
> > index 955844d..d0a1c5c 100755
> > --- a/tests/ts/tailf/simple
> > +++ b/tests/ts/tailf/simple
> > @@ -27,9 +27,9 @@ echo {a..z} > $INPUT
> >  
> >  $TS_CMD_TAILF $INPUT > $TS_OUTPUT 2>&1 &
> >  
> > -sleep 0.1
> > +sleep 0.5
> >  echo {0..9} >> $INPUT
> > -sleep 0.1
> > +sleep 0.5
> >  
> >  rm -f $INPUT
> 
> For such tests coreutils uses a helper function
> to apply a truncated exponential backoff,
> to run quickly in the common case, but also
> delay longer if necessary. See retry_delay_() at:
> 
> http://git.sv.gnu.org/gitweb/?p=coreutils.git;a=blob;f=init.cfg;h=725ee121;hb=HEAD#l608

 Andreas? (hint: send a new patch :-))

    Karel


-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: [PATCH] tests: allow non-inotify tailf to keep up

[Andreas Henriksson]

Hello!

On Mon, Jul 28, 2014 at 01:59:23PM +0200, Karel Zak wrote:
> On Sun, Jul 27, 2014 at 01:36:01PM +0100, Pádraig Brady wrote:
> > For such tests coreutils uses a helper function
> > to apply a truncated exponential backoff,
> > to run quickly in the common case, but also
> > delay longer if necessary. See retry_delay_() at:

Thanks for the pointer!

> > 
> > http://git.sv.gnu.org/gitweb/?p=coreutils.git;a=blob;f=init.cfg;h=725ee121;hb=HEAD#l608
> 
>  Andreas? (hint: send a new patch :-))

I fail to see how this is useful here. We either need a short delay (using
inotify) or a longer delay (not using inotify). All other steps seems
pointless to me, even harmful in non-inotify case!

Having a too short delay before appending data could make the testcase
succed even when tailf doesn't work properly (because it wakes up and
reads all data in first go).
And obviously having a too short delay before the file removal could
cause the test case to fail, but I consider false-positives worse
then false-negatives myself.

My hope was that 2*0.5s delay would be both low and high enough
to be good enough everywhere. Just wanted to warn about this maybe
not being 100% fail-proof.


If anyone else want to propose a solution/patch here, feel free!

Regards,
Andreas Henriksson

Re: [PATCH] tests: allow non-inotify tailf to keep up

[Karel Zak]

On Mon, Jul 28, 2014 at 02:21:15PM +0200, Andreas Henriksson wrote:
> Hello!
> 
> On Mon, Jul 28, 2014 at 01:59:23PM +0200, Karel Zak wrote:
> > On Sun, Jul 27, 2014 at 01:36:01PM +0100, Pádraig Brady wrote:
> > > For such tests coreutils uses a helper function
> > > to apply a truncated exponential backoff,
> > > to run quickly in the common case, but also
> > > delay longer if necessary. See retry_delay_() at:
> 
> Thanks for the pointer!
> 
> > > 
> > > http://git.sv.gnu.org/gitweb/?p=coreutils.git;a=blob;f=init.cfg;h=725ee121;hb=HEAD#l608
> > 
> >  Andreas? (hint: send a new patch :-))
> 
> I fail to see how this is useful here. We either need a short delay (using
> inotify) or a longer delay (not using inotify). All other steps seems
> pointless to me, even harmful in non-inotify case!

 OK, merged. Maybe someone will provide a better solution later.

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com