tailf, really needed?

tailf, really needed?

[Ruediger Meier]

Hi,

As far as I understood tailf's advantage over "tail -f" is that it does 
not access the file when it does not grow. But nowadays 
coreutils "tail -f" also does not seem to access the file. So do we 
really need tailf?

The point is that I've noticed that our tailf fails to deal with 
filesystems where inotify is broken. For example it does not work for 
overlayfs. coreutils tail code looks quite complicated and seems to 
manage such cases. Is it worth to fix our tailf or better just remove 
it and use "tail -f"?

BTW coreutils tail is much more comfortable. It has many important 
options. For example watching log files without -F or --retry does not 
make sense to me (because of logrotate).

Last but not least, is anybody using tailf at all? Google does not find 
much about people who are using this.

cu,
Rudi

Re: tailf, really needed?

[Sami Kerola]

On 13 March 2015 at 09:00, Ruediger Meier <sweet_f_a@gmx.de> wrote:
> As far as I understood tailf's advantage over "tail -f" is that it does
> not access the file when it does not grow. But nowadays
> coreutils "tail -f" also does not seem to access the file. So do we
> really need tailf?
>
> The point is that I've noticed that our tailf fails to deal with
> filesystems where inotify is broken. For example it does not work for
> overlayfs. coreutils tail code looks quite complicated and seems to
> manage such cases. Is it worth to fix our tailf or better just remove
> it and use "tail -f"?
>
> BTW coreutils tail is much more comfortable. It has many important
> options. For example watching log files without -F or --retry does not
> make sense to me (because of logrotate).
>
> Last but not least, is anybody using tailf at all? Google does not find
> much about people who are using this.

I'm in favor slapping deprecated note to tailf. When I double tab my
bash the result is:

Display all 3227 possibilities? (y or n)

and I think that's too many. Reducing number of commands where
similar functionality is provided elsewhere should result to better
overall software quality, and a system that is earlier to learn to use.
Less really is more in this case.

-- 
Sami Kerola
http://www.iki.fi/kerolasa/

Re: tailf, really needed?

[Pádraig Brady]

On 13/03/15 09:00, Ruediger Meier wrote:
> Hi,
> 
> As far as I understood tailf's advantage over "tail -f" is that it does 
> not access the file when it does not grow. But nowadays 
> coreutils "tail -f" also does not seem to access the file. So do we 
> really need tailf?
> 
> The point is that I've noticed that our tailf fails to deal with 
> filesystems where inotify is broken. For example it does not work for 
> overlayfs. coreutils tail code looks quite complicated and seems to 
> manage such cases. Is it worth to fix our tailf or better just remove 
> it and use "tail -f"?
> 
> BTW coreutils tail is much more comfortable. It has many important 
> options. For example watching log files without -F or --retry does not 
> make sense to me (because of logrotate).
> 
> Last but not least, is anybody using tailf at all? Google does not find 
> much about people who are using this.

tailf is a strange one. If there was an issue with tail(1) accessing
the files, then why not fix it? In any case it seems without inotify
that tailf(1) does access the file?

  nanosleep({0, 250000000}, NULL)         = 0
  open("file", O_RDONLY)    = 3
  fstat(3, {st_mode=S_IFREG|0664, st_size=7048, ...}) = 0
  close(3)

while tail -f does not:

  nanosleep({0, 1000000000}, NULL)         = 0
  fstat(3, {st_mode=S_IFREG|0664, st_size=7048, ...}) = 0

debian codesearch turned up just a single user in a test:
http://sources.debian.net/src/atheist/0.20110402-2/test/conditions/process-running.test/?hl=5#L5

A deprecation warning makes sense to me.

cheers,
Pádraig.

Re: tailf, really needed?

[Ruediger Meier]

On Friday 13 March 2015, Pádraig Brady wrote:
> On 13/03/15 09:00, Ruediger Meier wrote:
> > Hi,
> >
> > As far as I understood tailf's advantage over "tail -f" is that it
> > does not access the file when it does not grow. But nowadays
> > coreutils "tail -f" also does not seem to access the file. So do we
> > really need tailf?
> >
> > The point is that I've noticed that our tailf fails to deal with
> > filesystems where inotify is broken. For example it does not work
> > for overlayfs. coreutils tail code looks quite complicated and
> > seems to manage such cases. Is it worth to fix our tailf or better
> > just remove it and use "tail -f"?
> >
> > BTW coreutils tail is much more comfortable. It has many important
> > options. For example watching log files without -F or --retry does
> > not make sense to me (because of logrotate).
> >
> > Last but not least, is anybody using tailf at all? Google does not
> > find much about people who are using this.
>
> tailf is a strange one. If there was an issue with tail(1) accessing
> the files, then why not fix it? In any case it seems without inotify
> that tailf(1) does access the file?
>
>   nanosleep({0, 250000000}, NULL)         = 0
>   open("file", O_RDONLY)    = 3
>   fstat(3, {st_mode=S_IFREG|0664, st_size=7048, ...}) = 0
>   close(3)
>
> while tail -f does not:
>
>   nanosleep({0, 1000000000}, NULL)         = 0
>   fstat(3, {st_mode=S_IFREG|0664, st_size=7048, ...}) = 0

Yes, I also checked this. The case undefined HAVE_INOTIFY_INIT is broken 
since inotify support was added in 2007, fc7aeb09.

BTW I think there could be a minor improvement for coreutils tail for 
the cases "file is empty" or "-n 0". Maybe we could skip opening the 
file one time at the beginning. So that "tail -n0 -f logfile" would 
really never access the file unless it grows.

cu,
Rudi

Re: tailf, really needed?

[Pádraig Brady]

On 13/03/15 13:02, Ruediger Meier wrote:
> On Friday 13 March 2015, Pádraig Brady wrote:
>> On 13/03/15 09:00, Ruediger Meier wrote:
>>> Hi,
>>>
>>> As far as I understood tailf's advantage over "tail -f" is that it
>>> does not access the file when it does not grow. But nowadays
>>> coreutils "tail -f" also does not seem to access the file. So do we
>>> really need tailf?
>>>
>>> The point is that I've noticed that our tailf fails to deal with
>>> filesystems where inotify is broken. For example it does not work
>>> for overlayfs. coreutils tail code looks quite complicated and
>>> seems to manage such cases. Is it worth to fix our tailf or better
>>> just remove it and use "tail -f"?
>>>
>>> BTW coreutils tail is much more comfortable. It has many important
>>> options. For example watching log files without -F or --retry does
>>> not make sense to me (because of logrotate).
>>>
>>> Last but not least, is anybody using tailf at all? Google does not
>>> find much about people who are using this.
>>
>> tailf is a strange one. If there was an issue with tail(1) accessing
>> the files, then why not fix it? In any case it seems without inotify
>> that tailf(1) does access the file?
>>
>>   nanosleep({0, 250000000}, NULL)         = 0
>>   open("file", O_RDONLY)    = 3
>>   fstat(3, {st_mode=S_IFREG|0664, st_size=7048, ...}) = 0
>>   close(3)
>>
>> while tail -f does not:
>>
>>   nanosleep({0, 1000000000}, NULL)         = 0
>>   fstat(3, {st_mode=S_IFREG|0664, st_size=7048, ...}) = 0
> 
> Yes, I also checked this. The case undefined HAVE_INOTIFY_INIT is broken 
> since inotify support was added in 2007, fc7aeb09.
> 
> BTW I think there could be a minor improvement for coreutils tail for 
> the cases "file is empty" or "-n 0". Maybe we could skip opening the 
> file one time at the beginning. So that "tail -n0 -f logfile" would 
> really never access the file unless it grows.

I see what you mean, though I'm not convinced it's worth complicating GNU tail
for that edge case. Not updating atime is mainly a performance concern.
I.e. you want to avoid the continuous open() of the file
(though relatime alleviates a lot of that concern these days).

Are there valid cases where you really don't want to update atime
for logical reasons?  Maybe for forensics you might have logs that
only tail would access, and you want accurate atime for, but it does
seem like an extreme edge case.

thanks,
Pádraig.

Re: tailf, really needed?

[Ruediger Meier]

On Friday 13 March 2015, Pádraig Brady wrote:
> On 13/03/15 13:02, Ruediger Meier wrote:
> > On Friday 13 March 2015, Pdraig Brady wrote:
> >> On 13/03/15 09:00, Ruediger Meier wrote:
> >>> Hi,
> >>>
> >>> As far as I understood tailf's advantage over "tail -f" is that
> >>> it does not access the file when it does not grow. But nowadays
> >>> coreutils "tail -f" also does not seem to access the file. So do
> >>> we really need tailf?
> >>>
> >>> The point is that I've noticed that our tailf fails to deal with
> >>> filesystems where inotify is broken. For example it does not work
> >>> for overlayfs. coreutils tail code looks quite complicated and
> >>> seems to manage such cases. Is it worth to fix our tailf or
> >>> better just remove it and use "tail -f"?
> >>>
> >>> BTW coreutils tail is much more comfortable. It has many
> >>> important options. For example watching log files without -F or
> >>> --retry does not make sense to me (because of logrotate).
> >>>
> >>> Last but not least, is anybody using tailf at all? Google does
> >>> not find much about people who are using this.
> >>
> >> tailf is a strange one. If there was an issue with tail(1)
> >> accessing the files, then why not fix it? In any case it seems
> >> without inotify that tailf(1) does access the file?
> >>
> >>   nanosleep({0, 250000000}, NULL)         =0
> >>   open("file", O_RDONLY)    =3
> >>   fstat(3, {st_mode=S_IFREG|0664, st_sizep48, ...}) =0
> >>   close(3)
> >>
> >> while tail -f does not:
> >>
> >>   nanosleep({0, 1000000000}, NULL)         =0
> >>   fstat(3, {st_mode=S_IFREG|0664, st_sizep48, ...}) =0
> >
> > Yes, I also checked this. The case undefined HAVE_INOTIFY_INIT is
> > broken since inotify support was added in 2007, fc7aeb09.
> >
> > BTW I think there could be a minor improvement for coreutils tail
> > for the cases "file is empty" or "-n 0". Maybe we could skip
> > opening the file one time at the beginning. So that "tail -n0 -f
> > logfile" would really never access the file unless it grows.
>
> I see what you mean, though I'm not convinced it's worth complicating
> GNU tail for that edge case. Not updating atime is mainly a
> performance concern. I.e. you want to avoid the continuous open() of
> the file
> (though relatime alleviates a lot of that concern these days).
>
> Are there valid cases where you really don't want to update atime
> for logical reasons?

Not personally for me. Actually most filesystems are mounted noatime or 
relatime per default anyways.

I just thought about the use case of the original author of tailf. tailf 
was made to not wake up HD during the "read loop". But why waking it up 
initially?

Another thought would be to generally look for such edge cases in any 
commands where we could avoid some initial open or read calls. Maybe 
this could safe some HD "start stop counts" even on a regular existing 
system. Would be just a minor contribution to save some energy.

cu,
Rudi

Re: tailf, really needed?

[Ángel González]

On 13/03/15 14:02, Ruediger Meier wrote:
> BTW I think there could be a minor improvement for coreutils tail for
> the cases "file is empty" or "-n 0". Maybe we could skip opening the
> file one time at the beginning. So that "tail -n0 -f logfile" would
> really never access the file unless it grows.
>
> cu,
> Rudi

I suspect that might break filesystems lying about the file size, such 
as proc or some compressed fs. There may also be issues with some 
network fs.

A tail <file> with <file> of len 0, would usually be empty. However, it 
would be a problem not to provide an output for tail /proc/$$/status



PS: tail -f already acts a bit funny in that case, as it tails the end 
of the file, then thinks it has been truncated and shows it in full.

Re: tailf, really needed?

[Peter Cordes]

On Fri, Mar 13, 2015 at 11:37:35AM +0000, Pádraig Brady wrote:
> On 13/03/15 09:00, Ruediger Meier wrote:
> > 
> > Last but not least, is anybody using tailf at all? Google does not find 
> > much about people who are using this.
>
> A deprecation warning makes sense to me.

 I haven't checked the semantics carefully, but could we just turn
tailf into a wrapper for tail -f, to avoid maintaining code that's
gotten old and crufty?  And if there aren't corner cases where someone
might still want the old tailf, until it's finally removed from
util-linux.


something like:

#!/bin/sh
if [ "$(tty <&2)" != "not a tty" ];then
    echo "warning: tailf is deprecated, and just a wrapper for tail -f" >&2
fi
exec tail -f "$@"

Or is this just asking for gratuitous breakage and confusion beyond
what any existing users will get when they upgrade to a util-linux
where it's finally dropped after a few years of deprecation?

-- 
#define X(x,y) x##y
Peter Cordes ;  e-mail: X(peter@cor , des.ca)

"The gods confound the man who first found out how to distinguish the hours!
 Confound him, too, who in this place set up a sundial, to cut and hack
 my day so wretchedly into small pieces!" -- Plautus, 200 BC