Re: Beagle and logging inotify events

[Andreas Dilger <adilger@sun.com>]

On Nov 14, 2007  11:32 -0500, Chuck Lever wrote:
> I disagree: we don't need a "bullet-proof" log.  We can get a significant 
> performance improvement even with a permanent dnotify log implemented in 
> user-space.  We already have well-defined fallback behavior if such a log 
> is missing or incomplete.
>
> The problem with a permanent inotify log is that it can become unmanageably 
> enormous, and a performance problem to boot.  Recording at that level of 
> detail makes it more likely that the logger won't be able to keep up with 
> file system activity.
>
> A lightweight solution gets us most of the way there, is simple to 
> implement, and doesn't introduce many new issues.  As long as it can tell 
> us precisely where the holes are, it shouldn't be a problem.

Jan Kara is working on a patch for ext4 which would store a recursive
timestamp for each directory that gives the latest time that a file in
that directory was modified.  ZFS has a similar mechanism by virtue of
doing full-tree updates during COW of all the metadata blocks and storing
the most recent transaction number in each block.  I suspect btrfs could
do the same thing easily.

That would allow recursive-descent filesystem traversal to be much more
efficient because whole chunks of the filesystem tree can be ignored during
scans.

Cheers, Andreas
--
Andreas Dilger
Sr. Software Engineer, Lustre Group
Sun Microsystems of Canada, Inc.