From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andreas Dilger Subject: Re: Beagle and logging inotify events Date: Wed, 14 Nov 2007 12:32:45 -0700 Message-ID: <20071114193245.GE3966@webber.adilger.int> References: <9e4733910711131604k1290d4e1s5ee9808cbb61c2b6@mail.gmail.com> <45578746-916A-4F59-9A92-E7CEEFBC09B0@oracle.com> <9e4733910711140544l3f311868n96d753ce0b70cee5@mail.gmail.com> <473B0908.1060304@oracle.com> <9e4733910711140701q36cc168awbe060ed5537c491e@mail.gmail.com> <473B2329.8000407@oracle.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Jon Smirl , linux-fsdevel@vger.kernel.org, Jan Kara To: Chuck Lever Return-path: Received: from mail.clusterfs.com ([74.0.229.162]:41622 "EHLO mail.clusterfs.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752697AbXKNTcu (ORCPT ); Wed, 14 Nov 2007 14:32:50 -0500 Content-Disposition: inline In-Reply-To: <473B2329.8000407@oracle.com> Sender: linux-fsdevel-owner@vger.kernel.org List-Id: linux-fsdevel.vger.kernel.org On Nov 14, 2007 11:32 -0500, Chuck Lever wrote: > I disagree: we don't need a "bullet-proof" log. We can get a significant > performance improvement even with a permanent dnotify log implemented in > user-space. We already have well-defined fallback behavior if such a log > is missing or incomplete. > > The problem with a permanent inotify log is that it can become unmanageably > enormous, and a performance problem to boot. Recording at that level of > detail makes it more likely that the logger won't be able to keep up with > file system activity. > > A lightweight solution gets us most of the way there, is simple to > implement, and doesn't introduce many new issues. As long as it can tell > us precisely where the holes are, it shouldn't be a problem. Jan Kara is working on a patch for ext4 which would store a recursive timestamp for each directory that gives the latest time that a file in that directory was modified. ZFS has a similar mechanism by virtue of doing full-tree updates during COW of all the metadata blocks and storing the most recent transaction number in each block. I suspect btrfs could do the same thing easily. That would allow recursive-descent filesystem traversal to be much more efficient because whole chunks of the filesystem tree can be ignored during scans. Cheers, Andreas -- Andreas Dilger Sr. Software Engineer, Lustre Group Sun Microsystems of Canada, Inc.