Re: Recursive directory accounting for size, ctime, etc.

Pavel Machek
2008-08-07T20:10:38Z

On Tue 2008-07-15 11:28:22, Sage Weil wrote:
> All-
>
> Ceph is a new distributed file system for Linux designed for scalability
> (terabytes to exabytes, tens to thousands of storage nodes), reliability,
> and performance. The latest release (v0.3), aside from xattr support and
> the usual slew of bugfixes, includes a unique (?) recursive accounting
> infrastructure that allows statistics about all metadata nested beneath a
> point in the directory hierarchy to be efficiently propagated up the tree.
> Currently this includes a file and directory count, total bytes (summation
> over file sizes), and most recent inode ctime. For example, for a
> directory like /home, Ceph can efficiently report the total number of
> files, directories, and bytes contained by that entire subtree of the
> directory hierarchy.
>
> The file size summation is the most interesting, as it effectively gives
> you directory-based quota space accounting with fine granularity. In many
> deployments, the quota _accounting_ is more important than actual
> enforcement. Anybody who has had to figure out what has filled/is filling
> up a large volume will appreciate how cumbersome and inefficient 'du' can
> be for that purpose--especially when you're in a hurry.
>
> There are currently two ways to access the recursive stats via a standard
> shell. The first simply sets the directory st_size value to the
> _recursive_ bytes ('rbytes') value (when the client is mounted with -o
> rbytes). For example (watch the directory sizes),
...

> Naturally, there are a few caveats:
>
> - There is some built-in delay before statistics fully propagate up
> toward the root of the hierarchy. Changes are propagated
> opportunistically when lock/lease state allows, with an upper bound of (by
> default) ~30 seconds for each level of directory nesting.

Having instant rctime would be very nice -- for stuff like locate and
speeding up kde startup.

> I'm extremely interested in what people think of overloading the file
> system interface in this way. Handy? Crufty? Dangerous? Does anybody

Too ugly to live.

What about new rstat() syscall?

--
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html