Re: Recursive directory accounting for size, ctime, etc.

[Andreas Dilger <adilger@sun.com>]

On Jul 15, 2008  11:28 -0700, Sage Weil wrote:
> unique (?) recursive accounting 
> infrastructure that allows statistics about all metadata nested beneath a 
> point in the directory hierarchy to be efficiently propagated up the tree.  
> Currently this includes a file and directory count, total bytes (summation 
> over file sizes), and most recent inode ctime.

Interesting...

> Note that st_blocks is _not_ recursively defined, so 'du' still behaves as 
> expected.  If mounted with -o norbytes instead, the directory st_size is 
> the number of entries in the directory.

Is it possible to extract an environment variable from the process
in the kernel to decide what behaviour to have (e.g. like LS_COLORS)?

> The second interface takes advantage of the fact (?) that read() on a 
> directory is more or less undefined.  (Okay, that's not really true, but 
> it used to return encoded dirents or something similar, and more recently 
> returns -EISDIR.  As far as I know, no sane application expects meaningful 
> data from read() on a directory...)  So, assuming Ceph is mounted with -o 
> dirstat,

Hmm, what about just creating a virtual xattr that can be had with
getfattr user.dirstats?

>  - The 'rbytes' summation is over i_size, not blocks used.  That means 
> sparse files "appear" larger than the storage space they actually consume.

I'd think that in many cases it is more important to accumulate the
blocks count and not the size, since a single core file would throw
off the whole "hunt for the worst space consumer" approach.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.