Re: ChunkFS - measuring cross-chunk references

[Andreas Dilger <adilger@clusterfs.com>]

On Apr 23, 2007  15:04 +0530, Kalpak Shah wrote:
> On Mon, 2007-04-23 at 12:49 +0530, Karuna sagar K wrote:
> > The tool estimates the cross-chunk references from an extt2/3 file
> > system. It considers a block group as one chunk and calcuates how many
> > block groups does a file span across. So, the block group size gives
> > the estimate of chunk size.
> > 
> > The file systems were aged for about 3-4 months on a developers laptop.
> 
> With a blocksize of 4KB, a block group would be 128 MB. In the original
> Chunkfs paper, Valh had mentioned 1GB chunks and I believe it will be
> possible to use 2GB, 4GB or 8GB chunks in the future. As the chunk size
> increases the number of cross-chunk references will reduce and hence it
> might be a good idea to present these statistics considering different
> chunk sizes starting from 512MB upto 2GB.

Also, given that cross-chunk references will be more expensive to fix, I
can imagine the allocation policy for chunkfs will try to avoid this if
possible, further reducing the number of cross-chunk inodes.  I guess it
should be more clear whether the cross-chunk references are due to inode
block references, or because of e.g. directories referencing inodes in
another chunk.

Also, is it considered a cross-chunk reference if a directory entry is
referencing an inode in another group?  Should there be a continuation
inode in the local group, or is the directory entry itself enough?

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.