Re: ChunkFS - measuring cross-chunk references

[Theodore Tso <tytso@mit.edu>]

On Mon, Apr 23, 2007 at 02:53:33PM -0600, Andreas Dilger wrote:
> > With a blocksize of 4KB, a block group would be 128 MB. In the original
> > Chunkfs paper, Valh had mentioned 1GB chunks and I believe it will be
> > possible to use 2GB, 4GB or 8GB chunks in the future. As the chunk size
> > increases the number of cross-chunk references will reduce and hence it
> > might be a good idea to present these statistics considering different
> > chunk sizes starting from 512MB upto 2GB.
> 
> Also, given that cross-chunk references will be more expensive to fix, I
> can imagine the allocation policy for chunkfs will try to avoid this if
> possible, further reducing the number of cross-chunk inodes.  I guess it
> should be more clear whether the cross-chunk references are due to inode
> block references, or because of e.g. directories referencing inodes in
> another chunk.

It would also be good to distinguish between directories referencing
files in another chunk, and directories referencing subdirectories in
another chunk (which would be simpler to handle, given the topological
restrictions on directories, as compared to files and hard links).

There may also be special things we will need to do to handle
scenarios such as BackupPC, where if it looks like a directory
contains a huge number of hard links to a particular chunk, we'll need
to make sure that directory is either created in the right chunk
(possibly with hints from the application) or migrated to the right
chunk (but this might cause the inode number of the directory to
change --- maybe we allow this as long as the directory has never been
stat'ed, so that the inode number has never been observed).

The other thing which we should consider is that chunkfs really
requires a 64-bit inode number space, which means either we only allow
it on 64-bit systems, or we need to consider a migration so that even
on 32-bit platforms, stat() functions like stat64(), insofar that it
uses a stat structure which returns a 64-bit ino_t.

						- Ted