Re: topics for the file system mini-summit

[Valerie Henson <val_henson@linux.intel.com>]

On Thu, May 25, 2006 at 02:44:50PM -0700, Ric Wheeler wrote:
> 
>    (1) repair/fsck time can take hours or even days depending on the
> health of the file system and its underlying disk as well as the number
> of files.  This does not work well for large servers and is a disaster
> for "appliances" that need to run these commands buried deep in some
> data center without a person watching...
>    (2) most file system performance testing is done on "pristine" file
> systems with very few files.  Performance over time, especially with
> very high file counts, suffers very noticeable performance degradation
> with very large file systems.
>     (3) very poor fault containment for these very large devices - it
> would be great to be able to ride through a failure of a segment of the
> underlying storage without taking down the whole file system.
> 
> The obvious alternative to this is to break up these big disks into
> multiple small file systems, but there again we hit several issues.

1 and 3 are some of my main concerns, and what I want to focus a lot
of the workshop discussion on.  I view the question as: How do we keep
file system management simple while splitting the underlying storage
into isolated failure domains that can be repaired individually
online? (Say that three times fast.) Just splitting up into multiple
file systems only solves the second problem, and only if you have
forced umount, as you noted.

The approach we took in ZFS was to separate namespace management and
allocation management.  File systems aren't a fixed size, they take up
as much space as they need from a shared underlying pool.  You can
think of a file system in ZFS as a movable directory with management
bits attached.  I don't think this is the direction we should go, but
it's an example of separating your namespace management from a lot of
other stuff it doesn't really need to be attached to.

I don't think a block group is a good enough fault isolation domain -
think hard links.  What I think we need is normal file system
structures when you are referencing stuff inside your fault isolation
domain, and something more complicated if you have to reference stuff
outside.  One of Arjan's ideas involves something we're calling
continuation inodes - if the file's data is stored in multiple
domains, it has a separate continuation inode in each domain, and each
continuation inode has all the information necessary to run a full
fsck on the data inside that domain.  Similarly, if a directory has a
hard link to a file outside its domain, we'll have to allocate a
continuation inode and dir entry block in the domain containing the
file.  The idea is that you can run fsck on a domain without having to
go look outside that domain.  You may have to clean up a few things in
other domains, but they are easy to find and don't require an fsck in
other domains.

-VAL