Re: topics for the file system mini-summit

[Ric Wheeler <ric@emc.com>]

Matthew Wilcox wrote:

>On Thu, May 25, 2006 at 02:44:50PM -0700, Ric Wheeler wrote:
>  
>
>>The obvious alternative to this is to break up these big disks into
>>multiple small file systems, but there again we hit several issues.
>>
>>As an example, in one of the boxes that I work with we have 4 drives,
>>each 500GBs, with limited memory and CPU resources. To address the
>>issues above, we break each drive into 100GB chunks which gives us 20
>>(reiserfs) file systems per box.  The set of new problems that arise
>>from this include:
>>
>>   (1) no forced unmount - one file system goes down, you have to
>>reboot the box to recover.
>>   (2) worst case memory consumption for the journal scales linearly
>>with the number of file systems (32MB/per file system).
>>   (3) we take away the ability of the file system to do intelligent
>>head movement on the drives (i.e., I end up begging the application team
>>to please only use one file system per drive at a time for ingest ;-)).
>>The same goes for allocation - we basically have to push this up to the
>>application to use the capacity in an even way.
>>   (4) pain of administration of multiple file systems.
>>
>>I know that other file systems deal with scale better, but the question
>>is really how to move the mass of linux users onto these large and
>>increasingly common storage devices in a way that handles these challenges.
>>    
>>
>
>How do you handle the inode number space?  Do you partition it across
>the sub-filesystems, or do you prohibit hardlinks between the sub-fses?
>  
>
I think that the namespace needs to present a normal file system set of 
operations - support for hardlinks, no magic directories, etc. so that 
applications don't need to load balance (or even be aware) of the 
sub-units that provide storage.  If we removed that requirement, we 
would be back to today's collection of various file systems mounted on a 
single host.

I know that lustre aggregates full file systems, but you could build a 
file system on top of a collection of disk partitions/LUN's and then 
your inode would could be extended to encode the partition number and 
the internal mapping. You could even harden the block groups to the 
point that fsck could heal one group while the file system was (mostly?) 
online backed up by the rest of the block groups...

ric