Re: topics for the file system mini-summit

[Ric Wheeler <ric@emc.com>]

Andreas Dilger wrote:

>On May 26, 2006  20:49 -0400, Ric Wheeler wrote:
>  
>
>>Andreas Dilger wrote:
>>    
>>
>>>In a way what you describe is Lustre - it aggregates multiple "smaller"
>>>filesystems into a single large filesystem from the application POV
>>>(though in many cases "smaller" filesystems are 2TB).  It runs e2fsck
>>>in parallel if needed, has smart object allocation (clients do delayed
>>>allocation, can load balance across storage targets, etc), can run with
>>>down storage targets.
>>>      
>>>
>>The approach that lustre takes here is great -  distributed systems 
>>typically  take into account subcomponent failures as a fact of life & 
>>do this better than many single system designs...
>>
>>The challenge is still there on the "smaller" file systems that make up 
>>Lustre - you can spend a lot of time waiting for just one fsck to finish ;-)
>>    
>>
>
>CFS is actually quite interested in improving the health and reliability
>of the component filesystems also.  That is the reason for our interest
>in the U. Wisconsin IRON filesystem work, which we are (slowly) working
>to include into ext3.
>  
>
We actually were the sponsors of the Wisconsin work, so I am glad to 
hear that it has a real impact.  I think that the Iron FS ideas will 
help, but they still don't eliminate the issues of scalability with fsck 
(and some of the scalability issues I see where performance dips with 
high object count file systems).

>This will also be our focus for upcoming filesystem work.  It is
>relatively easy to make filesystems with 64-bit structures, but the
>ability to run such large filesystems in the face of corruption
>environments is the real challenge.  It isn't practical to need a
>17-year e2fsck time, extrapolating 2TB e2fsck times to 2^48 block
>filesystems.  A lot of the features in ZFS make sense in this regard.
>
>Cheers, Andreas
>--
>
>  
>
Absolutely agree - I wonder if there is some value in trying to go back 
to profiling fsck if someone has not already done that.  It won't get 
rid of the design limitations, but we might be able to make some 
significant improvements...

ric