From mboxrd@z Thu Jan 1 00:00:00 1970 From: Ric Wheeler Subject: Re: topics for the file system mini-summit Date: Fri, 26 May 2006 20:49:58 -0400 Message-ID: <4477A236.3040208@emc.com> References: <44762552.8000906@emc.com> <20060526164856.GQ5964@schatzie.adilger.int> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: linux-fsdevel@vger.kernel.org Return-path: Received: from mexforward.lss.emc.com ([168.159.213.200]:47657 "EHLO mexforward.lss.emc.com") by vger.kernel.org with ESMTP id S932782AbWE0A6C (ORCPT ); Fri, 26 May 2006 20:58:02 -0400 To: Andreas Dilger In-Reply-To: <20060526164856.GQ5964@schatzie.adilger.int> Sender: linux-fsdevel-owner@vger.kernel.org List-Id: linux-fsdevel.vger.kernel.org Andreas Dilger wrote: >On May 25, 2006 14:44 -0700, Ric Wheeler wrote: > > >>With both ext3 and with reiserfs, running a single large file system >>translates into several practical limitations before we even hit the >>existing size limitations: >> >> .... >> >>I know that other file systems deal with scale better, but the question >>is really how to move the mass of linux users onto these large and >>increasingly common storage devices in a way that handles these challenges. >> >> > >In a way what you describe is Lustre - it aggregates multiple "smaller" >filesystems into a single large filesystem from the application POV >(though in many cases "smaller" filesystems are 2TB). It runs e2fsck >in parallel if needed, has smart object allocation (clients do delayed >allocation, can load balance across storage targets, etc), can run with >down storage targets. > >Cheers, Andreas >-- >Andreas Dilger >Principal Software Engineer >Cluster File Systems, Inc. > > > > The approach that lustre takes here is great - distributed systems typically take into account subcomponent failures as a fact of life & do this better than many single system designs... The challenge is still there on the "smaller" file systems that make up Lustre - you can spend a lot of time waiting for just one fsck to finish ;-) ric