Re: creating a new 80 TB XFS

[Joe Landman <landman@scalableinformatics.com>]

On 02/26/2012 11:08 AM, Emmanuel Florac wrote:
> Le Sat, 25 Feb 2012 20:57:05 -0600 vous écriviez:
>
>> As others mentioned, an xfs_[check|repair] can take many hours or even
>> days on a multi-terabyte huge metadata filesystem.
>
> Just nitpicking, but I never had such a problem. I've run quite a lot
> of xfs_repair on 40TB+ filesystems, and it rarely was longer than 10 to
> 20 minutes. The important part is having enough RAM if the system hits
> swap it makes the check much slower).

We've found that adding the -m X and -P options seem to fix many of the 
longer running issues for large nearly full many TB file systems. 
Biggest one we've repaired has been 108TB and its taken a few hours, 
with ~80% utilization of the underlying file system.

I don't know if the sparse file bit we reported last year (with more 
data reported to the list in January this year) has had much attention 
(hard to reproduce I would imagine).  But apart from this, repair seems 
to work reasonably quickly.  I've not seen an instance after using the 
-m X -P options, or "days" for repair, even on heavily fragmented file 
systems.  Possibly Peter has seen this, and he might describe his 
observations in this regard.

Repair time is important.  There's no doubt of that.  To some degree, 
repair performance will be related to the speed of accessing the data on 
the drives, so if your best case IO speeds are low, performance on 
repair won't be terribly good.  Memory size is also important ...  we've 
had some repairs start swapping (not good) during repair.  Hence the -m 
X option (for suitable values of X).

Joe

-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics Inc.
email: landman@scalableinformatics.com
web  : http://scalableinformatics.com
        http://scalableinformatics.com/sicluster
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs