Re: 30 TB RAID6 + XFS slow write performance

From: Michael Monnerie <michael.monnerie@is.it-management.at>
To: xfs@oss.sgi.com
Cc: Stan Hoeppner <stan@hardwarefreak.com>,
	John Bokma <contact@johnbokma.com>
Subject: Re: 30 TB RAID6 + XFS slow write performance
Date: Fri, 22 Jul 2011 08:10:00 +0200	[thread overview]
Message-ID: <201107220810.01889@zmi.at> (raw)
In-Reply-To: <20110721064838.GA13963@dastard>

[-- Attachment #1.1: Type: Text/Plain, Size: 2684 bytes --]

On Donnerstag, 21. Juli 2011 Dave Chinner wrote:
> If you are writing files that grow like this, then you are doing
> something wrong. If the app can't do it's IO differently, then this
> is exactly the reason we have userspace-controlled preallocation
> interfaces.
> 
> Filesystems cannot prevent user stupidity from screwing something
> up....

This can happen if you copy a syslog server over to a new disk, then let 
it start it's work again. Many files that start small and grow. Luckily, 
the logs are rotated latest monthly, so it shouldn't be too bad.

> > And files >64KiB are immediately fragmented
> > then. At this time, there are only 16384 * 2KiB = 32MiB used, which
> > is 3,125% of the disk. I can't believe my numbers, are they true?
> 
> No, because most filesystems have a 4k block size. 

I just meant pure disk usage. Of 1GB, only 32MB are used, and this worst 
case example hits us badly.

> Not to mention
> that fragmentation is likely to be limited to the single AG the files
> in the directory belong to. i.e. even if we can't allocation a sunit
> aligned chunk in an AG, we won't switch to another AG just to do
> sunit aligned allocation.

This is good to know also, thanks.

> > OK, this is a worst case scenario, and as you've said before, any
> > filesystem can be considered full at 85% fill grade. But it's
> > incredible how quickly you could fuck up a filesystem when using
> > su/sw and writing small files.
> 
> Well, don't use a filesystem that is optimised for storing large
> sizes, large files and high bandwidth for storing lots of small
> files, then.  Indeed, the point of not packing the files is so they
> -don't fragemnt as they grow-. XFS is not designed to be optimal
> for small filesystems or small files. In most cases it will deal
> with them just fine, so in reality your concerns are mostly
> unfounded...

Yes, I just wanted to know about the corner cases, and how XFS behaves. 
Actually, we're changing over to using NetApps, and with their WAFL 
anyway I should drop all su/sw usage and just use 4KB blocks.

And even when XFS is optimized for large files, there are often small 
ones. Think of a mysql server with hundreds of DBs and 
innodb_file_per_table set. Even when some DBs are large, there are many 
small files.

But this thread has drifted a bit. XFS does great work, and now I 
understand the background a bit more. Thanks, Dave.

-- 
mit freundlichen Grüssen,
Michael Monnerie, Ing. BSc

it-management Internet Services: Protéger
http://proteger.at [gesprochen: Prot-e-schee]
Tel: +43 660 / 415 6531

// Haus zu verkaufen: http://zmi.at/langegg/

[-- Attachment #1.2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs