Re: extremely slow write performance plaintext

From: Stan Hoeppner <stan@hardwarefreak.com>
To: xfs@oss.sgi.com
Subject: Re: extremely slow write performance plaintext
Date: Fri, 14 Jan 2011 16:02:27 -0600	[thread overview]
Message-ID: <4D30C7F3.3040105@hardwarefreak.com> (raw)
In-Reply-To: <27616_1295038110_4D30B69E_27616_233_1_4D30B69E.4020506@davisvision.com>

Cory Coager put forth on 1/14/2011 2:48 PM:
> On 01/14/2011 02:51 PM, Stan Hoeppner wrote:
>> Make sure the write cache on the P600 (what size is it BTW?) is enabled and that
>> the BBU is in working order.  Also make sure the P600 is disabling the write
>> caches on the drives themselves.  Then...
>>
> Write cache is enabled on the controller, the size is 512MB, BBU is in good
> conditioned (checked with the HP utility).  How do I check the write cache on
> the drives?

The controller should do this automatically.  You'll have to check the docs to
verify.  This is to safeguard data.  The BBWC protects unwritten data in the
controller cache only, not the drives' caches.  It won't negatively affect
performance if the drives' caches are enabled.  On the contrary, it would
probably increase performance a bit.  It's simply less safe having them enabled
in the event of a crash.

After rereading your original post I don't think there's any issue here anyway.
 You stated you have 24 drives in 2 arrays (although you didn't state if all the
disks are on one P600 or two).

>> Mount with 'nobarrier' so XFS isn't interfering with the hardware cache
>> performance of the P600.  With barriers enabled (the default) XFS will
>> periodically flush the cache on the RAID card causing write performance problems.
>>
> Already using nobarrier.

This was the important part I was looking for.  It's apparently not a cache
issue then, unless the utility is lying or querying the wrong controller or
something.

Nothing relevant in dmesg or any other logs?  No errors of any kind?  Does
iostat reveal anything even slightly odd?

I also just noticed you're testing writes with a 1k block size.  That seems
awefully small.  Does the write throughput increase any when you test with a
4k/8k/16k block size?

BTW, this is an old machine.  PCI-X is dead.  Did this slow write trouble just
start recently?  What has changed since it previously worked fine?

You're making it very difficult to assist you by not providing basic
troubleshooting information.  I.e.

What has changed since the system functioned properly?
When did it change?
Did it ever work properly?
Etc.

God I hate pulling teeth... :)

-- 
Stan

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs