Re: 30 TB RAID6 + XFS slow write performance

Eric Sandeen
2011-07-19T00:00:20Z

On 7/18/11 2:58 PM, John Bokma wrote:
> Dear list members,
>
> A customer of mine is currently struggling with the performance of a
> 30 TB RAID6 which uses XFS as the filing system. I am somewhat sure
> it's not XFS that's causing the performance issue but my expertise is
> not XFS nor RAID; I just wrote the software that after moving to the
> larger RAID (from a much smaller one, ~ 3TB, using ext3) suddenly
> seems to have a huge drop in write performance.
>
> The software I wrote writes many small (50-150K) files in parallel
> (100+ processes), thousands of times per hour. Writing a file of
> 50-150K now and then seems to take between 30 and 90 seconds, and
> more rarely can take over 200 seconds (several times an hour).
>
> When all processes are stopped and restarted again the 30-90 seconds
> delay start happening when about 16-20+ processes are running.
>
> To me this sounds like something has been configured wrong. I already
> recommended my customer to find someone who is capable of configuring
> the RAID correctly; to me it sounds like a hardware/configuration
> issue.
>
> Any insights are very welcome.
>
> Hardware: card: MegaRAID SAS 9260-16i disks: 14x Barracuda® XT
> ST33000651AS 3TB (2 hot spares). RAID6 ~ 30TB

My first suggestion would be to check the partition alignment on the raid (if it is partitioned), and be sure it is aligned with the underlying raid geometry.

And then make sure you give mkfs.xfs the proper geometry as well.

After that, does the raid card have a battery-backed write cache? If so, you can safely disable barriers.

More info is always good too, for starters what kernel & what xfsprogs version?

What mkfs & mount options?

-Eric

> Thanks for reading, John
>
> _______________________________________________ xfs mailing list
> xfs@oss.sgi.com http://oss.sgi.com/mailman/listinfo/xfs