New RAID-6 snapshot

New RAID-6 snapshot

[H. Peter Anvin]

Well, I implemented read-modify-write for RAID-6, and at least on my
6-disk system it was a small but measurable performance *loss*.
Therefore, I have cleaned up the patches slightly, but basically
RAID-6 seems to work and it now needs people to test it out and let me
know what bugs is there.

Additionally, the previously mentioned generic md bug relating to
recovery onto multiple disks should be fixed in the main md code (it's
also needed for RAID-1.)

The new snapshot is at:

    http://www.kernel.org/pub/linux/kernel/people/hpa/raid6-20031230.tar.gz

... although I reserve the right to upload newer snapshots on an
ongoing basis :)

	-hpa


-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
If you send me mail in HTML format I will assume it's spam.
"Unix gives you enough rope to shoot yourself in the foot."
Architectures needed: ia64 m68k mips64 ppc ppc64 s390 s390x sh v850 x86-64

Re: New RAID-6 snapshot

[Mike Fedyk]

On Tue, Dec 30, 2003 at 03:34:28AM -0800, H. Peter Anvin wrote:
> Well, I implemented read-modify-write for RAID-6, and at least on my
> 6-disk system it was a small but measurable performance *loss*.

What was your previous patch doing instead?

Re: New RAID-6 snapshot

[H. Peter Anvin]

Mike Fedyk wrote:
> On Tue, Dec 30, 2003 at 03:34:28AM -0800, H. Peter Anvin wrote:
> 
>>Well, I implemented read-modify-write for RAID-6, and at least on my
>>6-disk system it was a small but measurable performance *loss*.
> 
> What was your previous patch doing instead?

The published snapshots (including the latest, since I backed out 
read-modify-write) all do read-reconstruct-write.

	-hpa

Re: New RAID-6 snapshot

[Mike Fedyk]

On Tue, Dec 30, 2003 at 10:35:53AM -0800, H. Peter Anvin wrote:
> Mike Fedyk wrote:
> >On Tue, Dec 30, 2003 at 03:34:28AM -0800, H. Peter Anvin wrote:
> >
> >>Well, I implemented read-modify-write for RAID-6, and at least on my
> >>6-disk system it was a small but measurable performance *loss*.
> >
> >What was your previous patch doing instead?
> 
> The published snapshots (including the latest, since I backed out 
> read-modify-write) all do read-reconstruct-write.

Does that mean that RRW (as the name seems to imply) would need to read the
entire stripe before it could write back the updated stripe compared to a
partial read with RMW? 

Re: New RAID-6 snapshot

[H. Peter Anvin]

Mike Fedyk wrote:
> 
> Does that mean that RRW (as the name seems to imply) would need to read the
> entire stripe before it could write back the updated stripe compared to a
> partial read with RMW? 
 >

Sort of.  If you have, say, seven drives total: D D D D D P Q, the 
access pattern look like:

D D D D D P Q
     ^
     Write this drive

... then RCW needs to read:

D D - D D - -

... and RMW needs to read:

- - D - - P Q


Now, if this is a "small" write (smaller than a page) then the stripe 
being modified always needs to be written, which helps RMW.  On the 
other hands, large writes really help RCW:

D D D D D P Q
^^^^^^^
Write these four drives

... then RCW only needs to read one drive, whereas RMW needs to read six.

Now, you'd never use RMW only; it's only used when it should be faster 
than RCW, which is why I'm a bit surprised about the performance 
degradation.  With only six drives in my test system it's pretty clear 
that RMW should very rarely be invoked, however, the performance ought 
to have stayed the same rather than slip.

	-hpa

Re: New RAID-6 snapshot

[Mike Fedyk]

On Tue, Dec 30, 2003 at 10:51:10AM -0800, H. Peter Anvin wrote:
> Now, you'd never use RMW only; it's only used when it should be faster 
> than RCW, which is why I'm a bit surprised about the performance 
> degradation.  With only six drives in my test system it's pretty clear 
> that RMW should very rarely be invoked, however, the performance ought 
> to have stayed the same rather than slip.

You mentioned that none of your snapshots have RMW in them.  Can you put up
a patch that adds RMW to the snapshots?  Maybe someone will notice
something...

Did you get any data corruption with the RMW patch applied?

Re: New RAID-6 snapshot

[H. Peter Anvin]

Mike Fedyk wrote:
> 
> You mentioned that none of your snapshots have RMW in them.  Can you put up
> a patch that adds RMW to the snapshots?  Maybe someone will notice
> something...
> 

It would take considerable amounts of work to get it into releasable 
shape, because I'd have to add RMW functionality to all the implemented 
algorithms -- not just the one my machine uses.  However, if someone is 
interested in vgrepping through the source I can post the part of the 
diff patch that applies to raid6main.c.

I doubt it's actually wrong, though.  It probably just causes worse 
access patterns in my particular configuration.

> Did you get any data corruption with the RMW patch applied?
 >

No.

	-hpa