Re: [PATCH 00/19] Hardware Accelerated MD RAID5: Introduction

Jeff Garzik
2006-09-13T04:05:42Z

Dan Williams wrote:
> On 9/11/06, Jeff Garzik <jeff@garzik.org> wrote:
>> Dan Williams wrote:
>> > This is a frequently asked question, Alan Cox had the same one at OLS.
>> > The answer is "probably." The only complication I currently see is
>> > where/how the stripe cache is maintained. With the IOPs its easy
>> > because the DMA engines operate directly on kernel memory. With the
>> > Promise card I believe they have memory on the card and it's not clear
>> > to me if the XOR engines on the card can deal with host memory. Also,
>> > MD would need to be modified to handle a stripe cache located on a
>> > device, or somehow synchronize its local cache with card in a manner
>> > that is still able to beat software only MD.
>>
>> sata_sx4 operates through [standard PC] memory on the card, and you use
>> a DMA engine to copy memory to/from the card.
>>
>> [select chipsets supported by] sata_promise operates directly on host
>> memory.
>>
>> So, while sata_sx4 is farther away from your direct-host-memory model,
>> it also has much more potential for RAID acceleration: ideally, RAID1
>> just copies data to the card once, then copies the data to multiple
>> drives from there. Similarly with RAID5, you can eliminate copies and
>> offload XOR, presuming the drives are all connected to the same card.
> In the sata_promise case its straight forward, all that is needed is
> dmaengine drivers for the xor and memcpy engines. This would be
> similar to the current I/OAT model where dma resources are provided by
> a PCI function. The sata_sx4 case would need a different flavor of
> the dma_do_raid5_block_ops routine, one that understands where the
> cache is located. MD would also need the capability to bypass the
> block layer since the data will have already been transferred to the
> card by a stripe cache operation
>
> The RAID1 case give me pause because it seems any work along these
> lines requires that the implementation work for both MD and DM, which
> then eventually leads to being tasked with merging the two.

RAID5 has similar properties. If all devices in a RAID5 array are
attached to a single SX4 card, then a high level write to the RAID5
array is passed directly to the card, which then performs XOR, striping,
etc.

Jeff