Linux: Why software RAID?

Linux: Why software RAID?

[Jeff Garzik]

Mark Perkel wrote:
> Running Linux on an AMD AM2 nVidia chip ser that supports Raid 0
> striping on the motherboard. Just wondering if hardware raid (SATA2) is
> going to be faster that software raid and why? 


First, it sounds like you are confusing motherboard "RAID" with real 
RAID.  There's a FAQ for this sort of thing:

	http://linux-ata.org/faq-sata-raid.html

In particular, your motherboard's Raid 0 striping (a) is not done in 
hardware, and (b) has nothing to do with SATA2.

But anyway, to help answer the question of hardware vs. software RAID, I 
wrote up a page:

	http://linux.yyz.us/why-software-raid.html

Generally, you want software RAID unless your PCI bus (or more rarely, 
your CPU) is getting saturated.  With RAID-0, there is no duplication of 
data, and so, PCI bus and CPU usage should be about the same for 
hardware and software RAID.

	Jeff

Re: Linux: Why software RAID?

[Chris Friesen]

Jeff Garzik wrote:

> But anyway, to help answer the question of hardware vs. software RAID, I 
> wrote up a page:
> 
>     http://linux.yyz.us/why-software-raid.html

Just curious...with these guys 
(http://www.bigfootnetworks.com/KillerOverview.aspx) putting linux on a 
PCI NIC to allow them to bypass Windows' network stack, has anyone ever 
considered doing "hardware" raid by using an embedded cpu running linux 
software RAID, with battery-backed memory?

It would theoretically allow you to remain feature-compatible by 
downloading new kernels to your RAID card.

Chris

Re: Linux: Why software RAID?

[H. Peter Anvin]

Chris Friesen wrote:
> Jeff Garzik wrote:
> 
>> But anyway, to help answer the question of hardware vs. software RAID, 
>> I wrote up a page:
>>
>>     http://linux.yyz.us/why-software-raid.html
> 
> Just curious...with these guys 
> (http://www.bigfootnetworks.com/KillerOverview.aspx) putting linux on a 
> PCI NIC to allow them to bypass Windows' network stack, has anyone ever 
> considered doing "hardware" raid by using an embedded cpu running linux 
> software RAID, with battery-backed memory?
> 
> It would theoretically allow you to remain feature-compatible by 
> downloading new kernels to your RAID card.
> 

Yes.  In fact, I have been told by several RAID chip vendors that their 
customers are *strongly* demanding that their chips be able to run Linux 
  md (and still use whatever hardware offload features.)

So it's happening.

	-hpa

Re: Linux: Why software RAID?

[Dan Williams]

On 8/23/06, H. Peter Anvin <hpa@zytor.com> wrote:
> Chris Friesen wrote:
> > Jeff Garzik wrote:
> >
> >> But anyway, to help answer the question of hardware vs. software RAID,
> >> I wrote up a page:
> >>
> >>     http://linux.yyz.us/why-software-raid.html
> >
> > Just curious...with these guys
> > (http://www.bigfootnetworks.com/KillerOverview.aspx) putting linux on a
> > PCI NIC to allow them to bypass Windows' network stack, has anyone ever
> > considered doing "hardware" raid by using an embedded cpu running linux
> > software RAID, with battery-backed memory?
> >
> > It would theoretically allow you to remain feature-compatible by
> > downloading new kernels to your RAID card.
> >
>
> Yes.  In fact, I have been told by several RAID chip vendors that their
> customers are *strongly* demanding that their chips be able to run Linux
>   md (and still use whatever hardware offload features.)
>
> So it's happening.
Speaking of md with hardware offload features:

http://prdownloads.sourceforge.net/xscaleiop/ols_paper_2006.pdf?download

>         -hpa

Dan

Re: Linux: Why software RAID?

[Mattias Wadenstein]

On Wed, 23 Aug 2006, Chris Friesen wrote:

> Jeff Garzik wrote:
>
>> But anyway, to help answer the question of hardware vs. software RAID, I 
>> wrote up a page:
>>
>>     http://linux.yyz.us/why-software-raid.html
>
> Just curious...with these guys 
> (http://www.bigfootnetworks.com/KillerOverview.aspx) putting linux on a PCI 
> NIC to allow them to bypass Windows' network stack, has anyone ever 
> considered doing "hardware" raid by using an embedded cpu running linux 
> software RAID, with battery-backed memory?

I'd expect this to be the reason why md offload support to xor engines and 
whatever turns up. It makes very little sense for a modern server/desktop 
CPU, but for the embedded ones it does.

/Mattias Wadenstein

Re: Linux: Why software RAID?

[Adam Kropelin]

Jeff Garzik <jeff@garzik.org> wrote:
> But anyway, to help answer the question of hardware vs. software RAID, I 
> wrote up a page:
> 
> 	http://linux.yyz.us/why-software-raid.html
> 
> Generally, you want software RAID unless your PCI bus (or more rarely, 
> your CPU) is getting saturated.  With RAID-0, there is no duplication of 
> data, and so, PCI bus and CPU usage should be about the same for 
> hardware and software RAID.

Hardware RAID can be (!= is) more tolerant of serious drive failures
where a single drive locks up the bus. A high-end hardware RAID card 
may be designed with independent controllers so a single drive failure
cannot take other spindles down with it. The same can be accomplished 
with sw RAID of course if the builder is careful to use multiple PCI 
cards, etc. Sw RAID over your motherboard's onboard controllers leaves
you vulnerable.

--Adam

Re: Linux: Why software RAID?

[Alan Cox]

Ar Iau, 2006-08-24 am 09:07 -0400, ysgrifennodd Adam Kropelin:
> Jeff Garzik <jeff@garzik.org> wrote:
> with sw RAID of course if the builder is careful to use multiple PCI 
> cards, etc. Sw RAID over your motherboard's onboard controllers leaves
> you vulnerable.

Generally speaking the channels on onboard ATA are independant with any
vaguely modern card. And for newer systems well the motherboard tends to
be festooned with random SATA controllers, all separate!

Re: Linux: Why software RAID?

[Adam Kropelin]

On Thu, Aug 24, 2006 at 02:20:50PM +0100, Alan Cox wrote:
> Ar Iau, 2006-08-24 am 09:07 -0400, ysgrifennodd Adam Kropelin:
> > Jeff Garzik <jeff@garzik.org> wrote:
> > with sw RAID of course if the builder is careful to use multiple PCI 
> > cards, etc. Sw RAID over your motherboard's onboard controllers leaves
> > you vulnerable.
> 
> Generally speaking the channels on onboard ATA are independant with any
> vaguely modern card. 

Ahh, I did not know that. Does this apply to master/slave connections on
the same PATA cable as well? I know zero about PATA, but I assumed from
the terminology that master and slave needed to cooperate rather closely.

> And for newer systems well the motherboard tends to
> be festooned with random SATA controllers, all separate!

And how. You can't swing a dead cat without hitting a half-dozen ATA
ports these days. And most of them are those infuriatingly insecure SATA
connectors that pop off when you look at them cross-eyed...

--Adam

Re: Linux: Why software RAID?

[Mark Lord]

Adam Kropelin wrote:
> On Thu, Aug 24, 2006 at 02:20:50PM +0100, Alan Cox wrote:
>> Generally speaking the channels on onboard ATA are independant with any
>> vaguely modern card. 
> 
> Ahh, I did not know that. Does this apply to master/slave connections on
> the same PATA cable as well?

No, it doesn't.  Except for cards which use special cables,
such as the Pacific Digital ADMA cards (which can even run both
master and slave simultaneously on a cable, though not with
the current Linux drivers).

Cheers

Re: Linux: Why software RAID?

[Alan Cox]

Ar Iau, 2006-08-24 am 07:31 -0700, ysgrifennodd Marc Perkel:
> So - the bottom line answer to my question is that unless you are
> running raid 5 and you have a high powered raid card with cache and
> battery backup that there is no significant speed increase to use
> hardware raid. For raid 0 there is no advantage.
> 
If your raid is entirely on PCI plug in cards and you are doing RAID1
there is a speed up using hardware assisted raid because of the PCI bus
contention. If your controllers are PCI express, on internal high speed
busses (eg chipset controllers) or at least half of them are then
generally there is no win with hardware raid 0/1 and it is often slower.

Re: Linux: Why software RAID?

[Bill Davidsen]

Alan Cox wrote:

>Ar Iau, 2006-08-24 am 07:31 -0700, ysgrifennodd Marc Perkel:
>  
>
>>So - the bottom line answer to my question is that unless you are
>>running raid 5 and you have a high powered raid card with cache and
>>battery backup that there is no significant speed increase to use
>>hardware raid. For raid 0 there is no advantage.
>>
>>    
>>
>If your raid is entirely on PCI plug in cards and you are doing RAID1
>there is a speed up using hardware assisted raid because of the PCI bus
>contention.
>

I would expect to see this with RAID5 as well, for the same reason...

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979

Re: Linux: Why software RAID?

[Joel Jaeggli]

Bill Davidsen wrote:
> Alan Cox wrote:
> 
>> Ar Iau, 2006-08-24 am 07:31 -0700, ysgrifennodd Marc Perkel:
>>  
>>
>>> So - the bottom line answer to my question is that unless you are
>>> running raid 5 and you have a high powered raid card with cache and
>>> battery backup that there is no significant speed increase to use
>>> hardware raid. For raid 0 there is no advantage.
>>>
>>>   
>> If your raid is entirely on PCI plug in cards and you are doing RAID1
>> there is a speed up using hardware assisted raid because of the PCI bus
>> contention.
>>
> 
> I would expect to see this with RAID5 as well, for the same reason...

assuming you actually have lots of pci contention that might be a
consideration... if you're sitting on server class hardware with
multiple pci buses or using pci-express cards that won't be a
significant issue.

-- 
------------------------------------------------------------------------
Joel Jaeggli             Unix Consulting              joelja@uoregon.edu
GPG Key Fingerprint:   5C6E 0104 BAF0 40B0 5BD3 C38B F000 35AB B67F 56B2