IDE hotplug support?

IDE hotplug support?

[Roy Sigurd Karlsbakk]

hi

I came across a nice case from procase, supporting 16 IDE drives in 
(so-called?) hotplug frames. Problem is... How will linux trat this?

I plan to use 15 drives in a RAID-5, assigning the last 16th drive as a 
spare.

Any good ideas?

Sorry if this is OT

roy

-- 
Roy Sigurd Karlsbakk, Datavaktmester

Computers are like air conditioners.
They stop working when you open Windows.

Re: IDE hotplug support?

[Pavel Machek]

Hi!

> I came across a nice case from procase, supporting 16 IDE drives in 
> (so-called?) hotplug frames. Problem is... How will linux trat this?

Should be okay. Hdparm can force spindown and bus rescan, and that's 
basically what you need.

> I plan to use 15 drives in a RAID-5, assigning the last 16th drive as a 
> spare.

8 controllers? hmmm...

> Any good ideas?
> 
> Sorry if this is OT

It is not.

-- 
Philips Velo 1: 1"x4"x8", 300gram, 60, 12MB, 40bogomips, linux, mutt,
details at http://atrey.karlin.mff.cuni.cz/~pavel/velo/index.html.

Re: IDE hotplug support?

[Martin Dalecki]

Uz.ytkownik Pavel Machek napisa?:
> Hi!
> 
> 
>>I came across a nice case from procase, supporting 16 IDE drives in 
>>(so-called?) hotplug frames. Problem is... How will linux trat this?
> 
> 
> Should be okay. Hdparm can force spindown and bus rescan, and that's 
> basically what you need.
> 
> 
>>I plan to use 15 drives in a RAID-5, assigning the last 16th drive as a 
>>spare.
> 
> 
> 8 controllers? hmmm...

15 drives == 16 interfaces == 8 channels == 4 controllers
with primary and secondary channel.

He will have groups of about 4 drives on each channel wich
serialize each other due to excessive IRQ line sharing and
master slave issues.

8 x 130MBy/s >>>> PCI bus throughput... I would rather recommend
a classical RAID controller card for this kind of
setup.

The request aliasing effects will be almost for sure disasterous
to overall system performance.

Re: IDE hotplug support?

[Jakob Østergaard]

On Thu, May 02, 2002 at 08:19:04PM +0200, Martin Dalecki wrote:
...
> 15 drives == 16 interfaces == 8 channels == 4 controllers
> with primary and secondary channel.

Usually using both master and slave on an IDE channel spells disaster
performance wise, and I would be surprised if the hotplug stuff worked
with this as well...

> 
> He will have groups of about 4 drives on each channel wich
> serialize each other due to excessive IRQ line sharing and
> master slave issues.

Use 8 controllers for the 15 (16) drives.

> 
> 8 x 130MBy/s >>>> PCI bus throughput... I would rather recommend
> a classical RAID controller card for this kind of
> setup.

Because RAID controllers do not use the PCI bus ???    ;)

The bus-overhead on RAID-5 is not too bad unless you specifically construct
a workload to make it so (writes-only, scattered so that the kernel cannot
cache stripes to avoid read-in for parity calculation).

Sure, the PCI bus will be a bottleneck, and PCI overhead alone will decrease
the real-world performance to somewhere below the theoretical PCI bandwidth
limitations, but don't let this blind you  -  100 MB/sec sustained transfers
can still be "good enough" for many people.

By the way, has anyone tried such larger multi-controller setups, and tested
the bandwidth in configurations with multiple PCI busses on the board, versus a
single PCI bus ?

-- 
................................................................
:   jakob@unthought.net   : And I see the elder races,         :
:.........................: putrid forms of man                :
:   Jakob Østergaard      : See him rise and claim the earth,  :
:        OZ9ABN           : his downfall is at hand.           :
:.........................:............{Konkhra}...............:

Re: IDE hotplug support?

[Samuel Flory]

  Why not just grab a pair of 8 port 3ware cards?  Run raid 5 on each 
card, and throw 0 or linear via the md driver on top?

Jakob Østergaard wrote:

>On Thu, May 02, 2002 at 08:19:04PM +0200, Martin Dalecki wrote:
>...
>  
>
>>15 drives == 16 interfaces == 8 channels == 4 controllers
>>with primary and secondary channel.
>>    
>>
>
>Usually using both master and slave on an IDE channel spells disaster
>performance wise, and I would be surprised if the hotplug stuff worked
>with this as well...
>
>  
>
>>He will have groups of about 4 drives on each channel wich
>>serialize each other due to excessive IRQ line sharing and
>>master slave issues.
>>    
>>
>
>Use 8 controllers for the 15 (16) drives.
>
>  
>
>>8 x 130MBy/s >>>> PCI bus throughput... I would rather recommend
>>a classical RAID controller card for this kind of
>>setup.
>>    
>>
>
>Because RAID controllers do not use the PCI bus ???    ;)
>
>The bus-overhead on RAID-5 is not too bad unless you specifically construct
>a workload to make it so (writes-only, scattered so that the kernel cannot
>cache stripes to avoid read-in for parity calculation).
>
>Sure, the PCI bus will be a bottleneck, and PCI overhead alone will decrease
>the real-world performance to somewhere below the theoretical PCI bandwidth
>limitations, but don't let this blind you  -  100 MB/sec sustained transfers
>can still be "good enough" for many people.
>
>By the way, has anyone tried such larger multi-controller setups, and tested
>the bandwidth in configurations with multiple PCI busses on the board, versus a
>single PCI bus ?
>
>  
>

Re: IDE hotplug support?

[Roy Sigurd Karlsbakk]

>   Why not just grab a pair of 8 port 3ware cards?  Run raid 5 on each 
> card, and throw 0 or linear via the md driver on top?

because I don't need the speed. and because they cost $1200 a piece

-- 
Roy Sigurd Karlsbakk, Datavaktmester

Computers are like air conditioners.
They stop working when you open Windows.

Re: IDE hotplug support?

[jw schultz]

On Fri, May 03, 2002 at 02:31:40AM +0200, Roy Sigurd Karlsbakk wrote:
> >   Why not just grab a pair of 8 port 3ware cards?  Run raid 5 on each 
> > card, and throw 0 or linear via the md driver on top?
> 
> because I don't need the speed. and because they cost $1200 a piece
> 

Whoever gave you that price is over-charging.

I just now went to 3ware.com and clicked through
where-to-buy and the first mail-order place i tried had the
8 port 7000 for US$455.99 each.

-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw@pegasys.ws

		Remember Cernan and Schmitt

Re: IDE hotplug support?

[Alan Cox]

> >=20
> > 8 x 130MBy/s >>>> PCI bus throughput... I would rather recommend
> > a classical RAID controller card for this kind of
> > setup.
> 
> Because RAID controllers do not use the PCI bus ???    ;)

The raid card transfers the data once, software raid once per device for
Raid 1/5 - thats a killer.

> By the way, has anyone tried such larger multi-controller setups, and t=
> ested
> the bandwidth in configurations with multiple PCI busses on the board, =
> versus a
> single PCI bus ?

With 2.4 yes. With all the 2.5 changes no.

Re: IDE hotplug support?

[Jakob Østergaard]

On Thu, May 02, 2002 at 09:26:38PM +0100, Alan Cox wrote:
> > >=20
> > > 8 x 130MBy/s >>>> PCI bus throughput... I would rather recommend
> > > a classical RAID controller card for this kind of
> > > setup.
> > 
> > Because RAID controllers do not use the PCI bus ???    ;)
> 
> The raid card transfers the data once, software raid once per device for
> Raid 1/5 - thats a killer.

For RAID-1 it's a killer (for writes), I agree.

But I really doubt it would be so horrible for RAID-5 - after all, it's only
one extra block (the parity block) for each N-1 blocks written (for an N disk
RAID-5).  The penalty should be less, the more disks you have in the array.

But seriously, has anyone out there ever seen a hardware RAID controller with
a *sustained* RAID-5 thoughput of more than 60 MB/sec ?   Not that I think it
is impossible, but I've never heard about it.  Enlighten me, please, and not
with marketing numbers...

> 
> > By the way, has anyone tried such larger multi-controller setups, and t=
> > ested
> > the bandwidth in configurations with multiple PCI busses on the board, =
> > versus a
> > single PCI bus ?
> 
> With 2.4 yes. With all the 2.5 changes no.

Did you get any speedup ?  Were you close to PCI bus saturation in the one-bus
scenario ?

-- 
................................................................
:   jakob@unthought.net   : And I see the elder races,         :
:.........................: putrid forms of man                :
:   Jakob Østergaard      : See him rise and claim the earth,  :
:        OZ9ABN           : his downfall is at hand.           :
:.........................:............{Konkhra}...............:

Re: IDE hotplug support?

[Martin Dalecki]

Uz.ytkownik Jakob Østergaard napisa?:
> On Thu, May 02, 2002 at 09:26:38PM +0100, Alan Cox wrote:
> 
>>>>=20
>>>>8 x 130MBy/s >>>> PCI bus throughput... I would rather recommend
>>>>a classical RAID controller card for this kind of
>>>>setup.
>>>
>>>Because RAID controllers do not use the PCI bus ???    ;)
>>
>>The raid card transfers the data once, software raid once per device for
>>Raid 1/5 - thats a killer.
> 
> 
> For RAID-1 it's a killer (for writes), I agree.
> 
> But I really doubt it would be so horrible for RAID-5 - after all, it's only
> one extra block (the parity block) for each N-1 blocks written (for an N disk
> RAID-5).  The penalty should be less, the more disks you have in the array.
> 
> But seriously, has anyone out there ever seen a hardware RAID controller with
> a *sustained* RAID-5 thoughput of more than 60 MB/sec ?   Not that I think it
> is impossible, but I've never heard about it.  Enlighten me, please, and not
> with marketing numbers...


Go to Sun hardware and you will see it quite frequently even on a simple
E450 equipped with an external RAID box. I saw them frequently enough in
sar accounts when the system was configured to trash on a swap
partition, which resided on such a RAID.

64 bit buses win here by a huge margin.

Re: IDE hotplug support?

[Jeff Nguyen]

You can get a sustained read speed of 80MB/s on the Adaptec 2000S
Zero Channel RAID with 7 drives (RAID-5). But the sustained write
speed is only around 32MB/s.

On the other hand, the 3Ware Escalade 7850 can sustain a read speed
of 130MB/s with 8 drives (RAID-5). The write speed is 30MB/s.

Jeff

----- Original Message -----
From: "Jakob Østergaard" <jakob@unthought.net>
To: "Alan Cox" <alan@lxorguk.ukuu.org.uk>
Cc: "Martin Dalecki" <dalecki@evision-ventures.com>; "Pavel Machek"
<pavel@suse.cz>; "Roy Sigurd Karlsbakk" <roy@karlsbakk.net>;
<linux-kernel@vger.kernel.org>
Sent: Thursday, May 02, 2002 2:13 PM
Subject: Re: IDE hotplug support?


> On Thu, May 02, 2002 at 09:26:38PM +0100, Alan Cox wrote:
> > > >=20
> > > > 8 x 130MBy/s >>>> PCI bus throughput... I would rather recommend
> > > > a classical RAID controller card for this kind of
> > > > setup.
> > >
> > > Because RAID controllers do not use the PCI bus ???    ;)
> >
> > The raid card transfers the data once, software raid once per device for
> > Raid 1/5 - thats a killer.
>
> For RAID-1 it's a killer (for writes), I agree.
>
> But I really doubt it would be so horrible for RAID-5 - after all, it's
only
> one extra block (the parity block) for each N-1 blocks written (for an N
disk
> RAID-5).  The penalty should be less, the more disks you have in the
array.
>
> But seriously, has anyone out there ever seen a hardware RAID controller
with
> a *sustained* RAID-5 thoughput of more than 60 MB/sec ?   Not that I think
it
> is impossible, but I've never heard about it.  Enlighten me, please, and
not
> with marketing numbers...
>
> >
> > > By the way, has anyone tried such larger multi-controller setups, and
t=
> > > ested
> > > the bandwidth in configurations with multiple PCI busses on the board,
=
> > > versus a
> > > single PCI bus ?
> >
> > With 2.4 yes. With all the 2.5 changes no.
>
> Did you get any speedup ?  Were you close to PCI bus saturation in the
one-bus
> scenario ?
>
> --
> ................................................................
> :   jakob@unthought.net   : And I see the elder races,         :
> :.........................: putrid forms of man                :
> :   Jakob Østergaard      : See him rise and claim the earth,  :
> :        OZ9ABN           : his downfall is at hand.           :
> :.........................:............{Konkhra}...............:
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: IDE hotplug support?

[Jakob Østergaard]

On Thu, May 02, 2002 at 03:22:00PM -0700, Jeff Nguyen wrote:
> You can get a sustained read speed of 80MB/s on the Adaptec 2000S
> Zero Channel RAID with 7 drives (RAID-5). But the sustained write
> speed is only around 32MB/s.
> 
> On the other hand, the 3Ware Escalade 7850 can sustain a read speed
> of 130MB/s with 8 drives (RAID-5). The write speed is 30MB/s.

So for both cards, the solution with simple IDE controllers, all on
one PCI bus, would be faster not only on RAID-5, but also RAID-1+0.

Or am I missing something ?

(I know, the IDE cabling is going to be *hell* for 15 drives if you
 want to stay within spec, and I would go SCSI or FC before that myself)

-- 
................................................................
:   jakob@unthought.net   : And I see the elder races,         :
:.........................: putrid forms of man                :
:   Jakob Østergaard      : See him rise and claim the earth,  :
:        OZ9ABN           : his downfall is at hand.           :
:.........................:............{Konkhra}...............:

Re: IDE hotplug support?

[Alan Cox]

> Or am I missing something ?
> 
> (I know, the IDE cabling is going to be *hell* for 15 drives if you
>  want to stay within spec, and I would go SCSI or FC before that myself=
> )

3Ware have folks they can point you at who can provide cases, prebuilt
systems and the like on that kind of size/scale.

Re: IDE hotplug support?

[Roy Sigurd Karlsbakk]

> You can get a sustained read speed of 80MB/s on the Adaptec 2000S
> Zero Channel RAID with 7 drives (RAID-5). But the sustained write
> speed is only around 32MB/s.
> 
> On the other hand, the 3Ware Escalade 7850 can sustain a read speed
> of 130MB/s with 8 drives (RAID-5). The write speed is 30MB/s.

I've done 82MB/s with two WD 120GB 7200 IDE drives in sw raid 0

ok. that was outer rim, but still.

need I say more?

roy
--
Roy Sigurd Karlsbakk, Datavaktmester

Computers are like air conditioners.
They stop working when you open Windows.

Re: IDE hotplug support?

[Roy Sigurd Karlsbakk]

On Thu, 2 May 2002, Jakob Østergaard wrote:

> On Thu, May 02, 2002 at 09:26:38PM +0100, Alan Cox wrote:
> > > >=20
> > > > 8 x 130MBy/s >>>> PCI bus throughput... I would rather recommend
> > > > a classical RAID controller card for this kind of
> > > > setup.
> > > 
> > > Because RAID controllers do not use the PCI bus ???    ;)
> > 
> > The raid card transfers the data once, software raid once per device for
> > Raid 1/5 - thats a killer.
> 
> For RAID-1 it's a killer (for writes), I agree.
> 
> But I really doubt it would be so horrible for RAID-5 - after all, it's only
> one extra block (the parity block) for each N-1 blocks written (for an N disk
> RAID-5).  The penalty should be less, the more disks you have in the array.
> 
> But seriously, has anyone out there ever seen a hardware RAID controller with
> a *sustained* RAID-5 thoughput of more than 60 MB/sec ?   Not that I think it
> is impossible, but I've never heard about it.  Enlighten me, please, and not
> with marketing numbers...

I was doing some testing on a Compaq ProLiant 380 (perhaps G2 - don't 
remember) that was doing some 70-80MB/s on a RAID-5. That was, however, on 
12 drives, 6 on each a controller. I managed to do a little more with JBOD 
and Linux Software RAID-5. That said, this was 18GB drives, which aren't 
the fastest drives on earth...

roy

-- 
Roy Sigurd Karlsbakk, Datavaktmester

Computers are like air conditioners.
They stop working when you open Windows.

Re: IDE hotplug support?

[Roy Sigurd Karlsbakk]

> 15 drives == 16 interfaces == 8 channels == 4 controllers
> with primary and secondary channel.
> 
> He will have groups of about 4 drives on each channel wich
> serialize each other due to excessive IRQ line sharing and
> master slave issues.
> 
> 8 x 130MBy/s >>>> PCI bus throughput... I would rather recommend
> a classical RAID controller card for this kind of
> setup.
> 
> The request aliasing effects will be almost for sure disasterous
> to overall system performance.

hm. all I want is lots of space. I don't need speed here. What is 
'disasterous' here?
-- 
Roy Sigurd Karlsbakk, Datavaktmester

Computers are like air conditioners.
They stop working when you open Windows.

Re: IDE hotplug support?

[Alan Cox]

> > The request aliasing effects will be almost for sure disasterous
> > to overall system performance.
> 
> hm. all I want is lots of space. I don't need speed here. What is 
> 'disasterous' here?

Halve the expected throughput and subtract a bit. Since you can put 8
ports on a 3ware card one drive per port at 160Gb a drive I suspect you
don't need master/slave pairs 8)

Re: IDE hotplug support?

[Roy Sigurd Karlsbakk]

On Fri, 3 May 2002, Alan Cox wrote:

> > > The request aliasing effects will be almost for sure disasterous
> > > to overall system performance.
> > 
> > hm. all I want is lots of space. I don't need speed here. What is 
> > 'disasterous' here?
> 
> Halve the expected throughput and subtract a bit. Since you can put 8
> ports on a 3ware card one drive per port at 160Gb a drive I suspect you
> don't need master/slave pairs 8)
>
I do. I can't afford 3ware
-- 
Roy Sigurd Karlsbakk, Datavaktmester

Computers are like air conditioners.
They stop working when you open Windows.

Re: IDE hotplug support?

[Zwane Mwaikambo]

On Tue, 30 Apr 2002, Roy Sigurd Karlsbakk wrote:

> I came across a nice case from procase, supporting 16 IDE drives in 
> (so-called?) hotplug frames. Problem is... How will linux trat this?

Wasn't this flogged, subjected to Britney Spears, humiliated in front of 
friends and family and dragged through the streets with nigh but its 
birthday suit revealing the scurvy it contracted from spending 3 
years aboard a naval vessel recently?

-- 
http://function.linuxpower.ca
		

Re: IDE hotplug support?

[Ragnar Hojland Espinosa]

On Tue, Apr 30, 2002 at 06:22:06PM +0200, Zwane Mwaikambo wrote:
> On Tue, 30 Apr 2002, Roy Sigurd Karlsbakk wrote:
> 
> > I came across a nice case from procase, supporting 16 IDE drives in 
> > (so-called?) hotplug frames. Problem is... How will linux trat this?
> 
> Wasn't this flogged, subjected to Britney Spears, humiliated in front of 
> friends and family and dragged through the streets with nigh but its 
> birthday suit revealing the scurvy it contracted from spending 3 
> years aboard a naval vessel recently?

Actually no.  The conclusion of that thread was something like common OTS
IDE components aren't hot swappable, because even if we ignore the bus shock
problem and the controller supports proper power down, the hard disk
probabily won't.

Considering Roy is asking about specific hardware its not of much relevance
overall.
-- 
____/|  Ragnar Højland      Freedom - Linux - OpenGL | CCNA - Brainbench MVP
\ o.O|  PGP94C4B2F0D27DE025BE2302C104B78C56 B72F0822 |  for Unix Programming
 =(_)=  "Thou shalt not follow the NULL pointer for  |  (www.brainbench.com)
   U     chaos and madness await thee at its end."      [38 pend. Mar 10]