Re: Abysmal RAID 0 performance on 2.4.0-test10 for IDE?

Re: Abysmal RAID 0 performance on 2.4.0-test10 for IDE?

[Rik van Riel]

On Tue, 26 Dec 2000, Felix von Leitner wrote:
> Thus spake Rik van Riel (riel@conectiva.com.br):
> > > One more detail: top says the CPU is 50% system when reading from either
> > > one of the disk or raid devices.  That seems awfully high considering
> > > that the Promise controller claims to do UDMA.
> > > 
> > > Any comments?
> > Your program reads in data at 30MB/second, on a memory bus
> > that most likely supports something like 60 to 100MB/second.
> 
> 100.

So that's 30% for the UDMA controller and maybe
30% for the CPU (if your program reads in all the
data).

> > Part of this memory bandwidth is needed for the UDMA controller
> > to push the data to memory, probably between 30% and 50%.
> 
> That would be 30%.

Add to that the overhead of allocating and reclaiming
the memory, doing the RAID mapping, sending commands
to the hard disk, ...

> > Every time the UDMA controller has the memory bus for itself the
> > CPU will busy-wait on memory, which shows up as CPU busy time.
> 
> So, you are saying, when I add a gigabit ethernet card, CPU will hit
> 100% at about 30 MB/second?  That sounds like a weak architecture ;-)

Hey, there's a reason PCs are so cheap ;)

regards,

Rik
--
Hollywood goes for world dumbination,
	Trailer at 11.

		http://www.surriel.com/
http://www.conectiva.com/	http://distro.conectiva.com.br/

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Abysmal RAID 0 performance on 2.4.0-test10 for IDE?

[Ian Stirling]

> 
> On Tue, 26 Dec 2000, Felix von Leitner wrote:
> > Thus spake Rik van Riel (riel@conectiva.com.br):
> > > > One more detail: top says the CPU is 50% system when reading from either
> > > > one of the disk or raid devices.  That seems awfully high considering
> > > > that the Promise controller claims to do UDMA.
> > > > 
> > > > Any comments?
> > > Your program reads in data at 30MB/second, on a memory bus
> > > that most likely supports something like 60 to 100MB/second.
> > 
> > 100.
> 
> So that's 30% for the UDMA controller and maybe
> 30% for the CPU (if your program reads in all the
> data).

Where are you getting 100MB/s?
The PCI bus can move around 130MB/sec, but RAM is lots faster.
A single PC100 DIMM can move 800MB/sec.
This P100 laptop I'm typing on gets better than 100MB/s ram reads.


Anyway, in clarification, Rik mentioned that two reads from different
disk (arrays?) on the same controller at the same time get more or less
the same speed.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Abysmal RAID 0 performance on 2.4.0-test10 for IDE?

[Barry K. Nathan]

Ian Stirling wrote:
> Where are you getting 100MB/s?
> The PCI bus can move around 130MB/sec, but RAM is lots faster.

I'll clarify your clarification further. :) Your typical PC has 33MHz
32-bit PCI. Increasing it to 66MHz or 64-bit can double the transfer rate,
and doing both can quadruple it. (Perhaps I've overlooked a detail or
oversimplified something, in which case I'd appreciate being corrected.)

-Barry K. Nathan <barryn@pobox.com>
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Abysmal RAID 0 performance on 2.4.0-test10 for IDE?

[Paul Jakma]

On Tue, 26 Dec 2000, Ian Stirling wrote:

> The PCI bus can move around 130MB/sec,

in bursts yes, but sustained data bandwidth of PCI is a lot lower,
maybe 30 to 50MB/s. And you won't get sustained RAID performance >
sustained PCI performance.

> Anyway, in clarification, Rik mentioned that two reads from different
> disk (arrays?) on the same controller at the same time get more or less
> the same speed.

try scsi.

--paulj

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

RAID - IDE - here we go again...

[Jakob Østergaard]

On Wed, Dec 27, 2000 at 04:23:43PM +0000, Paul Jakma wrote:
> On Tue, 26 Dec 2000, Ian Stirling wrote:
> 
> > The PCI bus can move around 130MB/sec,
> 
> in bursts yes, but sustained data bandwidth of PCI is a lot lower,
> maybe 30 to 50MB/s. And you won't get sustained RAID performance >
> sustained PCI performance.

Much higher than 30-50 - but yes, the total bandwidth won't exceed
the slowest channel.

> 
> > Anyway, in clarification, Rik mentioned that two reads from different
> > disk (arrays?) on the same controller at the same time get more or less
> > the same speed.
> 
> try scsi.

SCSI won't get you a faster PCI bus.  Guys, *PLEASE*, - everyone on this list
knows what the respective virtues and horrors of SCSI and IDE are.   Configured
properly, both can perform well, configured wrongly, both will suck.

The timings below are from a dual PII-350, Asus P2B-DS, it has a six disk SCSI
RAID and a five disk IDE RAID.  The IDE raid is configured properly with one
channel per disk - which would have solved the performance problem in the array
that spawned this thread. 

(By the way: the SCSI RAID is configured with three controllers for the six
disks, because of the low SCSI bus bandwidth, reality rules in the SCSI world
as well)

Kernel is 2.2, RAID is 0.90, IDE is Andre's

---------------------------------------
Dual PII-350, 256 MB RAM, test on 1 GB file

Filesystem:  75 GB ext2fs
RAID:        Linux Softare RAID-5
Disks:       5 pcs. IBM Deskstar 75 GXP (30GB)
Controller:  3 pcs. Promise PDC-2067

              -------Sequential Output-------- ---Sequential Input-- --Random--
              -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
         1024  4998 98.0 28660 45.9 14586 50.4  5386 98.5 71468 79.1 338.6  5.2                               

28 MB/sec write, 71MB/sec read on RAID-5.

---------------------------------------
Same box, test on 1 GB file

Filesystem:  18 GB ext2fs
RAID:        Linux Softare RAID-0

This array is built on other partitions on the same disks as above.

              -------Sequential Output-------- ---Sequential Input-- --Random--
              -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks---
Machine    MB K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU  /sec %CPU
         1024  5381 99.7 44267 50.1 22613 62.1  5469 99.4 98075 91.4 311.0  5.0                               
44 MB/sec write, 98 MB/sec read on RAID-0.  That's thru *one* PCI bus folks.

Please, for further comments or another IDE-hotplug-can!-cannot!-can-so!, let's
take this to linux-raid or #offtopic   ;)

Enough said.

-- 
................................................................
:   jakob@unthought.net   : And I see the elder races,         :
:.........................: putrid forms of man                :
:   Jakob Østergaard      : See him rise and claim the earth,  :
:        OZ9ABN           : his downfall is at hand.           :
:.........................:............{Konkhra}...............:
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Abysmal RAID 0 performance on 2.4.0-test10 for IDE?

[Tim Wright]

On Wed, Dec 27, 2000 at 04:23:43PM +0000, Paul Jakma wrote:
> On Tue, 26 Dec 2000, Ian Stirling wrote:
> 
> > The PCI bus can move around 130MB/sec,
> 
> in bursts yes, but sustained data bandwidth of PCI is a lot lower,
> maybe 30 to 50MB/s. And you won't get sustained RAID performance >
> sustained PCI performance.
> 

No. A well-designed card and driver doing cache-line sized transfers can
achieve ~100MB/s. On the IBM (Sequent) NUMA machines, we achieved in excess
of 3GB/s sustained read I/O (database full table scan) on a 16-quad (32 PCI
bus) system. That works out at around 100MB/s per bus.

Regards,

Tim

-- 
Tim Wright - timw@splhi.com or timw@aracnet.com or twright@us.ibm.com
IBM Linux Technology Center, Beaverton, Oregon
"Nobody ever said I was charming, they said "Rimmer, you're a git!"" RD VI
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/

Re: Abysmal RAID 0 performance on 2.4.0-test10 for IDE?

[Mathieu Chouquet-Stringer]

timw@splhi.com (Tim Wright) writes:

> On Wed, Dec 27, 2000 at 04:23:43PM +0000, Paul Jakma wrote:
> > On Tue, 26 Dec 2000, Ian Stirling wrote:
> > 
> > > The PCI bus can move around 130MB/sec,
> > 
> > in bursts yes, but sustained data bandwidth of PCI is a lot lower,
> > maybe 30 to 50MB/s. And you won't get sustained RAID performance >
> > sustained PCI performance.
> > 
> 
> No. A well-designed card and driver doing cache-line sized transfers can
> achieve ~100MB/s. On the IBM (Sequent) NUMA machines, we achieved in excess
> of 3GB/s sustained read I/O (database full table scan) on a 16-quad (32 PCI
> bus) system. That works out at around 100MB/s per bus.

Sadly, I am sure that your "well-designed" system must be costly as
hell... :(

-- 
Mathieu CHOUQUET-STRINGER              E-Mail : mchouque@e-steel.com
     Learning French is trivial: the word for horse is cheval, and
               everything else follows in the same way.
                        -- Alan J. Perlis
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
Please read the FAQ at http://www.tux.org/lkml/