* RAID1 VS RAID5
@ 2003-10-26 14:45 Mario Giammarco
2003-10-26 16:16 ` maarten van den Berg
` (2 more replies)
0 siblings, 3 replies; 20+ messages in thread
From: Mario Giammarco @ 2003-10-26 14:45 UTC (permalink / raw)
To: linux-raid
Hello,
I would like to buy new SCSI hard disk to be used with lsi 160
controller in 32 bit pci.
I can choose:
- RAID5 of three Maxtor Atlas 10K II 9gb
- RAID1 of two Maxtro Atlas 10K III 18gb
My problem is: I have seen that RAID1 code does not interleave reads so
it does not improve performance very much putting two hard disks.
Is there a patch to change raid1 behaviour?
In linux 2.6.0 is it better?
Thanks in advance for any reply.
--
Mario Giammarco
^ permalink raw reply [flat|nested] 20+ messages in thread* Re: RAID1 VS RAID5 2003-10-26 14:45 RAID1 VS RAID5 Mario Giammarco @ 2003-10-26 16:16 ` maarten van den Berg 2003-10-26 18:22 ` Mario Giammarco 2003-10-27 8:27 ` Hermann Himmelbauer 2003-10-26 16:55 ` Matti Aarnio 2003-10-27 8:33 ` Hermann Himmelbauer 2 siblings, 2 replies; 20+ messages in thread From: maarten van den Berg @ 2003-10-26 16:16 UTC (permalink / raw) To: Mario Giammarco, linux-raid On Sunday 26 October 2003 15:45, Mario Giammarco wrote: > Hello, > My problem is: I have seen that RAID1 code does not interleave reads so > it does not improve performance very much putting two hard disks. After thinking about your question for a minute, I think I found the obvious reason for that. Raid1 being a mirror set it does not make sense to interleave anything. Either disk1 reads it first or disk2 reads it first. Once you get the data from either disk, then you're done; no need to wait for the second disk (giving you the identical datablock). Interleaving only makes sense with other raid levels, but not with level 1. Maarten -- Yes of course I'm sure it's the red cable. I guarante[^%!/+)F#0c|'NO CARRIER ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: RAID1 VS RAID5 2003-10-26 16:16 ` maarten van den Berg @ 2003-10-26 18:22 ` Mario Giammarco 2003-10-27 8:27 ` Hermann Himmelbauer 1 sibling, 0 replies; 20+ messages in thread From: Mario Giammarco @ 2003-10-26 18:22 UTC (permalink / raw) To: maarten van den Berg; +Cc: linux-raid Il dom, 2003-10-26 alle 17:16, maarten van den Berg ha scritto: > Raid1 being a mirror set it does not make sense to > interleave anything. Either disk1 reads it first or disk2 reads it first. > Once you get the data from either disk, then you're done; no need to wait for > the second disk (giving you the identical datablock). OK I have some difficulties to explain it, I will try again: usually linux do read prefetch so when you read block "n" after it linux reads block "n+1". So the "n+1" block can be read at the same time on the other disk. Reading past mailing lists posts it seems that raid1 behaviour is this: 1) disk idle 2) request for block "n" 3) request is passed to hard disk with head nearer to block "n" (is it true? I am not sure) 4) request to block "n+1" "n+2" etc. are on the same disk so THE OTHER DISK IS READY FOR A READ REQUEST FROM OTHER PROCESSES 5) if sequential reading continue after some minutes the other disk is chosen to not stress too much only one disk (is it true?) Obviously this behaviour (point 4) helps servers with a lot of multitasking and processes. I prefer a "sequential" optimization. Can it be done? Thanks again! ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: RAID1 VS RAID5 2003-10-26 16:16 ` maarten van den Berg 2003-10-26 18:22 ` Mario Giammarco @ 2003-10-27 8:27 ` Hermann Himmelbauer 2003-10-27 9:54 ` Lars Marowsky-Bree ` (2 more replies) 1 sibling, 3 replies; 20+ messages in thread From: Hermann Himmelbauer @ 2003-10-27 8:27 UTC (permalink / raw) To: maarten van den Berg, Mario Giammarco, linux-raid On Sunday 26 October 2003 17:16, maarten van den Berg wrote: > On Sunday 26 October 2003 15:45, Mario Giammarco wrote: > > Hello, > > > > > > My problem is: I have seen that RAID1 code does not interleave reads so > > it does not improve performance very much putting two hard disks. > > After thinking about your question for a minute, I think I found the > obvious reason for that. Raid1 being a mirror set it does not make sense > to interleave anything. Either disk1 reads it first or disk2 reads it > first. Once you get the data from either disk, then you're done; no need to > wait for the second disk (giving you the identical datablock). > Interleaving only makes sense with other raid levels, but not with level 1. It seems you're missing something: Interleaving means here, reading one part of the data from one disk and another part from the other disk, not reading the same data from both. When reading a file from the RAID1, you could e.g. read the first block from the first disk, the second from the second disk, the third from the first disk and so on. This would *theoretically* double the read speed - like with RAID0. Practically this speed doubling would not occur as you have to add the seek times when reading files, but I assume with TCQ and things like that there could be some tricks to optimize the read behavior. Anyway - it seems no RAID1 implementation - be it hardware or software RAID1 - seems to make use of this read performance increase. Best Regards, Hermann -- x1@aon.at GPG key ID: 299893C7 (on keyservers) FP: 0124 2584 8809 EF2A DBF9 4902 64B4 D16B 2998 93C7 ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: RAID1 VS RAID5 2003-10-27 8:27 ` Hermann Himmelbauer @ 2003-10-27 9:54 ` Lars Marowsky-Bree 2003-10-27 10:16 ` Jeff Woods 2003-10-27 11:08 ` maarten van den Berg 2 siblings, 0 replies; 20+ messages in thread From: Lars Marowsky-Bree @ 2003-10-27 9:54 UTC (permalink / raw) To: Hermann Himmelbauer, maarten van den Berg, Mario Giammarco, linux-raid On 2003-10-27T09:27:32, Hermann Himmelbauer <dusty@strike.wu-wien.ac.at> said: > When reading a file from the RAID1, you could e.g. read the first block from > the first disk, the second from the second disk, the third from the first > disk and so on. > > This would *theoretically* double the read speed - like with RAID0. For pure reads, that may be true. But for writes, the disks have to resynchronize their heads and then you would get a penalty there. It all depends ;-) Sincerely, Lars Marowsky-Brée <lmb@suse.de> -- High Availability & Clustering \ ever tried. ever failed. no matter. SUSE Labs | try again. fail again. fail better. Research & Development, SUSE LINUX AG \ -- Samuel Beckett - To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: RAID1 VS RAID5 2003-10-27 8:27 ` Hermann Himmelbauer 2003-10-27 9:54 ` Lars Marowsky-Bree @ 2003-10-27 10:16 ` Jeff Woods 2003-10-28 10:45 ` Mario Giammarco 2003-10-27 11:08 ` maarten van den Berg 2 siblings, 1 reply; 20+ messages in thread From: Jeff Woods @ 2003-10-27 10:16 UTC (permalink / raw) To: Hermann Himmelbauer; +Cc: maarten van den Berg, Mario Giammarco, linux-raid At 10/27/2003 09:27 AM +0100, Hermann Himmelbauer wrote: >It seems you're missing something: Interleaving means here, reading one >part of the data from one disk and another part from the other disk, not >reading the same data from both. > >When reading a file from the RAID1, you could e.g. read the first block >from the first disk, the second from the second disk, the third from the >first disk and so on. > >This would *theoretically* double the read speed - like with RAID0. > >Practically this speed doubling would not occur as you have to add the >seek times when reading files, but I assume with TCQ and things like that >there could be some tricks to optimize the read behavior. > >Anyway - it seems no RAID1 implementation - be it hardware or software >RAID1 - seems to make use of this read performance increase. Disc I/O performance is significantly more complicated than you seem to perceive it. Disc drive performance (in stand-alone, JBOD and RAID configurations) depends on many factors such as spindle speed, number of sectors per track and cylinder (which varies substantially between inner and outer cylinders) and other things. However, serial reads are generally limited by the rotation speed (measured in sectors passing the head(s) per second) and seek times of the portion of the drive being accessed. As performance benchmarks that test inner versus outer cylinders generally show, the drives are much faster at both random seeks and sustained transfer rates on the "larger" outer cylinders. That's because more data is flying under the heads on each rotation and head seeks are needed less often (especially for sequential reads and/or writes!). Alternating I/Os between two spindles on a RAID1 pair won't (in general) speed up the transfer rate. Worse, it will needlessly monopolize both spindles when only one provides the same performance. Leaving the other spindle available to seek to another part of the drive and provide double the bandwidth has major advantages in a multiprocessing environment (including typical Linux, Unix and Windows environments... even single user "desktop" workstations). It sounds to me like the current algorithm used likely provides near-optimal performance for RAID1 configuration. OTOH, you have access to the source code if you really want to try it another way. But I strongly urge you to actually benchmark any changes you make to prove that you've made an improvement, especially before submitting it as a patch to the current code. -- Jeff Woods <kazrak+kernel@cesmail.net> ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: RAID1 VS RAID5 2003-10-27 10:16 ` Jeff Woods @ 2003-10-28 10:45 ` Mario Giammarco 0 siblings, 0 replies; 20+ messages in thread From: Mario Giammarco @ 2003-10-28 10:45 UTC (permalink / raw) To: Jeff Woods; +Cc: Hermann Himmelbauer, maarten van den Berg, linux-raid Il lun, 2003-10-27 alle 11:16, Jeff Woods ha scritto: > Alternating I/Os between two spindles on a RAID1 pair won't (in general) > speed up the transfer rate. Are you sure? Have you done some tests? Because I think the opposite. > Leaving the other > spindle available to seek to another part of the drive and provide double > the bandwidth has major advantages in a multiprocessing environment Is good but I do not need it... > OTOH, you have access to the source code if you really want to try it > another way. So again my first question thats seems to go ignored: does someone have made a patch? Is there some line to put in raidtab to change raid1 behavior? -- Mario Giammarco ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: RAID1 VS RAID5 2003-10-27 8:27 ` Hermann Himmelbauer 2003-10-27 9:54 ` Lars Marowsky-Bree 2003-10-27 10:16 ` Jeff Woods @ 2003-10-27 11:08 ` maarten van den Berg 2003-10-27 12:03 ` Jeff Woods 2 siblings, 1 reply; 20+ messages in thread From: maarten van den Berg @ 2003-10-27 11:08 UTC (permalink / raw) To: linux-raid On Monday 27 October 2003 09:27, Hermann Himmelbauer wrote: > On Sunday 26 October 2003 17:16, maarten van den Berg wrote: > > On Sunday 26 October 2003 15:45, Mario Giammarco wrote: > > > Hello, > > > > > > > > > My problem is: I have seen that RAID1 code does not interleave reads so > > > it does not improve performance very much putting two hard disks. > > > > After thinking about your question for a minute, I think I found the > > obvious reason for that. Raid1 being a mirror set it does not make sense > > to interleave anything. Either disk1 reads it first or disk2 reads it > > first. Once you get the data from either disk, then you're done; no need > > to wait for the second disk (giving you the identical datablock). > > Interleaving only makes sense with other raid levels, but not with level > > 1. > > It seems you're missing something: Interleaving means here, reading one > part of the data from one disk and another part from the other disk, not > reading the same data from both. I know what interleaving means; I just think it is counterproductive in Raid1. > When reading a file from the RAID1, you could e.g. read the first block > from the first disk, the second from the second disk, the third from the > first disk and so on. The way it works now, AFAIK, is that concurrent read commands are issued to all the drives in Raid1. Because of seek times and the amount of rotation needed to get the right sector underneath the head(s), one of the disks will be first to transfer the data. This can be 'sooner' by a large amount. If you instead want to interleave reads you lose the possibility to gain speed from this mechanism described above. It would be a trade-off and I'm not sure interleaving would come out as the winner here. Interleaving comes from for instance RAM access. But a RAM chip has to 'recover' from a read, i.e. it needs a little bit of time in between each read. Interleaving there is very sensible since you can let one bank 'rest' while the other bank 'works'. For disks, this is not so; once the right sector is under the head the hard work is done; reading on onto following or adjacent sectors happens at or near the theoretical full speed. If you switch drives after 128 blocks you slow your transfer, not speed it up. This will depend on many many factors like how large a read it is and how scattered the data is across the platter. But, maybe I'm way off base here and new insights have changed this. :-) Greetings, Maarten > This would *theoretically* double the read speed - like with RAID0. > > Practically this speed doubling would not occur as you have to add the seek > times when reading files, but I assume with TCQ and things like that there > could be some tricks to optimize the read behavior. > > Anyway - it seems no RAID1 implementation - be it hardware or software > RAID1 - seems to make use of this read performance increase. > > Best Regards, > Hermann -- Yes of course I'm sure it's the red cable. I guarante[^%!/+)F#0c|'NO CARRIER ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: RAID1 VS RAID5 2003-10-27 11:08 ` maarten van den Berg @ 2003-10-27 12:03 ` Jeff Woods 0 siblings, 0 replies; 20+ messages in thread From: Jeff Woods @ 2003-10-27 12:03 UTC (permalink / raw) To: maarten van den Berg; +Cc: linux-raid maarten van den Berg wrote (in part): >The way it works now, AFAIK, is that concurrent read commands are issued >to all the drives in Raid1. Because of seek times and the amount of >rotation needed to get the right sector underneath the head(s), one of the >disks will be first to transfer the data. This can be 'sooner' by a large >amount. > >If you instead want to interleave reads you lose the possibility to gain >speed from this mechanism described above. It would be a trade-off and I'm >not sure interleaving would come out as the winner here. IIUC, neither duplicate reads nor interleaved reads closely matches how (Linux software) RAID1 works. I'm not sure what the local protocol is (and if I'm stepping way out of line here, someone please correct me offlist), but I'm going to quote another post sent via this listserver last month: >>From: "Bailey, Scott" <scott.bailey@eds.com> >>To: "'Andrew Herdman'" <andrew@whine.com> >>Cc: linux-raid@vger.kernel.org >>Subject: RE: RAID1: Disks alternating on reads >>Date: Tue, 23 Sep 2003 11:12:29 -0400 >> >>>Pardon me if my assumption was incorrect, but I was under the belief >>>that when using software RAID1, that when reads occurred on the RAID >>>device that it would read from both drives in a striped fashion similar >>>to how RAID0 works to improve the speed of the md devices. I am >>>actually seeing this, but it appears that the reads on each drive are >>>continuing for 10 to 20 seconds before moving onto the next drive and >>>then another 10-20 seconds and back again. This is not allowing for any >>>performance increase, it just lets the drives rest alternately. >> >>My impression from crawling through the code a few months back was that >>this behavior was a design feature. On a read request, the request is >>sent to the device with heads "closest" to the target sector, EXCEPT that >>for sequential reads, the device gets reused on the assumption that the >>disk can be kept streaming, EXCEPT that after some maximum number of >>reads, another drive gets chosen to give the previous device a rest. >> >>I speculate your files are contiguous (or nearly so) with the result that >>the initially closest drive gets hammered until its 'work quota' gets >>exceeded, and then the next drive will get pounded. And so on. To test >>this, I think you could reduce MAX_WORK_PER_DISK in raid1.c from the >>default 128 (in 2.4.21) to something smaller (perhaps 8 sectors for a >>filesystem with 4K block size?) and see if the load evens out. >> >>Good luck, >> >> Scott Bailey >> scott.bailey at eds.com I believe the method Scott says the present code uses to be near optimal and am skeptical that arbitrary changes to it would be improvements. Of course, someone actually making such changes and then multiple people benchmarking it in a variety of different, well-designed, controlled tests is the only way to really tell. -- Jeff Woods <kazrak+kernel@cesmail.net> ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: RAID1 VS RAID5 2003-10-26 14:45 RAID1 VS RAID5 Mario Giammarco 2003-10-26 16:16 ` maarten van den Berg @ 2003-10-26 16:55 ` Matti Aarnio 2003-10-28 10:46 ` Mario Giammarco 2003-10-27 8:33 ` Hermann Himmelbauer 2 siblings, 1 reply; 20+ messages in thread From: Matti Aarnio @ 2003-10-26 16:55 UTC (permalink / raw) To: Mario Giammarco; +Cc: linux-raid On Sun, Oct 26, 2003 at 03:45:19PM +0100, Mario Giammarco wrote: > Hello, > I would like to buy new SCSI hard disk to be used with lsi 160 > controller in 32 bit pci. > > I can choose: > > - RAID5 of three Maxtor Atlas 10K II 9gb > - RAID1 of two Maxtro Atlas 10K III 18gb > > My problem is: I have seen that RAID1 code does not interleave reads so > it does not improve performance very much putting two hard disks. Performance boosting isn't primary goal in RAID schemes involving data replication. For RAID1 a suitably chosen read-interleave could be used to improve _large_ file reads, of course. Another strategy is to distribute read operations to all online disks forming up the RAID1 set with elevators of their own. > Is there a patch to change raid1 behaviour? If it feels important, such can be made. > In linux 2.6.0 is it better? It can be made, if it isn't. > Thanks in advance for any reply. > -- > Mario Giammarco /Matti Aarnio ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: RAID1 VS RAID5 2003-10-26 16:55 ` Matti Aarnio @ 2003-10-28 10:46 ` Mario Giammarco 0 siblings, 0 replies; 20+ messages in thread From: Mario Giammarco @ 2003-10-28 10:46 UTC (permalink / raw) To: Matti Aarnio; +Cc: linux-raid Il dom, 2003-10-26 alle 17:55, Matti Aarnio ha scritto: > Performance boosting isn't primary goal in RAID schemes involving > data replication. > RAID5 offers a nice performance boosts. > For RAID1 a suitably chosen read-interleave could be used to improve > _large_ file reads, of course. Another strategy is to distribute read > operations to all online disks forming up the RAID1 set with elevators > of their own. > > > Is there a patch to change raid1 behaviour? > > If it feels important, such can be made. > Who will make it? > > In linux 2.6.0 is it better? > > It can be made, if it isn't. So? 2.6.0 have different raid1 code? -- Mario Giammarco.it ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: RAID1 VS RAID5 2003-10-26 14:45 RAID1 VS RAID5 Mario Giammarco 2003-10-26 16:16 ` maarten van den Berg 2003-10-26 16:55 ` Matti Aarnio @ 2003-10-27 8:33 ` Hermann Himmelbauer 2003-10-27 9:19 ` Gordon Henderson 2003-10-28 10:40 ` Mario Giammarco 2 siblings, 2 replies; 20+ messages in thread From: Hermann Himmelbauer @ 2003-10-27 8:33 UTC (permalink / raw) To: Mario Giammarco, linux-raid On Sunday 26 October 2003 15:45, Mario Giammarco wrote: > Hello, > I would like to buy new SCSI hard disk to be used with lsi 160 > controller in 32 bit pci. > > I can choose: > > - RAID5 of three Maxtor Atlas 10K II 9gb > - RAID1 of two Maxtro Atlas 10K III 18gb > > My problem is: I have seen that RAID1 code does not interleave reads so > it does not improve performance very much putting two hard disks. You only have to keep in mind that the RAID5 CPU overhead is larger than the one of RAID1. My experience is that software RAID5 is quite slow. Best Regards, Hermann -- x1@aon.at GPG key ID: 299893C7 (on keyservers) FP: 0124 2584 8809 EF2A DBF9 4902 64B4 D16B 2998 93C7 ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: RAID1 VS RAID5 2003-10-27 8:33 ` Hermann Himmelbauer @ 2003-10-27 9:19 ` Gordon Henderson 2003-10-27 11:01 ` Hermann Himmelbauer 2003-10-28 10:40 ` Mario Giammarco 1 sibling, 1 reply; 20+ messages in thread From: Gordon Henderson @ 2003-10-27 9:19 UTC (permalink / raw) To: linux-raid On Mon, 27 Oct 2003, Hermann Himmelbauer wrote: > My experience is that software RAID5 is quite slow. My experiences are the opposite to yours I'm afraid - I've not found it any slower than a single drive and in some cases a lot faster! A lot depends on exactly what you are doing with it though, but I'm willing to sacrifice some speed for data integrity. Most of my systems are network servers with 100Mb Network cards fitted - as long as my disk systems are faster than 12.5MB/sec I'm happy. In practice I can stream 50MB/sec+ out of some simple RAID5 IDE systems I have. Gordon ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: RAID1 VS RAID5 2003-10-27 9:19 ` Gordon Henderson @ 2003-10-27 11:01 ` Hermann Himmelbauer 2003-10-27 13:40 ` Gordon Henderson ` (2 more replies) 0 siblings, 3 replies; 20+ messages in thread From: Hermann Himmelbauer @ 2003-10-27 11:01 UTC (permalink / raw) To: Gordon Henderson, linux-raid On Monday 27 October 2003 10:19, Gordon Henderson wrote: > On Mon, 27 Oct 2003, Hermann Himmelbauer wrote: > > My experience is that software RAID5 is quite slow. > > My experiences are the opposite to yours I'm afraid - I've not found it > any slower than a single drive and in some cases a lot faster! > > A lot depends on exactly what you are doing with it though, but I'm > willing to sacrifice some speed for data integrity. > > Most of my systems are network servers with 100Mb Network cards fitted - > as long as my disk systems are faster than 12.5MB/sec I'm happy. In > practice I can stream 50MB/sec+ out of some simple RAID5 IDE systems I > have. Well - I have an old Dual P-II-266 System with an onboard SCSI-Controller with 3 Ultra SCSI-disks connected, building a RAID5. I did a simple Test with "hdparm -tT" to provide you with numbers: /dev/sdb: Timing buffer-cache reads: 128 MB in 1.46 seconds = 87.67 MB/sec Timing buffered disk reads: 64 MB in 5.07 seconds = 12.62 MB/sec /dev/sdc: Timing buffer-cache reads: 128 MB in 1.47 seconds = 87.07 MB/sec Timing buffered disk reads: 64 MB in 4.78 seconds = 13.39 MB/sec /dev/sdd: Timing buffer-cache reads: 128 MB in 1.49 seconds = 85.91 MB/sec Timing buffered disk reads: 64 MB in 5.05 seconds = 12.67 MB/sec So you see, the seperate disks achieve ~ 13MB/s. My RAID5 raidtab looks like this: raiddev /dev/md0 raid-level 5 nr-raid-disks 3 nr-spare-disks 0 chunk-size 4 persistent-superblock 1 parity-algorithm left-symmetric device /dev/sdb2 raid-disk 0 device /dev/sdc2 raid-disk 1 device /dev/sdd2 raid-disk 2 And "hdparm -tT" looks like this: /dev/md0: Timing buffer-cache reads: 128 MB in 1.45 seconds = 88.28 MB/sec Timing buffered disk reads: 64 MB in 13.85 seconds = 4.62 MB/sec So this is ~ 1/3rd of the read performance of a single disk. And this is what a appr. measure when copying files etc. My kernel version is 2.4.20 and the CPU-Load during the hdparm test is only at ~ 30%. Best Regards, Hermann -- x1@aon.at GPG key ID: 299893C7 (on keyservers) FP: 0124 2584 8809 EF2A DBF9 4902 64B4 D16B 2998 93C7 ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: RAID1 VS RAID5 2003-10-27 11:01 ` Hermann Himmelbauer @ 2003-10-27 13:40 ` Gordon Henderson 2003-10-27 15:34 ` Hermann Himmelbauer 2003-10-27 14:17 ` Lars Marowsky-Bree 2003-10-27 15:52 ` Andrew Herdman 2 siblings, 1 reply; 20+ messages in thread From: Gordon Henderson @ 2003-10-27 13:40 UTC (permalink / raw) To: Hermann Himmelbauer; +Cc: linux-raid On Mon, 27 Oct 2003, Hermann Himmelbauer wrote: > Well - I have an old Dual P-II-266 System with an onboard SCSI-Controller with > 3 Ultra SCSI-disks connected, building a RAID5. I did a simple Test with > "hdparm -tT" to provide you with numbers: > > /dev/sdb: > Timing buffer-cache reads: 128 MB in 1.46 seconds = 87.67 MB/sec > Timing buffered disk reads: 64 MB in 5.07 seconds = 12.62 MB/sec It's possibly the "old Dual P-II-266" that may be slowing things down here. On one of my systems: (Dual Athlon 2.4 with 2 Promise PCI IDE cards and 4 drives): From /proc/mdstat md4 : active raid5 hdk6[3] hdi6[1] hdg6[2] hde6[0] 176055936 blocks level 5, 32k chunk, algorithm 0 [4/4] [UUUU] Single drive: /dev/hdg6: Timing buffer-cache reads: 128 MB in 0.48 seconds =266.67 MB/sec Timing buffered disk reads: 64 MB in 1.49 seconds = 42.95 MB/sec The array: /dev/md4: Timing buffer-cache reads: 128 MB in 0.50 seconds =256.00 MB/sec Timing buffered disk reads: 64 MB in 0.97 seconds = 65.98 MB/sec On another server (Dual PIII/Xeon 700MHz with SCSI drives) md4 : active raid5 sdd6[3] sdc6[1] sdb6[2] sda6[0] 23663424 blocks level 5, 32k chunk, algorithm 0 [4/4] [UUUU] Single drive: /dev/sdc6: Timing buffer-cache reads: 128 MB in 0.59 seconds =216.95 MB/sec Timing buffered disk reads: 64 MB in 3.39 seconds = 18.88 MB/sec The array: /dev/md4: Timing buffer-cache reads: 128 MB in 0.57 seconds =224.56 MB/sec Timing buffered disk reads: 64 MB in 0.75 seconds = 85.33 MB/sec The array in this case is 4 disks, connected 2 disks to an on-board dual Adaptec controller (It's a Dell 6xxx box) It also has 2 PCI adaptec 7xxx cards connected to an external box with 8 drives: md6 : active raid5 sdl2[7] sdh2[6] sdk2[5] sdg2[4] sdj2[3] sdf2[2] sdi2[1] sde2[0] 62918016 blocks level 5, 32k chunk, algorithm 0 [8/8] [UUUUUUUU] Single drive: /dev/sdk2: Timing buffer-cache reads: 128 MB in 0.58 seconds =220.69 MB/sec Timing buffered disk reads: 64 MB in 4.00 seconds = 16.00 MB/sec The array: /dev/md6: Timing buffer-cache reads: 128 MB in 0.57 seconds =224.56 MB/sec Timing buffered disk reads: 64 MB in 1.03 seconds = 62.14 MB/sec fast RAID array! So my advice is that if you want speed, be prepared to spend the £££ to get that speed (and I don't consider these servers particularly fast, but they are fast enough for my application which is NFS & Samba serving a small company of engineers (software/hardware) via a single 100MB/sec Ethernet interface). Gordon - To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: RAID1 VS RAID5 2003-10-27 13:40 ` Gordon Henderson @ 2003-10-27 15:34 ` Hermann Himmelbauer 0 siblings, 0 replies; 20+ messages in thread From: Hermann Himmelbauer @ 2003-10-27 15:34 UTC (permalink / raw) To: Gordon Henderson; +Cc: linux-raid On Monday 27 October 2003 14:40, Gordon Henderson wrote: > On Mon, 27 Oct 2003, Hermann Himmelbauer wrote: > > Well - I have an old Dual P-II-266 System with an onboard SCSI-Controller > > with 3 Ultra SCSI-disks connected, building a RAID5. I did a simple Test > > with "hdparm -tT" to provide you with numbers: > > > > /dev/sdb: > > Timing buffer-cache reads: 128 MB in 1.46 seconds = 87.67 MB/sec > > Timing buffered disk reads: 64 MB in 5.07 seconds = 12.62 MB/sec > > It's possibly the "old Dual P-II-266" that may be slowing things down > here. I also thought this at first. But looking at the system load with e.g. "top" or a graphic CPU-monitor the CPU load is only ~ 15%. When reading from a single drive, the load is higher, something like 25%. Moreover the most CPU-hungry application is the "hdparm"-Utility. The raid5d uses only ~ 2%. > On one of my systems: (Dual Athlon 2.4 with 2 Promise PCI IDE cards and 4 > drives): > /dev/hdg6: > Timing buffer-cache reads: 128 MB in 0.48 seconds =266.67 MB/sec > Timing buffered disk reads: 64 MB in 1.49 seconds = 42.95 MB/sec > /dev/md4: > Timing buffer-cache reads: 128 MB in 0.50 seconds =256.00 MB/sec > Timing buffered disk reads: 64 MB in 0.97 seconds = 65.98 MB/sec > > On another server (Dual PIII/Xeon 700MHz with SCSI drives) > /dev/sdc6: > Timing buffer-cache reads: 128 MB in 0.59 seconds =216.95 MB/sec > Timing buffered disk reads: 64 MB in 3.39 seconds = 18.88 MB/sec > /dev/md4: > Timing buffer-cache reads: 128 MB in 0.57 seconds =224.56 MB/sec > Timing buffered disk reads: 64 MB in 0.75 seconds = 85.33 MB/sec That's interesting: both systems are 4-disk RAID5 arrays. The first gains ~50% read performance, the second gains ~ 470% (!) - probably there is caching involved. Probably hdparm is not the best tool for benchmarking RAIDs... > So my advice is that if you want speed, be prepared to spend the £££ to > get that speed (and I don't consider these servers particularly fast, but > they are fast enough for my application which is NFS & Samba serving a > small company of engineers (software/hardware) via a single 100MB/sec > Ethernet interface). Well, 5MB/s are o.k. for my server and my application - anyway it's interesting why this happens. Best Regards, Hermann -- x1@aon.at GPG key ID: 299893C7 (on keyservers) FP: 0124 2584 8809 EF2A DBF9 4902 64B4 D16B 2998 93C7 - To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: RAID1 VS RAID5 2003-10-27 11:01 ` Hermann Himmelbauer 2003-10-27 13:40 ` Gordon Henderson @ 2003-10-27 14:17 ` Lars Marowsky-Bree 2003-10-27 15:52 ` Andrew Herdman 2 siblings, 0 replies; 20+ messages in thread From: Lars Marowsky-Bree @ 2003-10-27 14:17 UTC (permalink / raw) To: Hermann Himmelbauer, Gordon Henderson, linux-raid On 2003-10-27T12:01:10, Hermann Himmelbauer <dusty@strike.wu-wien.ac.at> said: For reads, raid5 or not should not actually matter much, at least not negatively. The performance impact hits on write. Sincerely, Lars Marowsky-Brée <lmb@suse.de> -- High Availability & Clustering \ ever tried. ever failed. no matter. SUSE Labs | try again. fail again. fail better. Research & Development, SUSE LINUX AG \ -- Samuel Beckett - To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: RAID1 VS RAID5 2003-10-27 11:01 ` Hermann Himmelbauer 2003-10-27 13:40 ` Gordon Henderson 2003-10-27 14:17 ` Lars Marowsky-Bree @ 2003-10-27 15:52 ` Andrew Herdman 2 siblings, 0 replies; 20+ messages in thread From: Andrew Herdman @ 2003-10-27 15:52 UTC (permalink / raw) To: linux-raid My P4-2.4GHz with 3 WD80G 8MB caches does significantly better with RAID5 hdparm -tT /dev/md/3 /dev/md/3: Timing buffer-cache reads: 1844 MB in 2.00 seconds = 921.08 MB/sec Timing buffered disk reads: 166 MB in 3.00 seconds = 55.26 MB/sec md3 : active raid5 ide/host4/bus0/target0/lun0/part3[1] ide/host2/bus1/target0/lun0/part3[2] ide/host2/bus0/target0/lun0/part3[0] 143713280 blocks level 5, 64k chunk, algorithm 2 [3/3] [UUU] Linux why 2.4.22-ck2-blackbox-aph-21 #1 Wed Sep 17 09:41:14 EDT 2003 i686 GNU/Linux This kernel also has the low latency and preempt patches applied and running at 500hz. Andrew ----- Original Message ----- From: "Hermann Himmelbauer" <dusty@strike.wu-wien.ac.at> To: "Gordon Henderson" <gordon@drogon.net>; <linux-raid@vger.kernel.org> Sent: Monday, October 27, 2003 6:01 AM Subject: Re: RAID1 VS RAID5 > On Monday 27 October 2003 10:19, Gordon Henderson wrote: > > On Mon, 27 Oct 2003, Hermann Himmelbauer wrote: > > > My experience is that software RAID5 is quite slow. > > > > My experiences are the opposite to yours I'm afraid - I've not found it > > any slower than a single drive and in some cases a lot faster! > > > > A lot depends on exactly what you are doing with it though, but I'm > > willing to sacrifice some speed for data integrity. > > > > Most of my systems are network servers with 100Mb Network cards fitted - > > as long as my disk systems are faster than 12.5MB/sec I'm happy. In > > practice I can stream 50MB/sec+ out of some simple RAID5 IDE systems I > > have. > > Well - I have an old Dual P-II-266 System with an onboard SCSI-Controller with > 3 Ultra SCSI-disks connected, building a RAID5. I did a simple Test with > "hdparm -tT" to provide you with numbers: > > /dev/sdb: > Timing buffer-cache reads: 128 MB in 1.46 seconds = 87.67 MB/sec > Timing buffered disk reads: 64 MB in 5.07 seconds = 12.62 MB/sec > > /dev/sdc: > Timing buffer-cache reads: 128 MB in 1.47 seconds = 87.07 MB/sec > Timing buffered disk reads: 64 MB in 4.78 seconds = 13.39 MB/sec > > /dev/sdd: > Timing buffer-cache reads: 128 MB in 1.49 seconds = 85.91 MB/sec > Timing buffered disk reads: 64 MB in 5.05 seconds = 12.67 MB/sec > > So you see, the seperate disks achieve ~ 13MB/s. My RAID5 raidtab looks like > this: > raiddev /dev/md0 > raid-level 5 > nr-raid-disks 3 > nr-spare-disks 0 > chunk-size 4 > persistent-superblock 1 > parity-algorithm left-symmetric > device /dev/sdb2 > raid-disk 0 > device /dev/sdc2 > raid-disk 1 > device /dev/sdd2 > raid-disk 2 > > And "hdparm -tT" looks like this: > > /dev/md0: > Timing buffer-cache reads: 128 MB in 1.45 seconds = 88.28 MB/sec > Timing buffered disk reads: 64 MB in 13.85 seconds = 4.62 MB/sec > > So this is ~ 1/3rd of the read performance of a single disk. And this is what > a appr. measure when copying files etc. > > My kernel version is 2.4.20 and the CPU-Load during the hdparm test is only at > ~ 30%. > > Best Regards, > Hermann > > -- > x1@aon.at > GPG key ID: 299893C7 (on keyservers) > FP: 0124 2584 8809 EF2A DBF9 4902 64B4 D16B 2998 93C7 > > - > To unsubscribe from this list: send the line "unsubscribe linux-raid" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: RAID1 VS RAID5 2003-10-27 8:33 ` Hermann Himmelbauer 2003-10-27 9:19 ` Gordon Henderson @ 2003-10-28 10:40 ` Mario Giammarco 1 sibling, 0 replies; 20+ messages in thread From: Mario Giammarco @ 2003-10-28 10:40 UTC (permalink / raw) To: Hermann Himmelbauer; +Cc: linux-raid Il lun, 2003-10-27 alle 09:33, Hermann Himmelbauer ha scritto: > My experience is that software RAID5 is quite slow. put chunk size 128 in your raidtab ^ permalink raw reply [flat|nested] 20+ messages in thread
* RAID1 VS RAID5 @ 2003-10-26 11:24 Mario Giammarco 0 siblings, 0 replies; 20+ messages in thread From: Mario Giammarco @ 2003-10-26 11:24 UTC (permalink / raw) To: linux-raid Hello, I would like to buy new SCSI hard disk to be used with lsi 160 controller in 32 bit pci. I can choose: - RAID5 of three Maxtor Atlas 10K II 9gb - RAID1 of two Maxtro Atlas 10K III 18gb My problem is: I have seen that RAID1 code does not interleave reads so it does not improve performance very much putting two hard disks. Is there a patch to change raid1 behaviour? In linux 2.6.0 is it better? Thanks in advance for any reply. -- Mario Giammarco ^ permalink raw reply [flat|nested] 20+ messages in thread
end of thread, other threads:[~2003-10-28 10:46 UTC | newest] Thread overview: 20+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2003-10-26 14:45 RAID1 VS RAID5 Mario Giammarco 2003-10-26 16:16 ` maarten van den Berg 2003-10-26 18:22 ` Mario Giammarco 2003-10-27 8:27 ` Hermann Himmelbauer 2003-10-27 9:54 ` Lars Marowsky-Bree 2003-10-27 10:16 ` Jeff Woods 2003-10-28 10:45 ` Mario Giammarco 2003-10-27 11:08 ` maarten van den Berg 2003-10-27 12:03 ` Jeff Woods 2003-10-26 16:55 ` Matti Aarnio 2003-10-28 10:46 ` Mario Giammarco 2003-10-27 8:33 ` Hermann Himmelbauer 2003-10-27 9:19 ` Gordon Henderson 2003-10-27 11:01 ` Hermann Himmelbauer 2003-10-27 13:40 ` Gordon Henderson 2003-10-27 15:34 ` Hermann Himmelbauer 2003-10-27 14:17 ` Lars Marowsky-Bree 2003-10-27 15:52 ` Andrew Herdman 2003-10-28 10:40 ` Mario Giammarco -- strict thread matches above, loose matches on Subject: below -- 2003-10-26 11:24 Mario Giammarco
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).