* 2.6 device mapper performance? @ 2006-03-09 0:49 Ken Hwang 2006-03-10 16:59 ` Ming Zhang 0 siblings, 1 reply; 9+ messages in thread From: Ken Hwang @ 2006-03-09 0:49 UTC (permalink / raw) To: dm-devel Hi, I'm not sure I should ask this question here, if this is not the place then I apologize. I have a home made NAS with Linux. I use evms to create volume, put xfs on top of it, and then use samba to share it with Windows clients. When I was using kernel 2.4 with all the needed patches I could get netbench 106Mbps with 4 clients, and 95Mbps with 8 clients. Recently I upgraded the same hardware to kernel 2.6 (I also upgraded the related application such as samba, xfs utility, and dmsetup accordingly). Then I ran netbench again and got 80Mbps with 4 clients, 53Mbps with 8 clients. Which drop almost 80% (95 vs 53) in 8 clients case. I then make and mount xfs on another raid5 (which uses the same disk but on different partitions) and found it got better performance (95Mbps with 4 clients, 85Mbps with 8 clients). In brief: xfs volume on raid5 md/md1 on sda6/sdb6/sdc6/sdd6 netbench: 95/85Mbps xfs volume on EVMS volume /dev/evms/volume1 on raid5 md/md3 on sda8/sdb8/sdc8/sdd8: 80/53Mbps Do you think the slow down (85 to 53Mbps) was caused by device mapper? Please advice. Ken -- No virus found in this outgoing message. Checked by AVG Free Edition. Version: 7.1.375 / Virus Database: 268.2.0/276 - Release Date: 3/7/2006 ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: 2.6 device mapper performance? 2006-03-09 0:49 2.6 device mapper performance? Ken Hwang @ 2006-03-10 16:59 ` Ming Zhang 2006-03-10 17:11 ` Ming Zhang ` (2 more replies) 0 siblings, 3 replies; 9+ messages in thread From: Ming Zhang @ 2006-03-10 16:59 UTC (permalink / raw) To: ken.hwang, device-mapper development u have too many chances here. so hard to blame any one. suggest u to test it one by one if possible. for example, have same box run 2.4 and 2.6, test performance on volume first before run xfs, and samba. ming On Wed, 2006-03-08 at 16:49 -0800, Ken Hwang wrote: > Hi, > > I'm not sure I should ask this question here, if this is not the place then > I apologize. I have a home made NAS with Linux. I use evms to create volume, > put xfs on top of it, and then use samba to share it with Windows clients. > When I was using kernel 2.4 with all the needed patches I could get netbench > 106Mbps with 4 clients, and 95Mbps with 8 clients. Recently I upgraded the > same hardware to kernel 2.6 (I also upgraded the related application such as > samba, xfs utility, and dmsetup accordingly). Then I ran netbench again and > got 80Mbps with 4 clients, 53Mbps with 8 clients. Which drop almost 80% (95 > vs 53) in 8 clients case. > > I then make and mount xfs on another raid5 (which uses the same disk but on > different partitions) and found it got better performance (95Mbps with 4 > clients, 85Mbps with 8 clients). In brief: > xfs volume on raid5 md/md1 on sda6/sdb6/sdc6/sdd6 netbench: 95/85Mbps > xfs volume on EVMS volume /dev/evms/volume1 on raid5 md/md3 on > sda8/sdb8/sdc8/sdd8: 80/53Mbps > > Do you think the slow down (85 to 53Mbps) was caused by device mapper? > Please advice. > > Ken > > -- > No virus found in this outgoing message. > Checked by AVG Free Edition. > Version: 7.1.375 / Virus Database: 268.2.0/276 - Release Date: 3/7/2006 > > -- > dm-devel mailing list > dm-devel@redhat.com > https://www.redhat.com/mailman/listinfo/dm-devel ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: 2.6 device mapper performance? 2006-03-10 16:59 ` Ming Zhang @ 2006-03-10 17:11 ` Ming Zhang 2006-03-10 20:01 ` Ken Hwang 2006-03-11 5:58 ` Ken Hwang 2 siblings, 0 replies; 9+ messages in thread From: Ming Zhang @ 2006-03-10 17:11 UTC (permalink / raw) To: ken.hwang; +Cc: device-mapper development On Fri, 2006-03-10 at 11:59 -0500, Ming Zhang wrote: > u have too many chances here. so hard to blame any one. suggest u to > test it one by one if possible. sorry, typo. should be "too many changes" > > for example, have same box run 2.4 and 2.6, test performance on volume > first before run xfs, and samba. > > ming > > On Wed, 2006-03-08 at 16:49 -0800, Ken Hwang wrote: > > Hi, > > > > I'm not sure I should ask this question here, if this is not the place then > > I apologize. I have a home made NAS with Linux. I use evms to create volume, > > put xfs on top of it, and then use samba to share it with Windows clients. > > When I was using kernel 2.4 with all the needed patches I could get netbench > > 106Mbps with 4 clients, and 95Mbps with 8 clients. Recently I upgraded the > > same hardware to kernel 2.6 (I also upgraded the related application such as > > samba, xfs utility, and dmsetup accordingly). Then I ran netbench again and > > got 80Mbps with 4 clients, 53Mbps with 8 clients. Which drop almost 80% (95 > > vs 53) in 8 clients case. > > > > I then make and mount xfs on another raid5 (which uses the same disk but on > > different partitions) and found it got better performance (95Mbps with 4 > > clients, 85Mbps with 8 clients). In brief: > > xfs volume on raid5 md/md1 on sda6/sdb6/sdc6/sdd6 netbench: 95/85Mbps > > xfs volume on EVMS volume /dev/evms/volume1 on raid5 md/md3 on > > sda8/sdb8/sdc8/sdd8: 80/53Mbps > > > > Do you think the slow down (85 to 53Mbps) was caused by device mapper? > > Please advice. > > > > Ken > > > > -- > > No virus found in this outgoing message. > > Checked by AVG Free Edition. > > Version: 7.1.375 / Virus Database: 268.2.0/276 - Release Date: 3/7/2006 > > > > -- > > dm-devel mailing list > > dm-devel@redhat.com > > https://www.redhat.com/mailman/listinfo/dm-devel > ^ permalink raw reply [flat|nested] 9+ messages in thread
* RE: 2.6 device mapper performance? 2006-03-10 16:59 ` Ming Zhang 2006-03-10 17:11 ` Ming Zhang @ 2006-03-10 20:01 ` Ken Hwang 2006-03-10 20:08 ` Ming Zhang 2006-03-11 5:58 ` Ken Hwang 2 siblings, 1 reply; 9+ messages in thread From: Ken Hwang @ 2006-03-10 20:01 UTC (permalink / raw) To: mingz, device-mapper development Hi Ming, Thanks and I ran another block device read test by doing: sync; time dd if=/dev/evms/.nodes/md/md3 of=/dev/null bs=1024k count=1024 AND sync; time dd if=/dev/evms/volume1 of=/dev/null bs=1024k count=1024 the /dev/evms/volume1 is a evms volume on top of /dev/evms/.nodes/md/md3 And the results are (real/user/sys): /dev/evms/.nodes/md/md3: 29.757s/0.020s/15.861s /dev/evms/volume1: 41.697s/0.004s/25.586s /dev/evms/volume1 took 12 seconds more to read the same amount of blocks. Compare to 29.757 seconds it's about 40% more. Can we said it's introduced by device mapper? Ken -----Original Message----- From: Ming Zhang [mailto:mingz@ele.uri.edu] Sent: Friday, March 10, 2006 8:59 AM To: ken.hwang@zyxel.com; device-mapper development Subject: Re: [dm-devel] 2.6 device mapper performance? u have too many chances here. so hard to blame any one. suggest u to test it one by one if possible. for example, have same box run 2.4 and 2.6, test performance on volume first before run xfs, and samba. ming On Wed, 2006-03-08 at 16:49 -0800, Ken Hwang wrote: > Hi, > > I'm not sure I should ask this question here, if this is not the place then > I apologize. I have a home made NAS with Linux. I use evms to create volume, > put xfs on top of it, and then use samba to share it with Windows clients. > When I was using kernel 2.4 with all the needed patches I could get netbench > 106Mbps with 4 clients, and 95Mbps with 8 clients. Recently I upgraded the > same hardware to kernel 2.6 (I also upgraded the related application such as > samba, xfs utility, and dmsetup accordingly). Then I ran netbench again and > got 80Mbps with 4 clients, 53Mbps with 8 clients. Which drop almost 80% (95 > vs 53) in 8 clients case. > > I then make and mount xfs on another raid5 (which uses the same disk but on > different partitions) and found it got better performance (95Mbps with 4 > clients, 85Mbps with 8 clients). In brief: > xfs volume on raid5 md/md1 on sda6/sdb6/sdc6/sdd6 netbench: 95/85Mbps > xfs volume on EVMS volume /dev/evms/volume1 on raid5 md/md3 on > sda8/sdb8/sdc8/sdd8: 80/53Mbps > > Do you think the slow down (85 to 53Mbps) was caused by device mapper? > Please advice. > > Ken > > -- > No virus found in this outgoing message. > Checked by AVG Free Edition. > Version: 7.1.375 / Virus Database: 268.2.0/276 - Release Date: 3/7/2006 > > -- > dm-devel mailing list > dm-devel@redhat.com > https://www.redhat.com/mailman/listinfo/dm-devel -- No virus found in this incoming message. Checked by AVG Free Edition. Version: 7.1.375 / Virus Database: 268.2.1/278 - Release Date: 3/9/2006 -- No virus found in this outgoing message. Checked by AVG Free Edition. Version: 7.1.375 / Virus Database: 268.2.1/279 - Release Date: 3/10/2006 ^ permalink raw reply [flat|nested] 9+ messages in thread
* RE: 2.6 device mapper performance? 2006-03-10 20:01 ` Ken Hwang @ 2006-03-10 20:08 ` Ming Zhang 0 siblings, 0 replies; 9+ messages in thread From: Ming Zhang @ 2006-03-10 20:08 UTC (permalink / raw) To: ken.hwang; +Cc: device-mapper development if possible, (if u have no data on it.) can u do dd write? u now build evms over raid? can u do blockdev -getra /dev/evms/volume1? ming On Fri, 2006-03-10 at 12:01 -0800, Ken Hwang wrote: > Hi Ming, > > Thanks and I ran another block device read test by doing: > sync; time dd if=/dev/evms/.nodes/md/md3 of=/dev/null bs=1024k count=1024 > AND > sync; time dd if=/dev/evms/volume1 of=/dev/null bs=1024k count=1024 > the /dev/evms/volume1 is a evms volume on top of /dev/evms/.nodes/md/md3 > And the results are (real/user/sys): > /dev/evms/.nodes/md/md3: 29.757s/0.020s/15.861s > /dev/evms/volume1: 41.697s/0.004s/25.586s > > /dev/evms/volume1 took 12 seconds more to read the same amount of blocks. > Compare to 29.757 seconds it's about 40% more. Can we said it's introduced > by device mapper? > > Ken > > -----Original Message----- > From: Ming Zhang [mailto:mingz@ele.uri.edu] > Sent: Friday, March 10, 2006 8:59 AM > To: ken.hwang@zyxel.com; device-mapper development > Subject: Re: [dm-devel] 2.6 device mapper performance? > > > u have too many chances here. so hard to blame any one. suggest u to > test it one by one if possible. > > for example, have same box run 2.4 and 2.6, test performance on volume > first before run xfs, and samba. > > ming > > On Wed, 2006-03-08 at 16:49 -0800, Ken Hwang wrote: > > Hi, > > > > I'm not sure I should ask this question here, if this is not the place > then > > I apologize. I have a home made NAS with Linux. I use evms to create > volume, > > put xfs on top of it, and then use samba to share it with Windows clients. > > When I was using kernel 2.4 with all the needed patches I could get > netbench > > 106Mbps with 4 clients, and 95Mbps with 8 clients. Recently I upgraded the > > same hardware to kernel 2.6 (I also upgraded the related application such > as > > samba, xfs utility, and dmsetup accordingly). Then I ran netbench again > and > > got 80Mbps with 4 clients, 53Mbps with 8 clients. Which drop almost 80% > (95 > > vs 53) in 8 clients case. > > > > I then make and mount xfs on another raid5 (which uses the same disk but > on > > different partitions) and found it got better performance (95Mbps with 4 > > clients, 85Mbps with 8 clients). In brief: > > xfs volume on raid5 md/md1 on sda6/sdb6/sdc6/sdd6 netbench: 95/85Mbps > > xfs volume on EVMS volume /dev/evms/volume1 on raid5 md/md3 on > > sda8/sdb8/sdc8/sdd8: 80/53Mbps > > > > Do you think the slow down (85 to 53Mbps) was caused by device mapper? > > Please advice. > > > > Ken > > > > -- > > No virus found in this outgoing message. > > Checked by AVG Free Edition. > > Version: 7.1.375 / Virus Database: 268.2.0/276 - Release Date: 3/7/2006 > > > > -- > > dm-devel mailing list > > dm-devel@redhat.com > > https://www.redhat.com/mailman/listinfo/dm-devel > -- > No virus found in this incoming message. > Checked by AVG Free Edition. > Version: 7.1.375 / Virus Database: 268.2.1/278 - Release Date: 3/9/2006 > > -- > No virus found in this outgoing message. > Checked by AVG Free Edition. > Version: 7.1.375 / Virus Database: 268.2.1/279 - Release Date: 3/10/2006 > ^ permalink raw reply [flat|nested] 9+ messages in thread
* RE: 2.6 device mapper performance? 2006-03-10 16:59 ` Ming Zhang 2006-03-10 17:11 ` Ming Zhang 2006-03-10 20:01 ` Ken Hwang @ 2006-03-11 5:58 ` Ken Hwang 2006-03-11 23:16 ` Ming Zhang 2 siblings, 1 reply; 9+ messages in thread From: Ken Hwang @ 2006-03-11 5:58 UTC (permalink / raw) To: mingz, device-mapper development; +Cc: ken.hwang Hi Ming, I did more tests and list them at the following. Please help me analyze them if it's possible and thanks. Allow me describe the disk/volume layout. I had 4 120GB sata disks as sda, sdb, sdc, sdd. And they form 4 raid5 region, on top of the raid5 region there is a container, then I take 50% of the container to be a evms region, and then put a evms volume /dev/evms/volume1 on top of the region. And I believe the raid5 region I used can be found at /dev/evms/.nodes/md/md3. test: sync; time dd if=/dev/evms/volume1 of=/dev/null bs=1024k count=1024 kernel 2.4 got 33.649s/0.010s/20.430s (real/user/sys) kernel 2.6 got 46.690s/0.024s/26.902s test: sync; time dd if=/dev/evms/.nodes/md/md3 of=/dev/null bs=1024k count=1024 kernel 2.4: 19.471s/0.000s/13.480s kernel 2.6: 23.763s/0.008s/16.097s test: sync; time dd if=/dev/zero of=/dev/evms/volume1 bs=1024k count=1024 kernel 2.4: 69.183s/0.000s/15.430s krenel 2.6: 52.543s/0.004/7.640s test: sync; time dd if=/dev/zero of=/dev/evms/.nodes/md/md3 bs=1024k count=1024 kernel 2.4: 37.862s/0.000s/11.730s kernel 2.6: 23.628s/0.000s/5.536s test: blockdev --getra /dev/evms/volume1 kernel 2.4: 1024 kernel 2.6: 384 test: blockdev --getra /dev/evms/.nodes/md/md3 kernel 2.4: 64 kernel 2.6: 256 Looks like kernel 2.4 reading is faster than 2.6 but 2.6 writing is faster. By comparing read from md3 and volume1, kernel 2.4 drop from 19.471s to 33.649s (72.8%) kernel 2.6 drop from 23.763s to 46.690s (96.4%) By comparing write to md3 and volume1, kernel 2.4 drop from 37.862s to 69.183s (82.7%) kernel 2.6 drop from 23.628s to 52.543s (122.4%) What's the blockdev numbers mean? Thanks, Ken -----Original Message----- From: Ming Zhang [mailto:mingz@ele.uri.edu] Sent: Friday, March 10, 2006 8:59 AM To: ken.hwang@zyxel.com; device-mapper development Subject: Re: [dm-devel] 2.6 device mapper performance? u have too many chances here. so hard to blame any one. suggest u to test it one by one if possible. for example, have same box run 2.4 and 2.6, test performance on volume first before run xfs, and samba. ming On Wed, 2006-03-08 at 16:49 -0800, Ken Hwang wrote: > Hi, > > I'm not sure I should ask this question here, if this is not the place then > I apologize. I have a home made NAS with Linux. I use evms to create volume, > put xfs on top of it, and then use samba to share it with Windows clients. > When I was using kernel 2.4 with all the needed patches I could get netbench > 106Mbps with 4 clients, and 95Mbps with 8 clients. Recently I upgraded the > same hardware to kernel 2.6 (I also upgraded the related application such as > samba, xfs utility, and dmsetup accordingly). Then I ran netbench again and > got 80Mbps with 4 clients, 53Mbps with 8 clients. Which drop almost 80% (95 > vs 53) in 8 clients case. > > I then make and mount xfs on another raid5 (which uses the same disk but on > different partitions) and found it got better performance (95Mbps with 4 > clients, 85Mbps with 8 clients). In brief: > xfs volume on raid5 md/md1 on sda6/sdb6/sdc6/sdd6 netbench: 95/85Mbps > xfs volume on EVMS volume /dev/evms/volume1 on raid5 md/md3 on > sda8/sdb8/sdc8/sdd8: 80/53Mbps > > Do you think the slow down (85 to 53Mbps) was caused by device mapper? > Please advice. > > Ken > > -- > No virus found in this outgoing message. > Checked by AVG Free Edition. > Version: 7.1.375 / Virus Database: 268.2.0/276 - Release Date: 3/7/2006 > > -- > dm-devel mailing list > dm-devel@redhat.com > https://www.redhat.com/mailman/listinfo/dm-devel -- No virus found in this incoming message. Checked by AVG Free Edition. Version: 7.1.375 / Virus Database: 268.2.1/278 - Release Date: 3/9/2006 -- No virus found in this outgoing message. Checked by AVG Free Edition. Version: 7.1.375 / Virus Database: 268.2.1/279 - Release Date: 3/10/2006 ^ permalink raw reply [flat|nested] 9+ messages in thread
* RE: 2.6 device mapper performance? 2006-03-11 5:58 ` Ken Hwang @ 2006-03-11 23:16 ` Ming Zhang 2006-03-15 21:48 ` Ken Hwang 0 siblings, 1 reply; 9+ messages in thread From: Ming Zhang @ 2006-03-11 23:16 UTC (permalink / raw) To: ken.hwang; +Cc: device-mapper development On Fri, 2006-03-10 at 21:58 -0800, Ken Hwang wrote: > Hi Ming, > > I did more tests and list them at the following. Please help me analyze them > if it's possible and thanks. Allow me describe the disk/volume layout. I had > 4 120GB sata disks as sda, sdb, sdc, sdd. And they form 4 raid5 region, on > top of the raid5 region there is a container, then I take 50% of the > container to be a evms region, and then put a evms volume /dev/evms/volume1 > on top of the region. And I believe the raid5 region I used can be found at > /dev/evms/.nodes/md/md3. > > test: sync; time dd if=/dev/evms/volume1 of=/dev/null bs=1024k count=1024 > kernel 2.4 got 33.649s/0.010s/20.430s (real/user/sys) > kernel 2.6 got 46.690s/0.024s/26.902s > > test: sync; time dd if=/dev/evms/.nodes/md/md3 of=/dev/null bs=1024k > count=1024 > kernel 2.4: 19.471s/0.000s/13.480s > kernel 2.6: 23.763s/0.008s/16.097s > > test: sync; time dd if=/dev/zero of=/dev/evms/volume1 bs=1024k count=1024 > kernel 2.4: 69.183s/0.000s/15.430s > krenel 2.6: 52.543s/0.004/7.640s > > test: sync; time dd if=/dev/zero of=/dev/evms/.nodes/md/md3 bs=1024k > count=1024 > kernel 2.4: 37.862s/0.000s/11.730s > kernel 2.6: 23.628s/0.000s/5.536s > > test: blockdev --getra /dev/evms/volume1 > kernel 2.4: 1024 > kernel 2.6: 384 try to increase this number by --setra with blockdev > > test: blockdev --getra /dev/evms/.nodes/md/md3 > kernel 2.4: 64 > kernel 2.6: 256 > > Looks like kernel 2.4 reading is faster than 2.6 but 2.6 writing is faster. > > By comparing read from md3 and volume1, > kernel 2.4 drop from 19.471s to 33.649s (72.8%) > kernel 2.6 drop from 23.763s to 46.690s (96.4%) > > By comparing write to md3 and volume1, > kernel 2.4 drop from 37.862s to 69.183s (82.7%) > kernel 2.6 drop from 23.628s to 52.543s (122.4%) > > What's the blockdev numbers mean? yes, pretty big drop here. no idea why. i think both lvm and evms use dm and should not have such big difference. that is to query/set the readahead buffer window, useful for sequential workload. > > Thanks, > > Ken > > > -----Original Message----- > From: Ming Zhang [mailto:mingz@ele.uri.edu] > Sent: Friday, March 10, 2006 8:59 AM > To: ken.hwang@zyxel.com; device-mapper development > Subject: Re: [dm-devel] 2.6 device mapper performance? > > > u have too many chances here. so hard to blame any one. suggest u to > test it one by one if possible. > > for example, have same box run 2.4 and 2.6, test performance on volume > first before run xfs, and samba. > > ming > > On Wed, 2006-03-08 at 16:49 -0800, Ken Hwang wrote: > > Hi, > > > > I'm not sure I should ask this question here, if this is not the place > then > > I apologize. I have a home made NAS with Linux. I use evms to create > volume, > > put xfs on top of it, and then use samba to share it with Windows clients. > > When I was using kernel 2.4 with all the needed patches I could get > netbench > > 106Mbps with 4 clients, and 95Mbps with 8 clients. Recently I upgraded the > > same hardware to kernel 2.6 (I also upgraded the related application such > as > > samba, xfs utility, and dmsetup accordingly). Then I ran netbench again > and > > got 80Mbps with 4 clients, 53Mbps with 8 clients. Which drop almost 80% > (95 > > vs 53) in 8 clients case. > > > > I then make and mount xfs on another raid5 (which uses the same disk but > on > > different partitions) and found it got better performance (95Mbps with 4 > > clients, 85Mbps with 8 clients). In brief: > > xfs volume on raid5 md/md1 on sda6/sdb6/sdc6/sdd6 netbench: 95/85Mbps > > xfs volume on EVMS volume /dev/evms/volume1 on raid5 md/md3 on > > sda8/sdb8/sdc8/sdd8: 80/53Mbps > > > > Do you think the slow down (85 to 53Mbps) was caused by device mapper? > > Please advice. > > > > Ken > > > > -- > > No virus found in this outgoing message. > > Checked by AVG Free Edition. > > Version: 7.1.375 / Virus Database: 268.2.0/276 - Release Date: 3/7/2006 > > > > -- > > dm-devel mailing list > > dm-devel@redhat.com > > https://www.redhat.com/mailman/listinfo/dm-devel > -- > No virus found in this incoming message. > Checked by AVG Free Edition. > Version: 7.1.375 / Virus Database: 268.2.1/278 - Release Date: 3/9/2006 > > -- > No virus found in this outgoing message. > Checked by AVG Free Edition. > Version: 7.1.375 / Virus Database: 268.2.1/279 - Release Date: 3/10/2006 > ^ permalink raw reply [flat|nested] 9+ messages in thread
* RE: 2.6 device mapper performance? 2006-03-11 23:16 ` Ming Zhang @ 2006-03-15 21:48 ` Ken Hwang 2006-03-15 21:53 ` Ming Zhang 0 siblings, 1 reply; 9+ messages in thread From: Ken Hwang @ 2006-03-15 21:48 UTC (permalink / raw) To: mingz; +Cc: device-mapper development Hi Ming, I did blockdev --setra 1024 on kernel 2.6 system to make the same number as read from kernel 2.4 and the result: test:sync; time dd if=/dev/evms/volume1 of=/dev/null bs=1024k count=1024 kernel 2.6 got 44.444s/0.004s/24.254s, and it was 46.690s/0.024s/26.902s. It's still far away from kernel 2.4: 33.649s/0.010s/20.430s. blockdev --setra 64 /dev/evms.nodes/md/md3 test:sync; time dd if=/dev/evms/.nodes/md/md3 of=/dev/null bs=1024k count=1024 kernel 2.6 got 26.770s/0.016s/11.257s, and it was 23.763s/0.008s/16.097s. The real time gets slower but system time gets faster. And they are still not close to kernel 2.4 19.471s/0.000s/13.480s Do you have any advice I can do to tune the 2.6 performance? Thanks in advance, Ken -----Original Message----- From: Ming Zhang [mailto:mingz@ele.uri.edu] Sent: Saturday, March 11, 2006 3:17 PM To: ken.hwang@zyxel.com Cc: device-mapper development Subject: RE: [dm-devel] 2.6 device mapper performance? On Fri, 2006-03-10 at 21:58 -0800, Ken Hwang wrote: > Hi Ming, > > I did more tests and list them at the following. Please help me analyze them > if it's possible and thanks. Allow me describe the disk/volume layout. I had > 4 120GB sata disks as sda, sdb, sdc, sdd. And they form 4 raid5 region, on > top of the raid5 region there is a container, then I take 50% of the > container to be a evms region, and then put a evms volume /dev/evms/volume1 > on top of the region. And I believe the raid5 region I used can be found at > /dev/evms/.nodes/md/md3. > > test: sync; time dd if=/dev/evms/volume1 of=/dev/null bs=1024k count=1024 > kernel 2.4 got 33.649s/0.010s/20.430s (real/user/sys) > kernel 2.6 got 46.690s/0.024s/26.902s > > test: sync; time dd if=/dev/evms/.nodes/md/md3 of=/dev/null bs=1024k > count=1024 > kernel 2.4: 19.471s/0.000s/13.480s > kernel 2.6: 23.763s/0.008s/16.097s > > test: sync; time dd if=/dev/zero of=/dev/evms/volume1 bs=1024k count=1024 > kernel 2.4: 69.183s/0.000s/15.430s > krenel 2.6: 52.543s/0.004/7.640s > > test: sync; time dd if=/dev/zero of=/dev/evms/.nodes/md/md3 bs=1024k > count=1024 > kernel 2.4: 37.862s/0.000s/11.730s > kernel 2.6: 23.628s/0.000s/5.536s > > test: blockdev --getra /dev/evms/volume1 > kernel 2.4: 1024 > kernel 2.6: 384 try to increase this number by --setra with blockdev > > test: blockdev --getra /dev/evms/.nodes/md/md3 > kernel 2.4: 64 > kernel 2.6: 256 > > Looks like kernel 2.4 reading is faster than 2.6 but 2.6 writing is faster. > > By comparing read from md3 and volume1, > kernel 2.4 drop from 19.471s to 33.649s (72.8%) > kernel 2.6 drop from 23.763s to 46.690s (96.4%) > > By comparing write to md3 and volume1, > kernel 2.4 drop from 37.862s to 69.183s (82.7%) > kernel 2.6 drop from 23.628s to 52.543s (122.4%) > > What's the blockdev numbers mean? yes, pretty big drop here. no idea why. i think both lvm and evms use dm and should not have such big difference. that is to query/set the readahead buffer window, useful for sequential workload. > > Thanks, > > Ken > > > -----Original Message----- > From: Ming Zhang [mailto:mingz@ele.uri.edu] > Sent: Friday, March 10, 2006 8:59 AM > To: ken.hwang@zyxel.com; device-mapper development > Subject: Re: [dm-devel] 2.6 device mapper performance? > > > u have too many chances here. so hard to blame any one. suggest u to > test it one by one if possible. > > for example, have same box run 2.4 and 2.6, test performance on volume > first before run xfs, and samba. > > ming > > On Wed, 2006-03-08 at 16:49 -0800, Ken Hwang wrote: > > Hi, > > > > I'm not sure I should ask this question here, if this is not the place > then > > I apologize. I have a home made NAS with Linux. I use evms to create > volume, > > put xfs on top of it, and then use samba to share it with Windows clients. > > When I was using kernel 2.4 with all the needed patches I could get > netbench > > 106Mbps with 4 clients, and 95Mbps with 8 clients. Recently I upgraded the > > same hardware to kernel 2.6 (I also upgraded the related application such > as > > samba, xfs utility, and dmsetup accordingly). Then I ran netbench again > and > > got 80Mbps with 4 clients, 53Mbps with 8 clients. Which drop almost 80% > (95 > > vs 53) in 8 clients case. > > > > I then make and mount xfs on another raid5 (which uses the same disk but > on > > different partitions) and found it got better performance (95Mbps with 4 > > clients, 85Mbps with 8 clients). In brief: > > xfs volume on raid5 md/md1 on sda6/sdb6/sdc6/sdd6 netbench: 95/85Mbps > > xfs volume on EVMS volume /dev/evms/volume1 on raid5 md/md3 on > > sda8/sdb8/sdc8/sdd8: 80/53Mbps > > > > Do you think the slow down (85 to 53Mbps) was caused by device mapper? > > Please advice. > > > > Ken > > > > -- > > No virus found in this outgoing message. > > Checked by AVG Free Edition. > > Version: 7.1.375 / Virus Database: 268.2.0/276 - Release Date: 3/7/2006 > > > > -- > > dm-devel mailing list > > dm-devel@redhat.com > > https://www.redhat.com/mailman/listinfo/dm-devel > -- > No virus found in this incoming message. > Checked by AVG Free Edition. > Version: 7.1.375 / Virus Database: 268.2.1/278 - Release Date: 3/9/2006 > > -- > No virus found in this outgoing message. > Checked by AVG Free Edition. > Version: 7.1.375 / Virus Database: 268.2.1/279 - Release Date: 3/10/2006 > -- No virus found in this incoming message. Checked by AVG Free Edition. Version: 7.1.375 / Virus Database: 268.2.1/279 - Release Date: 3/10/2006 -- No virus found in this outgoing message. Checked by AVG Free Edition. Version: 7.1.385 / Virus Database: 268.2.4/282 - Release Date: 3/15/2006 ^ permalink raw reply [flat|nested] 9+ messages in thread
* RE: 2.6 device mapper performance? 2006-03-15 21:48 ` Ken Hwang @ 2006-03-15 21:53 ` Ming Zhang 0 siblings, 0 replies; 9+ messages in thread From: Ming Zhang @ 2006-03-15 21:53 UTC (permalink / raw) To: ken.hwang; +Cc: device-mapper development then i run out of idea. i guess only evms developer can tell u exactly why 2.6 evms has such performance loss compared with 2.4.x both evms and lvm use device mapper in kernel and i do not see such performance loss in lvm. so pretty strange, ming On Wed, 2006-03-15 at 13:48 -0800, Ken Hwang wrote: > Hi Ming, > > I did blockdev --setra 1024 on kernel 2.6 system to make the same number as > read from kernel 2.4 and the result: > test:sync; time dd if=/dev/evms/volume1 of=/dev/null bs=1024k count=1024 > kernel 2.6 got 44.444s/0.004s/24.254s, and it was 46.690s/0.024s/26.902s. > It's still far away from kernel 2.4: 33.649s/0.010s/20.430s. > > > blockdev --setra 64 /dev/evms.nodes/md/md3 > test:sync; time dd if=/dev/evms/.nodes/md/md3 of=/dev/null bs=1024k > count=1024 > kernel 2.6 got 26.770s/0.016s/11.257s, and it was 23.763s/0.008s/16.097s. > The real time gets slower but system time gets faster. And they are still > not close to > kernel 2.4 19.471s/0.000s/13.480s > > Do you have any advice I can do to tune the 2.6 performance? > > Thanks in advance, > > Ken > > -----Original Message----- > From: Ming Zhang [mailto:mingz@ele.uri.edu] > Sent: Saturday, March 11, 2006 3:17 PM > To: ken.hwang@zyxel.com > Cc: device-mapper development > Subject: RE: [dm-devel] 2.6 device mapper performance? > > > On Fri, 2006-03-10 at 21:58 -0800, Ken Hwang wrote: > > Hi Ming, > > > > I did more tests and list them at the following. Please help me analyze > them > > if it's possible and thanks. Allow me describe the disk/volume layout. I > had > > 4 120GB sata disks as sda, sdb, sdc, sdd. And they form 4 raid5 region, on > > top of the raid5 region there is a container, then I take 50% of the > > container to be a evms region, and then put a evms volume > /dev/evms/volume1 > > on top of the region. And I believe the raid5 region I used can be found > at > > /dev/evms/.nodes/md/md3. > > > > test: sync; time dd if=/dev/evms/volume1 of=/dev/null bs=1024k count=1024 > > kernel 2.4 got 33.649s/0.010s/20.430s (real/user/sys) > > kernel 2.6 got 46.690s/0.024s/26.902s > > > > test: sync; time dd if=/dev/evms/.nodes/md/md3 of=/dev/null bs=1024k > > count=1024 > > kernel 2.4: 19.471s/0.000s/13.480s > > kernel 2.6: 23.763s/0.008s/16.097s > > > > test: sync; time dd if=/dev/zero of=/dev/evms/volume1 bs=1024k count=1024 > > kernel 2.4: 69.183s/0.000s/15.430s > > krenel 2.6: 52.543s/0.004/7.640s > > > > test: sync; time dd if=/dev/zero of=/dev/evms/.nodes/md/md3 bs=1024k > > count=1024 > > kernel 2.4: 37.862s/0.000s/11.730s > > kernel 2.6: 23.628s/0.000s/5.536s > > > > test: blockdev --getra /dev/evms/volume1 > > kernel 2.4: 1024 > > kernel 2.6: 384 > > try to increase this number by --setra with blockdev > > > > > test: blockdev --getra /dev/evms/.nodes/md/md3 > > kernel 2.4: 64 > > kernel 2.6: 256 > > > > Looks like kernel 2.4 reading is faster than 2.6 but 2.6 writing is > faster. > > > > By comparing read from md3 and volume1, > > kernel 2.4 drop from 19.471s to 33.649s (72.8%) > > kernel 2.6 drop from 23.763s to 46.690s (96.4%) > > > > By comparing write to md3 and volume1, > > kernel 2.4 drop from 37.862s to 69.183s (82.7%) > > kernel 2.6 drop from 23.628s to 52.543s (122.4%) > > > > What's the blockdev numbers mean? > > yes, pretty big drop here. no idea why. i think both lvm and evms use dm > and should not have such big difference. > > that is to query/set the readahead buffer window, useful for sequential > workload. > > > > > > Thanks, > > > > Ken > > > > > > -----Original Message----- > > From: Ming Zhang [mailto:mingz@ele.uri.edu] > > Sent: Friday, March 10, 2006 8:59 AM > > To: ken.hwang@zyxel.com; device-mapper development > > Subject: Re: [dm-devel] 2.6 device mapper performance? > > > > > > u have too many chances here. so hard to blame any one. suggest u to > > test it one by one if possible. > > > > for example, have same box run 2.4 and 2.6, test performance on volume > > first before run xfs, and samba. > > > > ming > > > > On Wed, 2006-03-08 at 16:49 -0800, Ken Hwang wrote: > > > Hi, > > > > > > I'm not sure I should ask this question here, if this is not the place > > then > > > I apologize. I have a home made NAS with Linux. I use evms to create > > volume, > > > put xfs on top of it, and then use samba to share it with Windows > clients. > > > When I was using kernel 2.4 with all the needed patches I could get > > netbench > > > 106Mbps with 4 clients, and 95Mbps with 8 clients. Recently I upgraded > the > > > same hardware to kernel 2.6 (I also upgraded the related application > such > > as > > > samba, xfs utility, and dmsetup accordingly). Then I ran netbench again > > and > > > got 80Mbps with 4 clients, 53Mbps with 8 clients. Which drop almost 80% > > (95 > > > vs 53) in 8 clients case. > > > > > > I then make and mount xfs on another raid5 (which uses the same disk > but > > on > > > different partitions) and found it got better performance (95Mbps with 4 > > > clients, 85Mbps with 8 clients). In brief: > > > xfs volume on raid5 md/md1 on sda6/sdb6/sdc6/sdd6 netbench: 95/85Mbps > > > xfs volume on EVMS volume /dev/evms/volume1 on raid5 md/md3 on > > > sda8/sdb8/sdc8/sdd8: 80/53Mbps > > > > > > Do you think the slow down (85 to 53Mbps) was caused by device mapper? > > > Please advice. > > > > > > Ken > > > > > > -- > > > No virus found in this outgoing message. > > > Checked by AVG Free Edition. > > > Version: 7.1.375 / Virus Database: 268.2.0/276 - Release Date: 3/7/2006 > > > > > > -- > > > dm-devel mailing list > > > dm-devel@redhat.com > > > https://www.redhat.com/mailman/listinfo/dm-devel > > -- > > No virus found in this incoming message. > > Checked by AVG Free Edition. > > Version: 7.1.375 / Virus Database: 268.2.1/278 - Release Date: 3/9/2006 > > > > -- > > No virus found in this outgoing message. > > Checked by AVG Free Edition. > > Version: 7.1.375 / Virus Database: 268.2.1/279 - Release Date: 3/10/2006 > > > -- > No virus found in this incoming message. > Checked by AVG Free Edition. > Version: 7.1.375 / Virus Database: 268.2.1/279 - Release Date: 3/10/2006 > > -- > No virus found in this outgoing message. > Checked by AVG Free Edition. > Version: 7.1.385 / Virus Database: 268.2.4/282 - Release Date: 3/15/2006 > ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2006-03-15 21:53 UTC | newest] Thread overview: 9+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2006-03-09 0:49 2.6 device mapper performance? Ken Hwang 2006-03-10 16:59 ` Ming Zhang 2006-03-10 17:11 ` Ming Zhang 2006-03-10 20:01 ` Ken Hwang 2006-03-10 20:08 ` Ming Zhang 2006-03-11 5:58 ` Ken Hwang 2006-03-11 23:16 ` Ming Zhang 2006-03-15 21:48 ` Ken Hwang 2006-03-15 21:53 ` Ming Zhang
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.