From: Vivek Goyal <vgoyal@redhat.com>
To: Ryo Tsuruta <ryov@valinux.co.jp>
Cc: dm-devel@redhat.com,
linux kernel mailing list <linux-kernel@vger.kernel.org>
Subject: Re: Regarding dm-ioband tests
Date: Tue, 1 Sep 2009 13:47:24 -0400 [thread overview]
Message-ID: <20090901174724.GC3753@redhat.com> (raw)
In-Reply-To: <20090901165011.GB3753@redhat.com>
On Tue, Sep 01, 2009 at 12:50:11PM -0400, Vivek Goyal wrote:
> Hi Ryo,
>
> I decided to play a bit more with dm-ioband and started doing some
> testing. I am doing a simple two dd threads doing reads and don't seem
> to be gettting the fairness. So thought will ask you what's the issue
> here. Is there an issue with my testing procedure.
>
> I got one 40G SATA drive (no hardware queuing). I have created two
> partitions on that disk /dev/sdd1 and /dev/sdd2 and created two ioband
> devices ioband1 and ioband2 on partitions sdd1 and sdd2 respectively. The
> weights of ioband1 and ioband2 devices are 200 and 100 respectively.
>
> I am assuming that this setup will create two default groups and IO
> going to partition sdd1 should get double the BW of partition sdd2.
>
> But it looks like I am not gettting that behavior. Following is the output
> of "dmsetup table" command. This snapshot has been taken every 2 seconds
> while IO was going on. Column 9 seems to be containing how many sectors
> of IO has been done on a particular io band device and group. Looking at
> the snapshot, it does not look like that ioband1 default group got double
> the BW of ioband2 default group.
>
> Am I doing something wrong here?
>
I tried another variant of test. This time I also created two additional
groups on ioband1 devices and linked these to cgroups test1 and test2 and
launched two dd threads in two cgroups on device ioband1. There also I
don't seem to be getting the right fairness numbers for cgroup test1 and
test2.
Script to create ioband devices and additional groups
-----------------------------------------------------
echo "0 $(blockdev --getsize /dev/sdd1) ioband /dev/sdd1 1 0 0 none"
"weight 0 :200" | dmsetup create ioband1
echo "0 $(blockdev --getsize /dev/sdd2) ioband /dev/sdd2 1 0 0 none"
"weight 0 :100" | dmsetup create ioband2
# Some code to mount and create cgroups.
# Read group id
test1_id=`cat /cgroup/ioband/test1/blkio.id`
test2_id=`cat /cgroup/ioband/test2/blkio.id`
test1_weight=200
test2_weight=100
dmsetup message ioband1 0 type cgroup
dmsetup message ioband1 0 attach $test1_id
dmsetup message ioband1 0 attach $test2_id
dmsetup message ioband1 0 weight $test1_id:$test1_weight
dmsetup message ioband1 0 weight $test2_id:$test2_weight
mount /dev/mapper/ioband1 /mnt/sdd1
mount /dev/mapper/ioband2 /mnt/sdd2
-----------------------------------------------------------------
Following are two dd jobs
-------------------------
dd if=/mnt/sdd1/testzerofile1 of=/dev/null &
echo $! > /cgroup/ioband/test1/tasks
dd if=/mnt/sdd1/testzerofile2 of=/dev/null &
echo $! > /cgroup/ioband/test2/tasks
Following are "dmsetup status" results every 2 seconds
======================================================
ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0 2 0 0 0 0 0 0 3 0 0 0 0 0 0
ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0 2 0 0 0 0 0 0 3 0 0 0 0 0 0
ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0 2 689 0 86336 0 0 0 3 650 3
81472 0 0 0
ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0 2 1725 0 217024 0 0 0 3 1270
11 158912 0 0 0
ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0 2 2690 0 338744 0 0 0 3 1978
15 247856 0 0 0
ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0 2 3762 0 474040 0 0 0 3 2583
21 323736 0 0 0
ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0 2 4745 0 598064 0 0 0 3 3275
27 410392 0 0 0
ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0 2 5737 0 723120 0 0 0 3 3985
31 499592 0 0 0
ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0 2 6815 0 859184 0 0 0 3 4594
37 575864 0 0 0
ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0 2 7823 0 986288 0 0 0 3 5276
43 661360 0 0 0
"dmsetup table" output
======================
ioband2: 0 40355280 ioband 8:50 1 4 192 none weight 768 :100
ioband1: 0 37768752 ioband 8:49 1 4 192 cgroup weight 768 :200 2:200 3:100
Because I am using "weight" policy, I thought that test1 cgroup with id
"2" will issue double the number of requests of cgroup test2 with id "3".
But that does not seem to be happening here. Is there an issue with my
testing method.
Thanks
Vivek
> ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
> ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0
>
> ioband2: 0 40355280 ioband 1 -1 96 0 11528 0 0 0
> ioband1: 0 37768752 ioband 1 -1 82 0 9736 0 0 0
>
> ioband2: 0 40355280 ioband 1 -1 748 2 93032 0 0 0
> ioband1: 0 37768752 ioband 1 -1 896 0 112232 0 0 0
>
> ioband2: 0 40355280 ioband 1 -1 1326 5 165816 0 0 0
> ioband1: 0 37768752 ioband 1 -1 1816 0 228312 0 0 0
>
> ioband2: 0 40355280 ioband 1 -1 1943 6 243712 0 0 0
> ioband1: 0 37768752 ioband 1 -1 2692 0 338760 0 0 0
>
> ioband2: 0 40355280 ioband 1 -1 2461 10 308576 0 0 0
> ioband1: 0 37768752 ioband 1 -1 3618 0 455608 0 0 0
>
> ioband2: 0 40355280 ioband 1 -1 3118 11 391352 0 0 0
> ioband1: 0 37768752 ioband 1 -1 4406 0 555032 0 0 0
>
> ioband2: 0 40355280 ioband 1 -1 3734 15 468760 0 0 0
> ioband1: 0 37768752 ioband 1 -1 5273 0 664328 0 0 0
>
> ioband2: 0 40355280 ioband 1 -1 4307 17 540784 0 0 0
> ioband1: 0 37768752 ioband 1 -1 6181 0 778992 0 0 0
>
> ioband2: 0 40355280 ioband 1 -1 4930 19 619208 0 0 0
> ioband1: 0 37768752 ioband 1 -1 7028 0 885728 0 0 0
>
> ioband2: 0 40355280 ioband 1 -1 5599 22 703280 0 0 0
> ioband1: 0 37768752 ioband 1 -1 7815 0 985024 0 0 0
>
> ioband2: 0 40355280 ioband 1 -1 6586 27 827456 0 0 0
> ioband1: 0 37768752 ioband 1 -1 8327 0 1049624 0 0 0
>
> Following are details of my test setup.
> ---------------------------------------
> I took dm-ioband patch version 1.12.3 and applied on 2.6.31-rc6.
>
> Created ioband devices using following command.
> ----------------------------------------------
> echo "0 $(blockdev --getsize /dev/sdd1) ioband /dev/sdd1 1 0 0 none"
> "weight 0 :200" | dmsetup create ioband1
> echo "0 $(blockdev --getsize /dev/sdd2) ioband /dev/sdd2 1 0 0 none"
> "weight 0 :100" | dmsetup create ioband2
>
> mount /dev/mapper/ioband1 /mnt/sdd1
> mount /dev/mapper/ioband2 /mnt/sdd2
>
> Started two dd threads
> ======================
> dd if=/mnt/sdd1/testzerofile1 of=/dev/null &
> dd if=/mnt/sdd2/testzerofile1 of=/dev/null &
>
> Output of dmsetup table command
> ================================
> ioband2: 0 40355280 ioband 8:50 1 4 192 none weight 768 :100
> ioband1: 0 37768752 ioband 8:49 1 4 192 none weight 768 :200
>
> Thanks
> Vivek
WARNING: multiple messages have this Message-ID (diff)
From: Vivek Goyal <vgoyal@redhat.com>
To: Ryo Tsuruta <ryov@valinux.co.jp>
Cc: linux kernel mailing list <linux-kernel@vger.kernel.org>,
dm-devel@redhat.com
Subject: Re: Regarding dm-ioband tests
Date: Tue, 1 Sep 2009 13:47:24 -0400 [thread overview]
Message-ID: <20090901174724.GC3753@redhat.com> (raw)
In-Reply-To: <20090901165011.GB3753@redhat.com>
On Tue, Sep 01, 2009 at 12:50:11PM -0400, Vivek Goyal wrote:
> Hi Ryo,
>
> I decided to play a bit more with dm-ioband and started doing some
> testing. I am doing a simple two dd threads doing reads and don't seem
> to be gettting the fairness. So thought will ask you what's the issue
> here. Is there an issue with my testing procedure.
>
> I got one 40G SATA drive (no hardware queuing). I have created two
> partitions on that disk /dev/sdd1 and /dev/sdd2 and created two ioband
> devices ioband1 and ioband2 on partitions sdd1 and sdd2 respectively. The
> weights of ioband1 and ioband2 devices are 200 and 100 respectively.
>
> I am assuming that this setup will create two default groups and IO
> going to partition sdd1 should get double the BW of partition sdd2.
>
> But it looks like I am not gettting that behavior. Following is the output
> of "dmsetup table" command. This snapshot has been taken every 2 seconds
> while IO was going on. Column 9 seems to be containing how many sectors
> of IO has been done on a particular io band device and group. Looking at
> the snapshot, it does not look like that ioband1 default group got double
> the BW of ioband2 default group.
>
> Am I doing something wrong here?
>
I tried another variant of test. This time I also created two additional
groups on ioband1 devices and linked these to cgroups test1 and test2 and
launched two dd threads in two cgroups on device ioband1. There also I
don't seem to be getting the right fairness numbers for cgroup test1 and
test2.
Script to create ioband devices and additional groups
-----------------------------------------------------
echo "0 $(blockdev --getsize /dev/sdd1) ioband /dev/sdd1 1 0 0 none"
"weight 0 :200" | dmsetup create ioband1
echo "0 $(blockdev --getsize /dev/sdd2) ioband /dev/sdd2 1 0 0 none"
"weight 0 :100" | dmsetup create ioband2
# Some code to mount and create cgroups.
# Read group id
test1_id=`cat /cgroup/ioband/test1/blkio.id`
test2_id=`cat /cgroup/ioband/test2/blkio.id`
test1_weight=200
test2_weight=100
dmsetup message ioband1 0 type cgroup
dmsetup message ioband1 0 attach $test1_id
dmsetup message ioband1 0 attach $test2_id
dmsetup message ioband1 0 weight $test1_id:$test1_weight
dmsetup message ioband1 0 weight $test2_id:$test2_weight
mount /dev/mapper/ioband1 /mnt/sdd1
mount /dev/mapper/ioband2 /mnt/sdd2
-----------------------------------------------------------------
Following are two dd jobs
-------------------------
dd if=/mnt/sdd1/testzerofile1 of=/dev/null &
echo $! > /cgroup/ioband/test1/tasks
dd if=/mnt/sdd1/testzerofile2 of=/dev/null &
echo $! > /cgroup/ioband/test2/tasks
Following are "dmsetup status" results every 2 seconds
======================================================
ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0 2 0 0 0 0 0 0 3 0 0 0 0 0 0
ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0 2 0 0 0 0 0 0 3 0 0 0 0 0 0
ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0 2 689 0 86336 0 0 0 3 650 3
81472 0 0 0
ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0 2 1725 0 217024 0 0 0 3 1270
11 158912 0 0 0
ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0 2 2690 0 338744 0 0 0 3 1978
15 247856 0 0 0
ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0 2 3762 0 474040 0 0 0 3 2583
21 323736 0 0 0
ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0 2 4745 0 598064 0 0 0 3 3275
27 410392 0 0 0
ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0 2 5737 0 723120 0 0 0 3 3985
31 499592 0 0 0
ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0 2 6815 0 859184 0 0 0 3 4594
37 575864 0 0 0
ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0 2 7823 0 986288 0 0 0 3 5276
43 661360 0 0 0
"dmsetup table" output
======================
ioband2: 0 40355280 ioband 8:50 1 4 192 none weight 768 :100
ioband1: 0 37768752 ioband 8:49 1 4 192 cgroup weight 768 :200 2:200 3:100
Because I am using "weight" policy, I thought that test1 cgroup with id
"2" will issue double the number of requests of cgroup test2 with id "3".
But that does not seem to be happening here. Is there an issue with my
testing method.
Thanks
Vivek
> ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
> ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0
>
> ioband2: 0 40355280 ioband 1 -1 96 0 11528 0 0 0
> ioband1: 0 37768752 ioband 1 -1 82 0 9736 0 0 0
>
> ioband2: 0 40355280 ioband 1 -1 748 2 93032 0 0 0
> ioband1: 0 37768752 ioband 1 -1 896 0 112232 0 0 0
>
> ioband2: 0 40355280 ioband 1 -1 1326 5 165816 0 0 0
> ioband1: 0 37768752 ioband 1 -1 1816 0 228312 0 0 0
>
> ioband2: 0 40355280 ioband 1 -1 1943 6 243712 0 0 0
> ioband1: 0 37768752 ioband 1 -1 2692 0 338760 0 0 0
>
> ioband2: 0 40355280 ioband 1 -1 2461 10 308576 0 0 0
> ioband1: 0 37768752 ioband 1 -1 3618 0 455608 0 0 0
>
> ioband2: 0 40355280 ioband 1 -1 3118 11 391352 0 0 0
> ioband1: 0 37768752 ioband 1 -1 4406 0 555032 0 0 0
>
> ioband2: 0 40355280 ioband 1 -1 3734 15 468760 0 0 0
> ioband1: 0 37768752 ioband 1 -1 5273 0 664328 0 0 0
>
> ioband2: 0 40355280 ioband 1 -1 4307 17 540784 0 0 0
> ioband1: 0 37768752 ioband 1 -1 6181 0 778992 0 0 0
>
> ioband2: 0 40355280 ioband 1 -1 4930 19 619208 0 0 0
> ioband1: 0 37768752 ioband 1 -1 7028 0 885728 0 0 0
>
> ioband2: 0 40355280 ioband 1 -1 5599 22 703280 0 0 0
> ioband1: 0 37768752 ioband 1 -1 7815 0 985024 0 0 0
>
> ioband2: 0 40355280 ioband 1 -1 6586 27 827456 0 0 0
> ioband1: 0 37768752 ioband 1 -1 8327 0 1049624 0 0 0
>
> Following are details of my test setup.
> ---------------------------------------
> I took dm-ioband patch version 1.12.3 and applied on 2.6.31-rc6.
>
> Created ioband devices using following command.
> ----------------------------------------------
> echo "0 $(blockdev --getsize /dev/sdd1) ioband /dev/sdd1 1 0 0 none"
> "weight 0 :200" | dmsetup create ioband1
> echo "0 $(blockdev --getsize /dev/sdd2) ioband /dev/sdd2 1 0 0 none"
> "weight 0 :100" | dmsetup create ioband2
>
> mount /dev/mapper/ioband1 /mnt/sdd1
> mount /dev/mapper/ioband2 /mnt/sdd2
>
> Started two dd threads
> ======================
> dd if=/mnt/sdd1/testzerofile1 of=/dev/null &
> dd if=/mnt/sdd2/testzerofile1 of=/dev/null &
>
> Output of dmsetup table command
> ================================
> ioband2: 0 40355280 ioband 8:50 1 4 192 none weight 768 :100
> ioband1: 0 37768752 ioband 8:49 1 4 192 none weight 768 :200
>
> Thanks
> Vivek
next prev parent reply other threads:[~2009-09-01 17:47 UTC|newest]
Thread overview: 80+ messages / expand[flat|nested] mbox.gz Atom feed top
2009-09-01 16:50 Regarding dm-ioband tests Vivek Goyal
2009-09-01 16:50 ` Vivek Goyal
2009-09-01 17:47 ` Vivek Goyal [this message]
2009-09-01 17:47 ` Vivek Goyal
2009-09-03 13:11 ` Vivek Goyal
2009-09-03 13:11 ` Vivek Goyal
2009-09-04 1:12 ` Ryo Tsuruta
2009-09-15 21:40 ` dm-ioband fairness in terms of sectors seems to be killing disk (Was: Re: Regarding dm-ioband tests) Vivek Goyal
2009-09-15 21:40 ` Vivek Goyal
2009-09-16 11:10 ` dm-ioband fairness in terms of sectors seems to be killing disk Ryo Tsuruta
2009-09-16 11:10 ` Ryo Tsuruta
2009-09-04 4:02 ` Regarding dm-ioband tests Ryo Tsuruta
2009-09-04 4:02 ` Ryo Tsuruta
2009-09-04 23:11 ` Vivek Goyal
2009-09-04 23:11 ` Vivek Goyal
2009-09-07 11:02 ` Ryo Tsuruta
2009-09-07 11:02 ` Ryo Tsuruta
2009-09-07 13:53 ` Rik van Riel
2009-09-07 13:53 ` Rik van Riel
2009-09-08 3:01 ` Ryo Tsuruta
2009-09-08 3:01 ` Ryo Tsuruta
2009-09-08 3:22 ` Balbir Singh
2009-09-08 3:22 ` Balbir Singh
2009-09-08 5:05 ` Ryo Tsuruta
2009-09-08 5:05 ` Ryo Tsuruta
2009-09-08 13:49 ` Vivek Goyal
2009-09-08 13:49 ` Vivek Goyal
2009-09-09 5:17 ` Ryo Tsuruta
2009-09-09 5:17 ` Ryo Tsuruta
2009-09-09 13:34 ` Vivek Goyal
2009-09-09 13:34 ` Vivek Goyal
2009-09-08 13:42 ` Vivek Goyal
2009-09-08 13:42 ` Vivek Goyal
2009-09-08 16:30 ` Nauman Rafique
2009-09-08 16:30 ` Nauman Rafique
2009-09-08 16:47 ` Rik van Riel
2009-09-08 16:47 ` Rik van Riel
2009-09-08 17:54 ` Vivek Goyal
2009-09-08 17:54 ` Vivek Goyal
2009-09-15 23:37 ` ioband: Writer starves reader even without competitors (Re: Regarding dm-ioband tests) Vivek Goyal
2009-09-15 23:37 ` Vivek Goyal
2009-09-16 12:08 ` ioband: Writer starves reader even without competitors Ryo Tsuruta
2009-09-08 17:06 ` Regarding dm-ioband tests Dhaval Giani
2009-09-09 6:05 ` Ryo Tsuruta
2009-09-09 6:05 ` Ryo Tsuruta
2009-09-09 10:51 ` Dhaval Giani
2009-09-10 7:58 ` Ryo Tsuruta
2009-09-10 7:58 ` Ryo Tsuruta
2009-09-11 9:53 ` Dhaval Giani
2009-09-15 15:12 ` Ryo Tsuruta
2009-09-15 15:12 ` Ryo Tsuruta
2009-09-15 15:19 ` Balbir Singh
2009-09-15 15:19 ` Balbir Singh
2009-09-15 15:58 ` Rik van Riel
2009-09-15 15:58 ` Rik van Riel
2009-09-15 16:21 ` Ryo Tsuruta
2009-09-15 16:21 ` Ryo Tsuruta
2009-09-09 13:57 ` Vivek Goyal
2009-09-09 13:57 ` Vivek Goyal
2009-09-10 3:06 ` Ryo Tsuruta
2009-09-09 10:01 ` Ryo Tsuruta
2009-09-09 14:31 ` Vivek Goyal
2009-09-09 14:31 ` Vivek Goyal
2009-09-10 3:45 ` Ryo Tsuruta
2009-09-10 13:25 ` Vivek Goyal
2009-09-10 13:25 ` Vivek Goyal
2009-09-08 19:24 ` Rik van Riel
2009-09-08 19:24 ` Rik van Riel
2009-09-09 0:09 ` Fabio Checconi
2009-09-09 2:06 ` Vivek Goyal
2009-09-09 2:06 ` Vivek Goyal
2009-09-09 15:41 ` Fabio Checconi
2009-09-09 17:30 ` Vivek Goyal
2009-09-09 17:30 ` Vivek Goyal
2009-09-09 19:01 ` Fabio Checconi
2009-09-09 9:24 ` Ryo Tsuruta
2009-09-09 9:24 ` Ryo Tsuruta
2009-09-16 4:45 ` ioband: Limited fairness and weak isolation between groups (Was: Re: Regarding dm-ioband tests) Vivek Goyal
2009-09-16 4:45 ` Vivek Goyal
2009-09-18 7:33 ` ioband: Limited fairness and weak isolation between groups Ryo Tsuruta
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20090901174724.GC3753@redhat.com \
--to=vgoyal@redhat.com \
--cc=dm-devel@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=ryov@valinux.co.jp \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.