Re: Regarding dm-ioband tests

[Vivek Goyal <vgoyal@redhat.com>]

On Tue, Sep 01, 2009 at 01:47:24PM -0400, Vivek Goyal wrote:
> On Tue, Sep 01, 2009 at 12:50:11PM -0400, Vivek Goyal wrote:
> > Hi Ryo,
> > 
> > I decided to play a bit more with dm-ioband and started doing some
> > testing. I am doing a simple two dd threads doing reads and don't seem
> > to be gettting the fairness. So thought will ask you what's the issue
> > here. Is there an issue with my testing procedure.
> > 
> > I got one 40G SATA drive (no hardware queuing). I have created two
> > partitions on that disk /dev/sdd1 and /dev/sdd2 and created two ioband
> > devices ioband1 and ioband2 on partitions sdd1 and sdd2 respectively. The
> > weights of ioband1 and ioband2 devices are 200 and 100 respectively. 
> > 
> > I am assuming that this setup will create two default groups and IO
> > going to partition sdd1 should get double the BW of partition sdd2.
> > 
> > But it looks like I am not gettting that behavior. Following is the output
> > of "dmsetup table" command. This snapshot has been taken every 2 seconds
> > while IO was going on. Column 9 seems to be containing how many sectors
> > of IO has been done on a particular io band device and group. Looking at
> > the snapshot, it does not look like that ioband1 default group got double
> > the BW of ioband2 default group.  
> > 
> > Am I doing something wrong here?
> > 
> 

Hi Ryo,

Did you get a chance to look into it? Am I doing something wrong or it is
an issue with dm-ioband.

Thanks
Vivek

> I tried another variant of test. This time I also created two additional
> groups on ioband1 devices and linked these to cgroups test1 and test2 and
> launched two dd threads in two cgroups on device ioband1. There also I
> don't seem to be getting the right fairness numbers for cgroup test1 and
> test2.
> 
> Script to create ioband devices and additional groups
> -----------------------------------------------------
> echo "0 $(blockdev --getsize /dev/sdd1) ioband /dev/sdd1 1 0 0 none"
> "weight 0 :200" | dmsetup create ioband1
> echo "0 $(blockdev --getsize /dev/sdd2) ioband /dev/sdd2 1 0 0 none"
> "weight 0 :100" | dmsetup create ioband2
> 
> # Some code to mount and create cgroups.
> # Read group id
> test1_id=`cat /cgroup/ioband/test1/blkio.id`
> test2_id=`cat /cgroup/ioband/test2/blkio.id`
> 
> test1_weight=200
> test2_weight=100
> 
> dmsetup message ioband1 0 type cgroup 
> dmsetup message ioband1 0 attach $test1_id
> dmsetup message ioband1 0 attach $test2_id
> dmsetup message ioband1 0 weight $test1_id:$test1_weight
> dmsetup message ioband1 0 weight $test2_id:$test2_weight
> 
> mount /dev/mapper/ioband1 /mnt/sdd1
> mount /dev/mapper/ioband2 /mnt/sdd2
> -----------------------------------------------------------------
> 
> Following are two dd jobs
> -------------------------
> dd if=/mnt/sdd1/testzerofile1 of=/dev/null &
> echo $! > /cgroup/ioband/test1/tasks
> 
> dd if=/mnt/sdd1/testzerofile2 of=/dev/null &
> echo $! > /cgroup/ioband/test2/tasks
> 
> 
> Following are "dmsetup status" results every 2 seconds
> ======================================================
> 
> ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
> ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0 2 0 0 0 0 0 0 3 0 0 0 0 0 0
> 
> ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
> ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0 2 0 0 0 0 0 0 3 0 0 0 0 0 0
> 
> ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
> ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0 2 689 0 86336 0 0 0 3 650 3
> 81472 0 0 0
> 
> ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
> ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0 2 1725 0 217024 0 0 0 3 1270
> 11 158912 0 0 0
> 
> ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
> ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0 2 2690 0 338744 0 0 0 3 1978
> 15 247856 0 0 0
> 
> ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
> ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0 2 3762 0 474040 0 0 0 3 2583
> 21 323736 0 0 0
> 
> ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
> ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0 2 4745 0 598064 0 0 0 3 3275
> 27 410392 0 0 0
> 
> ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
> ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0 2 5737 0 723120 0 0 0 3 3985
> 31 499592 0 0 0
> 
> ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
> ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0 2 6815 0 859184 0 0 0 3 4594
> 37 575864 0 0 0
> 
> ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
> ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0 2 7823 0 986288 0 0 0 3 5276
> 43 661360 0 0 0
> 
> "dmsetup table" output
> ======================
> ioband2: 0 40355280 ioband 8:50 1 4 192 none weight 768 :100
> ioband1: 0 37768752 ioband 8:49 1 4 192 cgroup weight 768 :200 2:200 3:100
> 
> Because I am using "weight" policy, I thought that test1 cgroup with id
> "2" will issue double the number of requests of cgroup test2 with id "3".
> But that does not seem to be happening here. Is there an issue with my
> testing method.
> 
> Thanks
> Vivek
>  
> > ioband2: 0 40355280 ioband 1 -1 0 0 0 0 0 0
> > ioband1: 0 37768752 ioband 1 -1 0 0 0 0 0 0
> > 
> > ioband2: 0 40355280 ioband 1 -1 96 0 11528 0 0 0
> > ioband1: 0 37768752 ioband 1 -1 82 0 9736 0 0 0
> > 
> > ioband2: 0 40355280 ioband 1 -1 748 2 93032 0 0 0
> > ioband1: 0 37768752 ioband 1 -1 896 0 112232 0 0 0
> > 
> > ioband2: 0 40355280 ioband 1 -1 1326 5 165816 0 0 0
> > ioband1: 0 37768752 ioband 1 -1 1816 0 228312 0 0 0
> > 
> > ioband2: 0 40355280 ioband 1 -1 1943 6 243712 0 0 0
> > ioband1: 0 37768752 ioband 1 -1 2692 0 338760 0 0 0
> > 
> > ioband2: 0 40355280 ioband 1 -1 2461 10 308576 0 0 0
> > ioband1: 0 37768752 ioband 1 -1 3618 0 455608 0 0 0
> > 
> > ioband2: 0 40355280 ioband 1 -1 3118 11 391352 0 0 0
> > ioband1: 0 37768752 ioband 1 -1 4406 0 555032 0 0 0
> > 
> > ioband2: 0 40355280 ioband 1 -1 3734 15 468760 0 0 0
> > ioband1: 0 37768752 ioband 1 -1 5273 0 664328 0 0 0
> > 
> > ioband2: 0 40355280 ioband 1 -1 4307 17 540784 0 0 0
> > ioband1: 0 37768752 ioband 1 -1 6181 0 778992 0 0 0
> > 
> > ioband2: 0 40355280 ioband 1 -1 4930 19 619208 0 0 0
> > ioband1: 0 37768752 ioband 1 -1 7028 0 885728 0 0 0
> > 
> > ioband2: 0 40355280 ioband 1 -1 5599 22 703280 0 0 0
> > ioband1: 0 37768752 ioband 1 -1 7815 0 985024 0 0 0
> > 
> > ioband2: 0 40355280 ioband 1 -1 6586 27 827456 0 0 0
> > ioband1: 0 37768752 ioband 1 -1 8327 0 1049624 0 0 0
> > 
> > Following are details of my test setup.
> > ---------------------------------------
> > I took dm-ioband patch version 1.12.3 and applied on 2.6.31-rc6.
> > 
> > Created ioband devices using following command.
> > ----------------------------------------------
> > echo "0 $(blockdev --getsize /dev/sdd1) ioband /dev/sdd1 1 0 0 none"
> > "weight 0 :200" | dmsetup create ioband1
> > echo "0 $(blockdev --getsize /dev/sdd2) ioband /dev/sdd2 1 0 0 none"
> > "weight 0 :100" | dmsetup create ioband2
> > 
> > mount /dev/mapper/ioband1 /mnt/sdd1
> > mount /dev/mapper/ioband2 /mnt/sdd2
> > 
> > Started two dd threads
> > ======================
> > dd if=/mnt/sdd1/testzerofile1 of=/dev/null &
> > dd if=/mnt/sdd2/testzerofile1 of=/dev/null &
> > 
> > Output of dmsetup table command
> > ================================
> > ioband2: 0 40355280 ioband 8:50 1 4 192 none weight 768 :100
> > ioband1: 0 37768752 ioband 8:49 1 4 192 none weight 768 :200
> > 
> > Thanks
> > Vivek