[libvirt] cgroup limits only affect kvm guest under certain conditions

[Dominik Klein <dk@in-telegence.net>]

Hi

I am playing with cgroups and try to limit block io for guests.

The proof of concept is:

# mkdir /dev/cgroup/blkio
# mount -t cgroup -o blkio blkio /dev/cgroup/blkio/
# cd blkio/
# mkdir test
# cd test/
# ls -l /dev/vdisks/kirk
lrwxrwxrwx 1 root root 7 2011-01-06 13:46 /dev/vdisks/kirk -> ../dm-5
# ls -l /dev/dm-5
brw-rw---- 1 root disk 253, 5 2011-01-06 13:36 /dev/dm-5
# echo "253:5  1048576" > blkio.throttle.write_bps_device
# echo $$ > tasks
# dd if=/dev/zero of=/dev/dm-5 bs=1M count=20
20+0 records in
20+0 records out
20971520 bytes (21 MB) copied, 20.0223 s, 1.0 MB/s

So limit applies to the dd child of my shell.

Now I assign /dev/dm-5 (/dev/vdisks/kirk) to a vm and echo the qemu-kvm
pid into tasks. Limits are not applied, the guest can happily use max io
bandwidth.

However, if I start the guest manually like

# qemu-kvm <options like libvirt creates> & echo $! >
/dev/cgroup/blkio/test/tasks

The limits _are_ applied.

So, this looks like some sort of race condition to me.

I tried to get information on this on the kernel mailing list [1], but
either noone read it or at least no one replied. Maybe someone here can
shed some light and maybe even fix the issue, if it is an issue.

Need more information? Please ask for it. I don't know what else to
supply at this point.

My current lab is built by:
OpenSuSE 11.3 64bit
Vanilla Kernel 2.6.37
libvirt 0.8.7
qemu-kvm 0.13.0
vm is started like this:
/usr/bin/qemu-kvm -M pc-0.12 -enable-kvm -m 2048 -smp
2,sockets=2,cores=1,threads=1 -name cliff -uuid
a8247e1e-e3d2-d0fc-c5e5-47a173c3e460 -nodefconfig -nodefaults -chardev
socket,id=monitor,path=/var/lib/libvirt/qemu/cliff.monitor,server,nowait
-mon chardev=monitor,mode=readline -rtc base=utc -boot c -device
lsi,id=scsi0,bus=pci.0,addr=0x6 -drive
file=/dev/vdisks/cliff,if=none,id=drive-virtio-disk0,boot=on,format=raw
-device
virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0
-drive
file=/root/openSUSE-11.3-NET-x86_64.iso,if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw
-device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0
-drive file=/dev/vdisks/jason,if=none,id=drive-virtio-disk1,format=raw
-device
virtio-blk-pci,bus=pci.0,addr=0x7,drive=drive-virtio-disk1,id=virtio-disk1
-drive file=/dev/vdisks/rob,if=none,id=drive-ide0-0-0,format=raw -device
ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -drive
file=/dev/vdisks/james,if=none,id=drive-scsi0-0-0,format=raw -device
scsi-disk,bus=scsi0.0,scsi-id=0,drive=drive-scsi0-0-0,id=scsi0-0-0
-netdev tap,id=hostnet0 -device
virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:37:84:e0,bus=pci.0,addr=0x5
-usb -vnc 127.0.0.1:0 -vga cirrus -device
virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3

Regards
Dominik

[1]
http://help.lockergnome.com/linux/race-condition-net_cls-found-qemu-kvm-environment--ftopict529787.html