Using Linux's CPUSET for KVM VCPUs

[Andre Przywara <andre.przywara@amd.com>]

Hi all,

while working on NUMA host pinning, I experimented with vCPU affinity 
within QEMU, but left it alone as it would complicate the code and would 
not achieve better experience than using taskset with the monitor 
provided thread ids like it is done currently. During that I looked at 
Linux' CPUSET implementation 
(/src/linux-2.6/Documentation/cgroups/cpusets.txt).
In brief, this is a pseudo file system based, truly hierarchical 
implementation of restricting a set of processes (or threads, it uses 
PIDs) to a certain subset of the machine.
Sadly we cannot leverage this for true guest NUMA memory assignment, but 
it would work nice for pinning (or not) guest vCPUs. I had the following 
structure in mind:
For each guest there is a new CPUSET (mkdir $CPUSET_MNT/`cat 
/proc/$$/cpuset`/kvm_$guestname). One could then assign the guest global 
resources to this CPUSET.
For each vCPU there is a separate CPUSET located under this guest global 
one. This would allow for easy manipulation of the pinning of vCPUs, 
even from the console without any mgt app (although this could be easily 
implemented in libvirt).

/
|
+--/ kvm_guest_01
|  |
|  +-- VCPU0
|  |
|  +-- VCPU1
|
+--/ kvm_guest_02
...

What do you think about it? It is worth implementing this?

Regards,
Andre.


-- 
Andre Przywara
AMD-Operating System Research Center (OSRC), Dresden, Germany
Tel: +49 351 448-3567-12