From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andre Przywara Subject: Using Linux's CPUSET for KVM VCPUs Date: Thu, 22 Jul 2010 16:03:13 +0200 Message-ID: <4C484FA1.60701@amd.com> Mime-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1"; format=flowed Content-Transfer-Encoding: 7bit To: KVM list , QEMU devel Return-path: Received: from tx2ehsobe004.messaging.microsoft.com ([65.55.88.14]:43040 "EHLO TX2EHSOBE007.bigfish.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1759407Ab0GVOJT (ORCPT ); Thu, 22 Jul 2010 10:09:19 -0400 Sender: kvm-owner@vger.kernel.org List-ID: Hi all, while working on NUMA host pinning, I experimented with vCPU affinity within QEMU, but left it alone as it would complicate the code and would not achieve better experience than using taskset with the monitor provided thread ids like it is done currently. During that I looked at Linux' CPUSET implementation (/src/linux-2.6/Documentation/cgroups/cpusets.txt). In brief, this is a pseudo file system based, truly hierarchical implementation of restricting a set of processes (or threads, it uses PIDs) to a certain subset of the machine. Sadly we cannot leverage this for true guest NUMA memory assignment, but it would work nice for pinning (or not) guest vCPUs. I had the following structure in mind: For each guest there is a new CPUSET (mkdir $CPUSET_MNT/`cat /proc/$$/cpuset`/kvm_$guestname). One could then assign the guest global resources to this CPUSET. For each vCPU there is a separate CPUSET located under this guest global one. This would allow for easy manipulation of the pinning of vCPUs, even from the console without any mgt app (although this could be easily implemented in libvirt). / | +--/ kvm_guest_01 | | | +-- VCPU0 | | | +-- VCPU1 | +--/ kvm_guest_02 ... What do you think about it? It is worth implementing this? Regards, Andre. -- Andre Przywara AMD-Operating System Research Center (OSRC), Dresden, Germany Tel: +49 351 448-3567-12