From mboxrd@z Thu Jan 1 00:00:00 1970 From: Anthony Liguori Subject: Re: [PATCH 0/3][RFC] NUMA: add host side pinning Date: Wed, 23 Jun 2010 17:21:17 -0500 Message-ID: <4C2288DD.3020207@codemonkey.ws> References: <1277327377-29629-1-git-send-email-andre.przywara@amd.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: kvm@vger.kernel.org, agraf@suse.de To: Andre Przywara Return-path: Received: from mail-gy0-f174.google.com ([209.85.160.174]:56066 "EHLO mail-gy0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753241Ab0FWWVO (ORCPT ); Wed, 23 Jun 2010 18:21:14 -0400 Received: by gye5 with SMTP id 5so4009664gye.19 for ; Wed, 23 Jun 2010 15:21:14 -0700 (PDT) In-Reply-To: <1277327377-29629-1-git-send-email-andre.przywara@amd.com> Sender: kvm-owner@vger.kernel.org List-ID: On 06/23/2010 04:09 PM, Andre Przywara wrote: > Hi, > > these three patches add basic NUMA pinning to KVM. According to a user > provided assignment parts of the guest's memory will be bound to different > host nodes. This should increase performance in large virtual machines > and on loaded hosts. > These patches are quite basic (but work) and I send them as RFC to get > some feedback before implementing stuff in vain. > > To use it you need to provide a guest NUMA configuration, this could be > as simple as "-numa node -numa node" to give two nodes in the guest. Then > you pin these nodes on a separate command line option to different host > nodes: "-numa pin,nodeid=0,host=0 -numa pin,nodeid=1,host=2" > This separation of host and guest config sounds a bit complicated, but > was demanded last time I submitted a similar version. > I refrained from binding the vCPUs to physical CPUs for now, but this > can be added later with an "cpubind" option to "-numa pin,". Also this > could be done from a management application by using sched_setaffinity(). > > Please note that this is currently made for qemu-kvm, although I am not > up-to-date regarding the curent status of upstreams QEMU's true SMP > capabilities. The final patch will be made against upstream QEMU anyway. > Also this is currently for Linux hosts (any other KVM hosts alive?) and > for PC guests only. I think both can be fixed easily if someone requests > it (and gives me a pointer to further information). > > Please comment on the approach in general and the implementation. > If we extended integrated -mem-path with -numa such that a different path could be used with each numa node (and we let an explicit file be specified instead of just a directory), then if I understand correctly, we could use numactl without any specific integration in qemu. Does this sound correct? IOW: qemu -numa node,mem=1G,nodeid=0,cpus=0-1,memfile=/dev/shm/node0.mem -numa node,mem=2G,nodeid=1,cpus=1-2,memfile=/dev/shm/node1.mem It's then possible to say: numactl --file /dev/shm/node0.mem --interleave=0,1 numactl --file /dev/shm/node1.mem --membind=2 I think this approach is nicer because it gives the user a lot more flexibility without having us chase other tools like numactl. For instance, your patches only support pinning and not interleaving. Regards, Anthony Liguori > Thanks and Regards, > Andre. > >