From mboxrd@z Thu Jan  1 00:00:00 1970
From: Anthony Liguori <anthony@codemonkey.ws>
Subject: Re: [PATCH 0/3] KVM-userspace: add NUMA support for guests
Date: Mon, 01 Dec 2008 09:27:57 -0600
Message-ID: <4934027D.8080904@codemonkey.ws>
References: <492F1DD9.8030901@amd.com> <49318A10.7080801@redhat.com> <4933F177.5040802@amd.com> <4933F4B0.7040500@redhat.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Cc: Andre Przywara <andre.przywara@amd.com>, kvm@vger.kernel.org,
	"Daniel P. Berrange" <berrange@redhat.com>, Andi Kleen <ak@suse.de>
To: Avi Kivity <avi@redhat.com>
Return-path: <kvm-owner@vger.kernel.org>
Received: from yx-out-2324.google.com ([74.125.44.29]:43257 "EHLO
	yx-out-2324.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
	with ESMTP id S1751503AbYLAP2C (ORCPT <rfc822;kvm@vger.kernel.org>);
	Mon, 1 Dec 2008 10:28:02 -0500
Received: by yx-out-2324.google.com with SMTP id 8so952672yxm.1
        for <kvm@vger.kernel.org>; Mon, 01 Dec 2008 07:28:01 -0800 (PST)
In-Reply-To: <4933F4B0.7040500@redhat.com>
Sender: kvm-owner@vger.kernel.org
List-ID: <kvm.vger.kernel.org>

Avi Kivity wrote:
> Andre Przywara wrote:
>
>> Any other useful commands for the monitor? Maybe (temporary) VCPU 
>> migration without page migration?
>
> Right now vcpu migration is done externally (we export the thread IDs 
> so management can pin them as it wishes).  If we add numa support, I 
> think it makes sense do it internally as well.  I suggest using the 
> same syntax for the monitor as for the command line; that's simplest 
> to learn and to implement.

I see no compelling reason to do cpu placement internally.  It can be 
done quite effectively externally.

Memory allocation is tough, but I don't think it's out of reach.  
Looking at the numactl man page, you can do:

numactl  --offset=1G  --length=1G --membind=1 --file /dev/shm/A --touch
       Bind the second gigabyte in the tmpfs file /dev/shm/A to node 1.


Since we can already create VM's with the -mem-path argument, if you 
create a 2GB guest and want it to span two numa nodes, you could do:

numactl  --offset=0G  --length=1G --membind=0 --file /dev/shm/A --touch
numactl  --offset=1G  --length=1G --membind=1 --file /dev/shm/A --touch

And then create the VM with:

qemu-system-x86_64 -mem-path /dev/shm/A -mem 2G ...

What's best about this approach, is that you get full access to what 
numactl is capable of.  Interleaving, rebalancing, etc.

Regards,

Anthony Liguori