Re: [Qemu-devel] [RFC PATCH] Exporting Guest RAM information for NUMA binding

[Bharata B Rao <bharata@linux.vnet.ibm.com>]

On Mon, Nov 21, 2011 at 04:25:26PM +0100, Peter Zijlstra wrote:
> On Mon, 2011-11-21 at 20:48 +0530, Bharata B Rao wrote:
> 
> > I looked at Peter's recent work in this area.
> > (https://lkml.org/lkml/2011/11/17/204)
> > 
> > It introduces two interfaces:
> > 
> > 1. ms_tbind() to bind a thread to a memsched(*) group
> > 2. ms_mbind() to bind a memory region to memsched group
> > 
> > I assume the 2nd interface could be used by QEMU to create
> > memsched groups for each of guest NUMA node memory regions.
> 
> No, you would need both, you'll need to group vcpu threads _and_ some
> vaddress space together.
> 
> I understood QEMU currently uses a single big anonymous mmap() to
> allocate the guest memory, using this you could either use multiple or
> carve up the big alloc into virtual nodes by assigning different parts
> to different ms groups.
> 
> Example: suppose you want to create a 2 node guest with 8 vcpus, create
> 2 ms groups, each with 4 vcpu threads and assign half the total guest
> mmap to either.
> 
> > In the past, Anthony has said that NUMA binding should be done from outside
> > of QEMU (http://www.kerneltrap.org/mailarchive/linux-kvm/2010/8/31/6267041)
> 
> If you want to expose a sense of virtual NUMA to your guest you really
> have no choice there. The only thing you can do externally is run whole
> VMs inside one particular node.
> 
> > Though that was in a different context, may be we should re-look at that
> > and see if QEMU still sticks to that. I know its a bit early, but if needed
> > we should ask Peter to consider extending ms_mbind() to take a tid parameter
> > too instead of working on current task by default.
> 
> Uh, what for? ms_mbind() works on the current process, not task.

In the original post of this mail thread, I proposed a way to export
guest RAM ranges (Guest Physical Address-GPA) and their corresponding host
host virtual mappings (Host Virtual Address-HVA) from QEMU (via QEMU monitor).
The idea was to use this GPA to HVA mappings from tools like libvirt to bind
specific parts of the guest RAM to different host nodes. This needed an
extension to existing mbind() to allow binding memory of a process(QEMU) from a
different process(libvirt). This was needed since we wanted to do all this from
libvirt.

Hence I was coming from that background when I asked for extending
ms_mbind() to take a tid parameter. If QEMU community thinks that NUMA
binding should all be done from outside of QEMU, it is needed, otherwise
what you have should be sufficient.

Regards,
Bharata.