From mboxrd@z Thu Jan 1 00:00:00 1970 From: Anthony Liguori Subject: Re: [PATCH 4/4] NUMA: realize NUMA memory pinning Date: Mon, 23 Aug 2010 14:27:33 -0500 Message-ID: <4C72CBA5.1020805@codemonkey.ws> References: <1281534738-8310-1-git-send-email-andre.przywara@amd.com> <1281534738-8310-5-git-send-email-andre.przywara@amd.com> <20100823185958.GC32690@amt.cnet> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Andre Przywara , avi@redhat.com, kvm@vger.kernel.org To: Marcelo Tosatti Return-path: Received: from mail-iw0-f174.google.com ([209.85.214.174]:45933 "EHLO mail-iw0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753825Ab0HWT1h (ORCPT ); Mon, 23 Aug 2010 15:27:37 -0400 Received: by iwn5 with SMTP id 5so3780021iwn.19 for ; Mon, 23 Aug 2010 12:27:36 -0700 (PDT) In-Reply-To: <20100823185958.GC32690@amt.cnet> Sender: kvm-owner@vger.kernel.org List-ID: On 08/23/2010 01:59 PM, Marcelo Tosatti wrote: > On Wed, Aug 11, 2010 at 03:52:18PM +0200, Andre Przywara wrote: > >> According to the user-provided assignment bind the respective part >> of the guest's memory to the given host node. This uses Linux' >> mbind syscall (which is wrapped only in libnuma) to realize the >> pinning right after the allocation. >> Failures are not fatal, but produce a warning. >> >> Signed-off-by: Andre Przywara >> --- >> hw/pc.c | 58 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> 1 files changed, 58 insertions(+), 0 deletions(-) >> >> diff --git a/hw/pc.c b/hw/pc.c >> index 1b24409..dbfc082 100644 >> --- a/hw/pc.c >> +++ b/hw/pc.c >> @@ -42,6 +42,15 @@ >> #include "device-assignment.h" >> #include "kvm.h" >> >> +#ifdef CONFIG_NUMA >> +#include >> +#include >> +#ifndef MPOL_F_RELATIVE_NODES >> + #define MPOL_F_RELATIVE_NODES (1<< 14) >> + #define MPOL_F_STATIC_NODES (1<< 15) >> +#endif >> +#endif >> + >> /* output Bochs bios info messages */ >> //#define DEBUG_BIOS >> >> @@ -882,6 +891,53 @@ void pc_cpus_init(const char *cpu_model) >> } >> } >> >> +static void bind_numa(ram_addr_t ram_addr) >> +{ >> +#ifdef CONFIG_NUMA >> + int i; >> + char* ram_ptr; >> + ram_addr_t len, ram_offset; >> + int bind_mode; >> + >> + ram_ptr = qemu_get_ram_ptr(ram_addr); >> + >> + ram_offset = 0; >> + for (i = 0; i< nb_numa_nodes; i++) { >> + len = numa_info[i].guest_mem; >> + if (numa_info[i].flags != 0) { >> + switch (numa_info[i].flags& NODE_HOST_POLICY_MASK) { >> + case NODE_HOST_BIND: >> + bind_mode = MPOL_BIND; >> + break; >> + case NODE_HOST_INTERLEAVE: >> + bind_mode = MPOL_INTERLEAVE; >> + break; >> + case NODE_HOST_PREFERRED: >> + bind_mode = MPOL_PREFERRED; >> + break; >> + default: >> + bind_mode = MPOL_DEFAULT; >> + break; >> + } >> + bind_mode |= (numa_info[i].flags& NODE_HOST_RELATIVE) ? >> + MPOL_F_RELATIVE_NODES : MPOL_F_STATIC_NODES; >> + >> + /* This is a workaround for a long standing bug in Linux' >> + * mbind implementation, which cuts off the last specified >> + * node. To stay compatible should this bug be fixed, we >> + * specify one more node and zero this one out. >> + */ >> + clear_bit(numa_num_configured_nodes() + 1, numa_info[i].host_mem); >> + if (mbind(ram_ptr + ram_offset, len, bind_mode, >> + numa_info[i].host_mem, numa_num_configured_nodes() + 1, 0)) >> + perror("mbind"); >> + } >> + ram_offset += len; >> + } >> +#endif >> > Why is it not possible (or perhaps not desired) to change the binding > after the guest is started? > > Sounds unflexible. > We really need a solution that lets a user use a tool like numactl outside of the QEMU instance. Regards, Anthony Liguori