From mboxrd@z Thu Jan 1 00:00:00 1970 From: Steve Brown Subject: Can't get guests to recognize NUMA architecture as alluded to in Redhat marketing material Date: Sat, 21 Nov 2009 11:20:22 -0500 Message-ID: <4B081346.4010106@stellablue.org> Reply-To: nanog2@stellablue.org Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit To: kvm@vger.kernel.org Return-path: Received: from mail-yx0-f187.google.com ([209.85.210.187]:54477 "EHLO mail-yx0-f187.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752607AbZKUQUV (ORCPT ); Sat, 21 Nov 2009 11:20:21 -0500 Received: by yxe17 with SMTP id 17so3677429yxe.33 for ; Sat, 21 Nov 2009 08:20:27 -0800 (PST) Sender: kvm-owner@vger.kernel.org List-ID: So, based on the following lines from the Redhat PDF on KVM: ....support for large memory systems with NUMA and integrated memory controllers.... ....NUMA support allows virtual machines to efficiently access large amounts of memory.... I decided to try out KVM as an alternative to the Xen setup we have been using where guests are pinned to nodes and limited (by choice) to only the available RAM at said node. This is a two socket, eight core, 72GB system. So I installed CentOS 5.4 and proceeded to use virsh-install to create a guest, simply a CentOS 5.4 guest. I allocated it 40GB or so of RAM to be sure memory allocation would cross node boundaries. I tried using "vcpus=8", "cpuset=auto", "cpuset=1,2 vcpus=8" (that one caused all sorts of problems and CPU lockups), "cpuset=1,2 vcpus=2", "cpuset=1,2" No matter what I still see only one NUMA node in the guest from numastat So what's going on here. Is the PDF misleading? Does a guest not need to know about NUMA and all scheduling/NUMAness handled by KVM? Am I missing some magical configuration line in the XML so the guest understands it's NUMAness? When allocating memory to the guest does the virsh wrapper make all the right backend calls to allocate exactly 50% of requested memory from each physical socket's half of total system memory, in this case 20GB from one socket and 20GB from the other? Any useful comments appreciated. Thanks!