From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andrew Cooper Subject: Re: Xenheap placement in a NUMA system Date: Mon, 28 Sep 2015 14:29:51 +0100 Message-ID: <560940CF.70407@citrix.com> References: <5FF17635-EECE-4832-9AC6-DD64BF88F1E4@yahoo.it> Mime-Version: 1.0 Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: quoted-printable Return-path: In-Reply-To: <5FF17635-EECE-4832-9AC6-DD64BF88F1E4@yahoo.it> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Mario Lodde , xen-devel@lists.xen.org List-Id: xen-devel@lists.xenproject.org On 28/09/15 13:28, Mario Lodde wrote: > Hi everyone, > > I=92m new to Xen and I=92m trying to understand how physical memory is ma= naged. If I=92m understanding correctly the source code, the hypervisor all= ocates the memory it needs only from the xenheap, while the rest of the mem= ory is used by dom0 and the domUs. To a first approximation, yes. The difference between the xenheap and domheap is that the xenheap is always mapped into virtual address space, while the domheap might not be. It is therefore safe to create a pointer to an item in xenheap memory, but domheap pages may only be accessed using {,un}map_domheap_page() (and friends). > If we consider a NUMA system, is the entire xenheap concentrated in the b= oot node or is it spread across the nodes? This is architecture dependent. On x86, the xenheap limit is at the directmap end, which puts it at the 5TiB physical boundary. On a small machine (< 5TB of RAM), xenheap pages will be available across all NUMA nodes. On larger machines, this will not be the case. > In case it=92s all stored in the boot node, and dom0=92s memory is also c= ontained entirely in the same node, is it correct to say that a failure in = another node would just require to shut down the VMs that are running in th= at node but the rest of the system is still able to run? What do you mean by "failure" here? A full DIMM failure would almost certainly be completely fatal to Xen, while an individual cell failure does have logic for being propagated to the owning domain, in the hope that it can take some corrective action. ~Andrew