Re: [PATCH 3/4] x86: provide build time option to support up to 123Tb of memory

[Andrew Cooper <andrew.cooper3@citrix.com>]

On 28/01/15 08:11, Jan Beulich wrote:
> As this requires growing struct page_info from 32 to 48 bytes as well
> as shrinking the always accessible direct mapped memory range from 5Tb
> to 3.5Tb, this isn't being introduced as a general or default enabled
> feature.
>
> For now setting "bigmem=y" implies "shadow-paging=n", as the shadow
> paging code otherwise fails to build (see
> http://lists.xenproject.org/archives/html/xen-devel/2015-01/msg03165.html).

I presume that once the linked list issues are sorted, bigmem and
shadow-paging will no longer be mutually exclusive?

>
> A side effect of the change to x86's mm.h is that asm/mm.h may no
> longer be included directly. Hence in the few places where this was done,
> xen/mm.h is being substituted (indirectly in the hvm/mtrr.h case).
>
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

The changes look surprisingly uninvasive.  Have you got a 123TB machine
to hand?

On a separate tact, I wonder whether it might be an idea about setting
about stealing some virtual address space back from 64bit PV guests.  If
we make a start now, in a couple of years time, it might be a plausible
ABI breakage that vendors would choose to take, as current operating
systems start falling out of support.

>
> --- a/xen/arch/x86/Rules.mk
> +++ b/xen/arch/x86/Rules.mk
> @@ -32,7 +32,8 @@ x86 := y
>  x86_32 := n
>  x86_64 := y
>  
> -shadow-paging ?= y
> +bigmem        ?= n
> +shadow-paging ?= $(if $(filter y,$(bigmem)),n,y)
>  
>  CFLAGS += -mno-red-zone -mno-sse -fpic
>  CFLAGS += -fno-asynchronous-unwind-tables
> @@ -42,3 +43,4 @@ CFLAGS += -DGCC_HAS_VISIBILITY_ATTRIBUTE
>  endif
>  
>  CFLAGS-$(shadow-paging) += -DCONFIG_SHADOW_PAGING
> +CFLAGS-$(bigmem)        += -DCONFIG_BIGMEM
> --- a/xen/arch/x86/hvm/mtrr.c
> +++ b/xen/arch/x86/hvm/mtrr.c
> @@ -18,13 +18,11 @@
>   */
>  
>  #include <public/hvm/e820.h>
> -#include <xen/types.h>
> +#include <xen/domain_page.h>
>  #include <asm/e820.h>
>  #include <asm/iocap.h>
> -#include <asm/mm.h>
>  #include <asm/paging.h>
>  #include <asm/p2m.h>
> -#include <xen/domain_page.h>
>  #include <asm/mtrr.h>
>  #include <asm/hvm/support.h>
>  #include <asm/hvm/cacheattr.h>
> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -386,8 +386,13 @@ static void __init setup_max_pdx(unsigne
>      if ( max_pdx > FRAMETABLE_NR )
>          max_pdx = FRAMETABLE_NR;
>  
> +    if ( max_pdx > MPT_VIRT_SIZE / sizeof(unsigned long) )
> +        max_pdx = MPT_VIRT_SIZE / sizeof(unsigned long);
> +
> +#ifdef PAGE_LIST_NULL

Why does PAGE_LIST_NULL become conditional?  It looks to be
unconditionally available from xen/mm.h.

~Andrew