Re: vNUMA for PV guest: kernel and toolstack interaction regarding e820_host=1

[David Vrabel <david.vrabel@citrix.com>]

On 06/02/15 19:32, Wei Liu wrote:
> Hi all
> 
> I encounter a problem that I would like to get some advice. It's PV
> specific because of the P2M manipulation is only required by PV.
> 
> Current scheme of memory allocation scheme:
> 
> 1. Libxc populate contiguous chunk of pages and fill in initial P2M. The
>    holes in e820 map are in fact filled with pages.
> 
> 2. Guest kernel reads e820 map from Xen and remap pages in e820 holes if
>    there are holes, update P2M as it sees fit. (That is normally true when
>    e820_host=1 is set)
> 
> This is not very ideal for PV vNUMA, because those pages remapped may
> end up in the wrong vnode.
> 
> What I have in mind is:
> 
> 1. Libxc populates pages, but skips e820 holes. The initial P2M is the
>    final P2M guest sees.
> 2. Guest kernel skips remapping. But Linux still needs to setup 1-1
>    mapping for holes.
> 
> In order to avoid misconfiguration, we would need to introduce a new
> feature flag to indicate guest has the ability to skip remapping. Libxc
> will check that feature flag when building domain.
> 
> Does the above scheme make sense?

I really not keen on any additional complexity in PV guest memory setup.
 Particularly as I don't see a long term future for x86 PV guests.  I
also don't think we should bake into the Xen ABI the behaviour of one
particular guest.

Consider how you can fix this purely in the guest.

David