Re: [PATCH V3 1/1] expand x86 arch_shared_info to support >3 level p2m tree

[Juergen Gross <jgross@suse.com>]

On 09/09/2014 12:27 PM, Andrew Cooper wrote:
> On 09/09/14 10:58, Juergen Gross wrote:
>> The x86 struct arch_shared_info field pfn_to_mfn_frame_list_list
>> currently contains the mfn of the top level page frame of the 3 level
>> p2m tree, which is used by the Xen tools during saving and restoring
>> (and live migration) of pv domains. With three levels of the p2m tree
>> it is possible to support up to 512 GB of RAM for a 64 bit pv domain.
>> A 32 bit pv domain can support more, as each memory page can hold 1024
>> instead of 512 entries, leading to a limit of 4 TB. To be able to
>> support more RAM on x86-64 an additional level is to be added.
>>
>> This patch expands struct arch_shared_info with a new p2m tree root
>> and the number of levels of the p2m tree. The new information is
>> indicated by the domain to be valid by storing ~0UL into
>> pfn_to_mfn_frame_list_list (this should be done only if more than
>> three levels are needed, of course).
>
> A small domain feeling a little tight on space could easily opt for a 2
> or even 1 level p2m.  (After all, one advantage of virt is to cram many
> small VMs into a server).
>
> How is xen and toolstack support for n-level p2ms going to be advertised
> to guests?  Simply assuming the toolstack is capable of dealing with
> this new scheme wont work with a new pv guest running on an older Xen.

Is it really worth doing such an optimization? This would save only very
few pages.

If you think it should be done we can add another SIF_* flag to
start_info->flags. In this case a domain using this feature could not be
migrated to a server with old tools, however. So we would probably end
with the need to be able to suppress that flag on a per-domain base.

Juergen

>
> ~Andrew
>
>>
>> Signed-off-by: Juergen Gross <jgross@suse.com>
>> ---
>>   xen/include/public/arch-x86/xen.h | 7 ++++++-
>>   1 file changed, 6 insertions(+), 1 deletion(-)
>>
>> diff --git a/xen/include/public/arch-x86/xen.h b/xen/include/public/arch-x86/xen.h
>> index f35804b..2ca996c 100644
>> --- a/xen/include/public/arch-x86/xen.h
>> +++ b/xen/include/public/arch-x86/xen.h
>> @@ -224,7 +224,12 @@ struct arch_shared_info {
>>       /* Frame containing list of mfns containing list of mfns containing p2m. */
>>       xen_pfn_t     pfn_to_mfn_frame_list_list;
>>       unsigned long nmi_reason;
>> -    uint64_t pad[32];
>> +    /*
>> +     * Following two fields are valid if pfn_to_mfn_frame_list_list contains
>> +     * ~0UL.
>> +     */
>> +    unsigned long p2m_levels;   /* number of levels of p2m tree */
>> +    xen_pfn_t     p2m_root;     /* p2m tree top level mfn */
>>   };
>>   typedef struct arch_shared_info arch_shared_info_t;
>>
>
>
>