[PATCH] x86: add feature flags to shared_info page

[PATCH] x86: add feature flags to shared_info page

[Juergen Gross]

In order to indicate the Xen tools capability to support the virtual
mapped linear p2m list instead the 3 level mfn tree add feature flags
to the shared_info page.

Signed-off-by: Juergen Gross <jgross@suse.com>
---
 tools/include/xen-foreign/reference.size |  4 ++--
 xen/include/public/arch-x86/xen.h        | 10 ++++++++++
 2 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/tools/include/xen-foreign/reference.size b/tools/include/xen-foreign/reference.size
index a0409db..9fb1eda 100644
--- a/tools/include/xen-foreign/reference.size
+++ b/tools/include/xen-foreign/reference.size
@@ -9,6 +9,6 @@ vcpu_guest_context        |     344     344    2800    5168
 arch_vcpu_info            |       0       0      24      16
 vcpu_time_info            |      32      32      32      32
 vcpu_info                 |      48      48      64      64
-arch_shared_info          |       0       0      28      48
-shared_info               |    1088    1088    2344    3136
+arch_shared_info          |       0       0      32      56
+shared_info               |    1088    1088    2348    3144
 
diff --git a/xen/include/public/arch-x86/xen.h b/xen/include/public/arch-x86/xen.h
index 232d4a5..ee3b5da 100644
--- a/xen/include/public/arch-x86/xen.h
+++ b/xen/include/public/arch-x86/xen.h
@@ -231,6 +231,9 @@ struct arch_shared_info {
      * been set to invalid e.g. due to the p2m being too large for the 3-level
      * p2m tree. In this case the linear mapper p2m list anchored at p2m_vaddr
      * is to be used.
+     * There is no need for the guest to set pfn_to_mfn_frame_list_list when
+     * Xen indicates the support of the linear mapped p2m list by having set
+     * XEN_FEATURE_P2M_VIRTUAL in features.
      */
     xen_pfn_t     pfn_to_mfn_frame_list_list;
     unsigned long nmi_reason;
@@ -259,6 +262,13 @@ struct arch_shared_info {
     /* There's no room for this field in the generic structure. */
     uint32_t wc_sec_hi;
 #endif
+    /* Features supported by Xen (hypervisor and/or tools) */
+    uint32_t features;
+    /*
+     * Xen supports the linear mapped p2m list, pfn_to_mfn_frame_list_list
+     * isn't required if p2m_cr3 and p2m_vaddr are specified by the guest.
+     */
+#define XEN_FEATURE_P2M_VIRTUAL (1 << 0)
 };
 typedef struct arch_shared_info arch_shared_info_t;
 
-- 
2.1.4

Re: [PATCH] x86: add feature flags to shared_info page

[Jan Beulich]

>>> On 02.03.15 at 13:59, <"jgross@suse.com".non-mime.internet> wrote:
> In order to indicate the Xen tools capability to support the virtual
> mapped linear p2m list instead the 3 level mfn tree add feature flags
> to the shared_info page.

But why in the shared info page? They'd belong to start info or should
be obtainable via XENVER_get_features.

Jan

Re: [PATCH] x86: add feature flags to shared_info page

[Andrew Cooper]

On 02/03/15 13:15, Jan Beulich wrote:
>>>> On 02.03.15 at 13:59, <"jgross@suse.com".non-mime.internet> wrote:
>> In order to indicate the Xen tools capability to support the virtual
>> mapped linear p2m list instead the 3 level mfn tree add feature flags
>> to the shared_info page.
> But why in the shared info page? They'd belong to start info or should
> be obtainable via XENVER_get_features.

Furthermore in this case, the virtual linear p2m is purely a
guest->toolstack feature/ABI.

Xen deliberately has no knowledge of PV guest p2ms of either the 3level
or linear variety.

Is there genuinely no better interface than the hypervisor feature flags
to indicate a piece of toolstack support?

~Andrew

Re: [PATCH] x86: add feature flags to shared_info page

[Ian Campbell]

On Mon, 2015-03-02 at 13:44 +0000, Andrew Cooper wrote:
> On 02/03/15 13:15, Jan Beulich wrote:
> >>>> On 02.03.15 at 13:59, <"jgross@suse.com".non-mime.internet> wrote:
> >> In order to indicate the Xen tools capability to support the virtual
> >> mapped linear p2m list instead the 3 level mfn tree add feature flags
> >> to the shared_info page.
> > But why in the shared info page? They'd belong to start info or should
> > be obtainable via XENVER_get_features.
> 
> Furthermore in this case, the virtual linear p2m is purely a
> guest->toolstack feature/ABI.
> 
> Xen deliberately has no knowledge of PV guest p2ms of either the 3level
> or linear variety.
> 
> Is there genuinely no better interface than the hypervisor feature flags
> to indicate a piece of toolstack support?

Per docs/misc/xenstore-paths.markdown there are platform-feature flags
in ~/control/ (multiproc suspend and xs_reset support) but that might be
available to the guest too late for this use case?

Ian.

Re: [PATCH] x86: add feature flags to shared_info page

[Jan Beulich]

>>> On 02.03.15 at 14:44, <andrew.cooper3@citrix.com> wrote:
> On 02/03/15 13:15, Jan Beulich wrote:
>>>>> On 02.03.15 at 13:59, <"jgross@suse.com".non-mime.internet> wrote:
>>> In order to indicate the Xen tools capability to support the virtual
>>> mapped linear p2m list instead the 3 level mfn tree add feature flags
>>> to the shared_info page.
>> But why in the shared info page? They'd belong to start info or should
>> be obtainable via XENVER_get_features.
> 
> Furthermore in this case, the virtual linear p2m is purely a
> guest->toolstack feature/ABI.
> 
> Xen deliberately has no knowledge of PV guest p2ms of either the 3level
> or linear variety.
> 
> Is there genuinely no better interface than the hypervisor feature flags
> to indicate a piece of toolstack support?

As Ian indicated in his reply, much depends on whether any other
mechanism would allow the information to be retrieved early enough
in the guest.

Jan

Re: [PATCH] x86: add feature flags to shared_info page

[Juergen Gross]

On 03/02/2015 02:56 PM, Jan Beulich wrote:
>>>> On 02.03.15 at 14:44, <andrew.cooper3@citrix.com> wrote:
>> On 02/03/15 13:15, Jan Beulich wrote:
>>>>>> On 02.03.15 at 13:59, <"jgross@suse.com".non-mime.internet> wrote:
>>>> In order to indicate the Xen tools capability to support the virtual
>>>> mapped linear p2m list instead the 3 level mfn tree add feature flags
>>>> to the shared_info page.
>>> But why in the shared info page? They'd belong to start info or should
>>> be obtainable via XENVER_get_features.
>>
>> Furthermore in this case, the virtual linear p2m is purely a
>> guest->toolstack feature/ABI.
>>
>> Xen deliberately has no knowledge of PV guest p2ms of either the 3level
>> or linear variety.
>>
>> Is there genuinely no better interface than the hypervisor feature flags
>> to indicate a piece of toolstack support?
>
> As Ian indicated in his reply, much depends on whether any other
> mechanism would allow the information to be retrieved early enough
> in the guest.

Options that would work are, regarding the time the information is
needed:

- XENVER_get_features
- start info
- shared info
- any (other) hypercall

All other interfaces are available much too late.

I think start info is the best option, as it is built by the tools for
domUs and, as Jan already mentioned, would be a better place for the
information as shared info.

The last remaining question: what to do regarding dom0? Here I see the
following alternatives:

- do nothing: the 3 level mfn tree is built even if it is not needed,
   wastes up to 2MB memory and might slow down dom0 (I doubt the slow
   down would be detectable)
- set the flag based on a hypervisor boot parameter (not very nice)
- throw the 3 level mfn tree away during boot as soon as the tools can
   tell dom0 to do so (requires a new interface)

Any thoughts?


Juergen

Re: [PATCH] x86: add feature flags to shared_info page

[Andrew Cooper]

On 02/03/15 15:11, Juergen Gross wrote:
> On 03/02/2015 02:56 PM, Jan Beulich wrote:
>>>>> On 02.03.15 at 14:44, <andrew.cooper3@citrix.com> wrote:
>>> On 02/03/15 13:15, Jan Beulich wrote:
>>>>>>> On 02.03.15 at 13:59, <"jgross@suse.com".non-mime.internet> wrote:
>>>>> In order to indicate the Xen tools capability to support the virtual
>>>>> mapped linear p2m list instead the 3 level mfn tree add feature flags
>>>>> to the shared_info page.
>>>> But why in the shared info page? They'd belong to start info or should
>>>> be obtainable via XENVER_get_features.
>>>
>>> Furthermore in this case, the virtual linear p2m is purely a
>>> guest->toolstack feature/ABI.
>>>
>>> Xen deliberately has no knowledge of PV guest p2ms of either the 3level
>>> or linear variety.
>>>
>>> Is there genuinely no better interface than the hypervisor feature
>>> flags
>>> to indicate a piece of toolstack support?
>>
>> As Ian indicated in his reply, much depends on whether any other
>> mechanism would allow the information to be retrieved early enough
>> in the guest.
>
> Options that would work are, regarding the time the information is
> needed:
>
> - XENVER_get_features
> - start info
> - shared info
> - any (other) hypercall
>
> All other interfaces are available much too late.
>
> I think start info is the best option, as it is built by the tools for
> domUs and, as Jan already mentioned, would be a better place for the
> information as shared info.

I concur.  The start info seems like the best place for it.

>
> The last remaining question: what to do regarding dom0? Here I see the
> following alternatives:
>
> - do nothing: the 3 level mfn tree is built even if it is not needed,
>   wastes up to 2MB memory and might slow down dom0 (I doubt the slow
>   down would be detectable)
> - set the flag based on a hypervisor boot parameter (not very nice)
> - throw the 3 level mfn tree away during boot as soon as the tools can
>   tell dom0 to do so (requires a new interface)
>
> Any thoughts?

The toolstack only needs to access the p2m when doing suspend/resume for
migrate, or a coredump.  None of these are applicable to dom0, so I
think dom0 can get away with doing whatever it prefers in this regard.

~Andrew

Re: [PATCH] x86: add feature flags to shared_info page

[Juergen Gross]

On 03/02/2015 04:22 PM, Andrew Cooper wrote:
> On 02/03/15 15:11, Juergen Gross wrote:
>> On 03/02/2015 02:56 PM, Jan Beulich wrote:
>>>>>> On 02.03.15 at 14:44, <andrew.cooper3@citrix.com> wrote:
>>>> On 02/03/15 13:15, Jan Beulich wrote:
>>>>>>>> On 02.03.15 at 13:59, <"jgross@suse.com".non-mime.internet> wrote:
>>>>>> In order to indicate the Xen tools capability to support the virtual
>>>>>> mapped linear p2m list instead the 3 level mfn tree add feature flags
>>>>>> to the shared_info page.
>>>>> But why in the shared info page? They'd belong to start info or should
>>>>> be obtainable via XENVER_get_features.
>>>>
>>>> Furthermore in this case, the virtual linear p2m is purely a
>>>> guest->toolstack feature/ABI.
>>>>
>>>> Xen deliberately has no knowledge of PV guest p2ms of either the 3level
>>>> or linear variety.
>>>>
>>>> Is there genuinely no better interface than the hypervisor feature
>>>> flags
>>>> to indicate a piece of toolstack support?
>>>
>>> As Ian indicated in his reply, much depends on whether any other
>>> mechanism would allow the information to be retrieved early enough
>>> in the guest.
>>
>> Options that would work are, regarding the time the information is
>> needed:
>>
>> - XENVER_get_features
>> - start info
>> - shared info
>> - any (other) hypercall
>>
>> All other interfaces are available much too late.
>>
>> I think start info is the best option, as it is built by the tools for
>> domUs and, as Jan already mentioned, would be a better place for the
>> information as shared info.
>
> I concur.  The start info seems like the best place for it.
>
>>
>> The last remaining question: what to do regarding dom0? Here I see the
>> following alternatives:
>>
>> - do nothing: the 3 level mfn tree is built even if it is not needed,
>>    wastes up to 2MB memory and might slow down dom0 (I doubt the slow
>>    down would be detectable)
>> - set the flag based on a hypervisor boot parameter (not very nice)
>> - throw the 3 level mfn tree away during boot as soon as the tools can
>>    tell dom0 to do so (requires a new interface)
>>
>> Any thoughts?
>
> The toolstack only needs to access the p2m when doing suspend/resume for
> migrate, or a coredump.  None of these are applicable to dom0, so I
> think dom0 can get away with doing whatever it prefers in this regard.

Are core dumps of dom0 really no issue? I think crash is using the 3
level mfn tree today.

Juergen

Re: [PATCH] x86: add feature flags to shared_info page

[Jan Beulich]

>>> On 02.03.15 at 16:40, <JGross@suse.com> wrote:
> On 03/02/2015 04:22 PM, Andrew Cooper wrote:
>> The toolstack only needs to access the p2m when doing suspend/resume for
>> migrate, or a coredump.  None of these are applicable to dom0, so I
>> think dom0 can get away with doing whatever it prefers in this regard.
> 
> Are core dumps of dom0 really no issue? I think crash is using the 3
> level mfn tree today.

Perhaps, but communicating crash's (perhaps even running on
another host) capabilities by other than a command line option to
Dom0 would seem rather odd.

Jan

Re: [PATCH] x86: add feature flags to shared_info page

[Andrew Cooper]

On 02/03/15 15:40, Juergen Gross wrote:
> On 03/02/2015 04:22 PM, Andrew Cooper wrote:
>> On 02/03/15 15:11, Juergen Gross wrote:
>>> On 03/02/2015 02:56 PM, Jan Beulich wrote:
>>>>>>> On 02.03.15 at 14:44, <andrew.cooper3@citrix.com> wrote:
>>>>> On 02/03/15 13:15, Jan Beulich wrote:
>>>>>>>>> On 02.03.15 at 13:59, <"jgross@suse.com".non-mime.internet>
>>>>>>>>> wrote:
>>>>>>> In order to indicate the Xen tools capability to support the
>>>>>>> virtual
>>>>>>> mapped linear p2m list instead the 3 level mfn tree add feature
>>>>>>> flags
>>>>>>> to the shared_info page.
>>>>>> But why in the shared info page? They'd belong to start info or
>>>>>> should
>>>>>> be obtainable via XENVER_get_features.
>>>>>
>>>>> Furthermore in this case, the virtual linear p2m is purely a
>>>>> guest->toolstack feature/ABI.
>>>>>
>>>>> Xen deliberately has no knowledge of PV guest p2ms of either the
>>>>> 3level
>>>>> or linear variety.
>>>>>
>>>>> Is there genuinely no better interface than the hypervisor feature
>>>>> flags
>>>>> to indicate a piece of toolstack support?
>>>>
>>>> As Ian indicated in his reply, much depends on whether any other
>>>> mechanism would allow the information to be retrieved early enough
>>>> in the guest.
>>>
>>> Options that would work are, regarding the time the information is
>>> needed:
>>>
>>> - XENVER_get_features
>>> - start info
>>> - shared info
>>> - any (other) hypercall
>>>
>>> All other interfaces are available much too late.
>>>
>>> I think start info is the best option, as it is built by the tools for
>>> domUs and, as Jan already mentioned, would be a better place for the
>>> information as shared info.
>>
>> I concur.  The start info seems like the best place for it.
>>
>>>
>>> The last remaining question: what to do regarding dom0? Here I see the
>>> following alternatives:
>>>
>>> - do nothing: the 3 level mfn tree is built even if it is not needed,
>>>    wastes up to 2MB memory and might slow down dom0 (I doubt the slow
>>>    down would be detectable)
>>> - set the flag based on a hypervisor boot parameter (not very nice)
>>> - throw the 3 level mfn tree away during boot as soon as the tools can
>>>    tell dom0 to do so (requires a new interface)
>>>
>>> Any thoughts?
>>
>> The toolstack only needs to access the p2m when doing suspend/resume for
>> migrate, or a coredump.  None of these are applicable to dom0, so I
>> think dom0 can get away with doing whatever it prefers in this regard.
>
> Are core dumps of dom0 really no issue? I think crash is using the 3
> level mfn tree today.

The only coredump of dom0 you are ever going to find is via /proc/vmcore
of a kdump environment, which is a Xen coredump with dom0 being just
another domain.

I have not used `crash` in this scenario, but it would be actively
making work for itself to attempt to piece dom0 back into a regular
image similar to `xl dump-core`, rather than simply using cr3 available
from struct vcpu.

So long as there is a kernel command line option available, the user who
sets up the crash environment can force it back into a state that their
tools expect.

~Andrew