[PATCH for-4.19 v2] x86/pvh: declare PVH dom0 supported with caveats

[PATCH for-4.19 v2] x86/pvh: declare PVH dom0 supported with caveats

[Roger Pau Monne]

PVH dom0 is functionally very similar to PVH domU except for the domain
builder and the added set of hypercalls available to it.

The main concern with declaring it "Supported" is the lack of some features
when compared to classic PV dom0, hence switch it's status to supported with
caveats.  List the known missing features, there might be more features missing
or not working as expected apart from the ones listed.

Note there's some (limited) PVH dom0 testing on both osstest and gitlab.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
---
Changes since v1:
 - Remove boot warning.
---
 CHANGELOG.md                  |  1 +
 SUPPORT.md                    | 15 ++++++++++++++-
 xen/arch/x86/hvm/dom0_build.c |  1 -
 3 files changed, 15 insertions(+), 2 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 201478aa1c0e..1778419cae64 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -14,6 +14,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
    - HVM PIRQs are disabled by default.
    - Reduce IOMMU setup time for hardware domain.
    - Allow HVM/PVH domains to map foreign pages.
+   - Declare PVH dom0 supported with caveats.
  - xl/libxl configures vkb=[] for HVM domains with priority over vkb_device.
  - Increase the maximum number of CPUs Xen can be built for from 4095 to
    16383.
diff --git a/SUPPORT.md b/SUPPORT.md
index d5d60c62ec11..711aacf34662 100644
--- a/SUPPORT.md
+++ b/SUPPORT.md
@@ -161,7 +161,20 @@ Requires hardware virtualisation support (Intel VMX / AMD SVM).
 Dom0 support requires an IOMMU (Intel VT-d / AMD IOMMU).
 
     Status, domU: Supported
-    Status, dom0: Experimental
+    Status, dom0: Supported, with caveats
+
+PVH dom0 hasn't received the same test coverage as PV dom0, so it can exhibit
+unexpected behavior or issues on some hardware.
+
+At least the following features are missing on a PVH dom0:
+
+  * PCI SR-IOV and Resizable BARs.
+
+  * Native NMI forwarding (nmi=dom0 command line option).
+
+  * MCE handling.
+
+  * PCI Passthrough to any kind of domUs.
 
 ### ARM
 
diff --git a/xen/arch/x86/hvm/dom0_build.c b/xen/arch/x86/hvm/dom0_build.c
index 6acbaceb94c1..f3eddb684686 100644
--- a/xen/arch/x86/hvm/dom0_build.c
+++ b/xen/arch/x86/hvm/dom0_build.c
@@ -1360,7 +1360,6 @@ int __init dom0_construct_pvh(struct domain *d, const module_t *image,
         print_e820_memory_map(d->arch.e820, d->arch.nr_e820);
     }
 
-    printk("WARNING: PVH is an experimental mode with limited functionality\n");
     return 0;
 }
 
-- 
2.44.0

Re: [PATCH for-4.19 v2] x86/pvh: declare PVH dom0 supported with caveats

[George Dunlap]

On Mon, Jun 10, 2024 at 9:50 AM Roger Pau Monne <roger.pau@citrix.com> wrote:
>
> PVH dom0 is functionally very similar to PVH domU except for the domain
> builder and the added set of hypercalls available to it.
>
> The main concern with declaring it "Supported" is the lack of some features
> when compared to classic PV dom0, hence switch it's status to supported with
> caveats.  List the known missing features, there might be more features missing
> or not working as expected apart from the ones listed.
>
> Note there's some (limited) PVH dom0 testing on both osstest and gitlab.
>
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> ---
> Changes since v1:
>  - Remove boot warning.
> ---
>  CHANGELOG.md                  |  1 +
>  SUPPORT.md                    | 15 ++++++++++++++-
>  xen/arch/x86/hvm/dom0_build.c |  1 -
>  3 files changed, 15 insertions(+), 2 deletions(-)
>
> diff --git a/CHANGELOG.md b/CHANGELOG.md
> index 201478aa1c0e..1778419cae64 100644
> --- a/CHANGELOG.md
> +++ b/CHANGELOG.md
> @@ -14,6 +14,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
>     - HVM PIRQs are disabled by default.
>     - Reduce IOMMU setup time for hardware domain.
>     - Allow HVM/PVH domains to map foreign pages.
> +   - Declare PVH dom0 supported with caveats.
>   - xl/libxl configures vkb=[] for HVM domains with priority over vkb_device.
>   - Increase the maximum number of CPUs Xen can be built for from 4095 to
>     16383.
> diff --git a/SUPPORT.md b/SUPPORT.md
> index d5d60c62ec11..711aacf34662 100644
> --- a/SUPPORT.md
> +++ b/SUPPORT.md
> @@ -161,7 +161,20 @@ Requires hardware virtualisation support (Intel VMX / AMD SVM).
>  Dom0 support requires an IOMMU (Intel VT-d / AMD IOMMU).
>
>      Status, domU: Supported
> -    Status, dom0: Experimental
> +    Status, dom0: Supported, with caveats
> +
> +PVH dom0 hasn't received the same test coverage as PV dom0, so it can exhibit
> +unexpected behavior or issues on some hardware.

What's the criteria for removing this paragraph?

FAOD I'm OK with it being checked in as-is, but I feel like this
paragraph is somewhat anomalous, and would at least like to have an
idea what might trigger its removal.

 -George

Re: [PATCH for-4.19 v2] x86/pvh: declare PVH dom0 supported with caveats

[Roger Pau Monné]

On Mon, Jun 10, 2024 at 04:55:34PM +0100, George Dunlap wrote:
> On Mon, Jun 10, 2024 at 9:50 AM Roger Pau Monne <roger.pau@citrix.com> wrote:
> >
> > PVH dom0 is functionally very similar to PVH domU except for the domain
> > builder and the added set of hypercalls available to it.
> >
> > The main concern with declaring it "Supported" is the lack of some features
> > when compared to classic PV dom0, hence switch it's status to supported with
> > caveats.  List the known missing features, there might be more features missing
> > or not working as expected apart from the ones listed.
> >
> > Note there's some (limited) PVH dom0 testing on both osstest and gitlab.
> >
> > Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> > Acked-by: Andrew Cooper <andrew.cooper3@citrix.com>
> > Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>
> > ---
> > Changes since v1:
> >  - Remove boot warning.
> > ---
> >  CHANGELOG.md                  |  1 +
> >  SUPPORT.md                    | 15 ++++++++++++++-
> >  xen/arch/x86/hvm/dom0_build.c |  1 -
> >  3 files changed, 15 insertions(+), 2 deletions(-)
> >
> > diff --git a/CHANGELOG.md b/CHANGELOG.md
> > index 201478aa1c0e..1778419cae64 100644
> > --- a/CHANGELOG.md
> > +++ b/CHANGELOG.md
> > @@ -14,6 +14,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
> >     - HVM PIRQs are disabled by default.
> >     - Reduce IOMMU setup time for hardware domain.
> >     - Allow HVM/PVH domains to map foreign pages.
> > +   - Declare PVH dom0 supported with caveats.
> >   - xl/libxl configures vkb=[] for HVM domains with priority over vkb_device.
> >   - Increase the maximum number of CPUs Xen can be built for from 4095 to
> >     16383.
> > diff --git a/SUPPORT.md b/SUPPORT.md
> > index d5d60c62ec11..711aacf34662 100644
> > --- a/SUPPORT.md
> > +++ b/SUPPORT.md
> > @@ -161,7 +161,20 @@ Requires hardware virtualisation support (Intel VMX / AMD SVM).
> >  Dom0 support requires an IOMMU (Intel VT-d / AMD IOMMU).
> >
> >      Status, domU: Supported
> > -    Status, dom0: Experimental
> > +    Status, dom0: Supported, with caveats
> > +
> > +PVH dom0 hasn't received the same test coverage as PV dom0, so it can exhibit
> > +unexpected behavior or issues on some hardware.
> 
> What's the criteria for removing this paragraph?
> 
> FAOD I'm OK with it being checked in as-is, but I feel like this
> paragraph is somewhat anomalous, and would at least like to have an
> idea what might trigger its removal.

More testing is the only way this paragraph can be removed IMO.

For example I would be happy to remove it if dom0 PVH works on all
hardware in the XenRT lab.  So far the Linux dom0 version used by
XenServer is missing some required fixes for such testing to be
feasible.

Thanks, Roger.