[RFC PATCH] PVH Dom0 RMRR IOMMU mapping regression fix

[elena.ufimtseva@oracle.com]

From: Elena Ufimtseva <elena.ufimtseva@oracle.com>

This patch addresses a regression introduced by commit 5ae03990c120a7b3067a52d9784c9aa72c0705a6 in new set_identity_p2m_entry. RMRRs are not being mapped in
IOMMU for PVH Dom0. This causes pages faults and some long 'hang-like' delays
during Dom0 PVH boot and device assignments.

During construct_dom0, in PVH path p2m is being constructed and identity mapped
in IOMMU. The p2m type is p2m_mmio_direct and p2m access p2m_rwx.
New code used to map RMRRs invoked from rmrr_identity_mapping
checks if p2m entry exists with same type and access and if yes, skips iommu
mapping. Since there are p2m entries for pvh dom0 iomem, RMRRs are not being
mapped in IOMMU.

As was mentioned in the earlier discussion, the solution is far from perfect and
desired solution would be rework construction of PVH Dom0 code.
Also as Jan mentioned, there is no need in having 'x' permissions for p2m entry
of a mmio region, thus changed here.
Permissions for p2m entry of read-only mmio regions are left unchanged as leaving only 'r' cause page faults. I am not sure what the reason of it yet, will try to dig it further.

You comments and suggestions are welcome!
Thank you.

Elena

Signed-off-by: assert <assert@elena.ufimtseva>
---
 xen/arch/x86/domain_build.c |  2 +-
 xen/arch/x86/mm/p2m.c       | 12 +++++++++++-
 2 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/xen/arch/x86/domain_build.c b/xen/arch/x86/domain_build.c
index 18cf6aa..259dfd4 100644
--- a/xen/arch/x86/domain_build.c
+++ b/xen/arch/x86/domain_build.c
@@ -434,7 +434,7 @@ static __init void pvh_add_mem_mapping(struct domain *d, unsigned long gfn,
         if ( rangeset_contains_singleton(mmio_ro_ranges, mfn + i) )
             a = p2m_access_rx;
         else
-            a = p2m_access_rwx;
+            a = p2m_access_rw;
 
         if ( (rc = set_mmio_p2m_entry(d, gfn + i, _mfn(mfn + i), a)) )
             panic("pvh_add_mem_mapping: gfn:%lx mfn:%lx i:%ld rc:%d\n",
diff --git a/xen/arch/x86/mm/p2m.c b/xen/arch/x86/mm/p2m.c
index b2726bd..97a0986 100644
--- a/xen/arch/x86/mm/p2m.c
+++ b/xen/arch/x86/mm/p2m.c
@@ -971,7 +971,17 @@ int set_identity_p2m_entry(struct domain *d, unsigned long gfn,
         ret = p2m_set_entry(p2m, gfn, _mfn(gfn), PAGE_ORDER_4K,
                             p2m_mmio_direct, p2ma);
     else if ( mfn_x(mfn) == gfn && p2mt == p2m_mmio_direct && a == p2ma )
-        ret = 0;
+    {
+        /*
+         * PVH fixme: during Dom0 PVH construction, p2m entries are being set
+         * but iomem regions are not mapped with IOMMU. This makes sure that
+         * RMRRs are correctly mapped with IOMMU.
+         */
+        if ( is_hardware_domain(d) && !iommu_use_hap_pt(d) )
+            ret = iommu_map_page(d, gfn, gfn, IOMMUF_readable|IOMMUF_writable);
+        else
+            ret = 0;
+    }
     else
     {
         if ( flag & XEN_DOMCTL_DEV_RDM_RELAXED )
-- 
2.1.4