[PATCH v5 0/3] x86/iommu: improve setup time of hwdom IOMMU

[PATCH v5 0/3] x86/iommu: improve setup time of hwdom IOMMU

[Roger Pau Monne]

Hello,

The aim of the series is to reduce boot time setup of IOMMU page tables
for dom0.

This patches rework the hardware domain IOMMU setup to use a rangeset
instead of iterating over all addresses up to the max RAM page.  See
patch 2/3 for performance figures.

Jan: I've kept your RB in patch 1 with the off-by-one adjustments, and
dropped it for patch 2 with the addition of the default label in the
switch over the e820 ranges.  Such default label is required to continue
to deal equally with the  ACPI and NVS ranges.

Thanks, Roger.

Roger Pau Monne (3):
  x86/iommu: remove regions not to be mapped
  x86/iommu: switch hwdom IOMMU to use a rangeset
  x86/iommu: cleanup unused functions

 CHANGELOG.md                        |   2 +
 xen/arch/x86/hvm/io.c               |  15 ++-
 xen/arch/x86/include/asm/hvm/io.h   |   4 +-
 xen/arch/x86/include/asm/setup.h    |   2 +-
 xen/arch/x86/setup.c                |  81 ++++++------
 xen/arch/x86/tboot.c                |   2 +-
 xen/drivers/passthrough/x86/iommu.c | 196 ++++++++++++----------------
 7 files changed, 146 insertions(+), 156 deletions(-)

-- 
2.43.0

[PATCH v5 1/3] x86/iommu: remove regions not to be mapped

[Roger Pau Monne]

Introduce the code to remove regions not to be mapped from the rangeset
that will be used to setup the IOMMU page tables for the hardware domain.

This change also introduces two new functions: remove_xen_ranges() and
vpci_subtract_mmcfg() that copy the logic in xen_in_range() and
vpci_is_mmcfg_address() respectively and remove the ranges that would otherwise
be intercepted by the original functions.

Note that the rangeset is still not populated.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
Changes since v4:
 - Fix off-by-one when removing the Xen used ranges, as the rangesets are
   inclusive.

Changes since v3:
 - Remove unnecessary line wrapping.

Changes since v1:
 - Split from bigger patch.
---
 xen/arch/x86/hvm/io.c               | 16 ++++++++
 xen/arch/x86/include/asm/hvm/io.h   |  3 ++
 xen/arch/x86/include/asm/setup.h    |  1 +
 xen/arch/x86/setup.c                | 48 +++++++++++++++++++++++
 xen/drivers/passthrough/x86/iommu.c | 61 +++++++++++++++++++++++++++++
 5 files changed, 129 insertions(+)

diff --git a/xen/arch/x86/hvm/io.c b/xen/arch/x86/hvm/io.c
index d75af83ad01f..a42854c52b65 100644
--- a/xen/arch/x86/hvm/io.c
+++ b/xen/arch/x86/hvm/io.c
@@ -369,6 +369,22 @@ bool vpci_is_mmcfg_address(const struct domain *d, paddr_t addr)
     return vpci_mmcfg_find(d, addr);
 }
 
+int __hwdom_init vpci_subtract_mmcfg(const struct domain *d, struct rangeset *r)
+{
+    const struct hvm_mmcfg *mmcfg;
+
+    list_for_each_entry ( mmcfg, &d->arch.hvm.mmcfg_regions, next )
+    {
+        int rc = rangeset_remove_range(r, PFN_DOWN(mmcfg->addr),
+                                       PFN_DOWN(mmcfg->addr + mmcfg->size - 1));
+
+        if ( rc )
+            return rc;
+    }
+
+    return 0;
+}
+
 static unsigned int vpci_mmcfg_decode_addr(const struct hvm_mmcfg *mmcfg,
                                            paddr_t addr, pci_sbdf_t *sbdf)
 {
diff --git a/xen/arch/x86/include/asm/hvm/io.h b/xen/arch/x86/include/asm/hvm/io.h
index a97731657801..e1e5e6fe7491 100644
--- a/xen/arch/x86/include/asm/hvm/io.h
+++ b/xen/arch/x86/include/asm/hvm/io.h
@@ -156,6 +156,9 @@ void destroy_vpci_mmcfg(struct domain *d);
 /* Check if an address is between a MMCFG region for a domain. */
 bool vpci_is_mmcfg_address(const struct domain *d, paddr_t addr);
 
+/* Remove MMCFG regions from a given rangeset. */
+int vpci_subtract_mmcfg(const struct domain *d, struct rangeset *r);
+
 #endif /* __ASM_X86_HVM_IO_H__ */
 
 
diff --git a/xen/arch/x86/include/asm/setup.h b/xen/arch/x86/include/asm/setup.h
index 9a460e4db8f4..cd07d98101d8 100644
--- a/xen/arch/x86/include/asm/setup.h
+++ b/xen/arch/x86/include/asm/setup.h
@@ -37,6 +37,7 @@ void discard_initial_images(void);
 void *bootstrap_map(const module_t *mod);
 
 int xen_in_range(unsigned long mfn);
+int remove_xen_ranges(struct rangeset *r);
 
 extern uint8_t kbd_shift_flags;
 
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 897b7e92082e..c9f65c3a70b8 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -2138,6 +2138,54 @@ int __hwdom_init xen_in_range(unsigned long mfn)
     return 0;
 }
 
+int __hwdom_init remove_xen_ranges(struct rangeset *r)
+{
+    paddr_t start, end;
+    int rc;
+
+    /* S3 resume code (and other real mode trampoline code) */
+    rc = rangeset_remove_range(r, PFN_DOWN(bootsym_phys(trampoline_start)),
+                               PFN_DOWN(bootsym_phys(trampoline_end)) - 1);
+    if ( rc )
+        return rc;
+
+    /*
+     * This needs to remain in sync with the uses of the same symbols in
+     * - __start_xen()
+     * - is_xen_fixed_mfn()
+     * - tboot_shutdown()
+     */
+    /* hypervisor .text + .rodata */
+    rc = rangeset_remove_range(r, PFN_DOWN(__pa(&_stext)),
+                               PFN_DOWN(__pa(&__2M_rodata_end)) - 1);
+    if ( rc )
+        return rc;
+
+    /* hypervisor .data + .bss */
+    if ( efi_boot_mem_unused(&start, &end) )
+    {
+        ASSERT(__pa(start) >= __pa(&__2M_rwdata_start));
+        rc = rangeset_remove_range(r, PFN_DOWN(__pa(&__2M_rwdata_start)),
+                                   PFN_DOWN(__pa(start)) - 1);
+        if ( rc )
+            return rc;
+        ASSERT(__pa(end) <= __pa(&__2M_rwdata_end));
+        rc = rangeset_remove_range(r, PFN_DOWN(__pa(end)),
+                                   PFN_DOWN(__pa(&__2M_rwdata_end)) - 1);
+        if ( rc )
+            return rc;
+    }
+    else
+    {
+        rc = rangeset_remove_range(r, PFN_DOWN(__pa(&__2M_rwdata_start)),
+                                   PFN_DOWN(__pa(&__2M_rwdata_end)) - 1);
+        if ( rc )
+            return rc;
+    }
+
+    return 0;
+}
+
 static int __hwdom_init cf_check io_bitmap_cb(
     unsigned long s, unsigned long e, void *ctx)
 {
diff --git a/xen/drivers/passthrough/x86/iommu.c b/xen/drivers/passthrough/x86/iommu.c
index 59b0c7e980ca..fc5215a9dc40 100644
--- a/xen/drivers/passthrough/x86/iommu.c
+++ b/xen/drivers/passthrough/x86/iommu.c
@@ -370,6 +370,14 @@ static unsigned int __hwdom_init hwdom_iommu_map(const struct domain *d,
     return perms;
 }
 
+static int __hwdom_init cf_check map_subtract(unsigned long s, unsigned long e,
+                                              void *data)
+{
+    struct rangeset *map = data;
+
+    return rangeset_remove_range(map, s, e);
+}
+
 struct map_data {
     struct domain *d;
     unsigned int flush_flags;
@@ -533,6 +541,59 @@ void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
             goto commit;
     }
 
+    /* Remove any areas in-use by Xen. */
+    rc = remove_xen_ranges(map);
+    if ( rc )
+        panic("IOMMU failed to remove Xen ranges: %d\n", rc);
+
+    /* Remove any overlap with the Interrupt Address Range. */
+    rc = rangeset_remove_range(map, 0xfee00, 0xfeeff);
+    if ( rc )
+        panic("IOMMU failed to remove Interrupt Address Range: %d\n", rc);
+
+    /* If emulating IO-APIC(s) make sure the base address is unmapped. */
+    if ( has_vioapic(d) )
+    {
+        for ( i = 0; i < d->arch.hvm.nr_vioapics; i++ )
+        {
+            rc = rangeset_remove_singleton(map,
+                PFN_DOWN(domain_vioapic(d, i)->base_address));
+            if ( rc )
+                panic("IOMMU failed to remove IO-APIC: %d\n", rc);
+        }
+    }
+
+    if ( is_pv_domain(d) )
+    {
+        /*
+         * Be consistent with CPU mappings: Dom0 is permitted to establish r/o
+         * ones there (also for e.g. HPET in certain cases), so it should also
+         * have such established for IOMMUs.  Remove any read-only ranges here,
+         * since ranges in mmio_ro_ranges are already explicitly mapped below
+         * in read-only mode.
+         */
+        rc = rangeset_report_ranges(mmio_ro_ranges, 0, ~0UL, map_subtract, map);
+        if ( rc )
+            panic("IOMMU failed to remove read-only regions: %d\n", rc);
+    }
+
+    if ( has_vpci(d) )
+    {
+        /*
+         * TODO: runtime added MMCFG regions are not checked to make sure they
+         * don't overlap with already mapped regions, thus preventing trapping.
+         */
+        rc = vpci_subtract_mmcfg(d, map);
+        if ( rc )
+            panic("IOMMU unable to remove MMCFG areas: %d\n", rc);
+    }
+
+    /* Remove any regions past the last address addressable by the domain. */
+    rc = rangeset_remove_range(map, PFN_DOWN(1UL << domain_max_paddr_bits(d)),
+                               ~0UL);
+    if ( rc )
+        panic("IOMMU unable to remove unaddressable ranges: %d\n", rc);
+
     if ( iommu_verbose )
         printk(XENLOG_INFO "%pd: identity mappings for IOMMU:\n", d);
 
-- 
2.43.0

Re: [PATCH v5 1/3] x86/iommu: remove regions not to be mapped

[Jan Beulich]

On 24.01.2024 18:29, Roger Pau Monne wrote:
> --- a/xen/arch/x86/hvm/io.c
> +++ b/xen/arch/x86/hvm/io.c
> @@ -369,6 +369,22 @@ bool vpci_is_mmcfg_address(const struct domain *d, paddr_t addr)
>      return vpci_mmcfg_find(d, addr);
>  }
>  
> +int __hwdom_init vpci_subtract_mmcfg(const struct domain *d, struct rangeset *r)
> +{
> +    const struct hvm_mmcfg *mmcfg;
> +
> +    list_for_each_entry ( mmcfg, &d->arch.hvm.mmcfg_regions, next )
> +    {
> +        int rc = rangeset_remove_range(r, PFN_DOWN(mmcfg->addr),
> +                                       PFN_DOWN(mmcfg->addr + mmcfg->size - 1));

Along the lines of this, ...

> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -2138,6 +2138,54 @@ int __hwdom_init xen_in_range(unsigned long mfn)
>      return 0;
>  }
>  
> +int __hwdom_init remove_xen_ranges(struct rangeset *r)
> +{
> +    paddr_t start, end;
> +    int rc;
> +
> +    /* S3 resume code (and other real mode trampoline code) */
> +    rc = rangeset_remove_range(r, PFN_DOWN(bootsym_phys(trampoline_start)),
> +                               PFN_DOWN(bootsym_phys(trampoline_end)) - 1);

... did you perhaps mean

                               PFN_DOWN(bootsym_phys(trampoline_end) - 1));

here (and then similarly below, except there the difference is benign I
think, for the labels being page-aligned)?

Jan

Re: [PATCH v5 1/3] x86/iommu: remove regions not to be mapped

[Roger Pau Monné]

On Thu, Jan 25, 2024 at 09:34:40AM +0100, Jan Beulich wrote:
> On 24.01.2024 18:29, Roger Pau Monne wrote:
> > --- a/xen/arch/x86/hvm/io.c
> > +++ b/xen/arch/x86/hvm/io.c
> > @@ -369,6 +369,22 @@ bool vpci_is_mmcfg_address(const struct domain *d, paddr_t addr)
> >      return vpci_mmcfg_find(d, addr);
> >  }
> >  
> > +int __hwdom_init vpci_subtract_mmcfg(const struct domain *d, struct rangeset *r)
> > +{
> > +    const struct hvm_mmcfg *mmcfg;
> > +
> > +    list_for_each_entry ( mmcfg, &d->arch.hvm.mmcfg_regions, next )
> > +    {
> > +        int rc = rangeset_remove_range(r, PFN_DOWN(mmcfg->addr),
> > +                                       PFN_DOWN(mmcfg->addr + mmcfg->size - 1));
> 
> Along the lines of this, ...
> 
> > --- a/xen/arch/x86/setup.c
> > +++ b/xen/arch/x86/setup.c
> > @@ -2138,6 +2138,54 @@ int __hwdom_init xen_in_range(unsigned long mfn)
> >      return 0;
> >  }
> >  
> > +int __hwdom_init remove_xen_ranges(struct rangeset *r)
> > +{
> > +    paddr_t start, end;
> > +    int rc;
> > +
> > +    /* S3 resume code (and other real mode trampoline code) */
> > +    rc = rangeset_remove_range(r, PFN_DOWN(bootsym_phys(trampoline_start)),
> > +                               PFN_DOWN(bootsym_phys(trampoline_end)) - 1);
> 
> ... did you perhaps mean
> 
>                                PFN_DOWN(bootsym_phys(trampoline_end) - 1));
> 
> here (and then similarly below, except there the difference is benign I
> think, for the labels being page-aligned)?

They are all page aligned, so I didn't care much,  but now that you
point it might be safer to do the subtraction from the address instead
of the frame number, just in case.

Thanks, Roger.

Re: [PATCH v5 1/3] x86/iommu: remove regions not to be mapped

[Jan Beulich]

On 25.01.2024 09:47, Roger Pau Monné wrote:
> On Thu, Jan 25, 2024 at 09:34:40AM +0100, Jan Beulich wrote:
>> On 24.01.2024 18:29, Roger Pau Monne wrote:
>>> --- a/xen/arch/x86/hvm/io.c
>>> +++ b/xen/arch/x86/hvm/io.c
>>> @@ -369,6 +369,22 @@ bool vpci_is_mmcfg_address(const struct domain *d, paddr_t addr)
>>>      return vpci_mmcfg_find(d, addr);
>>>  }
>>>  
>>> +int __hwdom_init vpci_subtract_mmcfg(const struct domain *d, struct rangeset *r)
>>> +{
>>> +    const struct hvm_mmcfg *mmcfg;
>>> +
>>> +    list_for_each_entry ( mmcfg, &d->arch.hvm.mmcfg_regions, next )
>>> +    {
>>> +        int rc = rangeset_remove_range(r, PFN_DOWN(mmcfg->addr),
>>> +                                       PFN_DOWN(mmcfg->addr + mmcfg->size - 1));
>>
>> Along the lines of this, ...
>>
>>> --- a/xen/arch/x86/setup.c
>>> +++ b/xen/arch/x86/setup.c
>>> @@ -2138,6 +2138,54 @@ int __hwdom_init xen_in_range(unsigned long mfn)
>>>      return 0;
>>>  }
>>>  
>>> +int __hwdom_init remove_xen_ranges(struct rangeset *r)
>>> +{
>>> +    paddr_t start, end;
>>> +    int rc;
>>> +
>>> +    /* S3 resume code (and other real mode trampoline code) */
>>> +    rc = rangeset_remove_range(r, PFN_DOWN(bootsym_phys(trampoline_start)),
>>> +                               PFN_DOWN(bootsym_phys(trampoline_end)) - 1);
>>
>> ... did you perhaps mean
>>
>>                                PFN_DOWN(bootsym_phys(trampoline_end) - 1));
>>
>> here (and then similarly below, except there the difference is benign I
>> think, for the labels being page-aligned)?
> 
> They are all page aligned, so I didn't care much,  but now that you
> point it might be safer to do the subtraction from the address instead
> of the frame number, just in case.

Hmm, no, for me neither trampoline_end nor trampoline_start are page
aligned. While bootsym_phys(trampoline_start) is, I don't think
bootsym_phys(trampoline_end) normally would be (it might only be by
coincidence).

Jan

Re: [PATCH v5 1/3] x86/iommu: remove regions not to be mapped

[Roger Pau Monné]

On Thu, Jan 25, 2024 at 12:13:01PM +0100, Jan Beulich wrote:
> On 25.01.2024 09:47, Roger Pau Monné wrote:
> > On Thu, Jan 25, 2024 at 09:34:40AM +0100, Jan Beulich wrote:
> >> On 24.01.2024 18:29, Roger Pau Monne wrote:
> >>> --- a/xen/arch/x86/hvm/io.c
> >>> +++ b/xen/arch/x86/hvm/io.c
> >>> @@ -369,6 +369,22 @@ bool vpci_is_mmcfg_address(const struct domain *d, paddr_t addr)
> >>>      return vpci_mmcfg_find(d, addr);
> >>>  }
> >>>  
> >>> +int __hwdom_init vpci_subtract_mmcfg(const struct domain *d, struct rangeset *r)
> >>> +{
> >>> +    const struct hvm_mmcfg *mmcfg;
> >>> +
> >>> +    list_for_each_entry ( mmcfg, &d->arch.hvm.mmcfg_regions, next )
> >>> +    {
> >>> +        int rc = rangeset_remove_range(r, PFN_DOWN(mmcfg->addr),
> >>> +                                       PFN_DOWN(mmcfg->addr + mmcfg->size - 1));
> >>
> >> Along the lines of this, ...
> >>
> >>> --- a/xen/arch/x86/setup.c
> >>> +++ b/xen/arch/x86/setup.c
> >>> @@ -2138,6 +2138,54 @@ int __hwdom_init xen_in_range(unsigned long mfn)
> >>>      return 0;
> >>>  }
> >>>  
> >>> +int __hwdom_init remove_xen_ranges(struct rangeset *r)
> >>> +{
> >>> +    paddr_t start, end;
> >>> +    int rc;
> >>> +
> >>> +    /* S3 resume code (and other real mode trampoline code) */
> >>> +    rc = rangeset_remove_range(r, PFN_DOWN(bootsym_phys(trampoline_start)),
> >>> +                               PFN_DOWN(bootsym_phys(trampoline_end)) - 1);
> >>
> >> ... did you perhaps mean
> >>
> >>                                PFN_DOWN(bootsym_phys(trampoline_end) - 1));
> >>
> >> here (and then similarly below, except there the difference is benign I
> >> think, for the labels being page-aligned)?
> > 
> > They are all page aligned, so I didn't care much,  but now that you
> > point it might be safer to do the subtraction from the address instead
> > of the frame number, just in case.
> 
> Hmm, no, for me neither trampoline_end nor trampoline_start are page
> aligned. While bootsym_phys(trampoline_start) is, I don't think
> bootsym_phys(trampoline_end) normally would be (it might only be by
> coincidence).

Oh, so it had been a coincidence of the build I was using I guess then.

Thanks, Roger.

Re: [PATCH v5 1/3] x86/iommu: remove regions not to be mapped

[Andrew Cooper]

On 25/01/2024 12:37 pm, Roger Pau Monné wrote:
> On Thu, Jan 25, 2024 at 12:13:01PM +0100, Jan Beulich wrote:
>> On 25.01.2024 09:47, Roger Pau Monné wrote:
>>> On Thu, Jan 25, 2024 at 09:34:40AM +0100, Jan Beulich wrote:
>>>> On 24.01.2024 18:29, Roger Pau Monne wrote:
>>>>> --- a/xen/arch/x86/hvm/io.c
>>>>> +++ b/xen/arch/x86/hvm/io.c
>>>>> @@ -369,6 +369,22 @@ bool vpci_is_mmcfg_address(const struct domain *d, paddr_t addr)
>>>>>      return vpci_mmcfg_find(d, addr);
>>>>>  }
>>>>>  
>>>>> +int __hwdom_init vpci_subtract_mmcfg(const struct domain *d, struct rangeset *r)
>>>>> +{
>>>>> +    const struct hvm_mmcfg *mmcfg;
>>>>> +
>>>>> +    list_for_each_entry ( mmcfg, &d->arch.hvm.mmcfg_regions, next )
>>>>> +    {
>>>>> +        int rc = rangeset_remove_range(r, PFN_DOWN(mmcfg->addr),
>>>>> +                                       PFN_DOWN(mmcfg->addr + mmcfg->size - 1));
>>>> Along the lines of this, ...
>>>>
>>>>> --- a/xen/arch/x86/setup.c
>>>>> +++ b/xen/arch/x86/setup.c
>>>>> @@ -2138,6 +2138,54 @@ int __hwdom_init xen_in_range(unsigned long mfn)
>>>>>      return 0;
>>>>>  }
>>>>>  
>>>>> +int __hwdom_init remove_xen_ranges(struct rangeset *r)
>>>>> +{
>>>>> +    paddr_t start, end;
>>>>> +    int rc;
>>>>> +
>>>>> +    /* S3 resume code (and other real mode trampoline code) */
>>>>> +    rc = rangeset_remove_range(r, PFN_DOWN(bootsym_phys(trampoline_start)),
>>>>> +                               PFN_DOWN(bootsym_phys(trampoline_end)) - 1);
>>>> ... did you perhaps mean
>>>>
>>>>                                PFN_DOWN(bootsym_phys(trampoline_end) - 1));
>>>>
>>>> here (and then similarly below, except there the difference is benign I
>>>> think, for the labels being page-aligned)?
>>> They are all page aligned, so I didn't care much,  but now that you
>>> point it might be safer to do the subtraction from the address instead
>>> of the frame number, just in case.
>> Hmm, no, for me neither trampoline_end nor trampoline_start are page
>> aligned. While bootsym_phys(trampoline_start) is, I don't think
>> bootsym_phys(trampoline_end) normally would be (it might only be by
>> coincidence).
> Oh, so it had been a coincidence of the build I was using I guess then.

trampoline_start has to be page aligned because of constraints from SIPI
and S3 (cant remember which one is the 4k constraint, but it's in the
comments).

On APs (and indeed, in Xen's pagetables), the trampoline is only a
single 4k page.

However, trampoline_end is quite a lot longer because there's various
things that get done on the BSP only, including recovering the E820 map,
EDID/etc in 16bit mode.

That said, we don't edit the trampoline very often, so if it happened to
work for you first time around, it probably hasn't changed since.

~Andrew

Re: [PATCH v5 1/3] x86/iommu: remove regions not to be mapped

[Jan Beulich]

On 25.01.2024 13:55, Andrew Cooper wrote:
> On 25/01/2024 12:37 pm, Roger Pau Monné wrote:
>> On Thu, Jan 25, 2024 at 12:13:01PM +0100, Jan Beulich wrote:
>>> On 25.01.2024 09:47, Roger Pau Monné wrote:
>>>> On Thu, Jan 25, 2024 at 09:34:40AM +0100, Jan Beulich wrote:
>>>>> On 24.01.2024 18:29, Roger Pau Monne wrote:
>>>>>> --- a/xen/arch/x86/hvm/io.c
>>>>>> +++ b/xen/arch/x86/hvm/io.c
>>>>>> @@ -369,6 +369,22 @@ bool vpci_is_mmcfg_address(const struct domain *d, paddr_t addr)
>>>>>>      return vpci_mmcfg_find(d, addr);
>>>>>>  }
>>>>>>  
>>>>>> +int __hwdom_init vpci_subtract_mmcfg(const struct domain *d, struct rangeset *r)
>>>>>> +{
>>>>>> +    const struct hvm_mmcfg *mmcfg;
>>>>>> +
>>>>>> +    list_for_each_entry ( mmcfg, &d->arch.hvm.mmcfg_regions, next )
>>>>>> +    {
>>>>>> +        int rc = rangeset_remove_range(r, PFN_DOWN(mmcfg->addr),
>>>>>> +                                       PFN_DOWN(mmcfg->addr + mmcfg->size - 1));
>>>>> Along the lines of this, ...
>>>>>
>>>>>> --- a/xen/arch/x86/setup.c
>>>>>> +++ b/xen/arch/x86/setup.c
>>>>>> @@ -2138,6 +2138,54 @@ int __hwdom_init xen_in_range(unsigned long mfn)
>>>>>>      return 0;
>>>>>>  }
>>>>>>  
>>>>>> +int __hwdom_init remove_xen_ranges(struct rangeset *r)
>>>>>> +{
>>>>>> +    paddr_t start, end;
>>>>>> +    int rc;
>>>>>> +
>>>>>> +    /* S3 resume code (and other real mode trampoline code) */
>>>>>> +    rc = rangeset_remove_range(r, PFN_DOWN(bootsym_phys(trampoline_start)),
>>>>>> +                               PFN_DOWN(bootsym_phys(trampoline_end)) - 1);
>>>>> ... did you perhaps mean
>>>>>
>>>>>                                PFN_DOWN(bootsym_phys(trampoline_end) - 1));
>>>>>
>>>>> here (and then similarly below, except there the difference is benign I
>>>>> think, for the labels being page-aligned)?
>>>> They are all page aligned, so I didn't care much,  but now that you
>>>> point it might be safer to do the subtraction from the address instead
>>>> of the frame number, just in case.
>>> Hmm, no, for me neither trampoline_end nor trampoline_start are page
>>> aligned. While bootsym_phys(trampoline_start) is, I don't think
>>> bootsym_phys(trampoline_end) normally would be (it might only be by
>>> coincidence).
>> Oh, so it had been a coincidence of the build I was using I guess then.
> 
> trampoline_start has to be page aligned because of constraints from SIPI
> and S3 (cant remember which one is the 4k constraint, but it's in the
> comments).

What you're talking about is the copy of the trampoline code/data in
low memory. trampoline_{start,end} themselves point into the Xen image.

> On APs (and indeed, in Xen's pagetables), the trampoline is only a
> single 4k page.
> 
> However, trampoline_end is quite a lot longer because there's various
> things that get done on the BSP only, including recovering the E820 map,
> EDID/etc in 16bit mode.

And this BSP-only part really wouldn't need removing here, I think.
The issue is that the BSP-only and also-AP plus S3-wakeup parts aren't
properly delimited (hmm, maybe wakeup_stack can be used for this
purpose). But if, as you say, we map only a single page, we could as
well limit logic here to just that.

Jan

> That said, we don't edit the trampoline very often, so if it happened to
> work for you first time around, it probably hasn't changed since.
> 
> ~Andrew

Re: [PATCH v5 1/3] x86/iommu: remove regions not to be mapped

[Andrew Cooper]

On 25/01/2024 1:13 pm, Jan Beulich wrote:
> On 25.01.2024 13:55, Andrew Cooper wrote:
>> On 25/01/2024 12:37 pm, Roger Pau Monné wrote:
>>> On Thu, Jan 25, 2024 at 12:13:01PM +0100, Jan Beulich wrote:
>>>> On 25.01.2024 09:47, Roger Pau Monné wrote:
>>>>> On Thu, Jan 25, 2024 at 09:34:40AM +0100, Jan Beulich wrote:
>>>>>> On 24.01.2024 18:29, Roger Pau Monne wrote:
>>>>>>> --- a/xen/arch/x86/hvm/io.c
>>>>>>> +++ b/xen/arch/x86/hvm/io.c
>>>>>>> @@ -369,6 +369,22 @@ bool vpci_is_mmcfg_address(const struct domain *d, paddr_t addr)
>>>>>>>      return vpci_mmcfg_find(d, addr);
>>>>>>>  }
>>>>>>>  
>>>>>>> +int __hwdom_init vpci_subtract_mmcfg(const struct domain *d, struct rangeset *r)
>>>>>>> +{
>>>>>>> +    const struct hvm_mmcfg *mmcfg;
>>>>>>> +
>>>>>>> +    list_for_each_entry ( mmcfg, &d->arch.hvm.mmcfg_regions, next )
>>>>>>> +    {
>>>>>>> +        int rc = rangeset_remove_range(r, PFN_DOWN(mmcfg->addr),
>>>>>>> +                                       PFN_DOWN(mmcfg->addr + mmcfg->size - 1));
>>>>>> Along the lines of this, ...
>>>>>>
>>>>>>> --- a/xen/arch/x86/setup.c
>>>>>>> +++ b/xen/arch/x86/setup.c
>>>>>>> @@ -2138,6 +2138,54 @@ int __hwdom_init xen_in_range(unsigned long mfn)
>>>>>>>      return 0;
>>>>>>>  }
>>>>>>>  
>>>>>>> +int __hwdom_init remove_xen_ranges(struct rangeset *r)
>>>>>>> +{
>>>>>>> +    paddr_t start, end;
>>>>>>> +    int rc;
>>>>>>> +
>>>>>>> +    /* S3 resume code (and other real mode trampoline code) */
>>>>>>> +    rc = rangeset_remove_range(r, PFN_DOWN(bootsym_phys(trampoline_start)),
>>>>>>> +                               PFN_DOWN(bootsym_phys(trampoline_end)) - 1);
>>>>>> ... did you perhaps mean
>>>>>>
>>>>>>                                PFN_DOWN(bootsym_phys(trampoline_end) - 1));
>>>>>>
>>>>>> here (and then similarly below, except there the difference is benign I
>>>>>> think, for the labels being page-aligned)?
>>>>> They are all page aligned, so I didn't care much,  but now that you
>>>>> point it might be safer to do the subtraction from the address instead
>>>>> of the frame number, just in case.
>>>> Hmm, no, for me neither trampoline_end nor trampoline_start are page
>>>> aligned. While bootsym_phys(trampoline_start) is, I don't think
>>>> bootsym_phys(trampoline_end) normally would be (it might only be by
>>>> coincidence).
>>> Oh, so it had been a coincidence of the build I was using I guess then.
>> trampoline_start has to be page aligned because of constraints from SIPI
>> and S3 (cant remember which one is the 4k constraint, but it's in the
>> comments).
> What you're talking about is the copy of the trampoline code/data in
> low memory. trampoline_{start,end} themselves point into the Xen image.

True, but we're operating on bootsym_phys(trampoline_start) which had
better be aligned.

We hard-code (by virtue of only filling in 1 single 4k PTE in the
pagetables) that the AP trampoline is 4k.

The range here should be 4k only too, or we're (falsely) marking lowmem
adjacent to the AP trampoline as a Xen range when it's not.

~Andrew

Re: [PATCH v5 1/3] x86/iommu: remove regions not to be mapped

[Roger Pau Monné]

On Thu, Jan 25, 2024 at 01:22:15PM +0000, Andrew Cooper wrote:
> On 25/01/2024 1:13 pm, Jan Beulich wrote:
> > On 25.01.2024 13:55, Andrew Cooper wrote:
> >> On 25/01/2024 12:37 pm, Roger Pau Monné wrote:
> >>> On Thu, Jan 25, 2024 at 12:13:01PM +0100, Jan Beulich wrote:
> >>>> On 25.01.2024 09:47, Roger Pau Monné wrote:
> >>>>> On Thu, Jan 25, 2024 at 09:34:40AM +0100, Jan Beulich wrote:
> >>>>>> On 24.01.2024 18:29, Roger Pau Monne wrote:
> >>>>>>> --- a/xen/arch/x86/hvm/io.c
> >>>>>>> +++ b/xen/arch/x86/hvm/io.c
> >>>>>>> @@ -369,6 +369,22 @@ bool vpci_is_mmcfg_address(const struct domain *d, paddr_t addr)
> >>>>>>>      return vpci_mmcfg_find(d, addr);
> >>>>>>>  }
> >>>>>>>  
> >>>>>>> +int __hwdom_init vpci_subtract_mmcfg(const struct domain *d, struct rangeset *r)
> >>>>>>> +{
> >>>>>>> +    const struct hvm_mmcfg *mmcfg;
> >>>>>>> +
> >>>>>>> +    list_for_each_entry ( mmcfg, &d->arch.hvm.mmcfg_regions, next )
> >>>>>>> +    {
> >>>>>>> +        int rc = rangeset_remove_range(r, PFN_DOWN(mmcfg->addr),
> >>>>>>> +                                       PFN_DOWN(mmcfg->addr + mmcfg->size - 1));
> >>>>>> Along the lines of this, ...
> >>>>>>
> >>>>>>> --- a/xen/arch/x86/setup.c
> >>>>>>> +++ b/xen/arch/x86/setup.c
> >>>>>>> @@ -2138,6 +2138,54 @@ int __hwdom_init xen_in_range(unsigned long mfn)
> >>>>>>>      return 0;
> >>>>>>>  }
> >>>>>>>  
> >>>>>>> +int __hwdom_init remove_xen_ranges(struct rangeset *r)
> >>>>>>> +{
> >>>>>>> +    paddr_t start, end;
> >>>>>>> +    int rc;
> >>>>>>> +
> >>>>>>> +    /* S3 resume code (and other real mode trampoline code) */
> >>>>>>> +    rc = rangeset_remove_range(r, PFN_DOWN(bootsym_phys(trampoline_start)),
> >>>>>>> +                               PFN_DOWN(bootsym_phys(trampoline_end)) - 1);
> >>>>>> ... did you perhaps mean
> >>>>>>
> >>>>>>                                PFN_DOWN(bootsym_phys(trampoline_end) - 1));
> >>>>>>
> >>>>>> here (and then similarly below, except there the difference is benign I
> >>>>>> think, for the labels being page-aligned)?
> >>>>> They are all page aligned, so I didn't care much,  but now that you
> >>>>> point it might be safer to do the subtraction from the address instead
> >>>>> of the frame number, just in case.
> >>>> Hmm, no, for me neither trampoline_end nor trampoline_start are page
> >>>> aligned. While bootsym_phys(trampoline_start) is, I don't think
> >>>> bootsym_phys(trampoline_end) normally would be (it might only be by
> >>>> coincidence).
> >>> Oh, so it had been a coincidence of the build I was using I guess then.
> >> trampoline_start has to be page aligned because of constraints from SIPI
> >> and S3 (cant remember which one is the 4k constraint, but it's in the
> >> comments).
> > What you're talking about is the copy of the trampoline code/data in
> > low memory. trampoline_{start,end} themselves point into the Xen image.
> 
> True, but we're operating on bootsym_phys(trampoline_start) which had
> better be aligned.
> 
> We hard-code (by virtue of only filling in 1 single 4k PTE in the
> pagetables) that the AP trampoline is 4k.
> 
> The range here should be 4k only too, or we're (falsely) marking lowmem
> adjacent to the AP trampoline as a Xen range when it's not.

Hm, looking at zap_low_mappings() we do seem to possibly map more than
one page, in fact on my current build trampoline_end -
trampoline_start is 6528.

Thanks, Roger.

[PATCH v6 1/3] x86/iommu: remove regions not to be mapped

[Roger Pau Monne]

Introduce the code to remove regions not to be mapped from the rangeset
that will be used to setup the IOMMU page tables for the hardware domain.

This change also introduces two new functions: remove_xen_ranges() and
vpci_subtract_mmcfg() that copy the logic in xen_in_range() and
vpci_is_mmcfg_address() respectively and remove the ranges that would otherwise
be intercepted by the original functions.

Note that the rangeset is still not populated.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v6:
 - Fix subtraction to be done against the address, not the mfn.

Changes since v4:
 - Fix off-by-one when removing the Xen used ranges, as the rangesets are
   inclusive.

Changes since v3:
 - Remove unnecessary line wrapping.

Changes since v1:
 - Split from bigger patch.
---
 xen/arch/x86/hvm/io.c               | 16 ++++++++
 xen/arch/x86/include/asm/hvm/io.h   |  3 ++
 xen/arch/x86/include/asm/setup.h    |  1 +
 xen/arch/x86/setup.c                | 48 +++++++++++++++++++++++
 xen/drivers/passthrough/x86/iommu.c | 61 +++++++++++++++++++++++++++++
 5 files changed, 129 insertions(+)

diff --git a/xen/arch/x86/hvm/io.c b/xen/arch/x86/hvm/io.c
index d75af83ad01f..a42854c52b65 100644
--- a/xen/arch/x86/hvm/io.c
+++ b/xen/arch/x86/hvm/io.c
@@ -369,6 +369,22 @@ bool vpci_is_mmcfg_address(const struct domain *d, paddr_t addr)
     return vpci_mmcfg_find(d, addr);
 }
 
+int __hwdom_init vpci_subtract_mmcfg(const struct domain *d, struct rangeset *r)
+{
+    const struct hvm_mmcfg *mmcfg;
+
+    list_for_each_entry ( mmcfg, &d->arch.hvm.mmcfg_regions, next )
+    {
+        int rc = rangeset_remove_range(r, PFN_DOWN(mmcfg->addr),
+                                       PFN_DOWN(mmcfg->addr + mmcfg->size - 1));
+
+        if ( rc )
+            return rc;
+    }
+
+    return 0;
+}
+
 static unsigned int vpci_mmcfg_decode_addr(const struct hvm_mmcfg *mmcfg,
                                            paddr_t addr, pci_sbdf_t *sbdf)
 {
diff --git a/xen/arch/x86/include/asm/hvm/io.h b/xen/arch/x86/include/asm/hvm/io.h
index a97731657801..e1e5e6fe7491 100644
--- a/xen/arch/x86/include/asm/hvm/io.h
+++ b/xen/arch/x86/include/asm/hvm/io.h
@@ -156,6 +156,9 @@ void destroy_vpci_mmcfg(struct domain *d);
 /* Check if an address is between a MMCFG region for a domain. */
 bool vpci_is_mmcfg_address(const struct domain *d, paddr_t addr);
 
+/* Remove MMCFG regions from a given rangeset. */
+int vpci_subtract_mmcfg(const struct domain *d, struct rangeset *r);
+
 #endif /* __ASM_X86_HVM_IO_H__ */
 
 
diff --git a/xen/arch/x86/include/asm/setup.h b/xen/arch/x86/include/asm/setup.h
index 9a460e4db8f4..cd07d98101d8 100644
--- a/xen/arch/x86/include/asm/setup.h
+++ b/xen/arch/x86/include/asm/setup.h
@@ -37,6 +37,7 @@ void discard_initial_images(void);
 void *bootstrap_map(const module_t *mod);
 
 int xen_in_range(unsigned long mfn);
+int remove_xen_ranges(struct rangeset *r);
 
 extern uint8_t kbd_shift_flags;
 
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index 897b7e92082e..ee233c69f1cb 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -2138,6 +2138,54 @@ int __hwdom_init xen_in_range(unsigned long mfn)
     return 0;
 }
 
+int __hwdom_init remove_xen_ranges(struct rangeset *r)
+{
+    paddr_t start, end;
+    int rc;
+
+    /* S3 resume code (and other real mode trampoline code) */
+    rc = rangeset_remove_range(r, PFN_DOWN(bootsym_phys(trampoline_start)),
+                               PFN_DOWN(bootsym_phys(trampoline_end) - 1));
+    if ( rc )
+        return rc;
+
+    /*
+     * This needs to remain in sync with the uses of the same symbols in
+     * - __start_xen()
+     * - is_xen_fixed_mfn()
+     * - tboot_shutdown()
+     */
+    /* hypervisor .text + .rodata */
+    rc = rangeset_remove_range(r, PFN_DOWN(__pa(&_stext)),
+                               PFN_DOWN(__pa(&__2M_rodata_end) - 1));
+    if ( rc )
+        return rc;
+
+    /* hypervisor .data + .bss */
+    if ( efi_boot_mem_unused(&start, &end) )
+    {
+        ASSERT(__pa(start) >= __pa(&__2M_rwdata_start));
+        rc = rangeset_remove_range(r, PFN_DOWN(__pa(&__2M_rwdata_start)),
+                                   PFN_DOWN(__pa(start) - 1));
+        if ( rc )
+            return rc;
+        ASSERT(__pa(end) <= __pa(&__2M_rwdata_end));
+        rc = rangeset_remove_range(r, PFN_DOWN(__pa(end)),
+                                   PFN_DOWN(__pa(&__2M_rwdata_end) - 1));
+        if ( rc )
+            return rc;
+    }
+    else
+    {
+        rc = rangeset_remove_range(r, PFN_DOWN(__pa(&__2M_rwdata_start)),
+                                   PFN_DOWN(__pa(&__2M_rwdata_end) - 1));
+        if ( rc )
+            return rc;
+    }
+
+    return 0;
+}
+
 static int __hwdom_init cf_check io_bitmap_cb(
     unsigned long s, unsigned long e, void *ctx)
 {
diff --git a/xen/drivers/passthrough/x86/iommu.c b/xen/drivers/passthrough/x86/iommu.c
index 59b0c7e980ca..fc5215a9dc40 100644
--- a/xen/drivers/passthrough/x86/iommu.c
+++ b/xen/drivers/passthrough/x86/iommu.c
@@ -370,6 +370,14 @@ static unsigned int __hwdom_init hwdom_iommu_map(const struct domain *d,
     return perms;
 }
 
+static int __hwdom_init cf_check map_subtract(unsigned long s, unsigned long e,
+                                              void *data)
+{
+    struct rangeset *map = data;
+
+    return rangeset_remove_range(map, s, e);
+}
+
 struct map_data {
     struct domain *d;
     unsigned int flush_flags;
@@ -533,6 +541,59 @@ void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
             goto commit;
     }
 
+    /* Remove any areas in-use by Xen. */
+    rc = remove_xen_ranges(map);
+    if ( rc )
+        panic("IOMMU failed to remove Xen ranges: %d\n", rc);
+
+    /* Remove any overlap with the Interrupt Address Range. */
+    rc = rangeset_remove_range(map, 0xfee00, 0xfeeff);
+    if ( rc )
+        panic("IOMMU failed to remove Interrupt Address Range: %d\n", rc);
+
+    /* If emulating IO-APIC(s) make sure the base address is unmapped. */
+    if ( has_vioapic(d) )
+    {
+        for ( i = 0; i < d->arch.hvm.nr_vioapics; i++ )
+        {
+            rc = rangeset_remove_singleton(map,
+                PFN_DOWN(domain_vioapic(d, i)->base_address));
+            if ( rc )
+                panic("IOMMU failed to remove IO-APIC: %d\n", rc);
+        }
+    }
+
+    if ( is_pv_domain(d) )
+    {
+        /*
+         * Be consistent with CPU mappings: Dom0 is permitted to establish r/o
+         * ones there (also for e.g. HPET in certain cases), so it should also
+         * have such established for IOMMUs.  Remove any read-only ranges here,
+         * since ranges in mmio_ro_ranges are already explicitly mapped below
+         * in read-only mode.
+         */
+        rc = rangeset_report_ranges(mmio_ro_ranges, 0, ~0UL, map_subtract, map);
+        if ( rc )
+            panic("IOMMU failed to remove read-only regions: %d\n", rc);
+    }
+
+    if ( has_vpci(d) )
+    {
+        /*
+         * TODO: runtime added MMCFG regions are not checked to make sure they
+         * don't overlap with already mapped regions, thus preventing trapping.
+         */
+        rc = vpci_subtract_mmcfg(d, map);
+        if ( rc )
+            panic("IOMMU unable to remove MMCFG areas: %d\n", rc);
+    }
+
+    /* Remove any regions past the last address addressable by the domain. */
+    rc = rangeset_remove_range(map, PFN_DOWN(1UL << domain_max_paddr_bits(d)),
+                               ~0UL);
+    if ( rc )
+        panic("IOMMU unable to remove unaddressable ranges: %d\n", rc);
+
     if ( iommu_verbose )
         printk(XENLOG_INFO "%pd: identity mappings for IOMMU:\n", d);
 
-- 
2.43.0

Re: [PATCH v6 1/3] x86/iommu: remove regions not to be mapped

[Jan Beulich]

On 25.01.2024 14:26, Roger Pau Monne wrote:
> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -2138,6 +2138,54 @@ int __hwdom_init xen_in_range(unsigned long mfn)
>      return 0;
>  }
>  
> +int __hwdom_init remove_xen_ranges(struct rangeset *r)
> +{
> +    paddr_t start, end;
> +    int rc;
> +
> +    /* S3 resume code (and other real mode trampoline code) */
> +    rc = rangeset_remove_range(r, PFN_DOWN(bootsym_phys(trampoline_start)),
> +                               PFN_DOWN(bootsym_phys(trampoline_end) - 1));

As per further v5 comments this wants to be

    rc = rangeset_remove_range(r, PFN_DOWN(bootsym_phys(trampoline_start)),
                               PFN_DOWN(bootsym_phys(trampoline_start)));

or some such, with suitable cross-referencing comments added here and on
the other side as to this only being a single page (unless this is already
somehow abstracted; I can't spot any respective assertion in xen.lds.S
though, for example).

Jan

Re: [PATCH v6 1/3] x86/iommu: remove regions not to be mapped

[Jan Beulich]

On 25.01.2024 14:26, Roger Pau Monne wrote:
> --- a/xen/arch/x86/setup.c
> +++ b/xen/arch/x86/setup.c
> @@ -2138,6 +2138,54 @@ int __hwdom_init xen_in_range(unsigned long mfn)
>      return 0;
>  }
>  
> +int __hwdom_init remove_xen_ranges(struct rangeset *r)
> +{
> +    paddr_t start, end;
> +    int rc;
> +
> +    /* S3 resume code (and other real mode trampoline code) */
> +    rc = rangeset_remove_range(r, PFN_DOWN(bootsym_phys(trampoline_start)),
> +                               PFN_DOWN(bootsym_phys(trampoline_end) - 1));

With the understanding that we've settled on us not quite being at the
point where only one page of the trampoline is kept mapped:
Reviewed-by: Jan Beulich <jbeulich@suse.com>

Jan

Re: [PATCH v6 1/3] x86/iommu: remove regions not to be mapped

[Paul Durrant]

On 25/01/2024 13:26, Roger Pau Monne wrote:
> Introduce the code to remove regions not to be mapped from the rangeset
> that will be used to setup the IOMMU page tables for the hardware domain.
> 
> This change also introduces two new functions: remove_xen_ranges() and
> vpci_subtract_mmcfg() that copy the logic in xen_in_range() and
> vpci_is_mmcfg_address() respectively and remove the ranges that would otherwise
> be intercepted by the original functions.
> 
> Note that the rangeset is still not populated.
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> ---
> Changes since v6:
>   - Fix subtraction to be done against the address, not the mfn.
> 
> Changes since v4:
>   - Fix off-by-one when removing the Xen used ranges, as the rangesets are
>     inclusive.
> 
> Changes since v3:
>   - Remove unnecessary line wrapping.
> 
> Changes since v1:
>   - Split from bigger patch.
> ---
>   xen/arch/x86/hvm/io.c               | 16 ++++++++
>   xen/arch/x86/include/asm/hvm/io.h   |  3 ++
>   xen/arch/x86/include/asm/setup.h    |  1 +
>   xen/arch/x86/setup.c                | 48 +++++++++++++++++++++++
>   xen/drivers/passthrough/x86/iommu.c | 61 +++++++++++++++++++++++++++++
>   5 files changed, 129 insertions(+)
> 

Reviewed-by: Paul Durrant <paul@xen.org>

[PATCH v5 2/3] x86/iommu: switch hwdom IOMMU to use a rangeset

[Roger Pau Monne]

The current loop that iterates from 0 to the maximum RAM address in order to
setup the IOMMU mappings is highly inefficient, and it will get worse as the
amount of RAM increases.  It's also not accounting for any reserved regions
past the last RAM address.

Instead of iterating over memory addresses, iterate over the memory map regions
and use a rangeset in order to keep track of which ranges need to be identity
mapped in the hardware domain physical address space.

On an AMD EPYC 7452 with 512GiB of RAM, the time to execute
arch_iommu_hwdom_init() in nanoseconds is:

x old
+ new
    N           Min           Max        Median           Avg        Stddev
x   5 2.2364154e+10  2.338244e+10 2.2474685e+10 2.2622409e+10 4.2949869e+08
+   5       1025012       1033036       1026188     1028276.2     3623.1194
Difference at 95.0% confidence
        -2.26214e+10 +/- 4.42931e+08
        -99.9955% +/- 9.05152e-05%
        (Student's t, pooled s = 3.03701e+08)

Execution time of arch_iommu_hwdom_init() goes down from ~22s to ~0.001s.

Note there's a change for HVM domains (ie: PVH dom0) that get switched to
create the p2m mappings using map_mmio_regions() instead of
p2m_add_identity_entry(), so that ranges can be mapped with a single function
call if possible.  Note that the interface of map_mmio_regions() doesn't
allow creating read-only mappings, but so far there are no such mappings
created for PVH dom0 in arch_iommu_hwdom_init().

No change intended in the resulting mappings that a hardware domain gets.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v4:
 - Add default case to handle ACPI and NVS regions (which are not mapped)
   unless in the low 4GB and when inclusive mode is set.
 - Add changelog entry.
 - Dropped Jans RB.

Changes since v3:
 - Remove unnecessary line wraps.

Changes since v2:
 - Simplify a bit the logic related to inclusive option, at the cost of making
   some no-op calls on some cases.

Changes since v1:
 - Split from bigger patch.
 - Remove unneeded default case.

x86/iommu: add CHANGELOG entry for hwdom setup time improvements

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
 CHANGELOG.md                        |   2 +
 xen/drivers/passthrough/x86/iommu.c | 149 ++++++----------------------
 2 files changed, 35 insertions(+), 116 deletions(-)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 723d06425431..3e8b996e4718 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -9,6 +9,8 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 ### Changed
  - Changed flexible array definitions in public I/O interface headers to not
    use "1" as the number of array elements.
+ - On x86:
+   - Reduce IOMMU setup time for hardware domain.
 
 ### Added
  - On x86:
diff --git a/xen/drivers/passthrough/x86/iommu.c b/xen/drivers/passthrough/x86/iommu.c
index fc5215a9dc40..c90755ff58fa 100644
--- a/xen/drivers/passthrough/x86/iommu.c
+++ b/xen/drivers/passthrough/x86/iommu.c
@@ -300,76 +300,6 @@ void iommu_identity_map_teardown(struct domain *d)
     }
 }
 
-static unsigned int __hwdom_init hwdom_iommu_map(const struct domain *d,
-                                                 unsigned long pfn,
-                                                 unsigned long max_pfn)
-{
-    mfn_t mfn = _mfn(pfn);
-    unsigned int i, type, perms = IOMMUF_readable | IOMMUF_writable;
-
-    /*
-     * Set up 1:1 mapping for dom0. Default to include only conventional RAM
-     * areas and let RMRRs include needed reserved regions. When set, the
-     * inclusive mapping additionally maps in every pfn up to 4GB except those
-     * that fall in unusable ranges for PV Dom0.
-     */
-    if ( (pfn > max_pfn && !mfn_valid(mfn)) || xen_in_range(pfn) )
-        return 0;
-
-    switch ( type = page_get_ram_type(mfn) )
-    {
-    case RAM_TYPE_UNUSABLE:
-        return 0;
-
-    case RAM_TYPE_CONVENTIONAL:
-        if ( iommu_hwdom_strict )
-            return 0;
-        break;
-
-    default:
-        if ( type & RAM_TYPE_RESERVED )
-        {
-            if ( !iommu_hwdom_inclusive && !iommu_hwdom_reserved )
-                perms = 0;
-        }
-        else if ( is_hvm_domain(d) )
-            return 0;
-        else if ( !iommu_hwdom_inclusive || pfn > max_pfn )
-            perms = 0;
-    }
-
-    /* Check that it doesn't overlap with the Interrupt Address Range. */
-    if ( pfn >= 0xfee00 && pfn <= 0xfeeff )
-        return 0;
-    /* ... or the IO-APIC */
-    if ( has_vioapic(d) )
-    {
-        for ( i = 0; i < d->arch.hvm.nr_vioapics; i++ )
-            if ( pfn == PFN_DOWN(domain_vioapic(d, i)->base_address) )
-                return 0;
-    }
-    else if ( is_pv_domain(d) )
-    {
-        /*
-         * Be consistent with CPU mappings: Dom0 is permitted to establish r/o
-         * ones there (also for e.g. HPET in certain cases), so it should also
-         * have such established for IOMMUs.
-         */
-        if ( iomem_access_permitted(d, pfn, pfn) &&
-             rangeset_contains_singleton(mmio_ro_ranges, pfn) )
-            perms = IOMMUF_readable;
-    }
-    /*
-     * ... or the PCIe MCFG regions.
-     * TODO: runtime added MMCFG regions are not checked to make sure they
-     * don't overlap with already mapped regions, thus preventing trapping.
-     */
-    if ( has_vpci(d) && vpci_is_mmcfg_address(d, pfn_to_paddr(pfn)) )
-        return 0;
-
-    return perms;
-}
-
 static int __hwdom_init cf_check map_subtract(unsigned long s, unsigned long e,
                                               void *data)
 {
@@ -455,8 +385,7 @@ static int __hwdom_init cf_check identity_map(unsigned long s, unsigned long e,
 
 void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
 {
-    unsigned long i, top, max_pfn, start, count;
-    unsigned int start_perms = 0;
+    unsigned int i;
     struct rangeset *map;
     struct map_data map_data = { .d = d };
     int rc;
@@ -487,58 +416,46 @@ void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
     if ( !map )
         panic("IOMMU init: unable to allocate rangeset\n");
 
-    max_pfn = (GB(4) >> PAGE_SHIFT) - 1;
-    top = max(max_pdx, pfn_to_pdx(max_pfn) + 1);
+    if ( iommu_hwdom_inclusive )
+    {
+        /* Add the whole range below 4GB, UNUSABLE regions will be removed. */
+        rc = rangeset_add_range(map, 0, PFN_DOWN(GB(4)) - 1);
+        if ( rc )
+            panic("IOMMU inclusive mappings can't be added: %d\n", rc);
+    }
 
-    for ( i = 0, start = 0, count = 0; i < top; )
+    for ( i = 0; i < e820.nr_map; i++ )
     {
-        unsigned long pfn = pdx_to_pfn(i);
-        unsigned int perms = hwdom_iommu_map(d, pfn, max_pfn);
+        const struct e820entry entry = e820.map[i];
 
-        if ( !perms )
-            /* nothing */;
-        else if ( paging_mode_translate(d) )
+        switch ( entry.type )
         {
-            int rc;
-
-            rc = p2m_add_identity_entry(d, pfn,
-                                        perms & IOMMUF_writable ? p2m_access_rw
-                                                                : p2m_access_r,
-                                        0);
+        case E820_UNUSABLE:
+            /* Only relevant for inclusive mode, otherwise this is a no-op. */
+            rc = rangeset_remove_range(map, PFN_DOWN(entry.addr),
+                                       PFN_DOWN(entry.addr + entry.size - 1));
             if ( rc )
-                printk(XENLOG_WARNING
-                       "%pd: identity mapping of %lx failed: %d\n",
-                       d, pfn, rc);
-        }
-        else if ( pfn != start + count || perms != start_perms )
-        {
-            long rc;
+                panic("IOMMU failed to remove unusable memory: %d\n", rc);
+            continue;
 
-        commit:
-            while ( (rc = iommu_map(d, _dfn(start), _mfn(start), count,
-                                    start_perms | IOMMUF_preempt,
-                                    &map_data.flush_flags)) > 0 )
-            {
-                start += rc;
-                count -= rc;
-                process_pending_softirqs();
-            }
-            if ( rc )
-                printk(XENLOG_WARNING
-                       "%pd: IOMMU identity mapping of [%lx,%lx) failed: %ld\n",
-                       d, start, start + count, rc);
-            start = pfn;
-            count = 1;
-            start_perms = perms;
-        }
-        else
-            ++count;
+        case E820_RESERVED:
+            if ( !iommu_hwdom_inclusive && !iommu_hwdom_reserved )
+                continue;
+            break;
 
-        if ( !(++i & 0xfffff) )
-            process_pending_softirqs();
+        case E820_RAM:
+            if ( iommu_hwdom_strict )
+                continue;
+            break;
 
-        if ( i == top && count )
-            goto commit;
+        default:
+            continue;
+        }
+
+        rc = rangeset_add_range(map, PFN_DOWN(entry.addr),
+                                PFN_DOWN(entry.addr + entry.size - 1));
+        if ( rc )
+            panic("IOMMU failed to add identity range: %d\n", rc);
     }
 
     /* Remove any areas in-use by Xen. */
-- 
2.43.0

Re: [PATCH v5 2/3] x86/iommu: switch hwdom IOMMU to use a rangeset

[Jan Beulich]

On 24.01.2024 18:29, Roger Pau Monne wrote:
> The current loop that iterates from 0 to the maximum RAM address in order to
> setup the IOMMU mappings is highly inefficient, and it will get worse as the
> amount of RAM increases.  It's also not accounting for any reserved regions
> past the last RAM address.
> 
> Instead of iterating over memory addresses, iterate over the memory map regions
> and use a rangeset in order to keep track of which ranges need to be identity
> mapped in the hardware domain physical address space.
> 
> On an AMD EPYC 7452 with 512GiB of RAM, the time to execute
> arch_iommu_hwdom_init() in nanoseconds is:
> 
> x old
> + new
>     N           Min           Max        Median           Avg        Stddev
> x   5 2.2364154e+10  2.338244e+10 2.2474685e+10 2.2622409e+10 4.2949869e+08
> +   5       1025012       1033036       1026188     1028276.2     3623.1194
> Difference at 95.0% confidence
>         -2.26214e+10 +/- 4.42931e+08
>         -99.9955% +/- 9.05152e-05%
>         (Student's t, pooled s = 3.03701e+08)
> 
> Execution time of arch_iommu_hwdom_init() goes down from ~22s to ~0.001s.
> 
> Note there's a change for HVM domains (ie: PVH dom0) that get switched to
> create the p2m mappings using map_mmio_regions() instead of
> p2m_add_identity_entry(), so that ranges can be mapped with a single function
> call if possible.  Note that the interface of map_mmio_regions() doesn't
> allow creating read-only mappings, but so far there are no such mappings
> created for PVH dom0 in arch_iommu_hwdom_init().
> 
> No change intended in the resulting mappings that a hardware domain gets.
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

Reviewed-by: Jan Beulich <jbeulich@suse.com>

Re: [PATCH v5 2/3] x86/iommu: switch hwdom IOMMU to use a rangeset

[Paul Durrant]

On 24/01/2024 17:29, Roger Pau Monne wrote:
> The current loop that iterates from 0 to the maximum RAM address in order to
> setup the IOMMU mappings is highly inefficient, and it will get worse as the
> amount of RAM increases.  It's also not accounting for any reserved regions
> past the last RAM address.
> 
> Instead of iterating over memory addresses, iterate over the memory map regions
> and use a rangeset in order to keep track of which ranges need to be identity
> mapped in the hardware domain physical address space.
> 
> On an AMD EPYC 7452 with 512GiB of RAM, the time to execute
> arch_iommu_hwdom_init() in nanoseconds is:
> 
> x old
> + new
>      N           Min           Max        Median           Avg        Stddev
> x   5 2.2364154e+10  2.338244e+10 2.2474685e+10 2.2622409e+10 4.2949869e+08
> +   5       1025012       1033036       1026188     1028276.2     3623.1194
> Difference at 95.0% confidence
>          -2.26214e+10 +/- 4.42931e+08
>          -99.9955% +/- 9.05152e-05%
>          (Student's t, pooled s = 3.03701e+08)
> 
> Execution time of arch_iommu_hwdom_init() goes down from ~22s to ~0.001s.
> 
> Note there's a change for HVM domains (ie: PVH dom0) that get switched to
> create the p2m mappings using map_mmio_regions() instead of
> p2m_add_identity_entry(), so that ranges can be mapped with a single function
> call if possible.  Note that the interface of map_mmio_regions() doesn't
> allow creating read-only mappings, but so far there are no such mappings
> created for PVH dom0 in arch_iommu_hwdom_init().
> 
> No change intended in the resulting mappings that a hardware domain gets.
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> ---
> Changes since v4:
>   - Add default case to handle ACPI and NVS regions (which are not mapped)
>     unless in the low 4GB and when inclusive mode is set.
>   - Add changelog entry.
>   - Dropped Jans RB.
> 
> Changes since v3:
>   - Remove unnecessary line wraps.
> 
> Changes since v2:
>   - Simplify a bit the logic related to inclusive option, at the cost of making
>     some no-op calls on some cases.
> 
> Changes since v1:
>   - Split from bigger patch.
>   - Remove unneeded default case.
> 
> x86/iommu: add CHANGELOG entry for hwdom setup time improvements
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

Reviewed-by: Paul Durrant <paul@xen.org>

[PATCH v5 3/3] x86/iommu: cleanup unused functions

[Roger Pau Monne]

Remove xen_in_range() and vpci_is_mmcfg_address() now that hey are unused.

Adjust comments to point to the new functions that replace the existing ones.

No functional change.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
Changes since v2:
 - Do remove vpci_is_mmcfg_address().
---
Can be squashed with the previous patch if desired, split as a separate patch
for clarity.
---
 xen/arch/x86/hvm/io.c             |  5 ---
 xen/arch/x86/include/asm/hvm/io.h |  3 --
 xen/arch/x86/include/asm/setup.h  |  1 -
 xen/arch/x86/setup.c              | 53 ++-----------------------------
 xen/arch/x86/tboot.c              |  2 +-
 5 files changed, 3 insertions(+), 61 deletions(-)

diff --git a/xen/arch/x86/hvm/io.c b/xen/arch/x86/hvm/io.c
index a42854c52b65..06283b41c463 100644
--- a/xen/arch/x86/hvm/io.c
+++ b/xen/arch/x86/hvm/io.c
@@ -364,11 +364,6 @@ static const struct hvm_mmcfg *vpci_mmcfg_find(const struct domain *d,
     return NULL;
 }
 
-bool vpci_is_mmcfg_address(const struct domain *d, paddr_t addr)
-{
-    return vpci_mmcfg_find(d, addr);
-}
-
 int __hwdom_init vpci_subtract_mmcfg(const struct domain *d, struct rangeset *r)
 {
     const struct hvm_mmcfg *mmcfg;
diff --git a/xen/arch/x86/include/asm/hvm/io.h b/xen/arch/x86/include/asm/hvm/io.h
index e1e5e6fe7491..24d1b6134f02 100644
--- a/xen/arch/x86/include/asm/hvm/io.h
+++ b/xen/arch/x86/include/asm/hvm/io.h
@@ -153,9 +153,6 @@ int register_vpci_mmcfg_handler(struct domain *d, paddr_t addr,
 /* Destroy tracked MMCFG areas. */
 void destroy_vpci_mmcfg(struct domain *d);
 
-/* Check if an address is between a MMCFG region for a domain. */
-bool vpci_is_mmcfg_address(const struct domain *d, paddr_t addr);
-
 /* Remove MMCFG regions from a given rangeset. */
 int vpci_subtract_mmcfg(const struct domain *d, struct rangeset *r);
 
diff --git a/xen/arch/x86/include/asm/setup.h b/xen/arch/x86/include/asm/setup.h
index cd07d98101d8..1ced1299c77b 100644
--- a/xen/arch/x86/include/asm/setup.h
+++ b/xen/arch/x86/include/asm/setup.h
@@ -36,7 +36,6 @@ unsigned long initial_images_nrpages(nodeid_t node);
 void discard_initial_images(void);
 void *bootstrap_map(const module_t *mod);
 
-int xen_in_range(unsigned long mfn);
 int remove_xen_ranges(struct rangeset *r);
 
 extern uint8_t kbd_shift_flags;
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index c9f65c3a70b8..8082aac303e0 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -1343,7 +1343,7 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p)
         relocated = true;
 
         /*
-         * This needs to remain in sync with xen_in_range() and the
+         * This needs to remain in sync with remove_xen_ranges() and the
          * respective reserve_e820_ram() invocation below. No need to
          * query efi_boot_mem_unused() here, though.
          */
@@ -1495,7 +1495,7 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p)
     if ( using_2M_mapping() )
         efi_boot_mem_unused(NULL, NULL);
 
-    /* This needs to remain in sync with xen_in_range(). */
+    /* This needs to remain in sync with remove_xen_ranges(). */
     if ( efi_boot_mem_unused(&eb_start, &eb_end) )
     {
         reserve_e820_ram(&boot_e820, __pa(_stext), __pa(eb_start));
@@ -2089,55 +2089,6 @@ void arch_get_xen_caps(xen_capabilities_info_t *info)
     }
 }
 
-int __hwdom_init xen_in_range(unsigned long mfn)
-{
-    paddr_t start, end;
-    int i;
-
-    enum { region_s3, region_ro, region_rw, region_bss, nr_regions };
-    static struct {
-        paddr_t s, e;
-    } xen_regions[nr_regions] __hwdom_initdata;
-
-    /* initialize first time */
-    if ( !xen_regions[0].s )
-    {
-        /* S3 resume code (and other real mode trampoline code) */
-        xen_regions[region_s3].s = bootsym_phys(trampoline_start);
-        xen_regions[region_s3].e = bootsym_phys(trampoline_end);
-
-        /*
-         * This needs to remain in sync with the uses of the same symbols in
-         * - __start_xen() (above)
-         * - is_xen_fixed_mfn()
-         * - tboot_shutdown()
-         */
-
-        /* hypervisor .text + .rodata */
-        xen_regions[region_ro].s = __pa(&_stext);
-        xen_regions[region_ro].e = __pa(&__2M_rodata_end);
-        /* hypervisor .data + .bss */
-        xen_regions[region_rw].s = __pa(&__2M_rwdata_start);
-        xen_regions[region_rw].e = __pa(&__2M_rwdata_end);
-        if ( efi_boot_mem_unused(&start, &end) )
-        {
-            ASSERT(__pa(start) >= xen_regions[region_rw].s);
-            ASSERT(__pa(end) <= xen_regions[region_rw].e);
-            xen_regions[region_rw].e = __pa(start);
-            xen_regions[region_bss].s = __pa(end);
-            xen_regions[region_bss].e = __pa(&__2M_rwdata_end);
-        }
-    }
-
-    start = (paddr_t)mfn << PAGE_SHIFT;
-    end = start + PAGE_SIZE;
-    for ( i = 0; i < nr_regions; i++ )
-        if ( (start < xen_regions[i].e) && (end > xen_regions[i].s) )
-            return 1;
-
-    return 0;
-}
-
 int __hwdom_init remove_xen_ranges(struct rangeset *r)
 {
     paddr_t start, end;
diff --git a/xen/arch/x86/tboot.c b/xen/arch/x86/tboot.c
index 86c4c22cacb8..4c254b4e34b4 100644
--- a/xen/arch/x86/tboot.c
+++ b/xen/arch/x86/tboot.c
@@ -321,7 +321,7 @@ void tboot_shutdown(uint32_t shutdown_type)
 
         /*
          * Xen regions for tboot to MAC. This needs to remain in sync with
-         * xen_in_range().
+         * remove_xen_ranges().
          */
         g_tboot_shared->num_mac_regions = 3;
         /* S3 resume code (and other real mode trampoline code) */
-- 
2.43.0

Re: [PATCH v5 3/3] x86/iommu: cleanup unused functions

[Paul Durrant]

On 24/01/2024 17:29, Roger Pau Monne wrote:
> Remove xen_in_range() and vpci_is_mmcfg_address() now that hey are unused.
> 
> Adjust comments to point to the new functions that replace the existing ones.
> 
> No functional change.
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> Reviewed-by: Jan Beulich <jbeulich@suse.com>
> ---
> Changes since v2:
>   - Do remove vpci_is_mmcfg_address().
> ---
> Can be squashed with the previous patch if desired, split as a separate patch
> for clarity.
> ---
>   xen/arch/x86/hvm/io.c             |  5 ---
>   xen/arch/x86/include/asm/hvm/io.h |  3 --
>   xen/arch/x86/include/asm/setup.h  |  1 -
>   xen/arch/x86/setup.c              | 53 ++-----------------------------
>   xen/arch/x86/tboot.c              |  2 +-
>   5 files changed, 3 insertions(+), 61 deletions(-)
> 

Reviewed-by: Paul Durrant <paul@xen.org>