Re: More than 255 vcpus Windows VM setup without viommu ?

["Daniel P. Berrangé" <berrange@redhat.com>]

On Sat, Sep 28, 2024 at 03:59:32PM +0100, David Woodhouse wrote:
> On Tue, 2024-07-02 at 05:17 +0000, Sandesh Patel wrote:
> > 
> > The error is due to invalid MSIX routing entry passed to KVM.
> > 
> > The VM boots fine if we attach a vIOMMU but adding a vIOMMU can
> > potentially result in IO performance loss in guest.
> > I was interested to know if someone could boot a large Windows VM by
> > some other means like kvm-msi-ext-dest-id.
> 
> I think I may (with Alex Graf's suggestion) have found the Windows bug
> with Intel IOMMU.
> 
> It looks like when interrupt remapping is enabled with an AMD CPU,
> Windows *assumes* it can generate AMD-style MSI messages even if the
> IOMMU is an Intel one. If we put a little hack into the IOMMU interrupt
> remapping to make it interpret an AMD-style message, Windows seems to
> boot at least a little bit further than it did before...

Rather than filling the intel IOMMU impl with hacks to make Windows
boot on AMD virtualized CPUs, shouldn't we steer people to use the
amd-iommu that QEMU already ships [1] ?

Even if we hack the intel iommu, so current Windows boots, can we
have confidence that future Windows releases will correctly boot
on an intel iommu with AMD CPUs virtualized ?

> --- a/hw/i386/intel_iommu.c
> +++ b/hw/i386/intel_iommu.c
> @@ -3550,9 +3550,14 @@ static int vtd_interrupt_remap_msi(IntelIOMMUState *iommu,
>  
>      /* This is compatible mode. */
>      if (addr.addr.int_mode != VTD_IR_INT_FORMAT_REMAP) {
> -        memcpy(translated, origin, sizeof(*origin));
> -        goto out;
> -    }
> +        if (0) {
> +            memcpy(translated, origin, sizeof(*origin));
> +            goto out;
> +        }
> +        /* Pretend it's an AMD-format remappable MSI (Yay Windows!) */
> +        index = origin->data & 0x7ff;
> +        printf("Compat mode index 0x%x\n", index);
> +    } else
>  
>      index = addr.addr.index_h << 15 | addr.addr.index_l;


With regards,
Daniel

[1] the AMD IOMMU is not perfect, because currently it has a significant
    QEMU impl flaw in that it secretly creates an extra PCI device behind
    the scenes. This makes it impossible for libvirt to manage the PCI
    resources from the AMD IOMMU. I feel like this ought to be solvable
    though, as it is just a QEMU impl decison that can be corrected.
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|