Virtio-IOMMU interrupt remapping design

[Demi Marie Obenour <demiobenour@gmail.com>]

[-- Attachment #1.1.1: Type: text/plain, Size: 1798 bytes --]

Virtio-IOMMU interrupt remapping turned out to be much harder than I
realized.  The main problem is that interrupt remapping is set up
very early in boot.  In fact, Linux calls the interrupt remapping probe
function from the APIC initialization code: x86_64_probe_apic ->
enable_IR_x2apic -> irq_remapping_prepare().  This is almost certainly
much before PCI has been initialized.  Also, the order in which devices
will be initialized is not something Linux guarantees at all, which is a
problem because interrupt remapping must be initialized before drivers
start setting up interrupts.  Otherwise, the interrupt remapping table
won't include entries for already-existing interrupts, and things will
either break badly, not get the benefit of interrupt remapping
security-wise, or both.

The reason I expect this doesn't cause problems for address translation
is that the IOMMU probably starts in bypass mode by default, meaning
that all DMA is permitted.  If the IOMMU is only used by VFIO or
IOMMUFD, it will not be needed until userspace starts up, which is after
the IOMMU has been initialized.  This isn't ideal, though, as it means
that kernel drivers operate without DMA protection.

Is a paravirtualized IOMMU with interrupt remapping something that makes
sense?  Absolutely!  However, the IOMMU should be considered a platform
device that must be initialized very early in boot.  Using virtio-IOMMU
with MMIO transport as the interface might be a reasonable option, but
the IOMMU needs to be enumerated via ACPI, device tree, or kernel
command line argument.  This allows it to be brought up before anything
capable of DMA is initialized.

Is this the right path to go down?  What do others think about this?
-- 
Sincerely,
Demi Marie Obenour (she/her/hers)

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 7253 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]