why is iommu_platform set to off by default?

why is iommu_platform set to off by default?

[Peter Maydell]

I've just spent a day or so trying to track down why PCI passthrough
of a virtio-blk-pci device wasn't working. The problem turns out to be
that by default virtio pci devices don't use the IOMMU, even when the
machine model has created an IOMMU and arranged for the PCI bus to
be underneath it. So when the L2 guest tries to program the virtio device,
the virtio device treats the IPAs it writes as if they were PAs and
of course the data structures it's looking for aren't there.

Why do we default this to 'off'? It seems pretty unhelpful not to
honour the existence of the IOMMU, and the failure mode is pretty
opaque (L2 guest just hangs)...

thanks
-- PMM

Re: why is iommu_platform set to off by default?

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 1187 bytes --]

On Tue, Nov 30, 2021 at 02:32:49PM +0000, Peter Maydell wrote:
> I've just spent a day or so trying to track down why PCI passthrough
> of a virtio-blk-pci device wasn't working. The problem turns out to be
> that by default virtio pci devices don't use the IOMMU, even when the
> machine model has created an IOMMU and arranged for the PCI bus to
> be underneath it. So when the L2 guest tries to program the virtio device,
> the virtio device treats the IPAs it writes as if they were PAs and
> of course the data structures it's looking for aren't there.
> 
> Why do we default this to 'off'? It seems pretty unhelpful not to
> honour the existence of the IOMMU, and the failure mode is pretty
> opaque (L2 guest just hangs)...

Historically VIRTIO used guest physical addresses instead of bus
addresses (IOVA). I think when IOMMU support was added, a QEMU -device
virtio-* parameter was added and it's simply off by default:

  iommu_platform=<bool>  - on/off (default: false)

Maybe this behavior is for backwards compatibility? Existing guests with
IOMMUs shouldn't change behavior, although this could be fixed with
machine type compat properties.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

Re: why is iommu_platform set to off by default?

[Michael S. Tsirkin]

On Tue, Nov 30, 2021 at 04:38:11PM +0000, Stefan Hajnoczi wrote:
> On Tue, Nov 30, 2021 at 02:32:49PM +0000, Peter Maydell wrote:
> > I've just spent a day or so trying to track down why PCI passthrough
> > of a virtio-blk-pci device wasn't working. The problem turns out to be
> > that by default virtio pci devices don't use the IOMMU, even when the
> > machine model has created an IOMMU and arranged for the PCI bus to
> > be underneath it. So when the L2 guest tries to program the virtio device,
> > the virtio device treats the IPAs it writes as if they were PAs and
> > of course the data structures it's looking for aren't there.
> > 
> > Why do we default this to 'off'? It seems pretty unhelpful not to
> > honour the existence of the IOMMU, and the failure mode is pretty
> > opaque (L2 guest just hangs)...
> 
> Historically VIRTIO used guest physical addresses instead of bus
> addresses (IOVA). I think when IOMMU support was added, a QEMU -device
> virtio-* parameter was added and it's simply off by default:
> 
>   iommu_platform=<bool>  - on/off (default: false)
> 
> Maybe this behavior is for backwards compatibility? Existing guests with
> IOMMUs shouldn't change behavior, although this could be fixed with
> machine type compat properties.
> 
> Stefan

Unfortunately iommu is special. Bypassing it makes the config
insecure for nested virt, so it's important that we do not
allow guest driver to disable the iommu through the
feature bit.
This means normal compat machinery (make feature on by default
but disable for legacy drivers) does not work here.

-- 
MST

Re: why is iommu_platform set to off by default?

[Michael S. Tsirkin]

On Tue, Nov 30, 2021 at 02:32:49PM +0000, Peter Maydell wrote:
> I've just spent a day or so trying to track down why PCI passthrough
> of a virtio-blk-pci device wasn't working. The problem turns out to be
> that by default virtio pci devices don't use the IOMMU, even when the
> machine model has created an IOMMU and arranged for the PCI bus to
> be underneath it. So when the L2 guest tries to program the virtio device,
> the virtio device treats the IPAs it writes as if they were PAs and
> of course the data structures it's looking for aren't there.

Because this is what legacy guests expect, and legacy configs are
much more common than nested.

> Why do we default this to 'off'? It seems pretty unhelpful not to
> honour the existence of the IOMMU, and the failure mode is pretty
> opaque (L2 guest just hangs)...
> 
> thanks
> -- PMM

This should be handled by VFIO in L1 really, it can check for a
device quirk and refuse binding if the feature bit
is disabled.


-- 
MST