Re: [RFC] Legacy Virtio Driver with Device Has Limited Memory Access

[Manivannan Sadhasivam <mani@kernel.org>]

On Mon, May 20, 2024 at 07:24:32PM +0900, Shunsuke Mie wrote:
> 2024年5月16日(木) 21:59 Manivannan Sadhasivam <mani@kernel.org>:
> >
> > On Thu, May 16, 2024 at 01:38:40PM +0900, Shunsuke Mie wrote:
> > > Hi virtio folks,
> > >
> >
> > You forgot to CC the actual Virtio folks. I've CCed them now.
> Oops. thank you.
> > > I'm writing to discuss finding a workaround with Virtio drivers and legacy
> > > devices with limited memory access.
> > >
> > > # Background
> > > The Virtio specification defines a feature (VIRTIO_F_ACCESS_PLATFORM) to
> > > indicate devices requiring restricted memory access or IOMMU translation. This
> > > feature bit resides at position 33 in the 64-bit Features register on modern
> > > interfaces. When the linux virtio driver finds the flag, the driver uses DMA
> > > API that handles to use of appropriate memory.
> > >
> > > # Problem
> > > However, legacy devices only have a 32-bit register for the features bits.
> > > Consequently, these devices cannot represent the ACCESS_PLATFORM bit. As a
> > > result, legacy devices with restricted memory access cannot function
> > > properly[1]. This is a legacy spec issue, but I'd like to find a workaround.
> > >
> > > # Proposed Solutions
> > > I know these are not ideal, but I propose the following solution.
> > > Driver-side:
> > >     - Implement special handling similar to xen_domain.
> > > In xen_domain, linux virtio driver enables to use the DMA API.
> > >     - Introduce a CONFIG option to adjust the DMA API behavior.
> > > Device-side:
> > > Due to indistinguishability from the guest's perspective, a device-side
> > > solution might be difficult.
> > >
> > > I'm open to any comments or suggestions you may have on this issue or
> > > alternative approaches.
> > >
> > > [1] virtio-net PCI endpoint function using PCIe Endpoint Framework,
> > > https://lore.kernel.org/lkml/54ee46c3-c845-3df3-8ba0-0ee79a2acab1@igel.co.jp/t/
> > > The Linux PCIe endpoint framework is used to implement the virtio-net device on
> > > a legacy interface. This is necessary because of the framework and hardware
> > > limitation.
> > >
> >
> > We can fix the endpoint framework limitation, but the problem lies with some
> > platforms where we cannot write to vendor capability registers and still have
> > IOMMU.
> I agree, this is a problem caused by the inability to set the
> capability. I'm not sure, but are there any chips that support this?

Most of the recent endpoint platforms should support this feature.

> Also, I wasn't aware of the IOMMU issue. I thought that if the Linux
> DMA subsystem could handle IOMMU properly, there wouldn't be any
> problems. Is that incorrect?

The issue is, legacy virtio PCI device only has 32bits. So they cannot support
VIRTIO_F_ACCESS_PLATFORM which is located at bit 33 as you explained.

And if this bit is not set, then DMA APIs won't be used by the virtio stack.

I think it is best to add support for modern virtio PCI device to make use of
IOMMU. Legacy devices can continue to use physical address.

- Mani

> 
> Shunsuke,
> Best
> > - Mani
> >
> > --
> > மணிவண்ணன் சதாசிவம்
> 

-- 
மணிவண்ணன் சதாசிவம்