Re: [RFC] Legacy Virtio Driver with Device Has Limited Memory Access

[Manivannan Sadhasivam <mani@kernel.org>]

On Mon, May 20, 2024 at 09:22:54AM -0400, Michael S. Tsirkin wrote:
> On Thu, May 16, 2024 at 02:59:13PM +0200, Manivannan Sadhasivam wrote:
> > On Thu, May 16, 2024 at 01:38:40PM +0900, Shunsuke Mie wrote:
> > > Hi virtio folks,
> > > 
> > 
> > You forgot to CC the actual Virtio folks. I've CCed them now.
> > 
> > > I'm writing to discuss finding a workaround with Virtio drivers and legacy
> > > devices with limited memory access.
> > > 
> > > # Background
> > > The Virtio specification defines a feature (VIRTIO_F_ACCESS_PLATFORM) to
> > > indicate devices requiring restricted memory access or IOMMU translation. This
> > > feature bit resides at position 33 in the 64-bit Features register on modern
> > > interfaces. When the linux virtio driver finds the flag, the driver uses DMA
> > > API that handles to use of appropriate memory.
> > > 
> > > # Problem
> > > However, legacy devices only have a 32-bit register for the features bits.
> > > Consequently, these devices cannot represent the ACCESS_PLATFORM bit. As a
> > > result, legacy devices with restricted memory access cannot function
> > > properly[1]. This is a legacy spec issue, but I'd like to find a workaround.
> > > 
> > > # Proposed Solutions
> > > I know these are not ideal, but I propose the following solution.
> > > Driver-side:
> > >     - Implement special handling similar to xen_domain.
> > > In xen_domain, linux virtio driver enables to use the DMA API.
> > >     - Introduce a CONFIG option to adjust the DMA API behavior.
> > > Device-side:
> > > Due to indistinguishability from the guest's perspective, a device-side
> > > solution might be difficult.
> > > 
> > > I'm open to any comments or suggestions you may have on this issue or
> > > alternative approaches.
> > > 
> > > [1] virtio-net PCI endpoint function using PCIe Endpoint Framework,
> > > https://lore.kernel.org/lkml/54ee46c3-c845-3df3-8ba0-0ee79a2acab1@igel.co.jp/t/
> > > The Linux PCIe endpoint framework is used to implement the virtio-net device on
> > > a legacy interface. This is necessary because of the framework and hardware
> > > limitation.
> > > 
> > 
> > We can fix the endpoint framework limitation, but the problem lies with some
> > platforms where we cannot write to vendor capability registers and still have
> > IOMMU.
> > 
> > - Mani
> 
> What are vendor capability registers and what do they have to do
> with the IOMMU?
> 

Virtio spec v1.2, sec 4.1.4 says,

"Each structure can be mapped by a Base Address register (BAR) belonging to the
function, or accessed via the special VIRTIO_PCI_CAP_PCI_CFG field in the PCI
configuration space.

The location of each structure is specified using a vendor-specific PCI
capability located on the capability list in PCI configuration space of the
device."

So this means the device has to expose the virtio structures through vendor
specific capability isn't it?

And only in that case, it can expose VIRTIO_F_ACCESS_PLATFORM bit for making
use of IOMMU translation.

- Mani

-- 
மணிவண்ணன் சதாசிவம்