Re: [PATCH v4 0/4] virtio: Clean up scatterlists and use the DMA API

Andy Lutomirski
2014-09-05T03:06:57Z

On Thu, Sep 4, 2014 at 7:32 PM, Rusty Russell <rusty@rustcorp.com.au> wrote:
> "Michael S. Tsirkin" <mst@redhat.com> writes:
>> On Wed, Sep 03, 2014 at 04:12:01PM +0930, Rusty Russell wrote:
>>> Andy Lutomirski <luto@amacapital.net> writes:
>>> > There really are virtio devices that are pieces of silicon and not
>>> > figments of a hypervisor's imagination [1].
>>>
>>> Hi Andy,
>>>
>>> As you're discovering, there's a reason no one has done the DMA
>>> API before.
>>>
>>> So the problem is that ppc64's IOMMU is a platform thing, not a bus
>>> thing. They really do carve out an exception for virtio devices,
>>> because performance (LOTS of performance). It remains to be seen if
>>> other platforms have the same performance issues, but in absence of
>>> other evidence, the answer is yes.
>>>
>>> It's a hack. But having specific virtual-only devices are an even
>>> bigger hack.
>>>
>>> Physical virtio devices have been talked about, but don't actually exist
>>> in Real Life. And someone a virtio PCI card is going to have serious
>>> performance issues: mainly because they'll want the rings in the card's
>>> MMIO region, not allocated by the driver.
>>
>> Why? What's wrong with rings in memory?
>
> AFAICT, the card would have to access guest memory to read it, using
> multiple DMA cycles. That's going to be slow.

I don't personally know all the considerations, but AFAICT NVMe puts
its rings in memory, and NMVe is very much focused on performance.

There might be an argument for trying to avoid using indirect rings on
real hardware to reduce the number of DMA round-trips needed for a
comment.

--Andy