[PATCH v5 00/10] virtio DMA API, yet again

[PATCH v5 00/10] virtio DMA API, yet again

[Andy Lutomirski]

This switches virtio to use the DMA API on Xen and if requested by
module option.

This fixes virtio on Xen, and it should break anything because it's
off by default on everything except Xen PV on x86.

To the Xen people: is this okay?  If it doesn't work on other Xen
variants (PVH? HVM?), can you submit follow-up patches to fix it?

To everyone else: we've waffled on this for way too long.  I think
we should to get DMA API implementation in with a conservative
policy like this rather than waiting until we achieve perfection.
I'm tired of carrying these patches around.

Michael, if these survive review, can you stage these in your tree?
Can you also take a look at tools/virtio?  I probably broke it, but I
couldn't get it to build without these patches either, so I'm stuck.

Changes from v4:
 - Bake vring_use_dma_api in from the beginning.
 - Automatically enable only on Xen.
 - Add module parameter.
 - Add s390 and alpha DMA API implementations.
 - Rebase to 4.5-rc1.

Changes from v3:
 - More big-endian fixes.
 - Added better virtio-ring APIs that handle allocation and use them in
   virtio-mmio and virtio-pci.
 - Switch to Michael's virtio-net patch.

Changes from v2:
 - Fix vring_mapping_error incorrect argument

Changes from v1:
 - Fix an endian conversion error causing a BUG to hit.
 - Fix a DMA ordering issue (swiotlb=force works now).
 - Minor cleanups.

Andy Lutomirski (7):
  vring: Introduce vring_use_dma_api()
  virtio_ring: Support DMA APIs
  virtio: Add improved queue allocation API
  virtio_mmio: Use the DMA API if enabled
  virtio_pci: Use the DMA API if enabled
  vring: Use the DMA API on Xen
  vring: Add a module parameter to force-enable the DMA API

Christian Borntraeger (3):
  dma: Provide simple noop dma ops
  alpha/dma: use common noop dma ops
  s390/dma: Allow per device dma ops

 arch/alpha/kernel/pci-noop.c        |  46 +---
 arch/s390/Kconfig                   |   6 +-
 arch/s390/include/asm/device.h      |   6 +-
 arch/s390/include/asm/dma-mapping.h |   6 +-
 arch/s390/pci/pci.c                 |   1 +
 arch/s390/pci/pci_dma.c             |   4 +-
 drivers/virtio/Kconfig              |   2 +-
 drivers/virtio/virtio_mmio.c        |  67 ++----
 drivers/virtio/virtio_pci_common.h  |   6 -
 drivers/virtio/virtio_pci_legacy.c  |  42 ++--
 drivers/virtio/virtio_pci_modern.c  |  61 ++----
 drivers/virtio/virtio_ring.c        | 412 ++++++++++++++++++++++++++++++------
 include/linux/dma-mapping.h         |   2 +
 include/linux/virtio.h              |  23 +-
 include/linux/virtio_ring.h         |  35 +++
 lib/Makefile                        |   1 +
 lib/dma-noop.c                      |  75 +++++++
 tools/virtio/linux/dma-mapping.h    |  17 ++
 18 files changed, 568 insertions(+), 244 deletions(-)
 create mode 100644 lib/dma-noop.c
 create mode 100644 tools/virtio/linux/dma-mapping.h

-- 
2.5.0

Re: [PATCH v5 00/10] virtio DMA API, yet again

[David Woodhouse]

[-- Attachment #1.1: Type: text/plain, Size: 1098 bytes --]

On Thu, 2016-01-28 at 18:31 -0800, Andy Lutomirski wrote:
> 
> To everyone else: we've waffled on this for way too long.  I think
> we should to get DMA API implementation in with a conservative
> policy like this rather than waiting until we achieve perfection.
> I'm tired of carrying these patches around.

Yeah, do it.

However,,, shouldn't the generic no-op DMA ops be checking the dma_mask
of the device and bitching if it can't reach the address in question?

Also, wasn't Christoph looking at making per-device DMA ops more
generic instead of an 'archdata' thing on basically every platform? Or
did I just imagine that part?

Not that I'm suggesting you make the s390 patch wait for that *instead*
of using archdata there, mind you. But I was kind of planning to let
the dust settle on this lot before I sort out the theoretical-except-
in-simulation issues with VT-d IOMMU covering *some* devices but not
all.

-- 
David Woodhouse                            Open Source Technology Centre
David.Woodhouse@intel.com                              Intel Corporation


[-- Attachment #1.2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5691 bytes --]

[-- Attachment #2: Type: text/plain, Size: 183 bytes --]

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [PATCH v5 00/10] virtio DMA API, yet again

[Christoph Hellwig]

On Fri, Jan 29, 2016 at 11:01:00AM +0000, David Woodhouse wrote:
> Also, wasn't Christoph looking at making per-device DMA ops more
> generic instead of an 'archdata' thing on basically every platform? Or
> did I just imagine that part?

What I've done for 4.5 is to switch all architectures to use DMA ops.
This should make it fairly easy to have a generic dma ops pointer in the
devie structure, but I have no need for that yet, and thus no short term
plans to do that work myself.

Re: [PATCH v5 00/10] virtio DMA API, yet again

[Michael S. Tsirkin]

On Thu, Jan 28, 2016 at 06:31:13PM -0800, Andy Lutomirski wrote:
> This switches virtio to use the DMA API on Xen and if requested by
> module option.
> 
> This fixes virtio on Xen, and it should break anything because it's
> off by default on everything except Xen PV on x86.
> 
> To the Xen people: is this okay?  If it doesn't work on other Xen
> variants (PVH? HVM?), can you submit follow-up patches to fix it?
> 
> To everyone else: we've waffled on this for way too long.  I think
> we should to get DMA API implementation in with a conservative
> policy like this rather than waiting until we achieve perfection.
> I'm tired of carrying these patches around.

I agree, thanks for working on this!

> Michael, if these survive review, can you stage these in your tree?

Yes, I'll stage everything except 10/10. I'd rather not maintain a
module option like this, things work for now and I'm working on a
clean solution for things like dpdk within guest.

So far I saw some comments on 9/10.

> Can you also take a look at tools/virtio?  I probably broke it, but I
> couldn't get it to build without these patches either, so I'm stuck.

Will do.

> Changes from v4:
>  - Bake vring_use_dma_api in from the beginning.
>  - Automatically enable only on Xen.
>  - Add module parameter.
>  - Add s390 and alpha DMA API implementations.
>  - Rebase to 4.5-rc1.
> 
> Changes from v3:
>  - More big-endian fixes.
>  - Added better virtio-ring APIs that handle allocation and use them in
>    virtio-mmio and virtio-pci.
>  - Switch to Michael's virtio-net patch.
> 
> Changes from v2:
>  - Fix vring_mapping_error incorrect argument
> 
> Changes from v1:
>  - Fix an endian conversion error causing a BUG to hit.
>  - Fix a DMA ordering issue (swiotlb=force works now).
>  - Minor cleanups.
> 
> Andy Lutomirski (7):
>   vring: Introduce vring_use_dma_api()
>   virtio_ring: Support DMA APIs
>   virtio: Add improved queue allocation API
>   virtio_mmio: Use the DMA API if enabled
>   virtio_pci: Use the DMA API if enabled
>   vring: Use the DMA API on Xen
>   vring: Add a module parameter to force-enable the DMA API
> 
> Christian Borntraeger (3):
>   dma: Provide simple noop dma ops
>   alpha/dma: use common noop dma ops
>   s390/dma: Allow per device dma ops
> 
>  arch/alpha/kernel/pci-noop.c        |  46 +---
>  arch/s390/Kconfig                   |   6 +-
>  arch/s390/include/asm/device.h      |   6 +-
>  arch/s390/include/asm/dma-mapping.h |   6 +-
>  arch/s390/pci/pci.c                 |   1 +
>  arch/s390/pci/pci_dma.c             |   4 +-
>  drivers/virtio/Kconfig              |   2 +-
>  drivers/virtio/virtio_mmio.c        |  67 ++----
>  drivers/virtio/virtio_pci_common.h  |   6 -
>  drivers/virtio/virtio_pci_legacy.c  |  42 ++--
>  drivers/virtio/virtio_pci_modern.c  |  61 ++----
>  drivers/virtio/virtio_ring.c        | 412 ++++++++++++++++++++++++++++++------
>  include/linux/dma-mapping.h         |   2 +
>  include/linux/virtio.h              |  23 +-
>  include/linux/virtio_ring.h         |  35 +++
>  lib/Makefile                        |   1 +
>  lib/dma-noop.c                      |  75 +++++++
>  tools/virtio/linux/dma-mapping.h    |  17 ++
>  18 files changed, 568 insertions(+), 244 deletions(-)
>  create mode 100644 lib/dma-noop.c
>  create mode 100644 tools/virtio/linux/dma-mapping.h
> 
> -- 
> 2.5.0

Re: [PATCH v5 00/10] virtio DMA API, yet again

[Andy Lutomirski]

On Sun, Jan 31, 2016 at 12:12 PM, Michael S. Tsirkin <mst@redhat.com> wrote:
> On Thu, Jan 28, 2016 at 06:31:13PM -0800, Andy Lutomirski wrote:
>> This switches virtio to use the DMA API on Xen and if requested by
>> module option.
>>
>> This fixes virtio on Xen, and it should break anything because it's
>> off by default on everything except Xen PV on x86.
>>
>> To the Xen people: is this okay?  If it doesn't work on other Xen
>> variants (PVH? HVM?), can you submit follow-up patches to fix it?
>>
>> To everyone else: we've waffled on this for way too long.  I think
>> we should to get DMA API implementation in with a conservative
>> policy like this rather than waiting until we achieve perfection.
>> I'm tired of carrying these patches around.
>
> I agree, thanks for working on this!
>
>> Michael, if these survive review, can you stage these in your tree?
>
> Yes, I'll stage everything except 10/10. I'd rather not maintain a
> module option like this, things work for now and I'm working on a
> clean solution for things like dpdk within guest.

The module option was mainly for testing, but patching in a "return
true" works just as well.

I ran the code through the DMA API debugging stuff and swiotlb=force
with the module option set under KVM (no Xen), and everything seemed
to work.

--Andy