Re: [Qemu-devel] [PATCH v4 00/18] spapr: vfio: Enable Dynamic DMA windows (DDW)

[Alexey Kardashevskiy <aik@ozlabs.ru>]

Hi!

Forgot to mention  - this is RFC only as 1) it depends on the host kernel
changes 2) does not support migration for emulated PHB + DDW, I think I
have to give David's idea with TCE table stub object a go.

Thanks


On 01/29/2015 08:27 PM, Alexey Kardashevskiy wrote:
> At the moment sPAPR PHB supports only a single 32bit window
> which is normally 1..2GB which is not enough for high performance devices.
> 
> PAPR spec enables creating an additional window(s) to support 64bit
> DMA and bigger page sizes.
> 
> This patchset adds DDW support for pseries. The host kernel changes are
> required, posted earlier today as:
> [PATCH v3 00/24] powerpc/iommu/vfio: Enable Dynamic DMA windows
> 
> This was tested on POWER8 system which allows one additional DMA window
> which is mapped at 0x800.0000.0000.0000 and supports 16MB pages.
> Existing guests check for DDW capabilities in PHB's device tree and if it
> is present, they request for an additional window and map entire guest RAM
> using H_PUT_TCE/... hypercalls once at boot time and switch to direct DMA
> operations.
> 
> TCE tables still may be big enough for guests backed with 64K pages but they
> are reasonably small for guests backed by 16MB pages.
> 
> 
> This does not contain PCI 64bit BAR support and VIO-TCE-bypass rework, these
> are required for this to work but they have been posted separately today.
> 
> 
> Please comment. Thanks!
> 
> Changes:
> v4:
> * (!) reimplemented the whole thing
> * machine reset and ddw-reset RTAS call both remove all TCE tables and
> create the default one
> * IOMMU group id is not needed to use VFIO PHB anymore, multiple groups
> are supported on the same VFIO container and virtual PHB
> 
> v3:
> * removed "reset" from API now
> * reworked machine versions
> * applied multiple comments
> * includes David's machine QOM rework as this patchset adds a new machine type
> 
> v2:
> * tested on emulated PHB
> * removed "ddw" machine property, now it is PHB property
> * disabled by default
> * defined "pseries-2.2" machine which enables DDW by default
> * fixed reset() and reference counting
> 
> 
> 
> 
> Alexey Kardashevskiy (18):
>   spapr_iommu: Disable in-kernel IOMMU tables for >4GB windows
>   spapr_iommu: Make H_PUT_TCE_INDIRECT endian-safe
>   spapr_pci: Introduce a liobn number generating macros
>   spapr_vio: Introduce a liobn number generating macros
>   spapr_pci: Make find_phb()/find_dev() public
>   spapr_iommu: Make spapr_tce_find_by_liobn() public
>   spapr_iommu: Implement free_table() helper
>   vfio: Add DMA memory registering
>   spapr_rtas: Reserve DDW RTAS token numbers
>   spapr_pci: Define DDW callbacks
>   spapr_pci/spapr_pci_vfio: Support Dynamic DMA Windows (DDW)
>   spapr_rtas: Add Dynamic DMA windows (DDW) RTAS handlers
>   spapr_pci: Advertise dynamic DMA windows to guest
>   vfio: Enable DDW ioctls to VFIO IOMMU driver
>   spapr_pci_vfio: Enable multiple groups per container
>   spapr_rtas_ddw: Workaround broken LE guests
>   target-ppc: kvm: make use of KVM_CREATE_SPAPR_TCE_64
>   vfio: Enable in-kernel acceleration via VFIO KVM device
> 
>  hw/ppc/Makefile.objs          |   3 +
>  hw/ppc/spapr.c                |   5 +
>  hw/ppc/spapr_iommu.c          |  23 ++-
>  hw/ppc/spapr_pci.c            | 209 ++++++++++++++++++++------
>  hw/ppc/spapr_pci_vfio.c       | 108 +++++++++-----
>  hw/ppc/spapr_rtas.c           |  29 ++++
>  hw/ppc/spapr_rtas_ddw.c       | 337 ++++++++++++++++++++++++++++++++++++++++++
>  hw/ppc/spapr_vio.c            |   2 +-
>  hw/vfio/common.c              | 167 ++++++++++++++++++---
>  include/hw/pci-host/spapr.h   |  38 ++++-
>  include/hw/ppc/spapr.h        |  15 +-
>  include/hw/vfio/vfio-common.h |   3 +-
>  include/hw/vfio/vfio.h        |   6 +-
>  target-ppc/kvm.c              |  48 ++++--
>  target-ppc/kvm_ppc.h          |  10 +-
>  trace-events                  |   4 +
>  16 files changed, 881 insertions(+), 126 deletions(-)
>  create mode 100644 hw/ppc/spapr_rtas_ddw.c
> 


-- 
Alexey