From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47216) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YpZBo-0004zK-5T for qemu-devel@nongnu.org; Tue, 05 May 2015 05:31:00 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1YpZBW-0006Pl-1N for qemu-devel@nongnu.org; Tue, 05 May 2015 05:30:56 -0400 Received: from mail-pd0-f179.google.com ([209.85.192.179]:35520) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1YpZBV-0006PH-Af for qemu-devel@nongnu.org; Tue, 05 May 2015 05:30:37 -0400 Received: by pdbqd1 with SMTP id qd1so190120214pdb.2 for ; Tue, 05 May 2015 02:30:35 -0700 (PDT) Message-ID: <55488DB4.4030305@ozlabs.ru> Date: Tue, 05 May 2015 19:30:28 +1000 From: Alexey Kardashevskiy MIME-Version: 1.0 References: <1429964684-23872-1-git-send-email-aik@ozlabs.ru> In-Reply-To: <1429964684-23872-1-git-send-email-aik@ozlabs.ru> Content-Type: text/plain; charset=koi8-r; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] [PATCH qemu v7 00/14] spapr: vfio: Enable Dynamic DMA windows (DDW) List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org Cc: Alex Williamson , qemu-ppc@nongnu.org, Alexander Graf , David Gibson On 04/25/2015 10:24 PM, Alexey Kardashevskiy wrote: > (cut-n-paste from kernel patchset) Anyone, ping? :) > Each Partitionable Endpoint (IOMMU group) has an address range on a PCI bus > where devices are allowed to do DMA. These ranges are called DMA windows. > By default, there is a single DMA window, 1 or 2GB big, mapped at zero > on a PCI bus. > > PAPR defines a DDW RTAS API which allows pseries guests > querying the hypervisor about DDW support and capabilities (page size mask > for now). A pseries guest may request an additional (to the default) > DMA windows using this RTAS API. > The existing pseries Linux guests request an additional window as big as > the guest RAM and map the entire guest window which effectively creates > direct mapping of the guest memory to a PCI bus. > > This patchset reworks PPC64 IOMMU code and adds necessary structures > to support big windows. > > Once a Linux guest discovers the presence of DDW, it does: > 1. query hypervisor about number of available windows and page size masks; > 2. create a window with the biggest possible page size (today 4K/64K/16M); > 3. map the entire guest RAM via H_PUT_TCE* hypercalls; > 4. switche dma_ops to direct_dma_ops on the selected PE. > > Once this is done, H_PUT_TCE is not called anymore for 64bit devices and > the guest does not waste time on DMA map/unmap operations. > > Note that 32bit devices won't use DDW and will keep using the default > DMA window so KVM optimizations will be required (to be posted later). > > This patchset adds DDW support for pseries. The host kernel changes are > required, posted as: > > [PATCH kernel v9 00/32] powerpc/iommu/vfio: Enable Dynamic DMA windows > > This patchset is based on git://github.com/dgibson/qemu.git spapr-next branch. > This is also pushed to git@github.com:aik/qemu.git > + a64ff6f...64ac9a4 64ac9a4 -> vfio-for-github (forced update) > > Please comment. Thanks! > > Changes: > v7: > * bunch of cleanups, renames after David+Thomas+Michael review > * patches are reorganized and those which do not need the host kernel headers > update are put first and can be pulled if these are good enough :) > > v6: > * spapr-pci-vfio-host-bridge is now a synonim of spapr-pci-host-bridge - > same PHB can host emulated and VFIO devices > * changed patches order > * lot of small changes > > v5: > * TCE tables got "enabled" state and are persistent, i.e. not recreated > every reboot > * added v2 of SPAPR_TCE_IOMMU > * fixed migration for emulated PHB with enabled DDW > * huge pile of other changes > > v4: > * reimplemented the whole thing > * machine reset and ddw-reset RTAS call both remove all TCE tables and > create the default one > * IOMMU group id is not needed to use VFIO PHB anymore, multiple groups > are supported on the same VFIO container and virtual PHB > > v3: > * removed "reset" from API now > * reworked machine versions > * applied multiple comments > * includes David's machine QOM rework as this patchset adds a new machine type > > v2: > * tested on emulated PHB > * removed "ddw" machine property, now it is PHB property > * disabled by default > * defined "pseries-2.2" machine which enables DDW by default > * fixed reset() and reference counting > > > > > Alexey Kardashevskiy (14): > spapr_pci: Finish making find_phb()/find_dev() public > vmstate: Define VARRAY with VMS_ALLOC > vfio: spapr: Move SPAPR-related code to a separate file > spapr_pci_vfio: Enable multiple groups per container > spapr_pci: Convert finish_realize() to > dma_capabilities_update()+dma_init_window() > spapr_iommu: Introduce "enabled" state for TCE table > spapr_iommu: Add root memory region > spapr_pci: Do complete reset of DMA config when resetting PHB > spapr_vfio_pci: Remove redundant spapr-pci-vfio-host-bridge > linux headers update for DDW on SPAPR > vfio: spapr: Add SPAPR IOMMU v2 support (DMA memory preregistering) > spapr: Add pseries-2.4 machine > spapr_pci/spapr_pci_vfio: Support Dynamic DMA Windows (DDW) > vfio: Enable DDW ioctls to VFIO IOMMU driver > > hw/ppc/Makefile.objs | 3 + > hw/ppc/spapr.c | 32 ++++- > hw/ppc/spapr_iommu.c | 144 +++++++++++++------ > hw/ppc/spapr_pci.c | 208 ++++++++++++++++++---------- > hw/ppc/spapr_pci_vfio.c | 147 ++++++++++++-------- > hw/ppc/spapr_rtas_ddw.c | 300 ++++++++++++++++++++++++++++++++++++++++ > hw/ppc/spapr_vio.c | 9 +- > hw/vfio/Makefile.objs | 1 + > hw/vfio/common.c | 180 +++++------------------- > hw/vfio/spapr.c | 312 ++++++++++++++++++++++++++++++++++++++++++ > include/hw/pci-host/spapr.h | 49 +++++-- > include/hw/ppc/spapr.h | 30 +++- > include/hw/vfio/vfio-common.h | 16 +++ > include/hw/vfio/vfio.h | 2 +- > include/migration/vmstate.h | 10 ++ > linux-headers/linux/vfio.h | 88 +++++++++++- > trace-events | 5 + > 17 files changed, 1188 insertions(+), 348 deletions(-) > create mode 100644 hw/ppc/spapr_rtas_ddw.c > create mode 100644 hw/vfio/spapr.c > -- Alexey