From mboxrd@z Thu Jan 1 00:00:00 1970 From: Alexey Kardashevskiy Date: Tue, 18 Feb 2020 07:36:45 +0000 Subject: [PATCH kernel 0/5] powerpc/powenv/ioda: Allow huge DMA window at 4GB Message-Id: <20200218073650.16149-1-aik@ozlabs.ru> List-Id: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: linuxppc-dev@lists.ozlabs.org Cc: Alexey Kardashevskiy , Alistair Popple , Alex Williamson , kvm-ppc@vger.kernel.org, David Gibson Here is an attempt to support bigger DMA space for devices supporting DMA masks less than 59 bits (GPUs come into mind first). POWER9 PHBs have an option to map 2 windows at 0 and select a windows based on DMA address being below or above 4GB. This adds the "iommu=iommu_bypass" kernel parameter and supports VFIO+pseries machine - current this requires telling upstream+unmodified QEMU about this via -global spapr-pci-host-bridge.dma64_win_addr=0x100000000 or per-phb property. 4/4 advertises the new option but there is no automation around it in QEMU (should it be?). For now it is either 1<<59 or 4GB mode; dynamic switching is not supported (could be via sysfs). This is a rebased version of https://lore.kernel.org/kvm/20191202015953.127902-1-aik@ozlabs.ru/ This is based on sha1 71c3a888cbca Linus Torvalds "Merge tag 'powerpc-5.6-1' of git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux". Please comment. Thanks. Alexey Kardashevskiy (5): powerpc/powernv/ioda: Move TCE bypass base to PE powerpc/powernv/ioda: Rework for huge DMA window at 4GB powerpc/powernv/ioda: Allow smaller TCE table levels powerpc/powernv/phb4: Add 4GB IOMMU bypass mode vfio/spapr_tce: Advertise and allow a huge DMA windows at 4GB arch/powerpc/include/asm/iommu.h | 2 + arch/powerpc/include/asm/opal-api.h | 9 +- arch/powerpc/include/asm/opal.h | 2 + arch/powerpc/platforms/powernv/pci.h | 2 +- include/uapi/linux/vfio.h | 2 + arch/powerpc/platforms/powernv/npu-dma.c | 1 + arch/powerpc/platforms/powernv/opal-call.c | 2 + arch/powerpc/platforms/powernv/pci-ioda-tce.c | 4 +- arch/powerpc/platforms/powernv/pci-ioda.c | 229 ++++++++++++++---- drivers/vfio/vfio_iommu_spapr_tce.c | 10 +- 10 files changed, 207 insertions(+), 56 deletions(-) -- 2.17.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.8 required=3.0 tests=HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E150C34031 for ; Tue, 18 Feb 2020 07:41:05 +0000 (UTC) Received: from lists.ozlabs.org (lists.ozlabs.org [203.11.71.2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 15B4821D7D for ; Tue, 18 Feb 2020 07:41:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 15B4821D7D Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=ozlabs.ru Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 48MCTC1vVMzDqY0 for ; Tue, 18 Feb 2020 18:41:03 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=ozlabs.ru (client-ip=107.174.27.60; helo=ozlabs.ru; envelope-from=aik@ozlabs.ru; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=ozlabs.ru Received: from ozlabs.ru (unknown [107.174.27.60]) by lists.ozlabs.org (Postfix) with ESMTP id 48MCP12sXczDqXw for ; Tue, 18 Feb 2020 18:37:25 +1100 (AEDT) Received: from fstn1-p1.ozlabs.ibm.com (localhost [IPv6:::1]) by ozlabs.ru (Postfix) with ESMTP id E2ECBAE80014; Tue, 18 Feb 2020 02:35:20 -0500 (EST) From: Alexey Kardashevskiy To: linuxppc-dev@lists.ozlabs.org Subject: [PATCH kernel 0/5] powerpc/powenv/ioda: Allow huge DMA window at 4GB Date: Tue, 18 Feb 2020 18:36:45 +1100 Message-Id: <20200218073650.16149-1-aik@ozlabs.ru> X-Mailer: git-send-email 2.17.1 X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexey Kardashevskiy , Alistair Popple , Alex Williamson , kvm-ppc@vger.kernel.org, David Gibson Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" Here is an attempt to support bigger DMA space for devices supporting DMA masks less than 59 bits (GPUs come into mind first). POWER9 PHBs have an option to map 2 windows at 0 and select a windows based on DMA address being below or above 4GB. This adds the "iommu=iommu_bypass" kernel parameter and supports VFIO+pseries machine - current this requires telling upstream+unmodified QEMU about this via -global spapr-pci-host-bridge.dma64_win_addr=0x100000000 or per-phb property. 4/4 advertises the new option but there is no automation around it in QEMU (should it be?). For now it is either 1<<59 or 4GB mode; dynamic switching is not supported (could be via sysfs). This is a rebased version of https://lore.kernel.org/kvm/20191202015953.127902-1-aik@ozlabs.ru/ This is based on sha1 71c3a888cbca Linus Torvalds "Merge tag 'powerpc-5.6-1' of git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux". Please comment. Thanks. Alexey Kardashevskiy (5): powerpc/powernv/ioda: Move TCE bypass base to PE powerpc/powernv/ioda: Rework for huge DMA window at 4GB powerpc/powernv/ioda: Allow smaller TCE table levels powerpc/powernv/phb4: Add 4GB IOMMU bypass mode vfio/spapr_tce: Advertise and allow a huge DMA windows at 4GB arch/powerpc/include/asm/iommu.h | 2 + arch/powerpc/include/asm/opal-api.h | 9 +- arch/powerpc/include/asm/opal.h | 2 + arch/powerpc/platforms/powernv/pci.h | 2 +- include/uapi/linux/vfio.h | 2 + arch/powerpc/platforms/powernv/npu-dma.c | 1 + arch/powerpc/platforms/powernv/opal-call.c | 2 + arch/powerpc/platforms/powernv/pci-ioda-tce.c | 4 +- arch/powerpc/platforms/powernv/pci-ioda.c | 229 ++++++++++++++---- drivers/vfio/vfio_iommu_spapr_tce.c | 10 +- 10 files changed, 207 insertions(+), 56 deletions(-) -- 2.17.1