From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3F5C5C28CAE for ; Thu, 16 Sep 2021 23:41:25 +0000 (UTC) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1C0D86128B for ; Thu, 16 Sep 2021 23:41:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 1C0D86128B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=deltatee.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 7C34683FC3; Thu, 16 Sep 2021 23:41:23 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KFWpVLmofQus; Thu, 16 Sep 2021 23:41:22 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp1.osuosl.org (Postfix) with ESMTPS id 0138383FBD; Thu, 16 Sep 2021 23:41:21 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id C9250C0026; Thu, 16 Sep 2021 23:41:21 +0000 (UTC) Received: from smtp4.osuosl.org (smtp4.osuosl.org [IPv6:2605:bc80:3010::137]) by lists.linuxfoundation.org (Postfix) with ESMTP id 26619C0021 for ; Thu, 16 Sep 2021 23:41:19 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id B6ED94159E for ; Thu, 16 Sep 2021 23:41:17 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp4.osuosl.org (amavisd-new); dkim=pass (2048-bit key) header.d=deltatee.com Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Mt69or9nsews for ; Thu, 16 Sep 2021 23:41:16 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from ale.deltatee.com (ale.deltatee.com [204.191.154.188]) by smtp4.osuosl.org (Postfix) with ESMTPS id C9DB2414CB for ; Thu, 16 Sep 2021 23:41:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=deltatee.com; s=20200525; h=Subject:MIME-Version:Message-Id:Date:Cc:To:From :references:content-disposition:in-reply-to; bh=2hK6AsHc/ysDKmL7f6dXUZSSeE0kBcwrFrtpZ34IiEA=; b=dv9GSoU5q89O6QAjG6UUhsk3GS +j/d9pDqmJ/2DTqV135u2egzDU+xfME9bSFt0gzd/BGOfamNWms6HmQznPJrqeDeQkwpLP0E2lxY/ q+z7pSlUuxGCM5Os4UOSwxQR918zGb3Ztoh71qOF8KH70g93rPysvHpD0pONYPjpFBEC7iyolydT7 AGMIsUK+1DL/9E5LAhIWj/PESbA3EtiPPUXe4b9zkfs8xM/0rmjpvYOvU6UGLijfJZ2BeiK4SCn5e Fz0SFsYdzYgf5R6I/uMXAb59wVgnOsuKJogy5Em5DV33/17yyoPYbh6HaZGTxefqWcNRE08KJMGrd sBI7uRAg==; Received: from cgy1-donard.priv.deltatee.com ([172.16.1.31]) by ale.deltatee.com with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1mR107-0008Hz-JR; Thu, 16 Sep 2021 17:41:09 -0600 Received: from gunthorp by cgy1-donard.priv.deltatee.com with local (Exim 4.94.2) (envelope-from ) id 1mR103-000Vqn-Ee; Thu, 16 Sep 2021 17:41:03 -0600 From: Logan Gunthorpe To: linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, linux-pci@vger.kernel.org, linux-mm@kvack.org, iommu@lists.linux-foundation.org Date: Thu, 16 Sep 2021 17:40:40 -0600 Message-Id: <20210916234100.122368-1-logang@deltatee.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 X-SA-Exim-Connect-IP: 172.16.1.31 X-SA-Exim-Rcpt-To: linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, linux-block@vger.kernel.org, linux-pci@vger.kernel.org, linux-mm@kvack.org, iommu@lists.linux-foundation.org, sbates@raithlin.com, hch@lst.de, jgg@ziepe.ca, christian.koenig@amd.com, jhubbard@nvidia.com, ddutile@redhat.com, willy@infradead.org, daniel.vetter@ffwll.ch, jason@jlekstrand.net, dave.hansen@linux.intel.com, helgaas@kernel.org, dan.j.williams@intel.com, andrzej.jakowski@intel.com, dave.b.minturn@intel.com, jianxin.xiong@intel.com, ira.weiny@intel.com, robin.murphy@arm.com, martin.oliveira@eideticom.com, ckulkarnilinux@gmail.com, logang@deltatee.com X-SA-Exim-Mail-From: gunthorp@deltatee.com Subject: [PATCH v3 00/20] Userspace P2PDMA with O_DIRECT NVMe devices X-SA-Exim-Version: 4.2.1 (built Wed, 08 May 2019 21:11:16 +0000) X-SA-Exim-Scanned: Yes (on ale.deltatee.com) Cc: Minturn Dave B , Martin Oliveira , Ira Weiny , John Hubbard , Dave Hansen , Robin Murphy , Matthew Wilcox , =?UTF-8?q?Christian=20K=C3=B6nig?= , Jason Gunthorpe , Logan Gunthorpe , Chaitanya Kulkarni , Jason Ekstrand , Daniel Vetter , Bjorn Helgaas , Dan Williams , Stephen Bates , Jakowski Andrzej , Christoph Hellwig , Xiong Jianxin X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" Hi, This patchset continues my work to add userspace P2PDMA access using O_DIRECT NVMe devices. My last posting[1] just included the first 13 patches in this series, but the early P2PDMA cleanup and map_sg error changes from that series have been merged into v5.15-rc1. To address concerns that that series did not add any new functionality, I've added back the userspcae functionality from the original RFC[2] (but improved based on the original feedback). The patchset enables userspace P2PDMA by allowing userspace to mmap() allocated chunks of the CMB. The resulting VMA can be passed only to O_DIRECT IO on NVMe backed files or block devices. A flag is added to GUP() in Patch 14, then Patches 15 through 17 wire this flag up based on whether the block queue indicates P2PDMA support. Patches 18 through 20 enable the CMB to be mapped into userspace by mmaping the nvme char device. This is relatively straightforward, however the one significant problem is that, presently, pci_p2pdma_map_sg() requires a homogeneous SGL with all P2PDMA pages or all regular pages. Enhancing GUP to support enforcing this rule would require a huge hack that I don't expect would be all that pallatable. So the first 13 patches add support for P2PDMA pages to dma_map_sg[table]() to the dma-direct and dma-iommu implementations. Thus systems without an IOMMU plus Intel and AMD IOMMUs are supported. (Other IOMMU implementations would then be unsupported, notably ARM and PowerPC but support would be added when they convert to dma-iommu). dma_map_sgtable() is preferred when dealing with P2PDMA memory as it will return -EREMOTEIO when the DMA device cannot map specific P2PDMA pages based on the existing rules in calc_map_type_and_dist(). The other issue is dma_unmap_sg() needs a flag to determine whether a given dma_addr_t was mapped regularly or as a PCI bus address. To allow this, a third flag is added to the page_link field in struct scatterlist. This effectively means support for P2PDMA will now depend on CONFIG_64BIT. Feedback welcome. This series is based on v5.15-rc1. A git branch is available here: https://github.com/sbates130272/linux-p2pmem/ p2pdma_user_cmb_v3 Thanks, Logan [1] https://lkml.kernel.org/r/20210513223203.5542-1-logang@deltatee.com [2] https://lkml.kernel.org/r/20201106170036.18713-1-logang@deltatee.com -- Logan Gunthorpe (20): lib/scatterlist: add flag for indicating P2PDMA segments in an SGL PCI/P2PDMA: attempt to set map_type if it has not been set PCI/P2PDMA: make pci_p2pdma_map_type() non-static PCI/P2PDMA: introduce helpers for dma_map_sg implementations dma-mapping: allow EREMOTEIO return code for P2PDMA transfers dma-direct: support PCI P2PDMA pages in dma-direct map_sg dma-mapping: add flags to dma_map_ops to indicate PCI P2PDMA support iommu/dma: support PCI P2PDMA pages in dma-iommu map_sg nvme-pci: check DMA ops when indicating support for PCI P2PDMA nvme-pci: convert to using dma_map_sgtable() RDMA/core: introduce ib_dma_pci_p2p_dma_supported() RDMA/rw: use dma_map_sgtable() PCI/P2PDMA: remove pci_p2pdma_[un]map_sg() mm: introduce FOLL_PCI_P2PDMA to gate getting PCI P2PDMA pages iov_iter: introduce iov_iter_get_pages_[alloc_]flags() block: set FOLL_PCI_P2PDMA in __bio_iov_iter_get_pages() block: set FOLL_PCI_P2PDMA in bio_map_user_iov() mm: use custom page_free for P2PDMA pages PCI/P2PDMA: introduce pci_mmap_p2pmem() nvme-pci: allow mmaping the CMB in userspace block/bio.c | 8 +- block/blk-map.c | 7 +- drivers/dax/super.c | 7 +- drivers/infiniband/core/rw.c | 75 +++---- drivers/iommu/dma-iommu.c | 68 +++++- drivers/nvme/host/core.c | 18 +- drivers/nvme/host/nvme.h | 4 +- drivers/nvme/host/pci.c | 98 +++++---- drivers/nvme/target/rdma.c | 2 +- drivers/pci/Kconfig | 3 +- drivers/pci/p2pdma.c | 402 +++++++++++++++++++++++++++++------ include/linux/dma-map-ops.h | 10 + include/linux/dma-mapping.h | 5 + include/linux/memremap.h | 4 +- include/linux/mm.h | 2 + include/linux/pci-p2pdma.h | 92 ++++++-- include/linux/scatterlist.h | 50 ++++- include/linux/uio.h | 21 +- include/rdma/ib_verbs.h | 30 +++ include/uapi/linux/magic.h | 1 + kernel/dma/direct.c | 44 +++- kernel/dma/mapping.c | 34 ++- lib/iov_iter.c | 28 +-- mm/gup.c | 28 ++- mm/huge_memory.c | 8 +- mm/memory-failure.c | 4 +- mm/memory_hotplug.c | 2 +- mm/memremap.c | 26 ++- 28 files changed, 834 insertions(+), 247 deletions(-) base-commit: 6880fa6c56601bb8ed59df6c30fd390cc5f6dd8f -- 2.30.2 _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu