From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B7BB0C369DC for ; Tue, 29 Apr 2025 20:27:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:In-Reply-To:Content-Type: MIME-Version:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=th0orYzeEy2a+Bk04Qy69a62SaixG+OtsUq4sJ7ZV0s=; b=xaS6I7wGXnHozmISWNad6kE7cv WdGfDs7DC5Q5rCWIZFuuvrf6JtOgST56voGNR5rIrpUdmzWu95p8l0Gat9IWw7frdBDZoOidts7sx 83mLVgQQtNLRXOaHzZYneMPZLGM+g7GuaS+4ojHwCflqvW/83c4yYH0Vcy5GUqnUkdMOyHVoGJzyC UTQCglxulNbqSjHMu3wiB8m1g/zqp6Ls1Ib8OL+EYtYWd1Vh1Ztpw+nyuasPQEEQNJixBKOlcyTsj T0RT9bGlXrm9EZH6y2+8A5PFu14ERVaPOQdfkLgcMb44Z0RoqQqWL2un5fNuFWvcrk0+CfaMjalXB Wyhy1e7g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98.2 #2 (Red Hat Linux)) id 1u9rXS-0000000AkJk-0quu; Tue, 29 Apr 2025 20:26:46 +0000 Received: from mail-pl1-x62e.google.com ([2607:f8b0:4864:20::62e]) by bombadil.infradead.org with esmtps (Exim 4.98.2 #2 (Red Hat Linux)) id 1u9rVV-0000000Ajx4-25dL for linux-arm-kernel@lists.infradead.org; Tue, 29 Apr 2025 20:24:47 +0000 Received: by mail-pl1-x62e.google.com with SMTP id d9443c01a7336-2263428c8baso69115ad.1 for ; Tue, 29 Apr 2025 13:24:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1745958284; x=1746563084; darn=lists.infradead.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=th0orYzeEy2a+Bk04Qy69a62SaixG+OtsUq4sJ7ZV0s=; b=YyDhnOjJq6LGi/T7E33uWI09dHMIwBygp+J4DLb7WSKW5CHxXcgXG427fMRFI9Rb2N PchbpiyN+Utdr/MZM9h70XGAa9GtDKV8Tnd9nap6lhOIood3vJP2Js6hBzie2V5dn5Pp Or3nxiLyMUnSw2bN75De9I677W6tYoduWoZAsdoDJAvM0YsDsueDRNNSoW3IFGEF77PA 0XG/aaLf/VGvj26yBnlNhR+jLeT+dajv/hCQCbrhVqho/ISTTuEeCwIqfTgxKq2I1bFa guf9qHoqix9XuWusxZa4rMTUnA3pweW2swK7HvRXmH4piddmxZSUdXE7DL+41YpL6VFs MvFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1745958284; x=1746563084; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=th0orYzeEy2a+Bk04Qy69a62SaixG+OtsUq4sJ7ZV0s=; b=mPuMe+hOE7czIrqwiy/eUmkVU2cz0QD7mJxAhKv7MbV0OPaWCgya82+Lpw1ekYZPaw X6G1hAUpZDRqyktcwfUSryoVGEA3ZQTpBYvzPzIFYygaBUZfdBjyXNfhliaWxByAONxw W+JB93jyDSbwSIs6XDjJg7fNHTnfgnMYA6ADYGNp4Ice5YNNdPt/DHDvBP2XsLo5GFay j4sEdZskDLaMz/HVTE8yF1hV1IHE/CaRhYwBMAIPBKWHIN0kON4w3h3jpZFOVPiODao6 fuglNuh62RuBDO00wxsYK3h9VrpsBY56SySseFnx5J6yUIZJYoYfl5R/d1u6qhQvvHzT 1ZKA== X-Forwarded-Encrypted: i=1; AJvYcCUfgchY8R6y9aQ2SqlJUwLrvw0BdzgJ1nzynI0g1or7YcYb/M7QKW6fcU0TwQHiaYab2q8WkWgVpds6LKcMt2aF@lists.infradead.org X-Gm-Message-State: AOJu0YyUQT1Yo5uC2aoPAiLGknem+N6CzHCE3IOjrLeVrE7H0zkFzXkR z83BonO0WcXnyU+dQlwV85rAqVaL+ZOMmpcDXFknryFR5DGhQH7chet62Uy2cQ== X-Gm-Gg: ASbGnctOGcKGkbPTBMLPv7Yco0M+qc6U+caomEaN/3v6zT70fPwO8+yzR98Gsy9qeF/ TgtNSOPa+fQ6kipBUaMJIlgN1iAPmDLIZTtIfKG2VZvf2h2nf8f5E4SluPjkXJGcK1GAOiyo1e5 9UKQ5pS01GxEavf1uAN7IV017+ZFUkpdQXQDXCx/S1nZK7U6v3HoXInFJU0ujbUznzpUgryHVy9 nomsG5HGlla6kUyixiiYcPaG2Ert+14Lf1CMaQk8BovjXrJx19v3o2SBhXqRpqiZh2KZp89fXL8 0hoPrQJeYHM35hgqPVLn4TDfpMG+iW9fNvQyBPnBED4wIWydC1kbcFfedmlWH/vGepoJP+iq X-Google-Smtp-Source: AGHT+IFEmxdIWmGebHZgDe3BPjAbf4Gr2zvdjQF+sAsv0SidwYPT51jhJKxYdYkU6a4RRgjjfiLv+w== X-Received: by 2002:a17:902:e746:b0:215:aca2:dc04 with SMTP id d9443c01a7336-22df4075fa8mr719485ad.26.1745958284303; Tue, 29 Apr 2025 13:24:44 -0700 (PDT) Received: from google.com (2.210.143.34.bc.googleusercontent.com. [34.143.210.2]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-74039a5e09bsm100828b3a.131.2025.04.29.13.24.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 29 Apr 2025 13:24:43 -0700 (PDT) Date: Tue, 29 Apr 2025 20:24:33 +0000 From: Pranjal Shrivastava To: Nicolin Chen Cc: jgg@nvidia.com, kevin.tian@intel.com, corbet@lwn.net, will@kernel.org, bagasdotme@gmail.com, robin.murphy@arm.com, joro@8bytes.org, thierry.reding@gmail.com, vdumpa@nvidia.com, jonathanh@nvidia.com, shuah@kernel.org, jsnitsel@redhat.com, nathan@kernel.org, peterz@infradead.org, yi.l.liu@intel.com, mshavit@google.com, zhangzekun11@huawei.com, iommu@lists.linux.dev, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-tegra@vger.kernel.org, linux-kselftest@vger.kernel.org, patches@lists.linux.dev, mochs@nvidia.com, alok.a.tiwari@oracle.com, vasant.hegde@amd.com Subject: Re: [PATCH v2 13/22] iommufd: Add mmap interface Message-ID: References: <7be26560c604b0cbc2fd218997b97a47e4ed11ff.1745646960.git.nicolinc@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <7be26560c604b0cbc2fd218997b97a47e4ed11ff.1745646960.git.nicolinc@nvidia.com> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250429_132445_539305_0F5A79DB X-CRM114-Status: GOOD ( 34.16 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, Apr 25, 2025 at 10:58:08PM -0700, Nicolin Chen wrote: > For vIOMMU passing through HW resources to user space (VMs), add an mmap > infrastructure to map a region of hardware MMIO pages. > > Maintain an mt_mmap per ictx for validations. To allow IOMMU drivers to > add and delete mmappable regions to/from the mt_mmap, add a pair of new > helpers: iommufd_ctx_alloc_mmap() and iommufd_ctx_free_mmap(). > > Signed-off-by: Nicolin Chen > --- > drivers/iommu/iommufd/iommufd_private.h | 8 +++++ > include/linux/iommufd.h | 15 ++++++++++ > drivers/iommu/iommufd/driver.c | 39 +++++++++++++++++++++++++ > drivers/iommu/iommufd/main.c | 39 +++++++++++++++++++++++++ > 4 files changed, 101 insertions(+) > > diff --git a/drivers/iommu/iommufd/iommufd_private.h b/drivers/iommu/iommufd/iommufd_private.h > index b974c207ae8a..db5b62ec4abb 100644 > --- a/drivers/iommu/iommufd/iommufd_private.h > +++ b/drivers/iommu/iommufd/iommufd_private.h > @@ -7,6 +7,7 @@ > #include > #include > #include > +#include > #include > #include > #include > @@ -44,6 +45,7 @@ struct iommufd_ctx { > struct xarray groups; > wait_queue_head_t destroy_wait; > struct rw_semaphore ioas_creation_lock; > + struct maple_tree mt_mmap; > > struct mutex sw_msi_lock; > struct list_head sw_msi_list; > @@ -55,6 +57,12 @@ struct iommufd_ctx { > struct iommufd_ioas *vfio_ioas; > }; > > +/* Entry for iommufd_ctx::mt_mmap */ > +struct iommufd_mmap { > + unsigned long pfn_start; > + unsigned long pfn_end; > +}; > + > /* > * The IOVA to PFN map. The map automatically copies the PFNs into multiple > * domains and permits sharing of PFNs between io_pagetable instances. This > diff --git a/include/linux/iommufd.h b/include/linux/iommufd.h > index 5dff154e8ce1..d63e2d91be0d 100644 > --- a/include/linux/iommufd.h > +++ b/include/linux/iommufd.h > @@ -236,6 +236,9 @@ int iommufd_object_depend(struct iommufd_object *obj_dependent, > struct iommufd_object *obj_depended); > void iommufd_object_undepend(struct iommufd_object *obj_dependent, > struct iommufd_object *obj_depended); > +int iommufd_ctx_alloc_mmap(struct iommufd_ctx *ictx, phys_addr_t base, > + size_t size, unsigned long *immap_id); > +void iommufd_ctx_free_mmap(struct iommufd_ctx *ictx, unsigned long immap_id); > struct device *iommufd_viommu_find_dev(struct iommufd_viommu *viommu, > unsigned long vdev_id); > int iommufd_viommu_get_vdev_id(struct iommufd_viommu *viommu, > @@ -262,11 +265,23 @@ static inline int iommufd_object_depend(struct iommufd_object *obj_dependent, > return -EOPNOTSUPP; > } > > +static inline int iommufd_ctx_alloc_mmap(struct iommufd_ctx *ictx, > + phys_addr_t base, size_t size, > + unsigned long *immap_id) > +{ > + return -EOPNOTSUPP; > +} > + > static inline void iommufd_object_undepend(struct iommufd_object *obj_dependent, > struct iommufd_object *obj_depended) > { > } > > +static inline void iommufd_ctx_free_mmap(struct iommufd_ctx *ictx, > + unsigned long immap_id) > +{ > +} > + > static inline struct device * > iommufd_viommu_find_dev(struct iommufd_viommu *viommu, unsigned long vdev_id) > { > diff --git a/drivers/iommu/iommufd/driver.c b/drivers/iommu/iommufd/driver.c > index fb7f8fe40f95..c55336c580dc 100644 > --- a/drivers/iommu/iommufd/driver.c > +++ b/drivers/iommu/iommufd/driver.c > @@ -78,6 +78,45 @@ void iommufd_object_undepend(struct iommufd_object *obj_dependent, > } > EXPORT_SYMBOL_NS_GPL(iommufd_object_undepend, "IOMMUFD"); > > +/* Driver should report the output @immap_id to user space for mmap() syscall */ > +int iommufd_ctx_alloc_mmap(struct iommufd_ctx *ictx, phys_addr_t base, > + size_t size, unsigned long *immap_id) > +{ > + struct iommufd_mmap *immap; > + int rc; > + > + if (WARN_ON_ONCE(!immap_id)) > + return -EINVAL; > + if (base & ~PAGE_MASK) > + return -EINVAL; > + if (!size || size & ~PAGE_MASK) > + return -EINVAL; > + > + immap = kzalloc(sizeof(*immap), GFP_KERNEL); > + if (!immap) > + return -ENOMEM; > + immap->pfn_start = base >> PAGE_SHIFT; > + immap->pfn_end = immap->pfn_start + (size >> PAGE_SHIFT) - 1; > + > + rc = mtree_alloc_range(&ictx->mt_mmap, immap_id, immap, sizeof(immap), I believe this should be sizeof(*immap) ? > + 0, LONG_MAX >> PAGE_SHIFT, GFP_KERNEL); > + if (rc < 0) { > + kfree(immap); > + return rc; > + } > + > + /* mmap() syscall will right-shift the immap_id to vma->vm_pgoff */ > + *immap_id <<= PAGE_SHIFT; > + return 0; > +} > +EXPORT_SYMBOL_NS_GPL(iommufd_ctx_alloc_mmap, "IOMMUFD"); > + > +void iommufd_ctx_free_mmap(struct iommufd_ctx *ictx, unsigned long immap_id) > +{ > + kfree(mtree_erase(&ictx->mt_mmap, immap_id >> PAGE_SHIFT)); > +} > +EXPORT_SYMBOL_NS_GPL(iommufd_ctx_free_mmap, "IOMMUFD"); > + > /* Caller should xa_lock(&viommu->vdevs) to protect the return value */ > struct device *iommufd_viommu_find_dev(struct iommufd_viommu *viommu, > unsigned long vdev_id) > diff --git a/drivers/iommu/iommufd/main.c b/drivers/iommu/iommufd/main.c > index ac51d5cfaa61..4b46ea47164d 100644 > --- a/drivers/iommu/iommufd/main.c > +++ b/drivers/iommu/iommufd/main.c > @@ -213,6 +213,7 @@ static int iommufd_fops_open(struct inode *inode, struct file *filp) > xa_init_flags(&ictx->objects, XA_FLAGS_ALLOC1 | XA_FLAGS_ACCOUNT); > xa_init(&ictx->groups); > ictx->file = filp; > + mt_init_flags(&ictx->mt_mmap, MT_FLAGS_ALLOC_RANGE); > init_waitqueue_head(&ictx->destroy_wait); > mutex_init(&ictx->sw_msi_lock); > INIT_LIST_HEAD(&ictx->sw_msi_list); > @@ -410,11 +411,49 @@ static long iommufd_fops_ioctl(struct file *filp, unsigned int cmd, > return ret; > } > > +/* > + * Kernel driver must first do iommufd_ctx_alloc_mmap() to register an mmappable > + * MMIO region to the iommufd core to receive an "immap_id". Then, driver should > + * report to user space this immap_id and the size of the registered MMIO region > + * for @vm_pgoff and @size of an mmap() call, via an IOMMU_VIOMMU_ALLOC ioctl in > + * the output fields of its driver-type data structure. > + * > + * Note the @size is allowed to be smaller than the registered size as a partial > + * mmap starting from the registered base address. > + */ > +static int iommufd_fops_mmap(struct file *filp, struct vm_area_struct *vma) > +{ > + struct iommufd_ctx *ictx = filp->private_data; > + size_t size = vma->vm_end - vma->vm_start; > + struct iommufd_mmap *immap; > + > + if (size & ~PAGE_MASK) > + return -EINVAL; > + if (!(vma->vm_flags & VM_SHARED)) > + return -EINVAL; > + if (vma->vm_flags & VM_EXEC) > + return -EPERM; > + > + /* vm_pgoff carries an index (immap_id) to an mtree entry (immap) */ > + immap = mtree_load(&ictx->mt_mmap, vma->vm_pgoff); > + if (!immap) > + return -ENXIO; > + if (size >> PAGE_SHIFT > immap->pfn_end - immap->pfn_start + 1) > + return -ENXIO; > + > + vma->vm_pgoff = 0; > + vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot); > + vm_flags_set(vma, VM_PFNMAP | VM_DONTEXPAND | VM_DONTDUMP | VM_IO); > + return remap_pfn_range(vma, vma->vm_start, immap->pfn_start, size, > + vma->vm_page_prot); > +} > + > static const struct file_operations iommufd_fops = { > .owner = THIS_MODULE, > .open = iommufd_fops_open, > .release = iommufd_fops_release, > .unlocked_ioctl = iommufd_fops_ioctl, > + .mmap = iommufd_fops_mmap, > }; > > /** Thanks, Praan > -- > 2.43.0 >