From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B5E1AC4345F for ; Wed, 17 Apr 2024 09:45:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:In-Reply-To:References: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=FMlzctufJkOC2+90gimd4bxJh8spuKVW68QDEgh1Lmg=; b=Kgx6YE6uPpLfbq OdC75vATWffC+bnMiE1VSB/poKp6LNbPMBPXFRbIeKpHCpskz7rUyaWS/gGXNbjI/jwo919Opw2PR wiawICKYds15EtK2GmWoUxCiwBNudvpON/ZRxOQIjwH1ObTjt4AiqIbBZYZ7AD4eTq1c/fL+HhdMR RfVhYcm4XfU9k0/3VeAQMMpYDqpDVpuSenMrjxG6Vc6IahJS0xdU74PNt8E6dNayuzjGJVFl1wSYT cUSOO0miMOu/O140FocLr4YD/+Hs/0tzsPTVrMeus7myeW1iWi83G5uY4k1BebqWmkObgaZrd+zZt /7nPrzzWz8LWq7oh0XJA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rx1rR-0000000FTDz-18uK; Wed, 17 Apr 2024 09:45:49 +0000 Received: from frasgout.his.huawei.com ([185.176.79.56]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rx1rN-0000000FTCI-3QaH for linux-arm-kernel@lists.infradead.org; Wed, 17 Apr 2024 09:45:48 +0000 Received: from mail.maildlp.com (unknown [172.18.186.231]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4VKGF43Vrnz6F8c6; Wed, 17 Apr 2024 17:43:36 +0800 (CST) Received: from lhrpeml100001.china.huawei.com (unknown [7.191.160.183]) by mail.maildlp.com (Postfix) with ESMTPS id F2387140B63; Wed, 17 Apr 2024 17:45:35 +0800 (CST) Received: from lhrpeml500005.china.huawei.com (7.191.163.240) by lhrpeml100001.china.huawei.com (7.191.160.183) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Wed, 17 Apr 2024 10:45:34 +0100 Received: from lhrpeml500005.china.huawei.com ([7.191.163.240]) by lhrpeml500005.china.huawei.com ([7.191.163.240]) with mapi id 15.01.2507.035; Wed, 17 Apr 2024 10:45:34 +0100 From: Shameerali Kolothum Thodi To: Jason Gunthorpe , Nicolin Chen CC: "will@kernel.org" , "robin.murphy@arm.com" , "joro@8bytes.org" , "thierry.reding@gmail.com" , "vdumpa@nvidia.com" , "jonathanh@nvidia.com" , "linux-kernel@vger.kernel.org" , "iommu@lists.linux.dev" , "linux-arm-kernel@lists.infradead.org" , "linux-tegra@vger.kernel.org" , Jerry Snitselaar Subject: RE: [PATCH v5 0/6] Add Tegra241 (Grace) CMDQV Support (part 1/2) Thread-Topic: [PATCH v5 0/6] Add Tegra241 (Grace) CMDQV Support (part 1/2) Thread-Index: AQHajVTpda5PtrGZ80+YonTPgO6cRrFphU4AgAKXr6CAAB7BMA== Date: Wed, 17 Apr 2024 09:45:34 +0000 Message-ID: References: <20240415171426.GF3637727@nvidia.com> In-Reply-To: Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.202.227.28] MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240417_024546_162123_EED2D04D X-CRM114-Status: GOOD ( 30.78 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org > -----Original Message----- > From: Shameerali Kolothum Thodi > Sent: Wednesday, April 17, 2024 9:01 AM > To: 'Jason Gunthorpe' ; Nicolin Chen > Cc: will@kernel.org; robin.murphy@arm.com; joro@8bytes.org; > thierry.reding@gmail.com; vdumpa@nvidia.com; jonathanh@nvidia.com; linux- > kernel@vger.kernel.org; iommu@lists.linux.dev; linux-arm- > kernel@lists.infradead.org; linux-tegra@vger.kernel.org; Jerry Snitselaar > > Subject: RE: [PATCH v5 0/6] Add Tegra241 (Grace) CMDQV Support (part 1/2) > > > > > -----Original Message----- > > From: Jason Gunthorpe > > Sent: Monday, April 15, 2024 6:14 PM > > To: Nicolin Chen > > Cc: will@kernel.org; robin.murphy@arm.com; joro@8bytes.org; > > thierry.reding@gmail.com; vdumpa@nvidia.com; jonathanh@nvidia.com; > > linux-kernel@vger.kernel.org; iommu@lists.linux.dev; linux-arm- > > kernel@lists.infradead.org; linux-tegra@vger.kernel.org; Jerry Snitselaar > > > > Subject: Re: [PATCH v5 0/6] Add Tegra241 (Grace) CMDQV Support (part 1/2) > > > > On Fri, Apr 12, 2024 at 08:43:48PM -0700, Nicolin Chen wrote: > > > > > The user-space support is to provide uAPIs (via IOMMUFD) for hypervisors > > > in user space to passthrough VCMDQs to VMs, allowing these VMs to > > access > > > the VCMDQs directly without trappings, i.e. no VM Exits. This gives huge > > > performance improvements: 70% to 90% reductions of TLB invalidation > > time > > > were measured by various DMA unmap tests running in a guest OS, > > compared > > > to a nested SMMU CMDQ (with trappings). > > > > So everyone is on the same page, this is the primary point of this > > series. The huge speed up of in-VM performance is necessary for the > > workloads this chip is expected to be running. This series is unique > > from all the rest because it runs inside a VM, often in the from of a > > distro release. > > > > It doesn't need the other series or it's own part 2 as it entirely > > stands alone on bare metal hardware or on top of commercial VM cloud > > instances runing who-knows-what in their hypervisors. > > > > The other parts are substantially about enabling qemu and the open > > ecosystem to have fully functional vSMMU3 virtualization. > > Hi, > > We do have plans to revive the SMMUv3 ECMDQ series posted a while back[0] > and looking at this series, I am just wondering whether it makes sense to have > a similar one with ECMDQ as well? I see that the NVIDIA VCMDQ has a special > bit > to restrict the commands that can be issued from user space. If we end up > assigning > a ECMDQ to user space, is there any potential risk in doing so? > > SMMUV3 spec does say, > "Arm expects that the Non-secure Stream table, Command queue, Event queue > and > PRI queue are controlled by the most privileged Non-secure system software. " > > Not clear to me what are the major concerns here and maybe we can come up > with > something to address that in kernel. Just to add to that. One idea could be like to have a case where when ECMDQs are detected, use that for issuing limited set of cmds(like stage 1 TLBIs) and use the normal cmdq for rest. Since we use stage 1 for both host and for Guest nested cases and TLBIs are the bottlenecks in most cases I think this should give performance benefits. Thanks, Shameer _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel