From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 80E58C48260 for ; Thu, 8 Feb 2024 16:14:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:In-Reply-To:References: Message-ID:Date:Subject:CC:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=2d1/LOw51QOpBfXIYqmTSJ2/rzQGN++A5RstRC4Eil0=; b=P1FFjWefgpobmZ 0z1cm18thLBUpI9z22igpELDhGlNf55dXIosAoeUPAu9IsHoYzdb3a10CppVBB4t70zQeDWgDkiRt eEHnV4senVEroiJOpT0a6QpwLlR3dXQUFBtJltOL73Jfnzu/Hj1yiq5nU8KQKEkkICTmpmheZDp3L 9iqplpLiI8oJEkSJFjlmcq7fsJhyxe3UR2yUEdwmTiGpBw0AF4e+SV9BbUVoPY2Q+6OLt8PRz5upM Q/NJCvQ7nwsXnCtd5vbOh4ro/bKIu0+wZtfnORqw+HO1CrOir0Jlc6PwNI8vgRiiCtHb8JyeIcDkr 8Lw4SJ2ge0hubvvtGkUg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rY72b-0000000EHOQ-2gLA; Thu, 08 Feb 2024 16:14:21 +0000 Received: from frasgout.his.huawei.com ([185.176.79.56]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rY72X-0000000EHNQ-1s25 for linux-arm-kernel@lists.infradead.org; Thu, 08 Feb 2024 16:14:19 +0000 Received: from mail.maildlp.com (unknown [172.18.186.216]) by frasgout.his.huawei.com (SkyGuard) with ESMTP id 4TW25f0rGCz67HRc; Fri, 9 Feb 2024 00:10:46 +0800 (CST) Received: from lhrpeml100006.china.huawei.com (unknown [7.191.160.224]) by mail.maildlp.com (Postfix) with ESMTPS id 26A5C140B2F; Fri, 9 Feb 2024 00:14:08 +0800 (CST) Received: from lhrpeml500005.china.huawei.com (7.191.163.240) by lhrpeml100006.china.huawei.com (7.191.160.224) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.35; Thu, 8 Feb 2024 16:14:07 +0000 Received: from lhrpeml500005.china.huawei.com ([7.191.163.240]) by lhrpeml500005.china.huawei.com ([7.191.163.240]) with mapi id 15.01.2507.035; Thu, 8 Feb 2024 16:14:07 +0000 From: Shameerali Kolothum Thodi To: Jason Gunthorpe CC: "kvmarm@lists.linux.dev" , "iommu@lists.linux.dev" , "linux-arm-kernel@lists.infradead.org" , Linuxarm , "kevin.tian@intel.com" , "alex.williamson@redhat.com" , "maz@kernel.org" , "oliver.upton@linux.dev" , "will@kernel.org" , "robin.murphy@arm.com" , "jean-philippe@linaro.org" , Jonathan Cameron Subject: RE: [RFC PATCH v2 6/7] iommu/arm-smmu-v3: Use KVM VMID for s2 stage Thread-Topic: [RFC PATCH v2 6/7] iommu/arm-smmu-v3: Use KVM VMID for s2 stage Thread-Index: AQHaWqKqc1WLEJmeWESanxfd3VFGjLEAmkKAgAABL7A= Date: Thu, 8 Feb 2024 16:14:07 +0000 Message-ID: <8bfca4c5bfe7424691efdae0f686ece4@huawei.com> References: <20240208151837.35068-1-shameerali.kolothum.thodi@huawei.com> <20240208151837.35068-7-shameerali.kolothum.thodi@huawei.com> <20240208155915.GR31743@ziepe.ca> In-Reply-To: <20240208155915.GR31743@ziepe.ca> Accept-Language: en-GB, en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.202.227.28] MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240208_081417_815303_E3840D12 X-CRM114-Status: GOOD ( 26.16 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org > -----Original Message----- > From: Jason Gunthorpe > Sent: Thursday, February 8, 2024 3:59 PM > To: Shameerali Kolothum Thodi > Cc: kvmarm@lists.linux.dev; iommu@lists.linux.dev; linux-arm- > kernel@lists.infradead.org; Linuxarm ; > kevin.tian@intel.com; alex.williamson@redhat.com; maz@kernel.org; > oliver.upton@linux.dev; will@kernel.org; robin.murphy@arm.com; jean- > philippe@linaro.org; Jonathan Cameron > Subject: Re: [RFC PATCH v2 6/7] iommu/arm-smmu-v3: Use KVM VMID for s2 > stage > > On Thu, Feb 08, 2024 at 03:18:36PM +0000, Shameer Kolothum wrote: > > If kvm is available make use of kvm pinned VMID interfaces to > > set the s2 stage VMID for nested domains. > > > > Signed-off-by: Shameer Kolothum > > --- > > drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 18 +++++++++++++----- > > drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 2 ++ > > 2 files changed, 15 insertions(+), 5 deletions(-) > > > > diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c > b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c > > index b41d77787a2f..18e3e04b50f4 100644 > > --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c > > +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c > > @@ -18,6 +18,7 @@ > > #include > > #include > > #include > > +#include > > #include > > #include > > #include > > @@ -2399,9 +2400,13 @@ int arm_smmu_domain_alloc_id(struct > arm_smmu_device *smmu, > > } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S2) { > > int vmid; > > > > - /* Reserve VMID 0 for stage-2 bypass STEs */ > > - vmid = ida_alloc_range(&smmu->vmid_map, 1, > > - (1 << smmu->vmid_bits) - 1, GFP_KERNEL); > > + if (smmu_domain->kvm) { > > + vmid = kvm_pinned_vmid_get(smmu_domain->kvm); > > + } else { > > + /* Reserve VMID 0 for stage-2 bypass STEs */ > > + vmid = ida_alloc_range(&smmu->vmid_map, 1, > > + (1 << smmu->vmid_bits) - 1, > GFP_KERNEL); > > + } > > We cannot allow the two different STEs to be programmed with the same > VMID but different translations, so somehow the two allocators have to > work together. My idea was, we only set smmu_domain->kvm in the nested parent case and that means SMMUv3 supports both S1 & S2. So all the non-nested domains are using S1. > > This is why the SVA BTM code has that complex ASID reassignment logic > so it can get away with two allocators. However ASID also has SMMU HW > ASET support to opt-in to the BTM broadcast. > > My suggestion is to avoid two allocators and make iommu instances that > support BTM always use the KVM owned VMID allocator by forbidding a S2 > domain from being created with a NULL KVM. This is what I was trying to do. But not sure we have systems out there where only S2 is supported and that may require a private VMID without KVM. > > IOW all the DMA API/etc will use S1 domains and the only way to get to > a S2 is to allocate a nesting parent via iommufd - for BTM systems > only. > > Since the S2 can't opt-out of the BTM broadcast this means the VMIDs > are cleanly assigned and we never get an issue where the KVM TLBI's > are flushing an unrelated S2. In the last patch we only enable BTM if only S1 is supported. Meaning we will always have a KVM VMID associated for S2(if it supports that). Did I miss something here? Thanks, Shameer _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel