From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2068.outbound.protection.outlook.com [40.107.244.68]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 554813FE4F; Wed, 20 Mar 2024 13:05:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=40.107.244.68 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710939939; cv=fail; b=OmnJ3XrN2LNYjtAhnj1L+3r2C/mdQ4ghSHXyQE2rwGffeItao3SPw2Brs+OoZR4BNRcsacX+rGITcJkzSEZqMCoG37o/kiZatM4aJpTO71nTiAoT+AxIRoUew9TyF6EH2SlePiJkIGaG4uXIwP0TTxoKOZEdTXQk6mIN08ezGBo= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1710939939; c=relaxed/simple; bh=DeDszcZrDzv2jU7mSdjGumxJHNJSw1209A8lbYz40IE=; h=Date:From:To:Cc:Subject:Message-ID:References:Content-Type: Content-Disposition:In-Reply-To:MIME-Version; b=AON6AiuEOE6ocEXFZfV5PaYNCcPA1E+H9aoTliIQVnmDzByKqiEOhZQ0qef9QC12TlMUGpFaSEZfuBfkeW//3HqmM3Qx2JlAi+voUXyk5osqfjl/WXqiAYU1BZoAjc8UGJDzXm8iJUjdskCgQmSizrtc+jpzwziPvEMFl2ka9Hs= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com; spf=fail smtp.mailfrom=nvidia.com; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b=nrvh2Yrv; arc=fail smtp.client-ip=40.107.244.68 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=nvidia.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=nvidia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=Nvidia.com header.i=@Nvidia.com header.b="nrvh2Yrv" ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cchIWv2K2a9fo3BsHr/GW18bKi/ox+kQPnpX530/BahL8n2OptdcNzsit0kZtJ7Lth0rRcD2W/PlY2ooDrpwGErivekCTrMzVwWCJzcxbG6o6RpN76whLRw5U3FGd4F7LkggAgoyUXU5WD0lH4wGbCAe+Cg+TUN1ifc4kJ6HCkhZR3tOyW/Xlt+LQq5eT99b1i01tpEhj5omRdJUVX3oT+Mra1uwLrVy0lowV0K92X1GLGX1RvApDE10p+CeTx0MTQ6FvxzV6Y6BjwdLh30twTXzcUXJJ5tfaUJ0D2joKCImaV5Ii+qq/38qf2z6zC+j0Lo5EyonL49+NGKqhM2F5g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Kyj2Fn8d6zaca2NHQqGTuk+CTFF51txeugMOlqbf3TU=; b=JD+6qPbJbgH0NdT7jopvF5rPZbNuEm+SE7ZPqDvXcQPWz1heUVhOlsTynOtf+PglMstN2g/3xKjPeHA+AVBxN9BONMIomVYZhFGy1cGnlIS0aCeYwtOMzinQmGLGY3GqlOD+HgPDFUEyFIiI29204I2FmcjULqJgzgpT+l+5VJnPZGBo/qq9xMeyW02FA697hNSigJtCd/M2w9RhgZUuwPeMGkImaDLM1v2BvrGmlLbjFn0Tc4dfGjA+e7RBOyhUsh4hkYUHI+ov+nG3jcKpn0DiPiXvTYeO+rJjgw11lPqKkwim2bNG0sOg6VpluTgADAC5NMur2L0g9iBM18kwDQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Kyj2Fn8d6zaca2NHQqGTuk+CTFF51txeugMOlqbf3TU=; b=nrvh2Yrvm/fAfTXTLg6XMeaeSYVqUQwKqu9YDyPV/zd63O/JYAsFKApq2QzaKIuXG3apE834UX0SfMzYvtZsecDCtPlyWb8t/pNJnKVi1PDp8u5fkfC7Sq/3EINAJi/esgd/tT4JlysAU2QHYpMMv6I8E+2GmsBweptn+CW6k8B7jpZs/gURuS2pVEEBo6+eLxecsI4QuPEgvW3fPY8rq3UsACADMp+1R3SYD1EOyK22sqVRUXWsEKOZHrgau96YpPh/XFWkkvk59CUD9nY0w5z5HRqlUzW7we8Xk18Csb2OElUs51L+8fmnQKVgQfZvQjxgvj8DOclhGUkGWykq4Q== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) by LV3PR12MB9119.namprd12.prod.outlook.com (2603:10b6:408:1a2::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7386.34; Wed, 20 Mar 2024 13:05:32 +0000 Received: from DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::6aec:dbca:a593:a222]) by DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::6aec:dbca:a593:a222%5]) with mapi id 15.20.7386.030; Wed, 20 Mar 2024 13:05:31 +0000 Date: Wed, 20 Mar 2024 10:05:29 -0300 From: Jason Gunthorpe To: Michael Shavit , Will Deacon Cc: iommu@lists.linux.dev, Joerg Roedel , linux-arm-kernel@lists.infradead.org, Robin Murphy , Eric Auger , Jean-Philippe Brucker , Moritz Fischer , Nicolin Chen , patches@lists.linux.dev, Shameerali Kolothum Thodi Subject: Re: [PATCH v5 24/27] iommu/arm-smmu-v3: Bring back SVA BTM support Message-ID: <20240320130529.GD1044235@nvidia.com> References: <0-v5-9a37e0c884ce+31e3-smmuv3_newapi_p2_jgg@nvidia.com> <24-v5-9a37e0c884ce+31e3-smmuv3_newapi_p2_jgg@nvidia.com> Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: X-ClientProxiedBy: SN4PR0501CA0114.namprd05.prod.outlook.com (2603:10b6:803:42::31) To DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) Precedence: bulk X-Mailing-List: iommu@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR12MB3849:EE_|LV3PR12MB9119:EE_ X-MS-Office365-Filtering-Correlation-Id: e537d9c4-3b77-4b31-716e-08dc48de6bd3 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: s/fwOgEU+Hs+Aqa5zFtMr/ru44PFwA5zR5QdqbaBKlAZFSs94MAbDYNAMLtv5ICe//gHOuiz+tGBJ8qvnuv3UH7K0EqcIgrOr3ycRFwaKVUXuqixDGeT08UsibyAsPF1eWwuPmRCSvm4JOmn0wqB68nq3PZoyltXp05QBpOGyFXzLVWqUaJwCaqZP1Wl3tlmMHwMIRHU+cV/gaxrgHKmph2F7HqHXq41geWR+acYaSG0BOMEb6hTpm95FJs9VU2JRYDVdkzYLF0ji7o+8dQ2vdA5IsK1jwjdMzLHkDfxxsjhzA4oRmwQ6MZbC7yhjNc6GZ+Maw3ODrXSTTJ2lUSQpeMCYgrVRqM50HHvuQOweQV7OKRsDf34cwLjzoVd5hpdogTl+dfHrGo7MY2w5xT4KLXgO26jUNXj4wXPr8kpaT0lMbIaOI3/xtQWtxjEBHn+rA/pUb/1EEB718kwxbCK0UwMV89jC+E1jJXNOiCy6ca2g8ApwdHVFYFWb3oPQDiNyAUNIofAY0nWU95JYCtKGufeQKuDx3Z2esxisw3NlLaNbZTIipcuP6eqRFFVdjB59UqGTue9gQXbzYlrnY99/QSZ8Ij3L1YPsBjLIVFJhYmdJmy/wWoELlUmzWafT04RlnZ/RuCeHnv/T2pLVfVbDdt9TJQxnLF4wqM2m/dDOQc= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(1800799015)(366007)(7416005)(376005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?MExHVHNuelJBUFVIUFZTV2Raci9NVkF2ektmL3R5UGRjRkxLSGtXREJpa0g3?= =?utf-8?B?MGUvTjdSOU02WWtmWkdGMUdGSGlNYzltelpnK0JEWENOY2FwbUJlb0pnYi9i?= =?utf-8?B?K3RMcm5tQTJKZWNxRFlyWkFWWGJCWDVtRDViYXVhMmFZRXVPRHVsOU9WQ21D?= =?utf-8?B?NGUxNllSL2didFZGV216NEk2dGtHekM4cGFsb0FyWThJMUtsYnc0dE1Va1R1?= =?utf-8?B?dTBWSlorQXRHQTEvV1JGMmRNeTZQTzAyYXp4VXJ2d2JRRmw4MUNiM2tCSGth?= =?utf-8?B?UHo1WTBQRzVaK0JQZU8vZ3FXbFlPc3pNWHMwdW9Wa2huZkNxanBlYjdEakQ1?= =?utf-8?B?TlhrY2VldE8yaHFiMGtvSmRhQUx2dGUwcG0zbGd4SHdjTnVVWXU0R1M5d3Er?= =?utf-8?B?V0dVajVxTXRYcUc4cUJYaXZCeGc5S0V1MUtydkI2amxNZUFJVDJ2dmRObHls?= =?utf-8?B?enZudk80NjR3aEMzT3ZUVGNGN0ZmbThYYUpaVTcySHlXY3R1eFVTNUV4cVQr?= =?utf-8?B?UjBGMG50ZmYxSU1CbTFXK0lKTjFDc2JXM0hiM0tjUnd0NGFGeWtLbVY4Mm04?= =?utf-8?B?aGVKUHNMNHArR3Rib292RjAyaGYvRUxhamVlS3ZsWW9Va3VzR3pDbGpqaWR3?= =?utf-8?B?aldZcTAzbkNuWm1WNzlqY3kwNFpxTDIyVUoyNVhReEtLYmlnU3FZeU9Mdjd2?= =?utf-8?B?c2JEandWSXFlQ2hyNGI5b2QrU0VOVEF3L1JOTkJsTlpTODVuTXN0cU1FWWlm?= =?utf-8?B?VHJNalo2UzNhd3FOVEVHaFlEclkvY1A2cFovbS9GbTBpQTBjeXA4M3NvdHZE?= =?utf-8?B?QitXY1JiSHRTSWx0cUcyQUtBOEFZOFcxTGpCWEJKc2lNSzljTGdXOHZ5NXNW?= =?utf-8?B?UjdrbVI5ZVY3d1hNYmFCbHNFamxwYTlsam5hTmt4YU1NV0ZvWTNDN3Roc0VE?= =?utf-8?B?czlKN0luUTFCYXhteDFNQzdmNGJtOUdiVTZ6QytIMEJPVml3SzBueHVuUk0r?= =?utf-8?B?UmRULzBvbVhJZzZuT290WmhVZEVXeUtnTWpyK2dHaU10WlFZMUNkOFJPc1JD?= =?utf-8?B?Q0x0UDFIdlpoaEdqcWoyU0hOdzN6djU4Z3I2Z2ZRODQrdlBuTW96cUlla2Ez?= =?utf-8?B?MjlrdFd4djJJaW9tUGxxSnl3ZGdJcTlMRG4yd0RTanNvZGZHSVpMRkViM1Y0?= =?utf-8?B?bTlJcWZEOXpIM2dZQzU1eWl2eStsRGxoNHVpaDJYS0hoTVlCTndFTlVyUGFr?= =?utf-8?B?VmRQaThrc2V3NVYrcTdGQjE5aW5lYzJUYzJURGM5WWVIZktGaTBkdDkrbEg4?= =?utf-8?B?Wnlrd2dkK0pkK3J1ckFhemJsUCtpdEJVWUVXUENuNEhIUHBUdmpaNU02TzFI?= =?utf-8?B?SWNlOGhzQmhRVVhqWjNZWFVjbHBsTGJBWkFmK3FRTExZd3ArRHppQXJRYlFh?= =?utf-8?B?c3k3UUpoQXdsMStYMHQzQ1h3djhKMDJRRjI1ckl3U3hpR3RyYTRYVTJlZHEx?= =?utf-8?B?TXh5TDh4ZTZXcWJJK2tuMDllSm1MYXI0NEVjUlp3QnpuTG9OU2h5NHRRZUNK?= =?utf-8?B?ZGJtSkFmUlpHcUhqL3JYZ2g0NjVBcHU5YnpSSDVLOTlqUnVRQ0pmU3VwTDJ5?= =?utf-8?B?b0pUajcweXF2R2RyLzJKR3NFdXlQa3N2eXM3aGpsZFVyQUlqV1JtVVFnMnVB?= =?utf-8?B?ZWFkK1V2c1dYQ2hGT29iNCs1a2xocGQ5Z3E1ZXEwQWhxOEtucFV2dWt2QmFS?= =?utf-8?B?SlNKRUJISXJYMHFiMEdQVWtnUEFRQVhQOEMyd1hYWVViM1JPZmxkMnpqSDc4?= =?utf-8?B?OVp4VXpmbzVSVU52Q25iSFJRUDVadnYzS0JmaFZGK3JyUmMwVENKc0xGYXRQ?= =?utf-8?B?YlVDdEhIZXc0R1orY0dkK1d0d1ZZNXV5T2pIUytRUHlFcGUvYkhqWnl0TUQv?= =?utf-8?B?ZDY2U2VQb0M1Rk9xWk1UeGh3WnUxbnlkbDYrYXJydjM0YmVwK3BIZE1uZFBi?= =?utf-8?B?QWFHUERlSy9vSGcyTmhudXVXRjVQb3VNTGhPU1pIR09YdVorc3BsQ0tGcUh6?= =?utf-8?B?Y25GNFEwMW5ZVllzdDdKeDZDNzBkb1BJa2dnRzZYQkhmY0lPVkhhTzU3N2VM?= =?utf-8?Q?bir5pcECFygkzzDyBzuF64mHS?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: e537d9c4-3b77-4b31-716e-08dc48de6bd3 X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Mar 2024 13:05:30.9832 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: rkxUbTvGJIMpmOIR++0o3Y99IsPpM5WTGfwCouvmHtNPJjmtdXU4//uE6KJeRdbx X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV3PR12MB9119 On Wed, Mar 20, 2024 at 01:07:01AM +0800, Michael Shavit wrote: > On Tue, Mar 5, 2024 at 7:44 AM Jason Gunthorpe wrote: > > > > BTM support is a feature where the CPU TLB invalidation can be forwarded > > to the IOMMU and also invalidate the IOTLB. For this to work the CPU and > > IOMMU ASID must be the same. > > > > Retain the prior SVA design here of keeping the ASID allocator for the > > IOMMU private to SMMU and force SVA domains to set an ASID that matches > > the CPU ASID. > > > > This requires changing the ASID assigned to a S1 domain if it happens to > > be overlapping with the required CPU ASID. We hold on to the CPU ASID so > > long as the SVA iommu_domain exists, so SVA domain conflict is not > > possible. > > > > With the asid per-smmu we no longer have a problem that two per-smmu > > iommu_domain's would need to share a CPU ASID entry in the IOMMU's xarray. > > > > Use the same ASID move algorithm for the S1 domains as before with some > > streamlining around how the xarray is being used. Do not synchronize the > > ASID's if BTM mode is not supported. Just leave BTM features off > > everywhere. > > > > Audit all the places that touch cd->asid and think carefully about how the > > locking works with the change to the cd->asid by the move algorithm. Use > > xarray internal locking during xa_alloc() instead of double locking. Add a > > note that concurrent S1 invalidation doesn't fully work. > > > > Note that this is all still dead code, ARM_SMMU_FEAT_BTM is never set. > > Clearly a lot of work/thought has gone into this patch, but I have to > wonder if this complexity is worth carrying forward for a feature that > was never enabled... Is moving this to part 3 or dropping it all > together an option? Yes, we can do any of those. If we drop it here then it will be moved to this series: https://lore.kernel.org/linux-iommu/20240208151837.35068-1-shameerali.kolothum.thodi@huawei.com/ Which is the final bit of work to actually enable the feature. We are getting quite close, it needs some of Nicolin's iommufd enablement work to cross the finish line, I think. I agree dead code like this should not be in the kernel.. Will? Please advise Jason From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1271BC54E67 for ; Wed, 20 Mar 2024 13:06:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:In-Reply-To:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=eFIoGIxz9FFpjkixXO3z04MFWwxxqUUW+g/xvZdrYu4=; b=oEM1Egfaqml0Cb WVTYOZwmm5120tVnnZzukUsQFSWTtSVTQdQUDjyruim/cNahd52D74Vx4TjIE2mhXKQ+IngQ3Ufhu neS3Ty6uwtpFd0SNtcVnwo+xpOKIEnoISAFwJHlK7/C4ZPwJZjl354wHSEAvuQ//42QZkzEVn3U3P kPH6bgcHLstwoebb6uhVlnci1RK0822L9rfUd71ljCLA4jQ6EMERnQFxBmKW78qNx+69/D2M/XH0p WBcwfy/Y8j1Tr18ftHOgZBFh4eIKpC1HGiKTokd/62iG3SxjU3vMOTvPVW7CJHMrsCGMzQtF+6LK7 2cIgjM/muFMKEhQfejcQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rmvdZ-0000000HCip-36IE; Wed, 20 Mar 2024 13:05:45 +0000 Received: from mail-dm6nam04on20600.outbound.protection.outlook.com ([2a01:111:f403:2409::600] helo=NAM04-DM6-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rmvdV-0000000HCiH-2VfG for linux-arm-kernel@lists.infradead.org; Wed, 20 Mar 2024 13:05:43 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=cchIWv2K2a9fo3BsHr/GW18bKi/ox+kQPnpX530/BahL8n2OptdcNzsit0kZtJ7Lth0rRcD2W/PlY2ooDrpwGErivekCTrMzVwWCJzcxbG6o6RpN76whLRw5U3FGd4F7LkggAgoyUXU5WD0lH4wGbCAe+Cg+TUN1ifc4kJ6HCkhZR3tOyW/Xlt+LQq5eT99b1i01tpEhj5omRdJUVX3oT+Mra1uwLrVy0lowV0K92X1GLGX1RvApDE10p+CeTx0MTQ6FvxzV6Y6BjwdLh30twTXzcUXJJ5tfaUJ0D2joKCImaV5Ii+qq/38qf2z6zC+j0Lo5EyonL49+NGKqhM2F5g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Kyj2Fn8d6zaca2NHQqGTuk+CTFF51txeugMOlqbf3TU=; b=JD+6qPbJbgH0NdT7jopvF5rPZbNuEm+SE7ZPqDvXcQPWz1heUVhOlsTynOtf+PglMstN2g/3xKjPeHA+AVBxN9BONMIomVYZhFGy1cGnlIS0aCeYwtOMzinQmGLGY3GqlOD+HgPDFUEyFIiI29204I2FmcjULqJgzgpT+l+5VJnPZGBo/qq9xMeyW02FA697hNSigJtCd/M2w9RhgZUuwPeMGkImaDLM1v2BvrGmlLbjFn0Tc4dfGjA+e7RBOyhUsh4hkYUHI+ov+nG3jcKpn0DiPiXvTYeO+rJjgw11lPqKkwim2bNG0sOg6VpluTgADAC5NMur2L0g9iBM18kwDQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Kyj2Fn8d6zaca2NHQqGTuk+CTFF51txeugMOlqbf3TU=; b=nrvh2Yrvm/fAfTXTLg6XMeaeSYVqUQwKqu9YDyPV/zd63O/JYAsFKApq2QzaKIuXG3apE834UX0SfMzYvtZsecDCtPlyWb8t/pNJnKVi1PDp8u5fkfC7Sq/3EINAJi/esgd/tT4JlysAU2QHYpMMv6I8E+2GmsBweptn+CW6k8B7jpZs/gURuS2pVEEBo6+eLxecsI4QuPEgvW3fPY8rq3UsACADMp+1R3SYD1EOyK22sqVRUXWsEKOZHrgau96YpPh/XFWkkvk59CUD9nY0w5z5HRqlUzW7we8Xk18Csb2OElUs51L+8fmnQKVgQfZvQjxgvj8DOclhGUkGWykq4Q== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) by LV3PR12MB9119.namprd12.prod.outlook.com (2603:10b6:408:1a2::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7386.34; Wed, 20 Mar 2024 13:05:32 +0000 Received: from DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::6aec:dbca:a593:a222]) by DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::6aec:dbca:a593:a222%5]) with mapi id 15.20.7386.030; Wed, 20 Mar 2024 13:05:31 +0000 Date: Wed, 20 Mar 2024 10:05:29 -0300 From: Jason Gunthorpe To: Michael Shavit , Will Deacon Cc: iommu@lists.linux.dev, Joerg Roedel , linux-arm-kernel@lists.infradead.org, Robin Murphy , Eric Auger , Jean-Philippe Brucker , Moritz Fischer , Nicolin Chen , patches@lists.linux.dev, Shameerali Kolothum Thodi Subject: Re: [PATCH v5 24/27] iommu/arm-smmu-v3: Bring back SVA BTM support Message-ID: <20240320130529.GD1044235@nvidia.com> References: <0-v5-9a37e0c884ce+31e3-smmuv3_newapi_p2_jgg@nvidia.com> <24-v5-9a37e0c884ce+31e3-smmuv3_newapi_p2_jgg@nvidia.com> Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: SN4PR0501CA0114.namprd05.prod.outlook.com (2603:10b6:803:42::31) To DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR12MB3849:EE_|LV3PR12MB9119:EE_ X-MS-Office365-Filtering-Correlation-Id: e537d9c4-3b77-4b31-716e-08dc48de6bd3 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: s/fwOgEU+Hs+Aqa5zFtMr/ru44PFwA5zR5QdqbaBKlAZFSs94MAbDYNAMLtv5ICe//gHOuiz+tGBJ8qvnuv3UH7K0EqcIgrOr3ycRFwaKVUXuqixDGeT08UsibyAsPF1eWwuPmRCSvm4JOmn0wqB68nq3PZoyltXp05QBpOGyFXzLVWqUaJwCaqZP1Wl3tlmMHwMIRHU+cV/gaxrgHKmph2F7HqHXq41geWR+acYaSG0BOMEb6hTpm95FJs9VU2JRYDVdkzYLF0ji7o+8dQ2vdA5IsK1jwjdMzLHkDfxxsjhzA4oRmwQ6MZbC7yhjNc6GZ+Maw3ODrXSTTJ2lUSQpeMCYgrVRqM50HHvuQOweQV7OKRsDf34cwLjzoVd5hpdogTl+dfHrGo7MY2w5xT4KLXgO26jUNXj4wXPr8kpaT0lMbIaOI3/xtQWtxjEBHn+rA/pUb/1EEB718kwxbCK0UwMV89jC+E1jJXNOiCy6ca2g8ApwdHVFYFWb3oPQDiNyAUNIofAY0nWU95JYCtKGufeQKuDx3Z2esxisw3NlLaNbZTIipcuP6eqRFFVdjB59UqGTue9gQXbzYlrnY99/QSZ8Ij3L1YPsBjLIVFJhYmdJmy/wWoELlUmzWafT04RlnZ/RuCeHnv/T2pLVfVbDdt9TJQxnLF4wqM2m/dDOQc= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(1800799015)(366007)(7416005)(376005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?MExHVHNuelJBUFVIUFZTV2Raci9NVkF2ektmL3R5UGRjRkxLSGtXREJpa0g3?= =?utf-8?B?MGUvTjdSOU02WWtmWkdGMUdGSGlNYzltelpnK0JEWENOY2FwbUJlb0pnYi9i?= =?utf-8?B?K3RMcm5tQTJKZWNxRFlyWkFWWGJCWDVtRDViYXVhMmFZRXVPRHVsOU9WQ21D?= =?utf-8?B?NGUxNllSL2didFZGV216NEk2dGtHekM4cGFsb0FyWThJMUtsYnc0dE1Va1R1?= =?utf-8?B?dTBWSlorQXRHQTEvV1JGMmRNeTZQTzAyYXp4VXJ2d2JRRmw4MUNiM2tCSGth?= =?utf-8?B?UHo1WTBQRzVaK0JQZU8vZ3FXbFlPc3pNWHMwdW9Wa2huZkNxanBlYjdEakQ1?= =?utf-8?B?TlhrY2VldE8yaHFiMGtvSmRhQUx2dGUwcG0zbGd4SHdjTnVVWXU0R1M5d3Er?= =?utf-8?B?V0dVajVxTXRYcUc4cUJYaXZCeGc5S0V1MUtydkI2amxNZUFJVDJ2dmRObHls?= =?utf-8?B?enZudk80NjR3aEMzT3ZUVGNGN0ZmbThYYUpaVTcySHlXY3R1eFVTNUV4cVQr?= =?utf-8?B?UjBGMG50ZmYxSU1CbTFXK0lKTjFDc2JXM0hiM0tjUnd0NGFGeWtLbVY4Mm04?= =?utf-8?B?aGVKUHNMNHArR3Rib292RjAyaGYvRUxhamVlS3ZsWW9Va3VzR3pDbGpqaWR3?= =?utf-8?B?aldZcTAzbkNuWm1WNzlqY3kwNFpxTDIyVUoyNVhReEtLYmlnU3FZeU9Mdjd2?= =?utf-8?B?c2JEandWSXFlQ2hyNGI5b2QrU0VOVEF3L1JOTkJsTlpTODVuTXN0cU1FWWlm?= =?utf-8?B?VHJNalo2UzNhd3FOVEVHaFlEclkvY1A2cFovbS9GbTBpQTBjeXA4M3NvdHZE?= =?utf-8?B?QitXY1JiSHRTSWx0cUcyQUtBOEFZOFcxTGpCWEJKc2lNSzljTGdXOHZ5NXNW?= =?utf-8?B?UjdrbVI5ZVY3d1hNYmFCbHNFamxwYTlsam5hTmt4YU1NV0ZvWTNDN3Roc0VE?= =?utf-8?B?czlKN0luUTFCYXhteDFNQzdmNGJtOUdiVTZ6QytIMEJPVml3SzBueHVuUk0r?= =?utf-8?B?UmRULzBvbVhJZzZuT290WmhVZEVXeUtnTWpyK2dHaU10WlFZMUNkOFJPc1JD?= =?utf-8?B?Q0x0UDFIdlpoaEdqcWoyU0hOdzN6djU4Z3I2Z2ZRODQrdlBuTW96cUlla2Ez?= =?utf-8?B?MjlrdFd4djJJaW9tUGxxSnl3ZGdJcTlMRG4yd0RTanNvZGZHSVpMRkViM1Y0?= =?utf-8?B?bTlJcWZEOXpIM2dZQzU1eWl2eStsRGxoNHVpaDJYS0hoTVlCTndFTlVyUGFr?= =?utf-8?B?VmRQaThrc2V3NVYrcTdGQjE5aW5lYzJUYzJURGM5WWVIZktGaTBkdDkrbEg4?= =?utf-8?B?Wnlrd2dkK0pkK3J1ckFhemJsUCtpdEJVWUVXUENuNEhIUHBUdmpaNU02TzFI?= =?utf-8?B?SWNlOGhzQmhRVVhqWjNZWFVjbHBsTGJBWkFmK3FRTExZd3ArRHppQXJRYlFh?= =?utf-8?B?c3k3UUpoQXdsMStYMHQzQ1h3djhKMDJRRjI1ckl3U3hpR3RyYTRYVTJlZHEx?= =?utf-8?B?TXh5TDh4ZTZXcWJJK2tuMDllSm1MYXI0NEVjUlp3QnpuTG9OU2h5NHRRZUNK?= =?utf-8?B?ZGJtSkFmUlpHcUhqL3JYZ2g0NjVBcHU5YnpSSDVLOTlqUnVRQ0pmU3VwTDJ5?= =?utf-8?B?b0pUajcweXF2R2RyLzJKR3NFdXlQa3N2eXM3aGpsZFVyQUlqV1JtVVFnMnVB?= =?utf-8?B?ZWFkK1V2c1dYQ2hGT29iNCs1a2xocGQ5Z3E1ZXEwQWhxOEtucFV2dWt2QmFS?= =?utf-8?B?SlNKRUJISXJYMHFiMEdQVWtnUEFRQVhQOEMyd1hYWVViM1JPZmxkMnpqSDc4?= =?utf-8?B?OVp4VXpmbzVSVU52Q25iSFJRUDVadnYzS0JmaFZGK3JyUmMwVENKc0xGYXRQ?= =?utf-8?B?YlVDdEhIZXc0R1orY0dkK1d0d1ZZNXV5T2pIUytRUHlFcGUvYkhqWnl0TUQv?= =?utf-8?B?ZDY2U2VQb0M1Rk9xWk1UeGh3WnUxbnlkbDYrYXJydjM0YmVwK3BIZE1uZFBi?= =?utf-8?B?QWFHUERlSy9vSGcyTmhudXVXRjVQb3VNTGhPU1pIR09YdVorc3BsQ0tGcUh6?= =?utf-8?B?Y25GNFEwMW5ZVllzdDdKeDZDNzBkb1BJa2dnRzZYQkhmY0lPVkhhTzU3N2VM?= =?utf-8?Q?bir5pcECFygkzzDyBzuF64mHS?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: e537d9c4-3b77-4b31-716e-08dc48de6bd3 X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Mar 2024 13:05:30.9832 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: rkxUbTvGJIMpmOIR++0o3Y99IsPpM5WTGfwCouvmHtNPJjmtdXU4//uE6KJeRdbx X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV3PR12MB9119 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240320_060541_767121_0F4A9ED2 X-CRM114-Status: GOOD ( 24.67 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org T24gV2VkLCBNYXIgMjAsIDIwMjQgYXQgMDE6MDc6MDFBTSArMDgwMCwgTWljaGFlbCBTaGF2aXQg d3JvdGU6Cj4gT24gVHVlLCBNYXIgNSwgMjAyNCBhdCA3OjQ04oCvQU0gSmFzb24gR3VudGhvcnBl IDxqZ2dAbnZpZGlhLmNvbT4gd3JvdGU6Cj4gPgo+ID4gQlRNIHN1cHBvcnQgaXMgYSBmZWF0dXJl IHdoZXJlIHRoZSBDUFUgVExCIGludmFsaWRhdGlvbiBjYW4gYmUgZm9yd2FyZGVkCj4gPiB0byB0 aGUgSU9NTVUgYW5kIGFsc28gaW52YWxpZGF0ZSB0aGUgSU9UTEIuIEZvciB0aGlzIHRvIHdvcmsg dGhlIENQVSBhbmQKPiA+IElPTU1VIEFTSUQgbXVzdCBiZSB0aGUgc2FtZS4KPiA+Cj4gPiBSZXRh aW4gdGhlIHByaW9yIFNWQSBkZXNpZ24gaGVyZSBvZiBrZWVwaW5nIHRoZSBBU0lEIGFsbG9jYXRv ciBmb3IgdGhlCj4gPiBJT01NVSBwcml2YXRlIHRvIFNNTVUgYW5kIGZvcmNlIFNWQSBkb21haW5z IHRvIHNldCBhbiBBU0lEIHRoYXQgbWF0Y2hlcwo+ID4gdGhlIENQVSBBU0lELgo+ID4KPiA+IFRo aXMgcmVxdWlyZXMgY2hhbmdpbmcgdGhlIEFTSUQgYXNzaWduZWQgdG8gYSBTMSBkb21haW4gaWYg aXQgaGFwcGVucyB0bwo+ID4gYmUgb3ZlcmxhcHBpbmcgd2l0aCB0aGUgcmVxdWlyZWQgQ1BVIEFT SUQuIFdlIGhvbGQgb24gdG8gdGhlIENQVSBBU0lEIHNvCj4gPiBsb25nIGFzIHRoZSBTVkEgaW9t bXVfZG9tYWluIGV4aXN0cywgc28gU1ZBIGRvbWFpbiBjb25mbGljdCBpcyBub3QKPiA+IHBvc3Np YmxlLgo+ID4KPiA+IFdpdGggdGhlIGFzaWQgcGVyLXNtbXUgd2Ugbm8gbG9uZ2VyIGhhdmUgYSBw cm9ibGVtIHRoYXQgdHdvIHBlci1zbW11Cj4gPiBpb21tdV9kb21haW4ncyB3b3VsZCBuZWVkIHRv IHNoYXJlIGEgQ1BVIEFTSUQgZW50cnkgaW4gdGhlIElPTU1VJ3MgeGFycmF5Lgo+ID4KPiA+IFVz ZSB0aGUgc2FtZSBBU0lEIG1vdmUgYWxnb3JpdGhtIGZvciB0aGUgUzEgZG9tYWlucyBhcyBiZWZv cmUgd2l0aCBzb21lCj4gPiBzdHJlYW1saW5pbmcgYXJvdW5kIGhvdyB0aGUgeGFycmF5IGlzIGJl aW5nIHVzZWQuIERvIG5vdCBzeW5jaHJvbml6ZSB0aGUKPiA+IEFTSUQncyBpZiBCVE0gbW9kZSBp cyBub3Qgc3VwcG9ydGVkLiBKdXN0IGxlYXZlIEJUTSBmZWF0dXJlcyBvZmYKPiA+IGV2ZXJ5d2hl cmUuCj4gPgo+ID4gQXVkaXQgYWxsIHRoZSBwbGFjZXMgdGhhdCB0b3VjaCBjZC0+YXNpZCBhbmQg dGhpbmsgY2FyZWZ1bGx5IGFib3V0IGhvdyB0aGUKPiA+IGxvY2tpbmcgd29ya3Mgd2l0aCB0aGUg Y2hhbmdlIHRvIHRoZSBjZC0+YXNpZCBieSB0aGUgbW92ZSBhbGdvcml0aG0uIFVzZQo+ID4geGFy cmF5IGludGVybmFsIGxvY2tpbmcgZHVyaW5nIHhhX2FsbG9jKCkgaW5zdGVhZCBvZiBkb3VibGUg bG9ja2luZy4gQWRkIGEKPiA+IG5vdGUgdGhhdCBjb25jdXJyZW50IFMxIGludmFsaWRhdGlvbiBk b2Vzbid0IGZ1bGx5IHdvcmsuCj4gPgo+ID4gTm90ZSB0aGF0IHRoaXMgaXMgYWxsIHN0aWxsIGRl YWQgY29kZSwgQVJNX1NNTVVfRkVBVF9CVE0gaXMgbmV2ZXIgc2V0Lgo+IAo+IENsZWFybHkgYSBs b3Qgb2Ygd29yay90aG91Z2h0IGhhcyBnb25lIGludG8gdGhpcyBwYXRjaCwgYnV0IEkgaGF2ZSB0 bwo+IHdvbmRlciBpZiB0aGlzIGNvbXBsZXhpdHkgaXMgd29ydGggY2FycnlpbmcgZm9yd2FyZCBm b3IgYSBmZWF0dXJlIHRoYXQKPiB3YXMgbmV2ZXIgZW5hYmxlZC4uLiBJcyBtb3ZpbmcgdGhpcyB0 byBwYXJ0IDMgb3IgZHJvcHBpbmcgaXQgYWxsCj4gdG9nZXRoZXIgYW4gb3B0aW9uPwoKWWVzLCB3 ZSBjYW4gZG8gYW55IG9mIHRob3NlLiBJZiB3ZSBkcm9wIGl0IGhlcmUgdGhlbiBpdCB3aWxsIGJl IG1vdmVkCnRvIHRoaXMgc2VyaWVzOgoKaHR0cHM6Ly9sb3JlLmtlcm5lbC5vcmcvbGludXgtaW9t bXUvMjAyNDAyMDgxNTE4MzcuMzUwNjgtMS1zaGFtZWVyYWxpLmtvbG90aHVtLnRob2RpQGh1YXdl aS5jb20vCgpXaGljaCBpcyB0aGUgZmluYWwgYml0IG9mIHdvcmsgdG8gYWN0dWFsbHkgZW5hYmxl IHRoZSBmZWF0dXJlLiBXZSBhcmUKZ2V0dGluZyBxdWl0ZSBjbG9zZSwgaXQgbmVlZHMgc29tZSBv ZiBOaWNvbGluJ3MgaW9tbXVmZCBlbmFibGVtZW50CndvcmsgdG8gY3Jvc3MgdGhlIGZpbmlzaCBs aW5lLCBJIHRoaW5rLgoKSSBhZ3JlZSBkZWFkIGNvZGUgbGlrZSB0aGlzIHNob3VsZCBub3QgYmUg aW4gdGhlIGtlcm5lbC4uCgpXaWxsPyBQbGVhc2UgYWR2aXNlCgpKYXNvbgoKX19fX19fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KbGludXgtYXJtLWtlcm5lbCBtYWls aW5nIGxpc3QKbGludXgtYXJtLWtlcm5lbEBsaXN0cy5pbmZyYWRlYWQub3JnCmh0dHA6Ly9saXN0 cy5pbmZyYWRlYWQub3JnL21haWxtYW4vbGlzdGluZm8vbGludXgtYXJtLWtlcm5lbAo=