From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 63254C52D7C for ; Mon, 19 Aug 2024 17:45:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:MIME-Version:In-Reply-To: Content-Type:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=XZcm7E1zk5TW7hYjPOq/Vbb3nopkdvLlRjaeyNr27SU=; b=nPwDON3JpP5LsULLCgeT9xMvIV W92gHupqKn93t/9ASpWSmixfkYDaCO7Yg0RICHyYdGSLgYEfRdI0HvV8IzJAcbltZaENon0SL7Q1m UwgGbsS4nhAwPJiIqk5120Kz4hY/apPeapb+HgVJT6nDYX2kuacELQbs8RNRZDSsuBOQ/Vp5jvXg9 +aBw55JtigfvOvhn2ihqBz7EPxjkzBbZYdN4SPNAPRnr1bNbeDHalHJggEaKYJsFeAOZoEvgW7rI7 9FPNynPToRgHammg6EyfWU2Sfb19Il538Z5MtOAaH0qZJbggv0cxW9rN5s43J9WKl5DabVrwWJItw RDNl1mpA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sg6Ra-00000002RwN-421X; Mon, 19 Aug 2024 17:45:26 +0000 Received: from mail-bn1nam02on20617.outbound.protection.outlook.com ([2a01:111:f403:2407::617] helo=NAM02-BN1-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sg6Ir-00000002Ql2-2XqC for linux-arm-kernel@lists.infradead.org; Mon, 19 Aug 2024 17:36:27 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=XH8PutGePsZv4yTkYnE3R/xVpJ6quoMBtXKSWRfN4/48vgh8lFt2Qsnc0xvdp5L4n1LyRrHvkHn2EqwIhaGMqW0lTmC0oSHXKGj1PmPIFbwOR25g8aug8T4MS+hSqCZ54dY/hczkeJS6yNmDN9DQEcZMJRudbzMgMiMz6rj8kjUWHXu/Mm6Ey3tP2/3wo01ZmH8gNNi41RwwfmKyNAAiEKMKTy4ajaQ1+E1kV3XRS5BLw9L855DbqdJUir4PgrUz5iIoJgPuXlM6tB30lHXPSdOIAiYs4uBt+uzNmQZGAFAcY4EvIVgChTVUWZeU+jhAK1tSLwJ49qPyuabUY2sxyQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XZcm7E1zk5TW7hYjPOq/Vbb3nopkdvLlRjaeyNr27SU=; b=wo+JznXlmBfLvaljRtoeRxzM0li4x9akf/0UeOhW9j3U0QM9St7HEAcx9qpN+3+mxAPloZvGvMY8NGlSTVaMfR7slKjSoCoH07daEKhTerOpe+xDaEcGJAvPDIzvPMlHQXpGhGOCnn3eBfFfyjzOLmakQ+4GO48MX6wpCeVJVNLlBtbNghARmZ+da96ukCiKBqx6KKdqbMWGDBYylg3fjFQ4RS/auZUhLHNPWGEZsRnKMNU2SkSlHiJczt7v2VDKnPQ8wgnHFjqa7lpMFXGMCHG+7b76OVYKlEa4VjvA3oQjeuxQJ5kHZI+SAGr/LEfzbWwXgHf1LyciXmykCB32nw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XZcm7E1zk5TW7hYjPOq/Vbb3nopkdvLlRjaeyNr27SU=; b=E34H+bcjF3QMGAchgc/QKa1lzFPpty+wpgCSvX6nGRkX4x1aQ66vbBgPUsxI/vrASOa6ipN6qmurIIS/oXE4Fr/yws3fOWkJJwQimN4+qnBMCxsF2RtwJ5IFa/Gnu/mlDfletKSCfUnc5kMcMtndAMdeFihfR0CdH19Xd4Bnuap1SWq6nay/K4XA3mRAcK7ecRJr8MWkK9awrKJt5sakphSvzSjtZkQ/RBTFR32Vv9fACAa0ZUTptwTZirI41kO85dFlT4TLqPX/xrtEFPHJI2tXjnQwRlvnlsGkT4u0xLESpn8s0oLXmG7JnHyVw2DVeJNsTxLG2Tg3+oM2zTrW8g== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB7763.namprd12.prod.outlook.com (2603:10b6:610:145::10) by MW3PR12MB4458.namprd12.prod.outlook.com (2603:10b6:303:5d::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7875.21; Mon, 19 Aug 2024 17:36:17 +0000 Received: from CH3PR12MB7763.namprd12.prod.outlook.com ([fe80::8b63:dd80:c182:4ce8]) by CH3PR12MB7763.namprd12.prod.outlook.com ([fe80::8b63:dd80:c182:4ce8%3]) with mapi id 15.20.7875.019; Mon, 19 Aug 2024 17:36:17 +0000 Date: Mon, 19 Aug 2024 14:36:15 -0300 From: Jason Gunthorpe To: Nicolin Chen Cc: kevin.tian@intel.com, will@kernel.org, joro@8bytes.org, suravee.suthikulpanit@amd.com, robin.murphy@arm.com, dwmw2@infradead.org, baolu.lu@linux.intel.com, shuah@kernel.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kselftest@vger.kernel.org Subject: Re: [PATCH v1 15/16] iommu/arm-smmu-v3: Add viommu cache invalidation support Message-ID: <20240819173615.GN2032816@nvidia.com> References: <729dfd0808f85d88fd3ef8bcea0168cc1d2c0d59.1723061378.git.nicolinc@nvidia.com> <20240815233635.GV2032816@nvidia.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: BN0PR03CA0039.namprd03.prod.outlook.com (2603:10b6:408:e7::14) To CH3PR12MB7763.namprd12.prod.outlook.com (2603:10b6:610:145::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB7763:EE_|MW3PR12MB4458:EE_ X-MS-Office365-Filtering-Correlation-Id: e43ae003-92c3-453e-7acb-08dcc0756e00 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|376014|7416014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?NRidhjGEarDgN1b5Kz5RkSh9eYanMwabWjwrYVASNXzVd1zDcI8kJQiye7DL?= =?us-ascii?Q?QO4kWcXttYzSzRGKj3aXxPj4JQzDBmXWhss2iBg55Jp5jr40ZK+uRndy/tt9?= =?us-ascii?Q?k0WDErQSh/Jzp6At3lgXdatsRFTfhHb9dD1VkseObUBZ8m+j9uVCU5oAyW9K?= =?us-ascii?Q?+qtVCSonhRjx30HJQx4I4cVMbXjFzqTHCsov56rgHJJkOo72mkRLWuGP+G7M?= =?us-ascii?Q?vYNCyoXvh9HUNZrphdrCQWmvwkJ24k0G2NpEpI659I2FD8cB+Udqn8GLOvg0?= =?us-ascii?Q?3tXOWWrbo7V6cuWdnUZc8ZFpy32S/orZhxwSwXOHgpW2rz7R+fC4VAx5CYVO?= =?us-ascii?Q?nEnmokx9ST3ZLwd21iRlzmbujaBXXChPe+nKykDgQlbFloFgISKbQDFbGTdO?= =?us-ascii?Q?2eb3cadvI8Mp5BChtbFzlq7IU990ZFwoW3QOjitar8TCDUNmvrqhQVOTlt4d?= =?us-ascii?Q?2zEGPlsMivNd03jklTeHbE/XUKaUlB+Hb9xtV7nCMkD7udbi6DxFgnx1X/9u?= =?us-ascii?Q?cYQQ0qOygYxIS9Y+W+md7psrtDMIgMM3Wb7VI0Hmd/7j2UKcgQCFagnVHEXq?= =?us-ascii?Q?yj9Rrb6ti0IY+xPtsN6MFNkOOI0degZgrlVqXGI6LFbt2dtjvPR6nxHhRx3F?= =?us-ascii?Q?4cA6MGwiEgrPmaybp6uWFm+46JOkIAL5xvk3hg6hu68j4SJsBntrH38BFc7N?= =?us-ascii?Q?mcTX/1X3px4/RvdA5bQA5Mbdq8Hag1Z17MsfRrnHrtmpjiPD0jF5FnN58gsf?= =?us-ascii?Q?uAiMMVg3toCXE91toYKp5QnZ4nuJUA/JzaxDEfK9NjAXl7PrRT94D58UQ1X1?= =?us-ascii?Q?GibkyqvtY+m54vh2mTllTy+Lo/+U0iTsFFD41nrlmoxO6bJ3koBB4Vil86YH?= =?us-ascii?Q?qgp1pzlhoSFIgkUozglup6sWhgYEVZCDUyTuYj9NKd3lbJ/bMH/pLm7kZZVC?= =?us-ascii?Q?RNbYJSa930KnLhbx4pblC7wQyuvYU97cgXW9p93BR43CnKen6B5DvG4as4Me?= =?us-ascii?Q?Pw8rruZGclFWeKX3wlCRdiw+kY19+yXitFoG/S19b3ifNsnZ2JLIhN7ooYjk?= =?us-ascii?Q?La/Q7geAu426+rGrHbciPxLo2oFkIPaEG76a+zAj4CV5NPh5fwv0RZTVSVkG?= =?us-ascii?Q?h7lybvOfww4AKdxUb+kS3rUtab1BJHnAN9NRn2AcwELRsh6oHgSyj2GiIgOH?= =?us-ascii?Q?LWHZuBWYRtSKE25nCkVDud2BG4YDo8kBVlom4rJ61urKTPAQ9UXARd/+GPIV?= =?us-ascii?Q?6SZymDZW/sVvweU8veGV5NWDE/5NCSfUtEphdrX5jNaJqQj+qZI46bCAAFTW?= =?us-ascii?Q?0cEOye/YX1RNnsfLppPBs9gFcOFtjlTyGUkKivxT/neaCQ=3D=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CH3PR12MB7763.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(376014)(7416014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?ClvKHtr2KGi6uS4hDtFyNZ6DluG0Xo2FAySXZy5k78mXR0yHD6v9rjTBBbKu?= =?us-ascii?Q?3jAtGZv+PQ5IpmsZp+aG67o39IA9LgrTDFJi6ePXzB1pz3e5ACrDnbyFSJ45?= =?us-ascii?Q?GYnARvYEES1V8rocw2ONL8oI4cVehcCIVzcin39E0SBzmpYn5I0hkTLXwKoh?= =?us-ascii?Q?JNzoZREUnE0F1VpLIRquB2tk90JDJIbyFILgJyOh8iOIkE/GFBW/HLgY08Fh?= =?us-ascii?Q?e5s0e5BdWdm/oQdh5kEJ+gT8YRa2EQ3WEwqrHd3QYnDQiopQjmyuQ5HNlY9o?= =?us-ascii?Q?TM96tW7HOayFknq+VLfcpkeW6sgSCeAzU5UGscT6ySAqcExUiFoWUXHpU4LN?= =?us-ascii?Q?YS2dvNrkJSYzjRlxTxgf7mdVzOZrnYYQRf7kirI1IgydTAqBE75qrcPxcq9d?= =?us-ascii?Q?QMtYmBQxL/Yo7mRpwsvlOEw8EPvtPysoEX4/FixifPXx41WANLFw997elNUV?= =?us-ascii?Q?ZK9h3HPNpuLepvk873vURF11lryG5yzHjUe8GtaIE1dhiFTjtGReE2WXVQDj?= =?us-ascii?Q?h0mIuJyi8wpF15TqT5jbsebU6pAW8XMbGLpaa4lfaR8U/uH0T87yxJkseWvi?= =?us-ascii?Q?KXUiWOai3jxWOLl1isTDQJZ6yRezJr8SseEDovXCilBp7cd6hgFdfnll4V3Z?= =?us-ascii?Q?xrzhRZqJ6ogdHM51UjOC2GEGOV6SGQ5+OVeGXRkc9I3z5Pfv/0YEoLPUdFIH?= =?us-ascii?Q?arnkCeB4dSslra74Bo26koe6aSEFfDlhyHYdvHBW7Cc3WqbPn5SrQaUpxh5f?= =?us-ascii?Q?YrWs9lwaw/FYSx3M4nlSlWsVNswxkHmwBra36oYIfBdpCjAP56q/GrWRf2Pi?= =?us-ascii?Q?JHcG8HP2BMpINIPGaFOMeaA9l1fGMWQJdEMbe0EwQanJJXuEt8Jwk3dHdDWM?= =?us-ascii?Q?1nCa0fmr1NCj/wiNBdsjGv42R9UkvS+BIy3EfA2D6uTk8Y4aHrpEm1oC72aa?= =?us-ascii?Q?bpesz/MOcrmDAoNIPFUu9gQwAMHZua2JEpxHVcczNPoh9I1hUUU16ylV9WL/?= =?us-ascii?Q?A3wgRuzsz3WugfOeoIP+9cKKxrUVEJFyS0iyI2Mwm1XO4zwL7Gp6Hsxu+Qlc?= =?us-ascii?Q?LBiBTO0XdQTMQTZ5FLxAE9VHBUsQRE8YAl14D3SvPXQEgO39OrJlX5ZgdjpE?= =?us-ascii?Q?zX4uEzor0gtF+kfi7oY49+NGTOTjz3/ZIL1zW+TRlzCw1kJAr/xXvPuLXHI5?= =?us-ascii?Q?U1Otp+2sRq0Ne2l1StfxwGsXr5rBp1+h2z+daOZPqh75zgl7eCmOijvurYDb?= =?us-ascii?Q?YHqLf7splVWePbIjuDuBgzqNpLxSxMrqb6O1JvtDAulSl3PnUUFWGAhyVDCM?= =?us-ascii?Q?7ns0lN3fQtJRtIRDKz++qtyEJxc80sqnDMjxukf8s4gVbCj8fg7Ckvk8zGKA?= =?us-ascii?Q?9jiPUdk2DeVx7PDfNkx7w8DMHLCU5h659zDn5XuCszXNalzzk8RUrlgBQRMn?= =?us-ascii?Q?AJNMpvTFOEaBqQ/mqfJk6AvKLsjN8Gse5e79r/Z7Q4racTCIIzeLsyopN9gp?= =?us-ascii?Q?9XhVpgf2/LTQlV7lfN6M3c+lHBjRlbAD+7VxNKBFcm8eELrMYohkxLge+ZvE?= =?us-ascii?Q?nn7FLP2JkQxrznabd/KeScaRJWqZrUHMdR4j/yif?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: e43ae003-92c3-453e-7acb-08dcc0756e00 X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB7763.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Aug 2024 17:36:17.0403 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 9d+xN3lTuBQ//JLPr3kHyZGOzqCKIYGFGeuxjNU6XfjPlA737SeVSOegac6jINGG X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW3PR12MB4458 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240819_103625_656636_06DCACDE X-CRM114-Status: GOOD ( 24.26 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Thu, Aug 15, 2024 at 05:50:06PM -0700, Nicolin Chen wrote: > Though only driver would know whether it would eventually access > the vdev_id list, I'd like to keep things in the way of having a > core-managed VIOMMU object (IOMMU_VIOMMU_TYPE_DEFAULT), so the > viommu invalidation handler could have a lock at its top level to > protect any potential access to the vdev_id list. It is a bit tortured to keep the xarray hidden. It would be better to find a way to expose the right struct to the driver. > > > @@ -3249,6 +3266,19 @@ arm_smmu_convert_user_cmd(struct arm_smmu_domain *s2_parent, > > > cmd->cmd[0] &= ~CMDQ_TLBI_0_VMID; > > > cmd->cmd[0] |= FIELD_PREP(CMDQ_TLBI_0_VMID, vmid); > > > break; > > > + case CMDQ_OP_ATC_INV: > > > + case CMDQ_OP_CFGI_CD: > > > + case CMDQ_OP_CFGI_CD_ALL: > > > > Oh, I didn't catch on that CD was needing this too.. :\ > > Well, viommu cache has a very wide range :) > > > That makes the other op much more useless than I expected. I really > > wanted to break these two series apart. > > HWPT invalidate and VIOMMU invalidate are somewhat duplicated in > both concept and implementation for SMMUv3. It's not a problem to > have both but practically I can't think of the reason why VMM not > simply stick to the wider VIOMMU invalidate uAPI alone.. > > > Maybe we need to drop the hwpt invalidation from the other series and > > Yea, the hwpt invalidate is just one patch in your series, it's > easy to move if we want to. > > aim to merge this all together through the iommufd tree. > > I have been hoping for that, as you can see those driver patches > are included here :) Well, this series has to go through iommufd of course I was hoping will could take the nesting enablement and we'd do the viommu next window. But nesting enablment with out viommu is alot less useful than I had thought :( So maybe Will acks the nesting patches and we take the bunch together. Jason