From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 41528CFB44F for ; Mon, 7 Oct 2024 17:45:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:MIME-Version:In-Reply-To: Content-Type:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=JwxBJ7SS7rJv5PazIHyCOhehomu7lxutMzzsbAtPd4Y=; b=4CL1WRrVEd2CAHITN3ZtjUQn+R rGz6IK4LyOTMQ1jnjWwdglHESFORF869gsv1SvFZodnwKtWccNl0tWr8ZW3uGj58Uo3RIJn7kJ1GU GrpLxNdAG8s10FONgiNTWZfafZYUeHa5ilSiTKpql7po6s1xDcBTI8abPPD4Wc8nOka0s3E7p4VpY 7diOBrWZ1m+HWNix3nYXELLow8lr6yCrbBYTC1pE2qeCopb33uyVB0Svx+lIpKZFapzfo/ehs2gFL 3Z00KukBdPDr3SjY/iBMfmSn2BAkVUZVE3QapeB7piC9FG4U/y+lWVUOT6wD9T4OKsnSnMpyursT1 YbZx0Sqw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1sxrn3-00000003N2T-2IFZ; Mon, 07 Oct 2024 17:45:01 +0000 Received: from mail-mw2nam04on2060b.outbound.protection.outlook.com ([2a01:111:f403:240a::60b] helo=NAM04-MW2-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1sxrlj-00000003Mgj-2tsD for linux-arm-kernel@lists.infradead.org; Mon, 07 Oct 2024 17:43:41 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=YPQ9A8lG2gg1HH+nZ0U2J8H8SlL0yeQpG+Vkkc9XNEKrqfPzGIleeSBCdGrHZCZ4bw5MsoO2IrbJGmxJRVmONj4sWfvMS52jlJKeHSCzlt0dykkImW5OhLunDgVS6o46Usk9WZ0z7CPMj/JbrxbJ1Yxtjrhi9vcLP8sfm5uI3TRsrmU+0vqV2GHlz+hNYDmS7Jbv8Gf6UrRHdcIim5VRwaZe4+MIJIeQK7/GVgZfv1QA62ngtNzU8S4ii80SLcZ72F5lo6a7+If11RiU9rgiA9qeIfKBj64E14aLnUSGSX+ktBUN5yTII5aWhlJgk5m0E1n9yNPMkOE/h3kJGTYhTQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=JwxBJ7SS7rJv5PazIHyCOhehomu7lxutMzzsbAtPd4Y=; b=XnOTrAyT6ovD9+cIzpobRQoxTZ3DgqCkcRRxYsSU0WCsCGGVgdKDu/5+miu9iRHMhYczlbiLgRxvBxQC6qWUjHsAVRRNNsr8NTG4hEukR8d0Q0RbXmg+WLa5+bnKkOGQeKg01ztzvQveAARkGdhxVgS3CK+4hGTsgk4dSGkZrgBmMzRwjUwCwn2ErtG1h60MoSHmv7KM/7n1FATPPDKRaLCPCEHKebSzvjvCh8qnSK1n6allj112oGmf5UtI2fdHkMuefVHtovCAFBaItgNH8RGyOOOVwionHkuahWO2CseNlWjvvsWFzbKMY2t50n0VexEHyQb6oU0bijQQMqwpYg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JwxBJ7SS7rJv5PazIHyCOhehomu7lxutMzzsbAtPd4Y=; b=NKs7Pqx+8IpHOhRcdmdOAQpb/RXKyA/SYp44Io4s9OPqUGRNAsKUKoTVKXgYU1AKCzhY5syzvWauOx0VxkLHCqQgGnaSowJWZzEZO/PcxvkjuH2hAn+HPm57leknnWtCC/OoXokClscHUrliqyPMcJkZ6e0ILXKULn6w3AZlVEuRKvIhCKbrXN5+wroqU6HrfKnfXzqtxEZzWjfy/wgLjFyDIo9bnt/AZ6ajaOxtMi0KUydlZy2EDTC3m07W63r/7XjpLBsR5ZjgQw8AKBLIWgIU1CwCP+8o9N0Bmo5zVGnzZbuiz4sT2H0yBVD3lrP6KYoFncZ/oH7bBjzCCZZ49g== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) by DS0PR12MB7725.namprd12.prod.outlook.com (2603:10b6:8:136::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8026.22; Mon, 7 Oct 2024 17:43:24 +0000 Received: from CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732]) by CH3PR12MB8659.namprd12.prod.outlook.com ([fe80::6eb6:7d37:7b4b:1732%4]) with mapi id 15.20.8026.020; Mon, 7 Oct 2024 17:43:24 +0000 Date: Mon, 7 Oct 2024 14:43:22 -0300 From: Jason Gunthorpe To: Will Deacon Cc: iommu@lists.linux.dev, Joerg Roedel , linux-arm-kernel@lists.infradead.org, Robin Murphy , Eric Auger , Jean-Philippe Brucker , Jerry Snitselaar , Moritz Fischer , Michael Shavit , Nicolin Chen , patches@lists.linux.dev, Shameerali Kolothum Thodi Subject: Re: [PATCH v9 02/14] iommu/arm-smmu-v3: Start building a generic PASID layer Message-ID: <20241007174322.GC1365916@nvidia.com> References: <0-v9-5cd718286059+79186-smmuv3_newapi_p2b_jgg@nvidia.com> <2-v9-5cd718286059+79186-smmuv3_newapi_p2b_jgg@nvidia.com> <20240702145705.GA4135@willie-the-truck> <20240709193905.GL107163@nvidia.com> <20240906140853.GA17187@willie-the-truck> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20240906140853.GA17187@willie-the-truck> X-ClientProxiedBy: BL0PR0102CA0072.prod.exchangelabs.com (2603:10b6:208:25::49) To CH3PR12MB8659.namprd12.prod.outlook.com (2603:10b6:610:17c::13) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB8659:EE_|DS0PR12MB7725:EE_ X-MS-Office365-Filtering-Correlation-Id: 47ec1f89-bc6f-49c5-8aea-08dce6f78aaf X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|1800799024|7416014|376014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?vg1RR+CaTz41+4zpHkcz4w6SnKChMJTzEJUv9IsodsNX2RXnvymkVNLlfWFn?= =?us-ascii?Q?W63ttrG87MFsqH62qW/Ra7oOvo1kiBA/xt3w2RY5LLZVxHvfwFdbQ59FX+KJ?= =?us-ascii?Q?9Pd6Rzo2aivX/nSX/Ij8dwz953nm0h7FRyOngEkqqMEBzpc+FNpyFVkbJQSl?= =?us-ascii?Q?Ym824sw3vunU0k1IMoBgRMfQ+Mumbc+RjxJiubWBh4T2nVzp5ci4hNGUX79/?= =?us-ascii?Q?FAVIABTw6bsv7xN+/LB6c+e/YNUZDmFYFH4cGzasNOxR252M4wJyC3iPRZUV?= =?us-ascii?Q?16buat9cWMWnxjeeIeRAcU3HkLJgHAKcXA8Bj+RCOy4QBu5XNr/6V4tlVgFL?= =?us-ascii?Q?4VCIix/QHN2wngCej52KDWN32jXZb2XJmpeC3B9XY9pQT/r6qJOVSd+BAb9R?= =?us-ascii?Q?lc5uiDqyxC9ER76n+Q2U2i2lQgzNjK+yQ14qs9NUY1QBGFwOzIuTUqAqRc7O?= =?us-ascii?Q?EyowmTOqOtSf0smaTRrqpvMgt5KvkPgt6I65gcaTXPCDsqWx/ZTMR2afWJ/t?= =?us-ascii?Q?oRgL5cjq3CRsYaH0jMBsMhHVNPRM61Pc4rD4Yk7lYysEHaFZAIwKG0WzC/r/?= =?us-ascii?Q?DEpvPzPxxOy/pmMjnoCtV3QUqA3DnfeSZsw2o1w2F6+4aJ9tmtLzs71go83W?= =?us-ascii?Q?+XhCKTTPrurWqIkGjYg2K5fi241awTCb0v1IXoNklWt91Lm4tCQ0Yg+cETBr?= =?us-ascii?Q?GbgiVS4f/eMrgqvMFcNMfyjpO9IyEYPtJx4tx0Q7ckS+u8gE+6eizslYEnwR?= =?us-ascii?Q?IHSLkvus2CmXxPL1vqwQDX3T0bUMbSUqec2R6JeiqaA2rhJyBay6MRuGV+1O?= =?us-ascii?Q?CnxNFAqHFmgMZ7sQORkTXG7iKg6clg0bS9Hgru6BANU5i30B1VjWBejeY/ya?= =?us-ascii?Q?OYa0caUepde7RBic7uJdtW80Z259pfwuBNMEzBuj+zqO2SzxFL0IHmbMW7Ev?= =?us-ascii?Q?wu+FuskkN57Bq3i9IGwUpj9n5xoFqbZz5sHKFci/5BNUxflJoHO46mLsDof6?= =?us-ascii?Q?gaWwIf2MDqXFt8PGJJeCiNANxhWCFmToZBxfitLcjeljyIsKKTGpBfSxEHLO?= =?us-ascii?Q?WNtmIga+sjIdJioFKkGEFoK8Jmi2jKX9vMduVc4jlYVtxi+u52awqa4cYjOh?= =?us-ascii?Q?/Rg/p2BVQsO6A3ohxPH9gIR2LhJ9+Esw7hahXYhWh5J7gnlZcsPeCxeKCebE?= =?us-ascii?Q?IA0Ayun7jbUbhFvOejbSnblez4MGMzMiqWBkDQuOEdw10U/RK/hDJqchZPrl?= =?us-ascii?Q?hmOhVyO8Pgl7rzM0W0yWHyIJ0Y8mDL22C/7CaSR/Jw=3D=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CH3PR12MB8659.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(1800799024)(7416014)(376014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?o/EC9APNDy4J6kWglR785bRKzYRb3IhFjckg6CkOwaUXhsdTWpJZfXHRTlti?= =?us-ascii?Q?MBkSFUSiTZ+4CU3J4m2Q3HZzJmuwaLgu0jlLWb9np6GmH+rfVQAlcuhtKcA9?= =?us-ascii?Q?KV1YtpvkGQFYtbqkHyZ/8b9kWoYVdnDtrOESG+62E3nSpXnnH+EpgKfiGB+A?= =?us-ascii?Q?1piFj6S5tCYSR9X6+TNIgu0LKF85qRcy6YdXsQ7AmUHHb4graOaP0IChh1Xc?= =?us-ascii?Q?tl/Xv+tnKwAFCEwfk7OeihcmqQClANt8Nryt9fec4XCp3azpovb1G0lqe+Cs?= =?us-ascii?Q?zJ/ImxSVp7FOnvGFMZ5NFPSNZg0JLKoVgKqZUPiwcgdGY0cQbhIvfx97CV7U?= =?us-ascii?Q?QpQKM6i6t9MtlqmxH3r+/pjVt0qeiDfdKhn5/vMQLRxM8DjvMdRK2QM3MGgu?= =?us-ascii?Q?UWv+wrohkSn769GCLIPvAAfGSPH4j/UZNRerB9RifgVcgH5KvMuGb289oHIm?= =?us-ascii?Q?iGKee9saG93M+0V/9HgJ2wnQuVZbI6xOtc5W7miP/gMu+gYhaLn6IAzr9tZY?= =?us-ascii?Q?jWrNhgDT+nnn3bm7s69/aZJiwXnkQp61WLK3u+G9uWFABIxmlsnI1DUIvhh+?= =?us-ascii?Q?adjiVDuPRizAV8aBMRK0pdC7lyOqtns58AFt1YVobIA9KPD//8EinIhojPu1?= =?us-ascii?Q?t24Tx7eOlVtU1CSrxi9+BW2oS28llin5KL4G8kWcYi3lFeqWhxwKTJRuFQZu?= =?us-ascii?Q?T2PYLA0QZX9I6URPAfXXh9LlctHAUXRFhkUjdbN0pQqbzagLZQHp53yenQPY?= =?us-ascii?Q?GbT6Zhkk0svipaw/mTi4vnAWJJNlKW+EkcWPg/cTyPgIufdgvhzn/HSjFLp/?= =?us-ascii?Q?TQ3MYQ7Z3vLgMmVnZjJ5LVwV+kHPI1Uv5vV+pZs4GM8IRBNzffP/HR+aHG9q?= =?us-ascii?Q?848kf05n9WkTdFASDAjoS5WmMvW7hXMLA3jnAAKuH/hqzFbRzTiNzQq8BAST?= =?us-ascii?Q?LfLkVcSwGmliiRoGFo44JVnqh8pCkG4lfy57+3XM96O8NcEjRp48W9RNPszV?= =?us-ascii?Q?1KKeBoStAYjWMu89VLy0R2i03NAdHLdB4Ptt2QuaAvmL/qNGpSVlWFQ6Zdy5?= =?us-ascii?Q?PzIfdX1cL6QOfsTUQkplQepCcOnqOq4b2Lpa7gmczipYq/iuLjw5VFR9rvRm?= =?us-ascii?Q?dqx+6rFqa6lD2KbvGVqFeAJjHbuVB8L5Mk4GJCHyta7EVE2pH9mULiSOlsZx?= =?us-ascii?Q?IK+D9teeA+wGijlfid7Pxs06WbxTGxPgMHPM7dP3/xw2THWj2L8IHUSNsn+4?= =?us-ascii?Q?qDcr3+umSkwCOFhuy+QMyUUpM+LGpD2hI+3oIxXXI2rlRCWcxE+sqTFal6tE?= =?us-ascii?Q?dqCCgmW6++U7xMeELDDgj/g/fEo3H0dpzghISB/jThB6DT1+ZIBjzU8KqFdr?= =?us-ascii?Q?NGoDhXjoqCCxdnCwpLrVwlFzUlWt2D37ehEo3BCZ4BjQXUfy9htdJWWGOcYY?= =?us-ascii?Q?+Crvq0H+e54nCw5oDcOEKGh1E0cOO6Cc7CDjT+KqWwjA/lvoYbrA1/WO2hL+?= =?us-ascii?Q?A+EkoofX58fjNhOk/rYYk4bzimtmp/uED35JRXfPN7YK5sRdZqPldwk/cRsH?= =?us-ascii?Q?ech1/pIAgrsGOgQCA9I=3D?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 47ec1f89-bc6f-49c5-8aea-08dce6f78aaf X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB8659.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Oct 2024 17:43:23.9502 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: blQiEcQxvjr4moK4wnfDlLDlx+dMR1MCwaiw4mykWdq/qddntVwQgYBFfU5zP59t X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR12MB7725 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241007_104339_756959_55191240 X-CRM114-Status: GOOD ( 16.62 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, Sep 06, 2024 at 03:08:54PM +0100, Will Deacon wrote: > - We have a bunch of comments around 'arm_smmu_asid_lock' that refer > to arm_smmu_share_asid(), which no longer exists. So, we can go ahead and put back the BTM support in the pre-existing slightly racy form. I think we can turn it on for guest support by operating it only if there is no S2 support in HW which would make it justified. Or we could delete the 'arm_smmu_asid_lock' for now. My idea to fix the race goes through making the domains sharable across instances because that will change the invalidation in a way that lets double invalidation happen during the ASID change. I was planning to wait for that, but it looks like it will be more time. It is linked to the viommu work which needs to go first. > - arm_smmu_attach_dev() takes/drops/re-takes the devices_lock indirectly > when it calls arm_smmu_attach_prepare() and arm_smmu_attach_commit(). This isn't retaking a lock, the operation being locked ran to completion, we just need to run two operations.. > - arm_smmu_attach_dev() takes/drops 'arm_smmu_asid_lock' via > arm_smmu_domain_finalise()) and then re-takes it before the attach. finalize won't be called from arm_smmu_attach_dev() very soon, they are unrelated operations. > Please note, I'm not saying that there's a bug here, just that it would > be easier to work with if we had some documentation and lock ordering > assertions. There are more assertions than there was now, I think most of the new code has them already, excluding the group mutex. Jason