From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3B22BC52D7F for ; Thu, 15 Aug 2024 19:54:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:In-Reply-To:Content-Type: MIME-Version:References:Message-ID:Subject:CC:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=eSlj4B9V7CXp7a3yeu+bc+Drt/hj2jA9ioL2qIyHoDg=; b=ypEw8+DV+DIVTEJ+9O4VEJ1fDv bQT1Xbz7ZF7TyxR4v8nk/94qrtsPdcx0Fvg5OPISUVN8mUSwYlkm40WvBmGR4GW1Vj22LMDIwTSNv I1995sEeMZy8oxkOiq+etpAh580syBlT7ETTmT2Xx3pSsntGhLoz0PaKvU8pF3BstF+zQ0JVCbBs5 wmseHygJP/pK1tV+mnCmReGmGMZur8qHSEa5qf1GNDk/KM9ayfw94rzRvxUy0ohnCMAIIrrq9sKu3 9j37d6XbCERwNxakKU3rOMg2XfK7or+F2FRLmHfOMM/Yj/I1xaCZMALHSjRJvZVm4O1AgIyd/hxlj 6VlNiDzQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1segXq-0000000AwA3-180w; Thu, 15 Aug 2024 19:54:02 +0000 Received: from mail-co1nam11on20627.outbound.protection.outlook.com ([2a01:111:f403:2416::627] helo=NAM11-CO1-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1segXC-0000000Aw3u-1cAA for linux-arm-kernel@lists.infradead.org; Thu, 15 Aug 2024 19:53:23 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=dx0aR4F0HVCGZvCgLrzGisFZvCJO1UkYGOIQTin64aX413XxeyXjNmOe3x7ZvSeIyukuGJR4Oeb1imi9Fawkdfo95LiIAawHIf1gtvdyUYkPgBefP+HQ/g/kSO3T59VzTYe+DXaD8zEM4PW+T1qrMAXlGf401SHUpqSabrg5zTGG5wYeeBqDL2DIZFrPXYlnraEZW6vYrJ5Z18x+2h4kwdIfbFvwuOdBlFI1mkJyPz4MfOZOKFzZ+AeGv1J5LQScYAJOqpnE3wj6/fDB/VmM4WuXYWWrKuCxqatUyFTH4pO9RuNEe9LH83DPjIsQgLWM1GVh8rlxp0lFs2RhU8z86Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=eSlj4B9V7CXp7a3yeu+bc+Drt/hj2jA9ioL2qIyHoDg=; b=Xni7wj1TZfHMA0FI+LpIjvYq+VwCwnAOlyb7gRQcNstndwRJ7rHAiVjgwOAzYwgn+KyD8c7/mwF3muTOdrhAKmp9O4FEq+CKhOTThbiEfR4JIqB548q4B12k/bUY2UWqJyF1g0EWJ2zOwbXZMu0QM0QxLkV/+MAgjhbhoCHd+syQTkg0PnEs9YNq9qcCTEMYE6yYWKI1AsLIYQWXL/4xsx+xavwDF1YQUX03cnROsNhSgzCBwdv8fjzRKW8trdrsddt0hVnsSyQPUuqsIG3BwP6byQZGG6Udt9T0pVY8RSbLNUG5msWWH/du8qk2fo2qcNzirCi/HcfVB0XsaE+iVw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.232) smtp.rcpttodomain=intel.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eSlj4B9V7CXp7a3yeu+bc+Drt/hj2jA9ioL2qIyHoDg=; b=MHA0wBSqHMyt0lzB5aKMh5xy2/KfMXE2QazqOp4wjO61zFfQ2ur6dYalcL6Bn0eRHimbb2cOW4Z7SArUgbns81AlY7EEJzozu5pPsGHtOPJIrik+esb/fDm5xMUSOrZdI+kTY9cMlWOcko0MKfwrWadUypjEyRPqAPOCmbUQuyxnhl6OGtsuOmZwJSqk+04uRdrU1CCe8rFeU9epNwpfwrsx0YHUCsRlBZKQ4Dfqix/W6CYFxHiDto7ytUm2JcELbP7Oaxl65KktTYx3umo8nW8UAUunv9c4Ekz99Xahd1oZeD1E57Sprc98uVWz1y7BcHiD809MkTTfNOeOjm3I7A== Received: from SJ0PR05CA0113.namprd05.prod.outlook.com (2603:10b6:a03:334::28) by CH3PR12MB9283.namprd12.prod.outlook.com (2603:10b6:610:1cd::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7875.19; Thu, 15 Aug 2024 19:53:13 +0000 Received: from SJ1PEPF000023CE.namprd02.prod.outlook.com (2603:10b6:a03:334:cafe::a2) by SJ0PR05CA0113.outlook.office365.com (2603:10b6:a03:334::28) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.10 via Frontend Transport; Thu, 15 Aug 2024 19:53:13 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.232) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.232 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.232; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.232) by SJ1PEPF000023CE.mail.protection.outlook.com (10.167.244.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.8 via Frontend Transport; Thu, 15 Aug 2024 19:53:13 +0000 Received: from drhqmail202.nvidia.com (10.126.190.181) by mail.nvidia.com (10.127.129.5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Thu, 15 Aug 2024 12:53:06 -0700 Received: from drhqmail202.nvidia.com (10.126.190.181) by drhqmail202.nvidia.com (10.126.190.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Thu, 15 Aug 2024 12:53:06 -0700 Received: from Asurada-Nvidia (10.127.8.12) by mail.nvidia.com (10.126.190.181) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4 via Frontend Transport; Thu, 15 Aug 2024 12:53:05 -0700 Date: Thu, 15 Aug 2024 12:53:04 -0700 From: Nicolin Chen To: Jason Gunthorpe CC: , , , , , , , , , , , Subject: Re: [PATCH v1 05/16] iommufd/viommu: Add IOMMU_VIOMMU_SET/UNSET_VDEV_ID ioctl Message-ID: References: <20240815190848.GP2032816@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SJ1PEPF000023CE:EE_|CH3PR12MB9283:EE_ X-MS-Office365-Filtering-Correlation-Id: 5027e89b-c9de-48e2-d889-08dcbd63e5e3 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|82310400026|376014|7416014|36860700013; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?LqOIVKWitqqgb7qppLiuv2j87bzIDxkmwhCiqTCKO/7bXCWOkcTxLezlCnHX?= =?us-ascii?Q?sU5/+RqippxP6a30bWwIgZBVx494cXD7u3zfwrlRatjS6XQVoCEde0B5omRi?= =?us-ascii?Q?2BIMxSQn39P5gpFHxlW5dxz4MTEFdMCBWWP1M1fusa607CtBp2QLNjO9I1QE?= =?us-ascii?Q?YE8Yb6D+bDwBg9rMWinTb4+q0CChhnebyrrreJdJfR1dq1M1XpifhxOxFZbM?= =?us-ascii?Q?N0Pi+9QEV/CIaFchCMd4wl13dCt6vDhWlvgje6w1xfXyBzPICIvG9haon1ae?= =?us-ascii?Q?D608XaanIBf/D2h3z2pr6No+9hy9uRS1DXDQK8bmN2qzM/EbgDfGmWaLynGx?= =?us-ascii?Q?8ByWzheD0d27Zg3HAsr0gadHouekj5I/HoAaejx0fMeyPhY/NLk7qdplH7/e?= =?us-ascii?Q?Jm73w+Q9SqX333tkHKafVpIlQFA1P1N3eLRG/oUznhAm0yoOvpwMKDBRtGPg?= =?us-ascii?Q?uFasaIgdyZE12+KPTWeC2bSNy1S027Qz4p/+PkDHY6ojHYZKOypVe73duAXh?= =?us-ascii?Q?pPKg4VEQmOkRpIslNa3b0S9bCqfoyXH3fwO4sdzZX8+X4SkZtHb1GdAQm3kE?= =?us-ascii?Q?rbGUqPMCD1Fy4iu5rjsb1Cu4qrGeWi/pTn1lDagAhpq6zvhEVIwco+Jbw1x2?= =?us-ascii?Q?qTo15aujaWyu87yZn/OsKn2lEEXCGRTnqLSgp8xMHZZl6tTtkmv+rhMPvxga?= =?us-ascii?Q?5Ew2ntmH9vOb7THrDNBLAH1TLaW1bEJyRRiy3J6DR3KkYm2knDB4tRvItsu0?= =?us-ascii?Q?Bfrvi3XP1K7LznMvaMUZGAH83OeZgYxMqj6L7oKUEoMcFUS0aCL/JLgmlyu1?= =?us-ascii?Q?BsWhba7abeAOS9/HhTDO7bVdsM0kEtLnqzZTHprzXdJA8MgO2zoadoaN4fDO?= =?us-ascii?Q?QJcJ3RrBT30AgUpcoZdAqEdwWNnPaT0p2HHBKGFuT0KgcJBGzCwK9Yx0GF5P?= =?us-ascii?Q?UW3qqBO3b9MBDyzLmV3aQbnpxHSjtWoz8NpQwV75i9DbYC8y/WUKaQUNdZFZ?= =?us-ascii?Q?De6e56anUgZjFjJXfJeYpDNgmPMWO/YbWHOJznZiN9Q7BibVeY92BOf+Vx3g?= =?us-ascii?Q?DSkQk8myj3tLY3TmYlHIK83pOWgPXzSJ5e3DrMfJkVwThA3wDKin3aCQH8hr?= =?us-ascii?Q?akatzTp91v2djqhqAcHowpK5SG57XoDvDgn8cvHIoEPb18xXjJ8b/T8k/fRm?= =?us-ascii?Q?YWLozzm2p8AmTCF9GuAASBVb+y4V8VFAr7SactuhAHwfs04xdQuJunL+NUM4?= =?us-ascii?Q?Ke+MrjyvShk23kB7bmgVIdpmByF9aZ8/04CTgzOFBhAnN+/YDm5ehpgq5j9b?= =?us-ascii?Q?R6hjB5gpXp2m5ctYA64+acLDiaC9YQ0BUyIzcK1RaUgfpVYpuI3zs+y2aqL5?= =?us-ascii?Q?i7xSwwbQgoLOXuEXM+/9UqoM6nheXh4zWVFRO2eSDteyL3oP3CibWlTgehrn?= =?us-ascii?Q?89xXawWyLUdI1a/zDtmGlTCL7v0J+S/T?= X-Forefront-Antispam-Report: CIP:216.228.118.232;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge1.nvidia.com;CAT:NONE;SFS:(13230040)(1800799024)(82310400026)(376014)(7416014)(36860700013);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Aug 2024 19:53:13.5781 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 5027e89b-c9de-48e2-d889-08dcbd63e5e3 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.232];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: SJ1PEPF000023CE.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB9283 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240815_125322_435715_BCB21532 X-CRM114-Status: GOOD ( 19.58 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Thu, Aug 15, 2024 at 12:46:29PM -0700, Nicolin Chen wrote: > > > +static struct device * > > > +iommufd_viommu_find_device(struct iommufd_viommu *viommu, u64 id) > > > +{ > > > + struct iommufd_vdev_id *vdev_id; > > > + > > > + xa_lock(&viommu->vdev_ids); > > > + vdev_id = xa_load(&viommu->vdev_ids, (unsigned long)id); > > > + xa_unlock(&viommu->vdev_ids); > > > > This lock doesn't do anything > > > > > + if (!vdev_id || vdev_id->vdev_id != id) > > > + return NULL; > > > > And this is unlocked > > > > > + return vdev_id->dev; > > > +} > > > > This isn't good.. We can't return the struct device pointer here as > > there is no locking for it anymore. We can't even know it is still > > probed to VFIO anymore. > > > > It has to work by having the iommu driver directly access the xarray > > and the entirely under the spinlock the iommu driver can translate the > > vSID to the pSID and the let go and push the invalidation to HW. No > > races. > > Maybe the iommufd_viommu_invalidate ioctl handler should hold that > xa_lock around the viommu->ops->cache_invalidate, and then add lock > assert in iommufd_viommu_find_device? xa_lock/spinlock might be too heavy. We can have a mutex to wrap around viommu ioctl handlers..