From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1C6F7C3DA4A for ; Wed, 14 Aug 2024 17:10:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:In-Reply-To:Content-Type: MIME-Version:References:Message-ID:Subject:CC:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=UD9xFFCT8bY4ME9Dcgqr7/VW5znu4rIB6+pO4Hzkz1o=; b=aFbXa0ybXP/s/5hC5Nl2OhiKl8 7CXrOrQ0DYiL9yfjb5HKRBskcVdua9G4cPVwfc3IsGIl5+0DFPSmE2IBYYR8QXDwDKGItSPS9hKey 7h3WqrJMTOk8aExLYiody8xPf7lfqiZClIkBY4QG41g9touWJyr2m/nBOgp1l+6lfVwjDcnbqLgcq Be3BeJBZ/ghhx+f7USVpofAtXEukDxaaFYHj23M84AlyLg/LeNzLj0DSIPtG4TAU6fAJhlk2Bqlfb o5O+FRAEinRLT3SBZswlKYp8nQbADORcAMT1R0hWK2htGD9Wux2p8NswaiS0zL2MLmTN4eYPUklML l7jmo+1A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1seHWA-00000007mM1-3yjw; Wed, 14 Aug 2024 17:10:38 +0000 Received: from mail-bn8nam04on20600.outbound.protection.outlook.com ([2a01:111:f403:2408::600] helo=NAM04-BN8-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1seHVW-00000007mFe-3SFq for linux-arm-kernel@lists.infradead.org; Wed, 14 Aug 2024 17:10:00 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=WQHj3S0NZQaNOhmUFkuo1edSiEjYACQlpqO5O276m2CVlhybvTdETp6fKUBbpykt/ys/cnmmheKHXhE3s3a8OkDFX1R94PyxGTrNg4fyS9+2RFtf8TKKMQ1a4tSxVsvcwVDO2cWTmLpqMMc02FKUd4jOwTR9u5kjXZB9Z5d8wilKTE0tVd1wfIYLjEyQEBXHmPDd+PL6epiynb76oVy7jLhvC/hH+2u7jlJtMZLzBK8PmvkxgARJto7v5CsF1PGfi1ilGs6mD27csf5SXqFFUXWOKsKfWk3FlLbE5QTUr36kjqxWAnXT+sASQbWJmPAyvYMhHKHWi5qIzpLRuUvMDw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=UD9xFFCT8bY4ME9Dcgqr7/VW5znu4rIB6+pO4Hzkz1o=; b=Bk2aA7/udxUmcRvFantq9PumQgOZcauUky33XxsIyNuvi5VZGyGTzZiuLHA6KKbg3QNGptWrIQieJKOwR3S68O6tQk0Z1OCjKc9oj1e+JZX2cUgpyL+FwuxdeisLL2yHnU6jKljNxozBroNEqZyANGDM2zZ09obLIQwsI5mylox1+XWiZHYVOie8D/bks/UuAi/5VVf0clZwibeovwAhSpczgVWrX1C8pShMZJQ+qTj1LE4FuWUEXDS6rRXho0o/israqSm/ebOBWYbqoCygBF2Tr5YuOnNG4E4tZzUox0WyHGinGf3uSd5JwGmuvTURsut+NGRm6rd0zED2rL86gg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.160) smtp.rcpttodomain=intel.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UD9xFFCT8bY4ME9Dcgqr7/VW5znu4rIB6+pO4Hzkz1o=; b=TEKAGWit2HKWXqaHKGocPu7ug96yGhFrwRfy4QaBwFwLYV/dTA3agqCUPaSJnkuPt7scKMNy/HniGe5t1Ahp981xCoFmPfKxgxWOVOjJXLybr7tjQQqff5gZHc28uAINfHBNC+n8XRwpKT3FoTyY+8g19QC+oTuBuysEJ67un+Rjs08n2itKC+4AkPF+HDTGvGcDEeYdl4EU+is5OCi2FKQ+F8k2ZaOVIylYC/TohwD2OOfCwIvCcJbSFE3zhMc5fVMcUoBl87asDqYyyqg1lYjP1cVyYrr0WwHmmEsFYf81/ZJvVRDmzcM0j6XuORKsyUhITkk4hB1HliQObSLXHg== Received: from BN0PR04CA0192.namprd04.prod.outlook.com (2603:10b6:408:e9::17) by LV8PR12MB9452.namprd12.prod.outlook.com (2603:10b6:408:200::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.23; Wed, 14 Aug 2024 17:09:49 +0000 Received: from BL02EPF0002992A.namprd02.prod.outlook.com (2603:10b6:408:e9:cafe::71) by BN0PR04CA0192.outlook.office365.com (2603:10b6:408:e9::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.22 via Frontend Transport; Wed, 14 Aug 2024 17:09:49 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.160) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.160 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.160; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.117.160) by BL02EPF0002992A.mail.protection.outlook.com (10.167.249.55) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7849.8 via Frontend Transport; Wed, 14 Aug 2024 17:09:49 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 14 Aug 2024 10:09:25 -0700 Received: from rnnvmail205.nvidia.com (10.129.68.10) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Wed, 14 Aug 2024 10:09:24 -0700 Received: from Asurada-Nvidia (10.127.8.12) by mail.nvidia.com (10.129.68.10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4 via Frontend Transport; Wed, 14 Aug 2024 10:09:23 -0700 Date: Wed, 14 Aug 2024 10:09:22 -0700 From: Nicolin Chen To: , , CC: , , , , , , , , , Subject: Re: [PATCH v1 05/16] iommufd/viommu: Add IOMMU_VIOMMU_SET/UNSET_VDEV_ID ioctl Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Disposition: inline In-Reply-To: X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BL02EPF0002992A:EE_|LV8PR12MB9452:EE_ X-MS-Office365-Filtering-Correlation-Id: 3cc4c600-dfb6-4e98-b7a2-08dcbc83e7c0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|36860700013|7416014|376014|1800799024; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?dENog2mSWI6j/sfkQ3ixoN2Sw6To8U6FUOp9JCjuVkXgxBPpG3Y7I+4J+t1S?= =?us-ascii?Q?FiQ735w4a/xa22HeCfmovdMa16zXhAfMlOdnLkn+XaEfBw9HK/S8xnquFCWw?= =?us-ascii?Q?1ydB38OPcalaxOa4vrwoVISL2Tm2yHf9caiD1E1Y7LYPixV9m9JjbqMA3deH?= =?us-ascii?Q?pE357S1FO773sOt41160XCP/IGZOOpMyc4breqgdBjYf8aS+5OUXgbESONo+?= =?us-ascii?Q?7NjqndnRq/Ugc8IAsHhntUaExF8clqmqo8E4jvD7cJxpS2NDQ8Kpu8UwmspJ?= =?us-ascii?Q?2QfGQg89bN2YkcfOqJ2TCfCAiNnsiX5Yq+pcVnXcc+MeL4nHi5kCc+35ZPkk?= =?us-ascii?Q?Zh9Mi6AfF47GyhHFXRh07kY/aP0K11ZiIMK2Sw3+leX2YfyttSyvC4r9ZRzO?= =?us-ascii?Q?0aXPj9m/40HPotP6xw7XhFSLQgSPr8in5MZHS5LMEArg34ZMvc6SjW89VwzN?= =?us-ascii?Q?5eZopBRTTaeqjHmf9yeVbyFVVUlidrxH8rdAUFvN8WdOSwtA7cyXPOusXONY?= =?us-ascii?Q?+gpXzZJ9Yh/JdRIEYUANQxrhY9rIru+xUqn+k5pwUEQRi44ZgdHivwcZQZgD?= =?us-ascii?Q?XLeNhFN/gXy2DZ1xWMZQ0aZrpdN7/pbJJLo64pb7zdN0x3NOa2WZpThP24sy?= =?us-ascii?Q?O357LhftuyoJvZnt+xDthXwT+wc4ozf6OMB+dm79u7WecUdFD0o1D6aWIJ73?= =?us-ascii?Q?eHKy34c0Zc3xTg1ZlS6C/7NZ2PiqcGL+YHI9Ee5NXJTCzXrc/XtY0VjvtIw0?= =?us-ascii?Q?3Tvhimr3/rhB2g1j+hUfd4SKobCfqLEFdJ3226ZfUEqXlXOj8nhimis+fzdA?= =?us-ascii?Q?AVu7A594P6iVlbBGHjVdrnT9/isfVRtymofXke+e8buT5tVr5uYaSMyS1eK7?= =?us-ascii?Q?TNRrYOjAyIOibbFXCugdCM0pmySmYsqU8Qp8/nzSKJlQH50qaoa6nt0VmL79?= =?us-ascii?Q?SMA7e7buppL0CwjHPKmmi7bVowRo/g9+UXguH5e1IDYfiIqQ6pqhduOljD9w?= =?us-ascii?Q?F9ciqSutMnt41RkeHToIjIV8xuKm4LktKJBQmWQ372aE1eHPsCZmWO/Zwp9Y?= =?us-ascii?Q?f/wTzOnlWzlDYjbku2TMjxfaBgEPXAscRfLmTkxHSAVFuyjPEaim+tc7jHNm?= =?us-ascii?Q?JlSUPao1oNWj7AntyAvdHLh7mOnrhUhgX5LpMdOQborA1gR6HboPpvD6ZDwy?= =?us-ascii?Q?PuNs3VgTmPCQ08pz0ELfv5/juj+rauuQyJF1r0QoHsTcDdLR7O8r0IXPK5zB?= =?us-ascii?Q?bJZ2RnN6gacbB5mivU2NxdTWmQTMXVBJFo7cTRQoiTBTSYomWAYCAoBQ7Dc8?= =?us-ascii?Q?sTIaSJX3mpbtrwSk2qL4xGdopXjlXzqg4BcXtQ+Vym7Gt2KSdcZ6H+st304h?= =?us-ascii?Q?kRYf8+N8LdAHHFi3Dg+bXgW27JT5YqQgWTT+NwFicPfojryn9s09A/07BL8g?= =?us-ascii?Q?X4T7xvvOTpbgEpw8lZoOFofde8GuOhVz?= X-Forefront-Antispam-Report: CIP:216.228.117.160;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc6edge1.nvidia.com;CAT:NONE;SFS:(13230040)(82310400026)(36860700013)(7416014)(376014)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Aug 2024 17:09:49.2797 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 3cc4c600-dfb6-4e98-b7a2-08dcbc83e7c0 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.117.160];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: BL02EPF0002992A.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV8PR12MB9452 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240814_100958_938004_CE5F8606 X-CRM114-Status: GOOD ( 13.10 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, Aug 07, 2024 at 01:10:46PM -0700, Nicolin Chen wrote: > @@ -135,7 +135,14 @@ void iommufd_device_destroy(struct iommufd_object *obj) > { > struct iommufd_device *idev = > container_of(obj, struct iommufd_device, obj); > + struct iommufd_vdev_id *vdev_id, *curr; > > + list_for_each_entry(vdev_id, &idev->vdev_id_list, idev_item) { > + curr = xa_cmpxchg(&vdev_id->viommu->vdev_ids, vdev_id->vdev_id, > + vdev_id, NULL, GFP_KERNEL); > + WARN_ON(curr != vdev_id); > + kfree(vdev_id); > + } Kevin already pointed out previously during the RFC review that we probably should do one vdev_id per idev. And Jason expressed okay to either way. I didn't plan to change this part until this week for the VIRQ series. My rethinking is that an idev is attached to one (and only one) nested HWPT. The nested HWPT is associated to one (and only one) VIOMMU object. So, it's unlikely we can a second vdev_id, i.e. idev->vdev_id is enough. This helps us to build a device-based virq report function: +void iommufd_device_report_virq(struct device *dev, unsigned int data_type, + void *data_ptr, size_t data_len); I built a link from device to viommu reusing Baolu's work: struct device -> struct iommu_group -> struct iommu_attach_handle -> struct iommufd_attach_handle -> struct iommufd_device (idev) -> struct iommufd_vdev_id (idev->vdev_id) The vdev_id struct holds viommu and virtual ID, so allowing us to add another two helpers: +struct iommufd_viommu *iommufd_device_get_viommu(struct device *dev); +u64 iommufd_device_get_virtual_id(struct device *dev); A driver that reports event/irq per device can use these helpers to report virq via the core-managed VIOMMU object. (If a driver has some non-per-device type of IRQs, it would have to allocate a driver-managed VIOMMU object instead.) I have both a revised VIOMMU series and a new VIRQ series ready. Will send in the following days after some testing/polishing. Thanks Nicolin