From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A17A8CFA745 for ; Fri, 4 Oct 2024 05:35:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:In-Reply-To: Content-Transfer-Encoding:Content-Type:MIME-Version:References:Message-ID: Subject:CC:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=CcdpJE9jFr2Flrs6gJDDo4yPMIAU+lNdut+ayE3A3NU=; b=flAyEKYiMaMdKjMCgtiqy5V5/c fnK2Dxz7Uri+RbbRnE4xiRXhq4h0TJDgDhsbpv1SfqMmoCmtEzn2B5Do4BY7RIk3Q7wY+gNrarLF+ ojD0vx+4jzJ29cjGiR7cojIAfOEbZcERYgLD2l47E8zxYx+uw0a2KnTMhku+3yuXrScjaPUbDUor+ nprnt5lCKiT+61wXyojsCdY15euhgZkMRg8zQ7o+eNPcOmLt+z0MBuCn3Y4PC+CaToAwFRZZEiqiP cYXtd7ek0zh6M2T+XAxqf9hHCszXFX5DRR6L9lY1Dh0IPpor3aG8IqNZ3QeRCf2D/ryMSnkB33e9M 2gTAcm7g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1swayQ-0000000B4fv-47Bn; Fri, 04 Oct 2024 05:35:30 +0000 Received: from mail-bn1nam02on20615.outbound.protection.outlook.com ([2a01:111:f403:2407::615] helo=NAM02-BN1-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1swawX-0000000B4Oh-3end for linux-arm-kernel@lists.infradead.org; Fri, 04 Oct 2024 05:33:35 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=IXTDYPOdQtl68kXBPGS+4fzEW69Sga+675VXxwumhO2auSRvrFjBjVa/xybJIfSEyW8z1KISEfXKLUSkHuan077UtupJuasKFkGzKgXQZQdD5IN3O0/REOOQvF8Y0cnZNPlZZQGAF6frjDOyUvHQEmGo5oqdonSckDLV1uKMPlGcBCNfOYUNigW36T2XcDUrXXq9RzYL/8mJIsVit/3wXKh1IlINuM9ShFukF5jAJw1z8rrRVT4cjpffOP9CzzcZ+OGJ/1MzdDcjvI+AxBxBVcSpALAOfMYfLOkk7K0Ey6JYZBoYJYZC6SbA2s78G9AR1to4nRzIuRluyQhPwbh15A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=CcdpJE9jFr2Flrs6gJDDo4yPMIAU+lNdut+ayE3A3NU=; b=uiF7QruESjBWwL56lZmwfyv8PMdFIIHHSbNsFizpFSRpTQ4Mz5SbWVAPY4LBKhwxLc/dkN8tLhxS6JZ9mYnZSq38C2i0nR070eO9lCuvgoZHOt7n9E2GWxjWe+JcAIMfa/RMAt6rAncOa92yRD941PH6fjnIu8oLM6As2kmRvgBI6bVneAKn/An7S9si+KZczbnUrWq2nr4Nvgt863xYgfo6yvdYQRltS3gOdalYncdcQwNWY/ispM7nkfMnOW1hr9kn1VG8lBrgWeRMtWzTzI5peQjZgvpHTiG2CPA6UO1fpUsJxoo7sE3BsBZ+dXt+2XLv9ITU8NM8iLY0OWo3Pw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.118.233) smtp.rcpttodomain=amd.com smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CcdpJE9jFr2Flrs6gJDDo4yPMIAU+lNdut+ayE3A3NU=; b=nBpC0nileSG9B2/bLNSIFlR4sDE+aqFYwYQt2W9UZRN/BuV+W8r2uwkqjOaLao7Gdr+U7dYhePvJkPGvFL/eDH8G0bZ6Trw8+GK6CrhHEtbu0cyBQL6eP+xjSSXZl7HeMp1Z1O+XZMRhdg0J2ck4h0bCFG9zCj+4Okien5qJ6MbkXO9KTPRCqgsx93HO4nmdq4Byr+JYnUAtpJFsVxAuSyPtg/sh+O4epaTNbBPV0o4ZpVgDVzcY5DUaf4wIUaA+7DBAw6O4+EpbkdrS9ccsuxU/lFYPo39GcF4a0aNQf8+kJFFEL/mowxQ+lnHlIztOSdAUO110gA27bpA0E1P3PA== Received: from SJ0PR03CA0056.namprd03.prod.outlook.com (2603:10b6:a03:33e::31) by BL3PR12MB6593.namprd12.prod.outlook.com (2603:10b6:208:38c::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8026.15; Fri, 4 Oct 2024 05:33:23 +0000 Received: from CO1PEPF000044EE.namprd05.prod.outlook.com (2603:10b6:a03:33e:cafe::b9) by SJ0PR03CA0056.outlook.office365.com (2603:10b6:a03:33e::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8026.18 via Frontend Transport; Fri, 4 Oct 2024 05:33:22 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.118.233) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.118.233 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.118.233; helo=mail.nvidia.com; pr=C Received: from mail.nvidia.com (216.228.118.233) by CO1PEPF000044EE.mail.protection.outlook.com (10.167.241.68) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8026.11 via Frontend Transport; Fri, 4 Oct 2024 05:33:22 +0000 Received: from drhqmail203.nvidia.com (10.126.190.182) by mail.nvidia.com (10.127.129.6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Thu, 3 Oct 2024 22:33:16 -0700 Received: from drhqmail203.nvidia.com (10.126.190.182) by drhqmail203.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4; Thu, 3 Oct 2024 22:33:15 -0700 Received: from Asurada-Nvidia (10.127.8.13) by mail.nvidia.com (10.126.190.182) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.4 via Frontend Transport; Thu, 3 Oct 2024 22:33:14 -0700 Date: Thu, 3 Oct 2024 22:33:13 -0700 From: Nicolin Chen To: Alexey Kardashevskiy CC: , , , , , , , , , , , , , , , , , , , Subject: Re: [PATCH v2 06/19] iommufd/viommu: Add IOMMU_VIOMMU_SET/UNSET_VDEV_ID ioctl Message-ID: References: <6348cc7a72ce9f2ac0e9caf9737e70177a01eb74.1724776335.git.nicolinc@nvidia.com> <35701c5e-030a-4f52-b6f6-ed18368fb2cd@amd.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <35701c5e-030a-4f52-b6f6-ed18368fb2cd@amd.com> X-NV-OnPremToCloud: ExternallySecured X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CO1PEPF000044EE:EE_|BL3PR12MB6593:EE_ X-MS-Office365-Filtering-Correlation-Id: 7e6ace7a-4352-4f39-95e5-08dce4360fe4 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|82310400026|376014|36860700013|1800799024; X-Microsoft-Antispam-Message-Info: =?utf-8?B?c3o1MTZEdjhRblJ3U2tSNUFFTVFZNmZhQkVYRlkzQlVtcjVNVXF1YWdseGJQ?= =?utf-8?B?dXZOTDljZndZSVZGdUpoMkN1SWJZeDV4T2YvK3ZkZGpaRmtNZ0JVelRJOUkr?= =?utf-8?B?MlFZTkIxUVhIVWt3ZFpOcUk4TTZvME93Yy8vd2tvMTRFeWhtTkFZbE0vSjc3?= =?utf-8?B?d1NDQkhUMnFYT3hBN3lJeG5ZNzJYdEkwcC9VRzdnRHpsQjdZQkV4a0xVNC9C?= =?utf-8?B?V21sZllCbDR0bFhyUXF4R0huTTZBUGhPVm4zR25yekErMzNGS1JuUDhlMUlW?= =?utf-8?B?TE9ENHBXMER1SFE3QmlUZUd1aU94enNSTGpQWkU4WGFKV0RyRldERHR3TWNC?= =?utf-8?B?QXFqNmdscTBKSzNYYnZNRllCV24vbzdFSmh2NmFac05qTThHbFBOUjMxeUlD?= =?utf-8?B?SnFJcnROQmZ5ZDZ5TE1xd0pXQUsxZlpDOUVkcTRBNUVXZndQWjhCTEFmZGZB?= =?utf-8?B?aVlrek5LeTZBd2lnd013UHlXbkZRZmViNDc3ZXgwVXI2eWZWaFNpWG54T3FD?= =?utf-8?B?NktIWFByV2FZVi8vdXB1dzFzNmttR0ViVkJwNlhKOE4vVTUzYVJsQ05hOWR5?= =?utf-8?B?OU50aThrVStWNFViZWlObjZ1M1ZUYzhtWERGMk85S3N4RVVKdGpEZVlhT1Fa?= =?utf-8?B?TW41L1ZxM2oyUU5COHd1VnNsWURhRUNTbHZ0SnhlVkM5Vjgzd1JsL1R3Mytt?= =?utf-8?B?RU1ZYmlaYXpjaDNYb25ldUdQVTkwa1YwL2RUa0JmWWFYdlk4WTBLOCs0TjJp?= =?utf-8?B?cHljd2xHVldhQkZaVW83ZVB5V0J1cDRYMEUwYnpGOWF2SkdCblIrR3FkMmVN?= =?utf-8?B?TjN4WS93OHNmYTZLc0tlU3Jkem5UdHdtU2ZhaUxlN2UvSDFScmdmbWsybHRK?= =?utf-8?B?dWlYbEdMVWVvWWM1L3FteE54UkdhNUR5UGZlaVBaakdOMnZkcTgwU1Q2WDRy?= =?utf-8?B?QlNWMGU3WFdZOTZScWZ4aGtJVzlVVzd6QTVGNTlYRlgrY1hKREQ4bDZDaFY4?= =?utf-8?B?aVBvMFVLUDIxbG1welRPOFZWYkE0Y1pqZXFQTEZTUStVU2haZklVZzZBTXZ1?= =?utf-8?B?dURrZ0VuZUtPOXQzd3BnK2kzcUl0bm1qYThsV3VxYWUycUtoeUpRcWRtWjBH?= =?utf-8?B?VzNwSkkvL0Y5RUQ5NTFWTFI4NjJQbVJJYVg2aWRNWkw5bUZDeW5mVU9rNFZZ?= =?utf-8?B?ZEREMk5RRjFDeW5pd2sxU3NzK1pVck1KQ0ZKRk9PeUZuZ0lNendnVnN2Z0pC?= =?utf-8?B?amtkSnFnb09yRk1TcUxsT2h0ejNNb3ZpYWhVdzJTQmswenhOYXVVOVpBNUxP?= =?utf-8?B?dk1Ua1FrQXRHczcxZEF5ekxSalhuOWtvNitQSFVKKzRGN1NIc2FVMFVjSWJ1?= =?utf-8?B?RkRsbVQrb3JwdjZkQmxLNzVvRXVNaFJVOUYxc3Q1aHoyYWpJUms2Y0N0N3Nx?= =?utf-8?B?ck04OEk5N0hxN0NUSTErV0ttSXFOT01vN3lLanBVcU9zbnlzaWh1KzNpaXUr?= =?utf-8?B?ZDkrcnVhWUJxTHhYWjRGZEZSKzdFRkNzeWRHcVUyL1M0N2YwMCsrbmUyTE9k?= =?utf-8?B?ZTFxTVdlbHlZYUFYL0U5M1B4VEE2c2FidU44eDczZU5lM2FOSHZ4RjRuNllQ?= =?utf-8?B?a3NsYnBwUFVMWWtsQ091OHRyK0E3NjhQVFdvdWNtNkE5OTlxTFJSMEluV1hn?= =?utf-8?B?WXdCNFVnT3dubGQyWUp1SExJZkJzODljcWFVWDFQcjZCUVVWcWY1YXQ4dDBO?= =?utf-8?B?Ykx4MXFTdU9HOWxmMmRrQ0VoTDhQMnVLQjRLSWkrR251RFlzcFVpL3ZORjNw?= =?utf-8?B?T0srOFFDc01EMnk0ZUR1dzM3SDgrUFZPWEJQSGNaUWZ5NGgzbmhVRjF6MmJU?= =?utf-8?Q?zCYw7B0eAvwCR?= X-Forefront-Antispam-Report: CIP:216.228.118.233;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:mail.nvidia.com;PTR:dc7edge2.nvidia.com;CAT:NONE;SFS:(13230040)(7416014)(82310400026)(376014)(36860700013)(1800799024);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 04 Oct 2024 05:33:22.5148 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7e6ace7a-4352-4f39-95e5-08dce4360fe4 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[216.228.118.233];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1PEPF000044EE.namprd05.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL3PR12MB6593 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241003_223333_950447_8E2CAA72 X-CRM114-Status: GOOD ( 26.93 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, Oct 04, 2024 at 02:32:28PM +1000, Alexey Kardashevskiy wrote: > > +/** > > + * struct iommu_viommu_set_vdev_id - ioctl(IOMMU_VIOMMU_SET_VDEV_ID) > > + * @size: sizeof(struct iommu_viommu_set_vdev_id) > > + * @viommu_id: viommu ID to associate with the device to store its virtual ID > > + * @dev_id: device ID to set its virtual ID > > + * @__reserved: Must be 0 > > + * @vdev_id: Virtual device ID > > + * > > + * Set a viommu-specific virtual ID of a device > > + */ > > +struct iommu_viommu_set_vdev_id { > > + __u32 size; > > + __u32 viommu_id; > > + __u32 dev_id; > > Is this ID from vfio_device_bind_iommufd.out_devid? Yes. > > + __u32 __reserved; > > + __aligned_u64 vdev_id; > > What is the nature of this id? It is not the guest's BDFn, is it? The Not exactly but certainly can be related. Explaining below.. > code suggests it is ARM's "SID" == "stream ID" and " Yes. That's the first use case of that. > a device might be > able to generate multiple StreamIDs" (how, why?) 🤯 And these streams > seem to have nothing to do with PCIe IDE streams, right? PCI device only has one stream ID per its SMMU. So the Stream ID is more like a channel ID or client ID from the SMMU (IOMMU) view. A PCI device's Stream ID can be calculated from the BDF numbers + the Stream-ID base of that PCI bus. That said, this is all about IOMMU. So, it is likely more natural to forward an IOMMU-specific ID (vStream ID for a vSMMU) v.s. BDF. > For my SEV-TIO exercise ("trusted IO"), I am looking for a kernel > interface to pass the guest's BDFs for a specific host device (which is > passed through) and nothing in the kernel has any knowledge of it atm, > is this the right place, or another ioctl() is needed here? > > Sorry, I am too ignorant about ARM :) We are reworking this ioctl to an IOMMU_VDEVICE_ALLOC cmd, meaning a virtual device allocation. A virtual device is another bond when an iommufd_device connects to an iommufd_viommu in the VM. The name "vDEVICE" and "virtual device" still need to go through discussion, so they aren't finalized. But the idea here is to have a structure to gather all virtualization information of the intersection of the device and the vIOMMU in the VM. On the other hand, BDF is very PCI specific yet IOMMU independent. E.g. it could exist for a PCI device even without a vIOMMU in the VM, i.e. there is no vDEVICE in such case. Right? So, if your use case relies on IOMMU and it is even a part of the IOMMU virtualization features, I think you are looking at the right place. And we should discuss how to incorporate that. Otherwise, I feel the struct vfio_pci might be the one to extend? > > +}; > > +#define IOMMU_VIOMMU_SET_VDEV_ID _IO(IOMMUFD_TYPE, IOMMUFD_CMD_VIOMMU_SET_VDEV_ID) > > + > > +/** > > + * struct iommu_viommu_unset_vdev_id - ioctl(IOMMU_VIOMMU_UNSET_VDEV_ID) > > + * @size: sizeof(struct iommu_viommu_unset_vdev_id) > > + * @viommu_id: viommu ID associated with the device to delete its virtual ID > > + * @dev_id: device ID to unset its virtual ID > > + * @__reserved: Must be 0 > > + * @vdev_id: Virtual device ID (for verification) > > + * > > + * Unset a viommu-specific virtual ID of a device > > + */ > > +struct iommu_viommu_unset_vdev_id { > > + __u32 size; > > + __u32 viommu_id; > > + __u32 dev_id; > > + __u32 __reserved; > > + __aligned_u64 vdev_id; > > +}; > > +#define IOMMU_VIOMMU_UNSET_VDEV_ID _IO(IOMMUFD_TYPE, IOMMUFD_CMD_VIOMMU_UNSET_VDEV_ID) > > #endif > > Nit: "git format-patch -O orderfile" makes patches nicer by putting the > documentation first (.h before .c, in this case) with the "ordefile" > looking like this: > > === > *.txt > configure > *Makefile* > *.json > *.h > *.c > === Interesting :) Will try it! Thanks Nicolin