From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5F732C27C5F for ; Fri, 7 Jun 2024 14:49:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:In-Reply-To:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=EsoeH0kyNuNFKQdNUA92F7eXY8GdVqms5vFSBCpQTsM=; b=B0pK9PxKBIFv3f cN4S+skvZOy44xPfntjko4PPLqjQA3gKC5LCq7HtRQu4bWNC/y+3rhAUVPhAl2q95Ktr0IrPYW/5F PwChGYDid5k9+q5eY7FzscnII2017+bCrw7IqxcVoHifzPjV+kKDUvWotmJxsteVenSt790eD4yry VPmRwRXOZiZ6ebAESgMsgZEcJjzBZMxfY49n6v9dMd4ivzMSfxU/aD45mcAcYf/brFuUoF0yhQrtb mzSodII0nW1dZyeHVeeNEiepBIeoG4yLugiYqSdO7bIK6enbG69utX3ums4cXcxSHNemtIv9dRZ0X M86d819H3EdwVVRD+SWQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sFauR-0000000EVuV-12QK; Fri, 07 Jun 2024 14:49:39 +0000 Received: from mail-dm6nam10on2060d.outbound.protection.outlook.com ([2a01:111:f400:7e88::60d] helo=NAM10-DM6-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sFauN-0000000EVrE-247h for linux-arm-kernel@lists.infradead.org; Fri, 07 Jun 2024 14:49:37 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eDKkf4bT0OYpkUTN2NLQ+9vy7JspJgLbwGnkpWqbXNkn8g2ZjoIlkHfdWvdRGYLhgEKQv4VC2RAMcEqj9YPJhns3sMZHXn7bOsPGqdEzuWZPw9ZMzmGuaNPs2SD+/J82CP3YYlo51/vdJJM6QL2Mfak21ZIKBkOrFxn8+ADmWcIHqVES6TOha9PUJrNlo0wM0iVCnStLQSPI+TDoMcbwZNNMMNGO4HsXBvVtw8w/29y61iH2eyYyCjhZ3qRlPC20vsWntRK+A3zmQjHZCG8EC6R6+ZpeMlAV32s/R5gdvi0O9dVakK8zjLQ+ywXmU1wndT5qKHGUFDa89GLwQ9JXiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9Yh89vploRfSJcxSon1/E2I94AyugCqbc8IScQp4OEI=; b=hlVh4aTtLwfySO1cTTfTvYCd2RYI4D59Kxt1vtu/Bo7cqAPK7cKFrWKI0Ep7KjVKNntLCsCffzExfCUrh15tWicpTH7ge8liSAuAOAbEFlRg71Uz6YfZYJ7VLlcwTW7NllKWcj+KOS7TPJpwl4IrfcqmCMzjdK0WOkmfucOA6vyQrdF5JeBl6X8kHg6qHyzreUNdeXUeON9RGU8xDw9gl5GDqCtsKuDPE8jho0Yj+JI4IT0ug4SCuij7ZHAa/uVH6V8YhGWyt8lyxoKC1E5Rzb30+Oli+aSQ31ZY4JpqwlUU/BCCgSNal0LgQehEKbfyppCoCMTsHTwH/WcPykqHjg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9Yh89vploRfSJcxSon1/E2I94AyugCqbc8IScQp4OEI=; b=PT8lUWud/71WtrisLYPSGGVXGpemODuwdrJYi8q/6kgW2YD9uvHTYRRmiMrtbfPOVFvi1h/VT3USiulWQRrYT7N7DoX7D33FgaDOUYKY90/iSlye9igqko7wjZ/GKn+LRfmV/epyStFTCjJj4UkPp7t+SCBzaFHOc08wVMRyrlsNiNpsN6IIcJx2HgKE5MEznyqOgR7D6YnLigs+lMGlS/jL3P9hI5t2Oeh6bZyXlBYLST3r38lgYwhKYY06nM6zHkohUMYs0MeDl8hZVJnwPCe5dzUe0uu/RYg8v3E6XNNpYEp4kESy/duDb50jGIkEjtTWnWfi79cHZSG75cYB2g== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) by MN2PR12MB4078.namprd12.prod.outlook.com (2603:10b6:208:1de::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7633.35; Fri, 7 Jun 2024 14:49:19 +0000 Received: from DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e]) by DM6PR12MB3849.namprd12.prod.outlook.com ([fe80::c296:774b:a5fc:965e%4]) with mapi id 15.20.7633.033; Fri, 7 Jun 2024 14:49:19 +0000 Date: Fri, 7 Jun 2024 11:49:17 -0300 From: Jason Gunthorpe To: "Tian, Kevin" Cc: Nicolin Chen , "will@kernel.org" , "robin.murphy@arm.com" , "suravee.suthikulpanit@amd.com" , "joro@8bytes.org" , "linux-kernel@vger.kernel.org" , "iommu@lists.linux.dev" , "linux-arm-kernel@lists.infradead.org" , "linux-tegra@vger.kernel.org" , "Liu, Yi L" , "eric.auger@redhat.com" , "vasant.hegde@amd.com" , "jon.grimm@amd.com" , "santosh.shukla@amd.com" , "Dhaval.Giani@amd.com" , "shameerali.kolothum.thodi@huawei.com" Subject: Re: [PATCH RFCv1 08/14] iommufd: Add IOMMU_VIOMMU_SET_DEV_ID ioctl Message-ID: <20240607144917.GK19897@nvidia.com> References: <20240606182423.GF19897@nvidia.com> <20240607002707.GJ19897@nvidia.com> Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: BL1PR13CA0108.namprd13.prod.outlook.com (2603:10b6:208:2b9::23) To DM6PR12MB3849.namprd12.prod.outlook.com (2603:10b6:5:1c7::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6PR12MB3849:EE_|MN2PR12MB4078:EE_ X-MS-Office365-Filtering-Correlation-Id: 192f64b3-b2bd-469d-643f-08dc87010255 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230031|366007|1800799015|7416005|376005; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?TPaoS8Y+Pipio0IfwrYZmDBKslF6vPpFVxeo3VBHnI2z8GDanUZ6DOCXAB8S?= =?us-ascii?Q?92ovzIuZri/S7l/j2UckEYYGoUCwWX7tT5XbZotwI1u5tqpKQAIhiyh7YEYc?= =?us-ascii?Q?3Wt6TlqkiyBg99J5g2paX4cpe/mn89kmd86W1HZeoQP5PJbjw5woVbelj+iF?= =?us-ascii?Q?sI22aIZX+xTIfLSlNjfNfzr/TqmbWDAAagNNYXhdlVPDcM5vBs0L1DTxLx9q?= =?us-ascii?Q?p+aXxeVC7cQYjXmqEcgGaSy5f5MiY4JpxadfGMKzWvy+Tv2oTAQvx1RzRlOe?= =?us-ascii?Q?gl3FvdnfQpJUKjaPhLiL0zK8nCmt/AXWkFQDZP/uBYmWKwJUdm+no7MGc/xd?= =?us-ascii?Q?TcTj1yJDYym6GBw/5ZSJGJ5PfiA4eot0KQtIgOsARRGw/vFDPdNQo/tWduC7?= =?us-ascii?Q?mXdZdM1ra5w5q333hLFADbOQe+AXjvJkjV38fianeGzrLHffcl3IJ1WC+kKQ?= =?us-ascii?Q?zNEyS1XYBU6vYP2QPbf0YixKreCUB4z2xyBlAUftuvJU0Irsaun12jRpduPv?= =?us-ascii?Q?lJACKB/OUIQbhzf0HYB9BN1madHEZc5AsxrEfCDuJiNnNqy0OG8l+UuiMqoN?= =?us-ascii?Q?zHqLCeA/XETFa3pwKmFtDgJbO1nwa+DoU4a9t9owpY26QU9vOIEj9Xu52HB/?= =?us-ascii?Q?htwNd9x4UZW5GRy7Ylvny5Tl1Umngq9ESDs619hEoY1sUE/lCPfe25Jgl8sR?= =?us-ascii?Q?O3Zt2uDW7vYHI2Ve6vzxLaSq99XVeTAaj7Hu1CJGPHS0CKGpEPBMKhGIGjMO?= =?us-ascii?Q?BxQmUPum9y/02rr8d/Tb8ZpCR7CO3NtZi4O8vxMpN5GRdpZiJ0HevJVYcVaP?= =?us-ascii?Q?9twd/qYYAu+DzLDq2cQBpOtb48RLcVAkKk67G7LRSzolfSJI3m/LkCuBLB+u?= =?us-ascii?Q?bN4T4ewHqr0zjOl5d53qr0Tgopp0V0dSGwIvDsF+S8P5XU8r+snPIbF0bpCz?= =?us-ascii?Q?e6uQlGIQVZ919ykH3RsENYlvGDLtGQrlIv53oMkONhyY0NhDCaHc7s6FP6L1?= =?us-ascii?Q?p4477mRRvoNh3KAl4kGI/tCZ119zPRnQh4/UeKXgxrgiR2eQZintFjPaO398?= =?us-ascii?Q?w2gfzr1m6CPmGXCLo7vrbYyGIqJ47Cvgd38jWz+SWPiisofMDBTaE/43Vkoi?= =?us-ascii?Q?UTxkDpLp2jNFXYX66I8bkwb8E8AK2Y9FAe8as0rUQ4Lr9ifxUv7RHCIxVrX8?= =?us-ascii?Q?kqUFvPYATlxBMkvaJimEvz4uUOGS+sITGQbXgAtO453FpbVjk85aTOjXasjn?= =?us-ascii?Q?0xTsLOMALGDFAhX/cSXow3ePaJKv/RcfLOx3IQqtdorLBqQsEQZCgJPbCXH3?= =?us-ascii?Q?7OhTmlctYXnsA3vhCRKuJJ7H?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DM6PR12MB3849.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(366007)(1800799015)(7416005)(376005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?uP+XD6whdwKn1KtTIspcqxH82fgnh1Ahhy0TncM9x4p2X9kP9arlgBVe605D?= =?us-ascii?Q?UpB8yi7Klae2RvuzLns8uhhQ8Inu7WBV44jtTnZt0it7Y61IN7qYBAlcL01t?= =?us-ascii?Q?gh49FHkks15/cNsM7xGg6FXx3ymK1ASJ0D2IjxOtoLnc/nI0PcgXIengvSsP?= =?us-ascii?Q?UyRQU1mN1hqiuuYhsWl4LjNd53EzmQRNBmjv5y+xGqepNINOksRn9GBLjzjo?= =?us-ascii?Q?tkNp5I63G5ayftaXRjdcHLqjLe7rLb/u53KLrajHqFP0oW+9+CG2GvSg1viO?= =?us-ascii?Q?vmjw2frJbPnjsSLlNSVK6Uswvovbzfcb2tGNrV6Oledtq57rSC9ORKJ8EAnN?= =?us-ascii?Q?oHE9qD7g7gwTe48Zni3/1ll2RNGKAfpTYPHblbAXX193YFpQP97dnUtOiwzg?= =?us-ascii?Q?GrrMpT5r6WM6itQzGLZFmlPMLDHXvn6prYnV+Q/gvURL6+TbQ7titbvUj0nH?= =?us-ascii?Q?n/h5xdby6/JvSnD9p0LFxjIuYqgympkhmR4DWGjkHTKj88MShpVi6SzZiIwR?= =?us-ascii?Q?UeDLYax30rOTfBqC8kHXBWgxDTsNjrcuNXeJ5c1m+NyN6ymOUlWN1SjrvoKy?= =?us-ascii?Q?kAubfghP942xDzxUZ6IftTdjN3p39DBFEcCwacglUDC3sqYIvQLHoTvpEvCm?= =?us-ascii?Q?nLIcPN130Z21UuNm+4O5lJ7eZoGvCgyolF8caxyE6YwjfWLD5/yYQk/nq1YV?= =?us-ascii?Q?rUymk9cykGo2j3wznBboX9nA+4/xtv/JNKb0TTOVafoC10MKuMMT4O/I879f?= =?us-ascii?Q?w3r13/sspKIA+8cbr0QlGwyc7K11pL7zYmsP2vytS/XgYtwH3Tglf7Jpli77?= =?us-ascii?Q?qby3C1uy24iWxbTUvyFBxa8xKysnHV5BN7WtwZBfXuIFzK8o1/eOcNMDqspK?= =?us-ascii?Q?gA5JNVOq6pMOB3aM5rneRQvKXF+3IU7Lr/ZlM5LvREni7ohA66FEeEyFAW8k?= =?us-ascii?Q?uAP0RlP5rL++mOlS2Tyk38PkzEqis1qSzOv2RYwOr/doh3x/s1wN33Praf6a?= =?us-ascii?Q?qIIZghBd7z0KjYPob7ya0V+R5fbKrfYBIs15pbeVVs+TOH+vCaLtAWrLwhZa?= =?us-ascii?Q?zh3sCX3qJzPD37TdIFEPTsWrVI7wSc3tHIzBx9pta9ajckjdsmaAX3/oDajE?= =?us-ascii?Q?ELullEPgCjXhE5mJduMQ6I9w2wp3w5ZxEWKQd8C6MdTfZgbQTf+ud120ObtE?= =?us-ascii?Q?JNs7HR7L5Ylb6ihwUmqZzj73j97yjZxvETBH/lN9jY33zt/DRDk7JkaDszBk?= =?us-ascii?Q?j0eVoLianCQDSnRH6MDiBlG7tqDkxld+37CHAwo4bOZmvxnHfDveQgD2ZP26?= =?us-ascii?Q?MosDHLmFU6Z5iyhFRMiT0Tkoy/BOvC+1EcJHDblWhQAWPj5F/vg5qSSJHL90?= =?us-ascii?Q?Kwct5gT+BgyZsPyjF2IOR1S54BwwVmYXKWqDTF9QriPg3DGluiuG1hka5QWe?= =?us-ascii?Q?BfWdLp+z56UNu4bmMvDgNnFGsYVxR+0sNf0SdNbRwOo9SC5ptVNQe2E4tyrO?= =?us-ascii?Q?ncWMciUB1DPLqsjvmelzmTIgiV0MiKeZrQCJm2QzUg2rVDTVpA1f9PDm9m3S?= =?us-ascii?Q?p7I7thhFSWy3ssDBnQ5h1mdBIWIwdQlJyxs97zWl?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 192f64b3-b2bd-469d-643f-08dc87010255 X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Jun 2024 14:49:18.5078 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: xPRHPqhnIwPmzm7B/mIDFFQUwIJ3SHRm4xlQWSEV+/kNWxEAfVKvDr9UAqubn8lC X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4078 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240607_074935_985697_4952DA72 X-CRM114-Status: GOOD ( 20.92 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, Jun 07, 2024 at 12:36:46AM +0000, Tian, Kevin wrote: > > From: Jason Gunthorpe > > Sent: Friday, June 7, 2024 8:27 AM > > > > On Thu, Jun 06, 2024 at 11:44:58AM -0700, Nicolin Chen wrote: > > > On Thu, Jun 06, 2024 at 03:24:23PM -0300, Jason Gunthorpe wrote: > > > > On Sun, Jun 02, 2024 at 08:25:34PM -0700, Nicolin Chen wrote: > > > > > > > > > > I understand the appeal of doing this has been to minimize qemu > > > > > > changes in its ACPI parts if we tackle that instead maybe we should > > > > > > just not implement viommu to multiple piommu. It is somewhat > > > > > > complicated. > > > > > > > > > > Would you please clarify that suggestion "not implement viommu > > > > > to multiple piommu"? > > > > > > > > > > For regular nesting (SMMU), we are still doing one vSMMU in the > > > > > VMM, though VCMDQ case would be an exception.... > > > > > > > > This is what I mean, always do multiple vSMMU if there are multiple > > > > physical pSMMUs. Don't replicate any virtual commands across pSMMUs. > > > > > > Thanks for clarifying. That also means you'd prefer putting the > > > command dispatcher in VMM, which is what we have at this moment. > > > > Unless someone knows a reason why we should strive hard to have only a > > single vSMMU and accept some invalidation inefficiency? > > > > migration? a single vSMMU provides better compatibility between > src/dest... Maybe, though I think we can safely split a single pSMMU into multiple vSMMUs using the IOMMUFD vIOMMU interface. So if your machine model has two vSMMUs and your physical HW has less we can still make that work. IOTLB efficiency will suffer though when splitting 1p -> 2v while invalidation performance will suffer when joining 2p -> 1v. Jason _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel