From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F1DC1CE7A81 for ; Thu, 5 Sep 2024 18:23:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:MIME-Version:In-Reply-To: Content-Type:References:Message-ID:Subject:Cc:To:From:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=nkbH/xW/9/ExEv0/zVk5vyP6gViaMeMxI4OE++laKv4=; b=TAhnQpSX63PuItKEUyBjAeaAmC a9Qx1C0GGMwE3e3kOsqLsS1kNMPiwv71JLIJNm0iigJn88rZh5eWoeV9YegdVt0HF7iI3iG5ylw7j JBx9SGGMcxPErP6hWFqgUlifWayPnJXCHMwDslntz5jxau7TzIQskye+61QG3l7Dy0mOBlK/FzxCs RxI7KUFsbZ8V4DFCoYOP5vEjwqoc80zcSHL+TjpawmCYPms0jwU4ITiybnmJUS4yhKTJUI1y3yn0o Qoyh+KPx8eZJUX1le8UTLYEqB0zOg0izhE9on3mr12K9od7hBzyT/cHpkbJCJS7Z0SilxnhAdJxTb zEAmM8OA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1smH8c-00000009Qha-2dJT; Thu, 05 Sep 2024 18:23:22 +0000 Received: from mail-bn7nam10on2060e.outbound.protection.outlook.com ([2a01:111:f403:2009::60e] helo=NAM10-BN7-obe.outbound.protection.outlook.com) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1smH7G-00000009QGO-1Cqy for linux-arm-kernel@lists.infradead.org; Thu, 05 Sep 2024 18:22:01 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=TNiXfb2JLBskYaZ5DuUASXaUgTHd6gEsLgJ0bG9ANC7xFXUDg7knLcQSoeMuLus2uiHKJ6s4atTXYFF1GFpX1P1Z9nTm5D5OC85KV5F5SVSWFQLoy6rx/HeLDBUF6U4q12r9VQOCW3DLlgryi47esGXWKxgR+1nXV+JWm9VQpQKxd2B7hex4T5qPlXPATuzvLjnjm6u909MZj5h33kVUY0VlHTfeDLDAgOuTDIsZYLKMLowpjonJHT+8CA4rCteiwX+tyhLNc2sp2npT1xfNFJSOT1bEElQyKIXdK7nH8BgJBqtjn6hd7zza79lNCzyiwy+WFt7bDR1rivMN2GPA7w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=nkbH/xW/9/ExEv0/zVk5vyP6gViaMeMxI4OE++laKv4=; b=pjEdukV/iHU8FeL4AwESxydUP1vKcxa2//NKqYGiRtglSdA5jAwr9rPOCvwReyqurgl6JgH/vrQ4lLkOm46GKb6Nm/YusNskuyJST6P4o5nqujBaU/i3tulOt8+qfFwcFAfyA5/NWIqNVt+9yIHJHUjLiVCE1PqjTqR/INZQDRVdHAsTGevSket+5O/Jp9Juba9qqFdgJOvkq13OydqGuX/XgJLc/q61MQpwSzylsc4OyP00/Jk/LDGE76yjlyqfygENGK5jk0lge2QphYfsWnQ+Bl2mMab5P4rsgm+dawOecbWVVvR3sJAh3xBq0Mc6lClbvQQ9ULxrZTvn22i/dQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nkbH/xW/9/ExEv0/zVk5vyP6gViaMeMxI4OE++laKv4=; b=lmBFd1A9TiV+y+NBMetElv0lbg8Ckw+sAr4F94dspjK8V6q4Gc39pfYtAe+73/MGzv7mODJL04edKgzGC8SLFDyiK603D2GoHkLXkFwOyJoqJglPyPCsgQcFt8hU0opu/KpRcXH7SXyLheH68/hg/OMfAo8fFy9lfN4IYxQnvzKigE0kwqwKhZB1iK//uvRCtskf+RUEElt1BJbKz7hz3Uj18h5ziK93urgC+sToA1tfB4zD/tHJL50ZO05HTDB/ErsMGzr0UAcYNmxuFXpvksSEKVGBzyVjr29vl54kKpcqYXl8hKJT7jxfcjhHThpHWdzgyv/yRZcH2oKv0ZNhSw== Authentication-Results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=nvidia.com; Received: from CH3PR12MB7763.namprd12.prod.outlook.com (2603:10b6:610:145::10) by CH3PR12MB8233.namprd12.prod.outlook.com (2603:10b6:610:129::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7918.27; Thu, 5 Sep 2024 18:21:50 +0000 Received: from CH3PR12MB7763.namprd12.prod.outlook.com ([fe80::8b63:dd80:c182:4ce8]) by CH3PR12MB7763.namprd12.prod.outlook.com ([fe80::8b63:dd80:c182:4ce8%3]) with mapi id 15.20.7918.024; Thu, 5 Sep 2024 18:21:49 +0000 Date: Thu, 5 Sep 2024 15:21:48 -0300 From: Jason Gunthorpe To: Nicolin Chen Cc: kevin.tian@intel.com, will@kernel.org, joro@8bytes.org, suravee.suthikulpanit@amd.com, robin.murphy@arm.com, dwmw2@infradead.org, baolu.lu@linux.intel.com, shuah@kernel.org, linux-kernel@vger.kernel.org, iommu@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linux-kselftest@vger.kernel.org, eric.auger@redhat.com, jean-philippe@linaro.org, mdf@kernel.org, mshavit@google.com, shameerali.kolothum.thodi@huawei.com, smostafa@google.com, yi.l.liu@intel.com Subject: Re: [PATCH v2 17/19] iommu/arm-smmu-v3: Add arm_smmu_viommu_cache_invalidate Message-ID: <20240905182148.GA1358970@nvidia.com> References: <4b61aba3bc6c1cce628d9db44d5b18ea567a8be1.1724776335.git.nicolinc@nvidia.com> <20240905162039.GT1358970@nvidia.com> Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: X-ClientProxiedBy: MN2PR07CA0015.namprd07.prod.outlook.com (2603:10b6:208:1a0::25) To CH3PR12MB7763.namprd12.prod.outlook.com (2603:10b6:610:145::10) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CH3PR12MB7763:EE_|CH3PR12MB8233:EE_ X-MS-Office365-Filtering-Correlation-Id: 5122fddb-8ce9-4bac-0776-08dccdd79bd5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|376014|7416014; X-Microsoft-Antispam-Message-Info: =?us-ascii?Q?Owcqf48mcWXj+YsgbHgCvl359+sSYMaxdEJpbtIRAjEmJbmXEllJzDOI5uYh?= =?us-ascii?Q?nfOqo8CqU+xWhCopgip5IsmChrbuDrGoOEd0m/nuN1/DwE5VVDJ2nA4Ia9sz?= =?us-ascii?Q?WkBVGZm1UpAEarO12HL10/h1uBy3YxauUkrFDDBVn6NjJjFk+M3bhOUFSsIq?= =?us-ascii?Q?ZDy6mF89cMuAsUvwNjUINJStVgwzxh3j4RhA5PHe4MEuxMbQqRwuUrTrXjej?= =?us-ascii?Q?yK4UVGe7IUoWzNJlUqUYFSRPlfp0AB2s0x/u0atGuQiHBasLhqeIjxQ2qhQZ?= =?us-ascii?Q?Y6JNwhk2nsGJHdHlIIkkoZcP59oXtwLKX9cO0jwfHyDaeQXXB+mTtBI2Wx1j?= =?us-ascii?Q?Q3I1/+CgkjF79UrfYrcIioGZVDMPYkdQfiP9MyS9DhEt2zBk3jYN0Yem1zQ0?= =?us-ascii?Q?hyctboBZm8ifjKT0FcdHpZnQKxcLjqE7TjYqHfDQFsX1CfXyGi066PQJ+R4L?= =?us-ascii?Q?ek/ex8lAvgavRokzWn+Apdgj/2XMaM7kANI+M9KLzSuaf197CvsC6zEvWBBI?= =?us-ascii?Q?hbsLvDwJ1NY1LA1RnPXjLDqJXoMhhiWqLaiEg6LjOF2Dcfg1FdnqnbaDc12v?= =?us-ascii?Q?/pim3+AFjGRUwcgxhXVw+Lp0z3xFbpAhS8uF/QfTf6kjtPDMkV7diIHYP0Yp?= =?us-ascii?Q?4HgDUu9r4tVgN2pG7Dlbkf9B48oYb7OGjkoYIUryUNUtye3M6Xcjfjxnjl8O?= =?us-ascii?Q?+SLFTdMG8qEOSPrAJUNP6oTiY2j53HlpIqZwJwvL76nE2gq+U5C4ds2Nh+wR?= =?us-ascii?Q?8s4D7n4SnksOOndk8Cg2Wt9Uc3a1L3m6l8VX52K7xwVrXw9TYqud2n6SsqAI?= =?us-ascii?Q?Nyr6zCzZ9tP15TWinZKXrTsRzdLr6kO1aabj5FriS4+opvQWI4MXW79mNklx?= =?us-ascii?Q?dPl8BZdHnt6+wGzjVcNUfh7uhApCGDFauy5utbC5z6HtevXPTlvNSUyZwaHP?= =?us-ascii?Q?GLk6UZ9R2hj9G9aD4MNDBtgYp842hHZQF1ujjBno7t/Qb3mYIDv/6Kmg863/?= =?us-ascii?Q?nojaI2pVWmdZWq9PDBXfCunAbX/9IB7itiMFqXi2Uy+2i7Q5fBNs+g3roLxS?= =?us-ascii?Q?YYgYJ3LRlHJvxMuIPMLUTWTLGexLFes0AKFQX+htoJCDJvpqU8bC+IqNfOxs?= =?us-ascii?Q?qkNrMYpS9EeLVCX/QZJNNrdO3Icpba/cPtYtxoOWnI/C1GS6UCTV0DS6lvdd?= =?us-ascii?Q?7/I0hq/aPCWCPJKAyhU55bQa7UzUVtqOuDZP+WnyZNytwweypf1QhehYjesU?= =?us-ascii?Q?YdBOcYflZvHm5VpRD+8GhSd5CcN0qXERJlFGRsUA/5W+q4GfirVGjvyw8FUy?= =?us-ascii?Q?4G7I77zk03TP6sBr0jJm49DPpEWeWbSx8S0WKG4Dta4L4g=3D=3D?= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CH3PR12MB7763.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(366016)(376014)(7416014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?+4lSrKytwL70UzYvgFvW/yky+Z2uH9U8dUNRRmt+vZRuxYNKQJGaVZ1TXgLP?= =?us-ascii?Q?Lxh9V2s6UnRjzEThsxBIC0BIMCvczsFQp7b+nxMtcHEGHj5UsiPXtlaZKg5j?= =?us-ascii?Q?5S+4LvP0TUrkvW6UUNzG8dJQX/t7BJsRwjl2Gw18aP3Zt7Len20pwjxi8DlP?= =?us-ascii?Q?yV8Hx4YJbfmS7kiI/GEXimZcc8SMR+7OULb/wt+F54vAtK0yXgmJ4DOJdTdh?= =?us-ascii?Q?ir3gNuzds/rliSD9QIqx7zoY9V9RczjSWnjbKGyvnEjzBPA8mD1ibLR8Z96V?= =?us-ascii?Q?N5i0XsHPyhlpZVUbH50IaaarYJfQwAo5EP7iB35J4begAeTpzVdaVT0AwvLS?= =?us-ascii?Q?6uVKcJO4hWT4Wr8UE/cIXb0HhBgGJsQaTkHQkvOhwLOMoK04s+EhFxe1Ewgc?= =?us-ascii?Q?34rJR3n3rMjWuV92bYfMd1TYlbaR/AZrarN7JSLdWlWV8Nv6C54xdNEmdvc3?= =?us-ascii?Q?1TzmKKB5+8U94jLJ1qfBbNykiwobELJM2FLrXxhs26l2H0vRIQWnKEGbTh2z?= =?us-ascii?Q?qEE+4UdEH4SvQCgTVf9ORO+3Fj5RqixKgU2lPrdhPPEej4S1+ZIjOHBAdw3Z?= =?us-ascii?Q?q8rNGGn6aupmqLMNNSfIRcANZIBAblbvvFyi4A+lga2kYBP2menDOwr4lf9+?= =?us-ascii?Q?1+8e8el7ittr5zjTZwnGpj3ljiTEpcnRVIQwJTFkW7bseFXyFni481ZOc1mI?= =?us-ascii?Q?xxgB0H257VfNQM1f4JHiuFIRBSd3nBmx0R62FqKNMBeFNgRLQ5i6+FQ8dsRt?= =?us-ascii?Q?qvI1v6OTqX6TTq1B8C+/3JJyETBXEbTjIw3udYxqkw5ybJsTX0Pf2CqcmFZn?= =?us-ascii?Q?XgtXJA/Uu5W843Yaz7BXOKPCFR/dY/alZgHjhp4ANw2CvgrS5a6KPpoNhqQM?= =?us-ascii?Q?pHNKis95et2PprGOEUlDxij8qgvlC5y5WxqEL7Vo6v6x7kKxpk2ebkNFVv24?= =?us-ascii?Q?rtHG5BdxElMoU82gyQGkL+FDqT1zVL48W5KO8HIFtMsDkLnLd9EDaVuab13J?= =?us-ascii?Q?TuA/8R6xXIQmmF6V0LZ/444Y3kjD5PPcV+gakdqOHkBl0JbIFrmqnRb/IEJB?= =?us-ascii?Q?tJNW/AXqmE3istjF/bioCoM4t3JRd/d/9CdjcYrb70ECOtE/LJnVzwOEk7vt?= =?us-ascii?Q?/Ig5VNska6zsrf9S5MrVE9PI/SR5LgAifwfr/tizDxEUrEU1j30HYqOTvXeW?= =?us-ascii?Q?dRBXETpco2IM0bXL59iOHymOyv8g7RJJst6IgJLcoRVVpzUK1eO1PaKUuLLD?= =?us-ascii?Q?Th0OdYpKXTPuPX7/yKzaH0/YDe7fv0EYwIjT865636EvFXYKp+mPzE0p/NGT?= =?us-ascii?Q?GcLKO9tLNvANxNiAKepIsmY1p/DVl1pwA2LirZryBa6bKUsxuk2ADehhRXGe?= =?us-ascii?Q?cIE9CFvd3L6gVom9xNti+ubnIqhInroaCXWUpAkkM4wehxBOxo27XacnZi1g?= =?us-ascii?Q?i5kuDEqQuFwTqBGaVJ0rxjDuBfj2M4uBwPiB7QuGfNxKVX4Oq1BrVus0c1hR?= =?us-ascii?Q?lrEqlR+W99xcBaCVIQL9Z2nTgY/8xLb+Ho7Zs2d5nhw2UR4YeUcCJbduoAc4?= =?us-ascii?Q?2HaUbTMop+8O85/apTfdLdtGEi5Ombksbj1Jtp6M?= X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5122fddb-8ce9-4bac-0776-08dccdd79bd5 X-MS-Exchange-CrossTenant-AuthSource: CH3PR12MB7763.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Sep 2024 18:21:49.6925 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 408RZ7i1pMC+HYlEL66AvCCJ10+lRQzS4OSz42SLnKWx4OzaQD/iViu2UaT3dIiw X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB8233 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240905_112158_457058_8E80A3C8 X-CRM114-Status: GOOD ( 21.52 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Thu, Sep 05, 2024 at 11:00:49AM -0700, Nicolin Chen wrote: > On Thu, Sep 05, 2024 at 01:20:39PM -0300, Jason Gunthorpe wrote: > > On Tue, Aug 27, 2024 at 09:59:54AM -0700, Nicolin Chen wrote: > > > > > +static int arm_smmu_viommu_cache_invalidate(struct iommufd_viommu *viommu, > > > + struct iommu_user_data_array *array) > > > +{ > > > + struct iommu_domain *domain = iommufd_viommu_to_parent_domain(viommu); > > > + > > > + return __arm_smmu_cache_invalidate_user( > > > + to_smmu_domain(domain), viommu, array); > > > > I'd like to have the viommu struct directly hold the VMID. The nested > > parent should be sharable between multiple viommus, it doesn't make > > any sense that it would hold the vmid. > > > > This is struggling because it is trying too hard to not have the > > driver allocate the viommu, and I think we should just go ahead and do > > that. Store the vmid, today copied from the nesting parent in the vmid > > private struct. No need for iommufd_viommu_to_parent_domain(), just > > rework the APIs to pass the vmid down not a domain. > > OK. When I designed all this stuff, we still haven't made mind > about sharing the s2 domain, i.e. moving the VMID, which might > need a couple of more patches to achieve. Yes, many more patches, and don't try to do it now.. But we can copy the vmid from the s2 and place it in the viommu struct during allocation time. Jason